Subscribe to and Manage Data Sources
Data Source API reference guide
Last updated
Was this helpful?
Data Source API reference guide
Last updated
Was this helpful?
This page describes the dataSource
endpoint, through which users can subscribe to data sources, make unmasking requests, and manage data source tasks. To create data sources, see the specific handler endpoints.
.
.
.
.
.
GET
Search for data sources.
GET
Get a data source based on the ID.
GET
Get data source based on the name.
GET
Get data source based on the short name.
GET
Get parent and child relationship records for derived data sources using a specified data source ID.
GET
Retrieve a blob.
GET
Get all users with the provided access level for this data source.
GET
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
GET
Retrieves a summary of total records, total visibilities, and visibilities a given user has access to.
GET
Retrieves a summary of total records, total visibilities, and visibilities the current user has access to for a specified data source.
GET /dataSource
Search for data sources that you own or are to see.
blobHandlerType
array[string]
Describes the type of underlying blob handler that will be used with this data source (e.g., Custom, MS SQL).
No
subscription
array[string]
The requesting user's subscription status: pending
, owner
, subscribed
, not_subscribed
, expert
, or ingest
.
No
status
array[string]
The data source status: passed
or failed
.
No
tag
array[string]
Filters data sources by tags associated with the data sources.
No
searchText
string
Searches for data source names using the provided string
.
No
column
array[string]
Searches for data source column names.
No
connectionString
array[string]
Searches by connection string.
No
schema
string
Searches for data source schema.
No
nameOnly
boolean
When true
, searchText
will only search data source names. Default is false
.
No
idOnly
boolean
When true
, only returns the ID Of the data source and the user's subscription status.
No
dataSourceIds
array[integer]
Searches for the provided data source IDs.
No
selectFields
array[string]
This field accepts the values id
, name
, and columnEvolutionEnabled
. When id
or name
are provided, the request will return only the ID or name of the data source and the subscription status. If columnEvolutionEnabled
is provided, the response will also include information about the policies, policy conflicts, and workspaces associated with the data sources.
No
offset
integer
Used in combination with size
to fetch pages. Default is 0
.
No
size
integer
The number of results to return per page. Default is 10
.
No
sortField
string
Used to sort results by field, which must be createdAt
, name
, blobHandlerType
, subscriptionStatus
, recordCount
, status
, policy
, or editable
.
No
sortOrder
string
Sorts results by order, which must be asc
or desc
.
No
excludedProjects
array[integer]
Filter out any data sources that belong to the specified projects.
No
ephemeral
boolean
When true
, returns ephemeral data sources.
No
clusterName
string
The name of the remote cluster the data source is connected to.
No
mode
integer
Specifies the query mode, which must be 0
(FULL
), 1
(COUNT
), 4
(TAG
), 5
(MIN_MAX
), or 6
(STATUS
).
No
globalPolicy
string
Filter by data sources that have this Global Policy applied.
No
hostname
string
Searches data sources by hostname.
No
determinePolicyContflicts
boolean
When true
, filters results to return the data sources with policy conflicts.
No
collectionId
string
Filter by data sources in the domain with this ID.
No
collectionExclude
string
When true
, filters results to return the data sources in the domain of the provided ID.
No
sddTemplateName
string
Filter by data sources that have the specified SDD identification framework applied.
No
excludeSddTemplateName
string
Filter by data sources that do not have the specified SDD identification framework applied.
No
The response will return data sources you own or are to see.
name
string
Data source name.
id
integer
Data source ID.
deleted
boolean
If true
the data source is a deleted data source.
description
string
The data source description.
createdAt
timestamp
The date and time the data source was created.
subscriptionPolicy
array
Details the type of Subscription Policy applied to the data source.
schemaEvolutionId
integer
The schema evolution ID.
recordCount
integer
The record count.
status
array[string]
Accepted statuses are passed
or failed
.
subscriptionStatus
array[string]
Accepted statuses are subscribed
or unsubscribed
.
blobHandlerType
array[string]
Describes the type of underlying blob handler of this data source (e.g., Custom, MS SQL).
subscriptionType
string
The type of subscription policy on the project. The type can be automatic
(which allows anyone to subscribe), approval
(which requires the subscriber to be manually approved), policy
(which only allows users with specific groups or attributes to subscribe), or manual
(which requires users to be manually added).
connectionString
string
The connection string information.
sqlSchemaName
string
The schema name.
policy
string
When this value is none
, there are no data policies applied to the data source. Otherwise, this field indicates whether or not there are policy conflicts among the data policies applied to the data source.
policyHandlerType
string
The policy handler type, such as None
or Builder
.
The following request returns 2 data sources.
GET
/dataSource/{dataSourceId}
Get a data source based on the ID.
Query parameters
dataSourceId
integer
The data source ID .
Yes
Response Schema
name
string
The data source name.
recordFormat
string
The data format of blobs in the data source, such as json
, xml
, html
, or jpeg
.
description
string
The description of the data source.
policyHandler
array
The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string
A string that represents this data source's schema name in Immuta.
sqlTableName
string
The SQL table name in Immuta.
blobHandler
array[object]
A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string
Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL
).
createdBy
integer
The ID of the profile creating the data source.
deleted
boolean
If true, the data source was deleted.
type
string
The data source type, such as queryable
or ingested
.
rowCount
integer
The number of rows.
documentation
string
Documentation associated with the data source.
id
integer
The data source ID.
policyHandlerType
string
The type of policy handler applied to the data source: Builder
.
subscriptionType
string
The type of subscription policy on the data source. The type can be automatic
(which allows anyone to subscribe), approval
(which requires the subscriber to be manually approved), policy
(which only allows users with specific groups or attributes to subscribe), or manual
(which requires users to be manually added).
subscriptionPolicy
array
Details about the Subscription Policy applied to the data source.
globalPolicies
string
Details about the Global Policies applied to the data source.
status
string
The data source health status.
The following request gets a data source based on the ID 22
.
GET
/dataSource/name/{dataSourceName}
Get a data source based on the name.
dataSourceName
string
The data source name.
Yes
name
string
The data source name.
recordFormat
string
The data format of blobs in the data source, such as json
, xml
, html
, or jpeg
.
description
string
The description of the data source.
policyHandler
array
The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string
A string that represents this data source's schema name in Immuta.
sqlTableName
string
The SQL table name in Immuta.
blobHandler
array[object]
A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string
Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL
).
createdBy
integer
The ID of the profile creating the data source.
deleted
boolean
If true, the data source was deleted.
type
string
The data source type, such as queryable
or ingested
.
rowCount
integer
The number of rows.
documentation
string
Documentation associated with the data source.
id
integer
The data source ID.
policyHandlerType
string
The type of policy handler applied to the data source: Builder
.
subscriptionType
string
The type of subscription policy on the data source. The type can be automatic
(which allows anyone to subscribe), approval
(which requires the subscriber to be manually approved), policy
(which only allows users with specific groups or attributes to subscribe), or manual
(which requires users to be manually added).
subscriptionPolicy
array
Details about the Subscription Policy applied to the data source.
globalPolicies
string
Details about the Global Policies applied to the data source.
status
string
The data source health status.
The following request gets a data source based on the name Public Barfoo
.
GET
/dataSource/sqlTableName/{shortName}
Get a data source based on the SQL table name.
shortName
string
The data source SQL table name.
Yes
name
string
The data source name.
recordFormat
string
The data format of blobs in the data source, such as json
, xml
, html
, or jpeg
.
description
string
The description of the data source.
policyHandler
array
The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string
A string that represents this data source's schema name in Immuta.
sqlTableName
string
The SQL table name in Immuta.
blobHandler
array[object]
A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string
Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL
).
createdBy
integer
The ID of the profile creating the data source.
deleted
boolean
If true, the data source was deleted.
type
string
The data source type, such as queryable
or ingested
.
rowCount
integer
The number of rows.
documentation
string
Documentation associated with the data source.
id
integer
The data source ID.
policyHandlerType
string
The type of policy handler applied to the data source: Builder
.
subscriptionType
string
The type of subscription policy on the data source. The type can be automatic
(which allows anyone to subscribe), approval
(which requires the subscriber to be manually approved), policy
(which only allows users with specific groups or attributes to subscribe), or manual
(which requires users to be manually added).
subscriptionPolicy
array
Details about the Subscription Policy applied to the data source.
globalPolicies
string
Details about the Global Policies applied to the data source.
status
string
The data source health status.
The following request gets a data source based on the SQL table name customer_data
.
GET
/dataSource/{dataSourceId}/lineage/{type}
Get parent and child relationship records for derived data sources using a specified data source ID.
type
string
The type of lineage records to return. Options include: parents
, children
, and all
.
Yes
dataSourceId
integer
The target data source ID.
Yes
children
array
Details of the child data source, including dataSourceId
, dataSourceName
, projectId
, policyHandlerDiff
, deleted
, createdBy
, and createdAt
.
parents
array
Details of the parent data source, including dataSourceId
, dataSourceName
, projectId
, policyHandlerDiff
, deleted
, createdBy
, and createdAt
.
The following request gets the parent relationship records for the derived data source with the data source ID 4
.
GET
/dataSource/{dataSourceId}/blob/{blobid*}
Retrieve a blob.
dataSourceId
integer
The data source ID.
Yes
blobId
string
The blob ID.
Yes
The response will download the blobs in a file you specify.
The following request retrieves a blob.
GET
/dataSource/{dataSourceId}/access
Get all users with the provided access level for this data source.
dataSourceId
integer
The data source ID.
Yes
states
Array[string]
The status levels to include when querying for user access.
No
approved
boolean
Denotes whether the returned access objects should be approved.
No
searchText
string
A string used to filter returned users. The query is executed with a wildcard prefix and suffix.
No
size
integer
The number of results to return.
No
offset
integer
The number of results to skip (for paging).
No
sortField
string
The field on which to sort the result set.
No
sortOrder
string
The order in which to sort the results.
No
expandGroups
boolean
If true will return individual members of any group subscribed.
No
ignoreSystemGenerated
boolean
If true, will not return system generated accounts.
No
filterBySchemaEvolution
boolean
If true, will only return users who have the specified level of access across ALL data sources within the same schema evolution group as this one.
No
count
integer
The number of users with access to the data source.
users
string
The metadata regarding the users with access to the data source.
The following request gets all users with the provided access level for this data source.
GET
/dataSource/{dataSourceId}/users/{profileId}/policyInfo
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
dataSourceId
integer
The data source ID.
Yes
profileId
integer
The profile ID of the user.
Yes
projectId
integer
The project ID. If provided, this project will be used when evaluating the user's visibilities.
No
visibilities
array
Details of the user's visibilities, including anyKey
.
visibilityRuleApplies
boolean
If true
, a visibility rule exists and the user is not excepted from it.
masked
array
Masking information for the data source, including metadata
, name
, type
, and actionType
.
additionalFilters
array
Policy information for the data source, including customWhere
, differentialPrivacy
, eventTimeColumn
, minimization
, time
, filterSeconds
, and isOlderOrNewer
.
The following request gets the visibility information for the user with the profile ID 2
on the data source with the data source ID 16
.
GET
/dataSource/{dataSourceId}/users/{profileId}/visibilityReport
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.
dataSourceId
integer
The data source ID.
Yes
profileId
integer
The profile ID of the user.
Yes
informationOnly
boolean
If true
, the query will just return information for the UI and will skip running some queries for ephemeral data sources.
No
includeNestedColumns
boolean
If true
, the query will return just information for the dictionary page, including the masking policies for nested columns.
No
noVisibilities
boolean
If true
, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer
The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer
The number of visibilities the current user can see for the given data source.
masked
array
Masking information for the data source, including metadata
, name
, type
, and actionType
.
dataSource
integer
The data source ID.
dataSourceName
string
The data source name.
additionalFilters
array
Policy information for the data source, including customWhere
, differentialPrivacy
, eventTimeColumn
, minimization
, time
, filterSeconds
, and isOlderOrNewer
.
allowMaskedJoins
boolean
If true
the data source allows masked joins.
policySet
array
Details about the policies on the data source.
The following request gets all of the visibility information for the user with the profile ID 2
on the data source with the data source ID 16
.
GET
/dataSource/{dataSourceId}/visibilityReport
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.
dataSourceId
integer
The data source ID.
Yes
noVisibilities
boolean
If true
, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer
The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer
The number of visibilities the current user can see for the given data source.
denialReason
string
Reason the user was denied visibility.
masked
array
Masking information for the data source, including metadata
, name
, type
, and actionType
.
dataSource
integer
The data source ID.
dataSourceName
string
The data source name.
additionalFilters
array
Policy information for the data source, including customWhere
, differentialPrivacy
, eventTimeColumn
, minimization
, time
, filterSeconds
, and isOlderOrNewer
.
allowMaskedJoins
boolean
If true
the data source allows masked joins.
policySet
array
Details about the policies on the data source.
The following request gets all of the visibility information for the current user on the data source with the data source ID 16
.
POST
Subscribe to a data source.
POST
Make a request for values to be unmasked.
POST
Add a user to a specific data source.
POST
/dataSource/subscribe
Subscribe to a data source.
dataSourceId
integer
Data source ID number.
Yes
dataSourceIds
array
The ID of the data source the user is subscribing to.
No
approvals
array
Includes details about the Subscription policy on the data source: requiredPermissions
, specificApproverRequired
, specificApprover
, and ownerModelId
.
No
body
array
Contains details about the data source, including the data source ID, subscription status of the user, the profile ID of the user, and the dates the data source was created and updated.
The following request subscribes to the data source with ID 22
.
Payload example
POST
/dataSource/{dataSourceId}/reverseMask
Makes a request for values to be unmasked.
dataSourceId
integer
The data source ID.
Yes
column
string
The column to unmask.
Yes
unmaskingReason
string
The reason the values need to be unmasked.
Yes
unmaskingUsers
array[integer]
The profile ID of the users who can unmask the values for the requestor.
Yes
projectId
integer
The ID of the associated project.
No
dataSourceId
integer
The data source ID.
No
id
integer
The ID of the request.
requestingUserProfile
integer
The requesting user profile ID.