This page describes the dataSource endpoint, through which users can subscribe to data sources, make unmasking requests, and manage data source tasks. To create data sources, see the specific handler endpoints.
Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.
Get parent and child relationship records for derived data sources using a specified data source ID.
GET
Retrieve a blob.
GET
Get all users with the provided access level for this data source.
GET
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
GET
Retrieves a summary of total records, total visibilities, and visibilities a given user has access to.
GET
Retrieves a summary of total records, total visibilities, and visibilities the current user has access to for a specified data source.
Search for data sources
GET /dataSource
Search for data sources.
Query parameters
Attribute
Description
Required
blobHandlerType
array[string] Describes the type of underlying blob handler that will be used with this data source (e.g., Custom, MS SQL).
No
subscription
array[string] The requesting user's subscription status: pending, owner, subscribed, not_subscribed, expert, or ingest.
No
status
array[string] The data source status: passed or failed.
No
tag
array[string] Filters data sources by tags associated with the data sources.
No
searchText
string Searches for data source names using the provided string.
No
column
array[string] Searches for data source column names.
No
connectionString
array[string] Searches by connection string.
No
schema
string Searches for data source schema.
No
nameOnly
boolean When true, searchText will only search data source names. Default is false.
No
idOnly
boolean When true, only returns the ID Of the data source and the user's subscription status.
No
dataSourceIds
array[integer] Searches for the provided data source IDs.
No
selectFields
array[string] This field accepts the values id, name, and columnEvolutionEnabled. When id or name are provided, the request will return only the ID or name of the data source and the subscription status. If columnEvolutionEnabled is provided, the response will also include information about the policies, policy conflicts, and workspaces associated with the data sources.
No
offset
integer Used in combination with size to fetch pages. Default is 0.
No
size
integer The number of results to return per page. Default is 10.
No
sortField
string Used to sort results by field, which must be createdAt, name, blobHandlerType, subscriptionStatus, recordCount, status, policy, or editable.
No
sortOrder
string Sorts results by order, which must be asc or desc.
No
excludedProjects
array[integer] Filter out any data sources that belong to the specified projects.
No
ephemeral
boolean When true, returns ephemeral data sources.
No
clusterName
string The name of the remote cluster the data source is connected to.
No
mode
integer Specifies the query mode, which must be 0 (FULL), 1 (COUNT), 4 (TAG), 5 (MIN_MAX), or 6 (STATUS).
No
globalPolicy
string Filter by data sources that have this Global Policy applied.
No
hostname
string Searches data sources by hostname.
No
determinePolicyContflicts
boolean When true, filters results to return the data sources with policy conflicts.
No
collectionId
string Filter by data sources in the domain with this ID.
No
collectionExclude
string When true, filters results to return the data sources in the domain of the provided ID.
No
sddTemplateName
string Filter by data sources that have the specified SDD identification framework applied.
No
excludeSddTemplateName
string Filter by data sources that do not have the specified SDD identification framework applied.
No
Response schema
Attribute
Description
name
string Data source name.
id
integer Data source ID.
deleted
boolean If true the data source is a deleted data source.
description
string The data source description.
createdAt
timestamp The date and time the data source was created.
subscriptionPolicy
array Details the type of Subscription Policy applied to the data source.
schemaEvolutionId
integer The schema evolution ID.
recordCount
integer The record count.
status
array[string] Accepted statuses are passed or failed.
subscriptionStatus
array[string] Accepted statuses are subscribed or unsubscribed.
blobHandlerType
array[string] Describes the type of underlying blob handler of this data source (e.g., Custom, MS SQL).
subscriptionType
string The type of subscription policy on the project. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
connectionString
string The connection string information.
sqlSchemaName
string The schema name.
policy
string When this value is none, there are no data policies applied to the data source. Otherwise, this field indicates whether or not there are policy conflicts among the data policies applied to the data source.
policyHandlerType
string The policy handler type, such as None or Builder.
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the ID 22.
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the name Public Barfoo.
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the SQL table name customer_data.
Get all users with the provided access level for this data source.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
states
Array[string] The status levels to include when querying for user access.
No
approved
boolean Denotes whether the returned access objects should be approved.
No
searchText
string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.
No
size
integer The number of results to return.
No
offset
integer The number of results to skip (for paging).
No
sortField
string The field on which to sort the result set.
No
sortOrder
string The order in which to sort the results.
No
expandGroups
boolean If true will return individual members of any group subscribed.
No
ignoreSystemGenerated
boolean If true, will not return system generated accounts.
No
filterBySchemaEvolution
boolean If true, will only return users who have the specified level of access across ALL data sources within the same schema evolution group as this one.
No
Response schema
Attribute
Description
count
integer The number of users with access to the data source.
users
string The metadata regarding the users with access to the data source.
Request example
The following request gets all users with the provided access level for this data source.
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
profileId
integer The profile ID of the user.
Yes
projectId
integer The project ID. If provided, this project will be used when evaluating the user's visibilities.
No
Response schema
Attribute
Description
visibilities
array Details of the user's visibilities, including anyKey.
visibilityRuleApplies
boolean If true, a visibility rule exists and the user is not excepted from it.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
Request example
The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
profileId
integer The profile ID of the user.
Yes
informationOnly
boolean If true, the query will just return information for the UI and will skip running some queries for ephemeral data sources.
No
includeNestedColumns
boolean If true, the query will return just information for the dictionary page, including the masking policies for nested columns.
No
Response schema
Attribute
Description
noVisibilities
boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer The number of visibilities the current user can see for the given data source.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
dataSource
integer The data source ID.
dataSourceName
string The data source name.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins
boolean If true the data source allows masked joins.
policySet
array Details about the policies on the data source.
Request example
The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
Response schema
Attribute
Description
noVisibilities
boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer The number of visibilities the current user can see for the given data source.
denialReason
string Reason the user was denied visibility.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
dataSource
integer The data source ID.
dataSourceName
string The data source name.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins
boolean If true the data source allows masked joins.
policySet
array Details about the policies on the data source.
Request example
The following request gets all of the visibility information for the current user on the data source with the data source ID 16.
array The ID of the data source the user is subscribing to.
No
approvals
array Includes details about the Subscription policy on the data source: requiredPermissions, specificApproverRequired, specificApprover, and ownerModelId.
No
Response schema
Attribute
Description
body
array Contains details about the data source, including the data source ID, subscription status of the user, the profile ID of the user, and the dates the data source was created and updated.
Request example
The following request subscribes to the data source with ID 22.
Add a user to a specific data source. Requestors cannot add themselves to a data source. To request access to a data source, use the /dataSource/subscribe endpoint.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
Payload parameters
Attribute
Description
Required
state
string The status of the user: subscribed, owner,expert, or ingest.
Yes
profileId
integer The profile ID of the user being added to the data source.
Yes
groupId
integer The ID of the group being added to the data source.
No
approvals
array Details about the user approving access: requiredPermission, specificApproverRequired, and specificApprover.
No
expiration
date The date the user's data source subscription ends.
No
Response schema
Attribute
Description
id
integer The user's subscription ID.
modelId
integer The model ID.
modelType
string The model type.
state
string The user's data source role, such as subscribed.
denialReasoning
string If the user was denied access, the reason for denial.
profile
integer The user's profile ID.
group
integer If a group was added, the group ID.
expiration
date The date the user's subscription to the data source will expire.
acknowledgeRequired
boolean If the data source is associated with a project, this value will be true if the user needs to confirm they have read the project acknowledgment.
createdAt
timestamp The date and time of creation.
updatedAt
timestamp The date and time of update.
approved
boolean When true, the user's request has been approved.
Request example
The following request adds a user (saved in example-payload.json) to this data source.
{"outgoing": [],"incoming": [{"fullCount":1,"dataSource": {"id":11,"name":"Public Customer" },"isOutgoing":false,"rows": [{"id":1,"state":"pending","type":"columnAdded","reason":"Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"","targetNames": ["Katie" ],"targetEmails": ["katie@immuta.com" ],"requester": {"name":"Immuta System Account","id":1,"email":null },"dataSource": {"id":11,"name":"Public Customer" },"createdAt":"2024-08-06T14:37:09.279+00:00","metadata": {"colName":"org","colType":"VARCHAR(16777216)" },"isOutgoing":false }] }]}
Mark tasks as complete
GET/dataSource/tasks/{taskId}
Handles the given task and marks it as complete.
Query parameters
Attribute
Description
Required
taskId
integer The task ID.
Yes
Response schema
Attribute
Description
result
array Includes details about the task.
Request example
The following request handles the given task and marks it as complete.
{"hits": [ {"fullCount":2,"id":6,"state":"completed","type":"columnAdded","reason":"Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"","targetNames": ["John" ],"targetEmails": ["john@example.com" ],"requester": {"name":"Immuta System Account","id":1, },"dataSource": {"id":11,"name":"Public Customer" },"createdAt":"2021-10-12T15:48:23.095Z" }, {"fullCount":2,"id":7,"state":"completed","type":"columnAdded", "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Fake Medical Claims 2017\"",
"targetNames": ["John" ],"targetEmails": ["john@example.com" ],"requester": {"name":"Immuta System Account","id":1 },"dataSource": {"id":3,"name":"Public Fake Medical Claims 2017" },"createdAt":"2021-10-12T18:56:22.954Z" } ],"count":2}
{"documentation":"# Public Credit Accounts\nThis request updates the data source documentation."}
Response example
{"name":"Public Credit Accounts","recordFormat":"Not Provided","description":null,"policyHandler": {"handlerId":21,"visibilitySchema": {"fields": [],"version":"2021-09-15T17:44:20.678Z" },"previousHandlerId":20,"dataSourceId":1 },"sqlSchemaName":"public","sqlTableName":"credit_accounts","blobHandler": {"url":"https://1.1.1.1:1111/snowflake/handler/1","ca": {"name":"Certificate Authority Bundle" },"manualDictionary":false },"createdBy":2,"deleted":false,"type":"queryable","recordCount":0,"rowCount":"100000","documentation":"# Public Credit Accounts\nThis request updates the data source documentation.","statsExpiration":"2021-09-22T13:51:46.646Z","id":1,"blobHandlerType":"Snowflake","policyHandlerType":"Builder","subscriptionType":"approval","subscriptionPolicy": {"type":"subscription","approvals": [{"requiredPermission":"OWNER","specificApproverRequired":false }] },"globalPolicies":null,"status":"failed","statusInfo": {"sql": {"status":"passed","message":"Passed" },"stats": {"status":"passed","lastAttempted":"2021-09-21T13:51:46.674Z" },"fingerprint": {"status":"passed","lastAttempted":"2021-09-09T14:12:25.177Z" },"lastAttempt": {"date":"2021-09-20T19:35:21.929Z" },"globalPolicy": {"status":"passed","lastAttempted":"2021-09-17T19:07:38.092Z" },"highCardinality": {"status":"failed","message":"Error could not connect to remote database","lastAttempted":"2021-09-20T16:43:19.426Z" } },"expiration":null,"catalogMetadata":null,"workspace":null,"seeded":false,"schemaEvolutionId":1,"columnEvolutionEnabled":true,"createdAt":"2021-09-09T14:12:09.511Z","updatedAt":"2021-09-21T13:52:42.908Z"}
Update multiple data sources
PUT/dataSource/bulk/{type}
Update data sources.
Query parameters
Attribute
Description
Required
type
string The action to perform on the data sources: add-users, disable, restore, delete, or tags.
Yes
Payload parameters
Attribute
Description
Required
ids
array[integer] The IDs of the data sources to update.
Yes
update
array[object] Only required for add-users (includes metadata about the users' profiles: id and state) and tags (includes metadata about the tags: name and source) types.
No
Response schema
Attribute
Description
bulkId
string The ID of the bulk data source update.
jobsCreated
integer The number of jobs created.
Request example
The following request adds the Address.email tag to two data sources.
Trigger the schema monitoring job for the specified detection group, or all groups if no payload parameters are given.
Payload parameters
Attribute
Description
Required
dataSourceIds
array[integer] The data source IDs to run the column detection job on. Leave empty to run this job globally on all data sources. This parameter cannot be included in the payload if schemaEvolutionId or any combination of hostname, database, port, or table is included.
No
hostname
string The hostname of the data sources. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
port
integer The port used to connect the data sources to Immuta. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
database
string The database name. This runs schema monitoring on the database provided. If data sources were initially registered via the V2 API, including this parameter will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, including this parameter will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
table
string The table name. This will run column detection to just update the columns in this table. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
schemaEvolutionId
No
skipColumnDetection
boolean When true, Immuta will only pull new tables from the source server. This parameter can only be paired with schemaEvolutionId.
No
overrides.httpPath
No
Response schema
Attribute
Description
schemaDetection
object Includes details about the resulting schema detection jobs.
columnDetection
object Includes details about the resulting column detection jobs.
bulkId
string The unique identifier of the jobs running schema monitoring and column detection.
Responses may include bulkId, schemaDetection, or columnDetection objects, depending on the payload.
Request example
The following request triggers the schema monitoring job for the specified detection group.
The tabs below illustrate payloads for triggering schema monitoring on a host, database, or table. The request will run schema monitoring for all databases registered under the hostname provided in the payload.
The request will run schema monitoring for all databases registered under the hostname provided in the payload.
The request will run schema monitoring on the database provided in the payload. If data sources were initially registered via the V2 API, this request will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, this request will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas.
{"database":"public"}
The request will run column detection and update the columns on the table specified in the payload.
{"database":"public","table":"healthcare"}
Response examples
The tabs below illustrate the example response for each example payload provided above.
Retrieve all blob handlers the current user is allowed to create.
GET
Get data sources that match a set of purposes.
GET
Retrieve all the data sources the current user has access to.
GET
Get all of the recent policy activities for a given data source.
GET
Get the profiles for the data source owners and experts.
GET
Get the tags for a data source.
GET
Return the users who can unmask the given column.
Run a data source health check
GET/dataSource/{dataSourceId}/test
Run a health check on the data source.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
Response schema
Attribute
Description
blob
object Indicates whether or not the blob was successfully crawled.
columnEvolution
object Indicates whether or not the job run to check for columns added or removed from the data source passed and when it was last run.
externalCatalog
object Indicates whether or not the external catalog was successfully linked to the data source.
fingerprint
object Indicates whether or not the fingerprint job was successful (passed) and when it was last run. The fingerprint captures summary statistics of the data source.
framework
object Indicates whether or not the classification was successfully run on the data source to determine its sensitivity.
globalPolicy
object Indicates whether or not global policies were successfully applied to the data source.
highCardinality
object Indicates whether or not the job run to calculate the data source's high cardinality column passed and when it was last run.
schemaEvolution
object Indicates whether or not the job run to check if a new table had been added in the remote database passed and when it was last run. If a new table was added, Immuta automatically creates a new data source. Correspondingly, if a remote table is removed, that data source will be disabled in the console.
sdd
object Indicates whether or not sensitive data discovery was successfully run on the data source.
sql
object Indicates whether or not the SQL query run to check the data source's health passed and when it was last run.
stats
object Indicates whether or not the job run to calculate the number of rows in the data source passed and when it was last run.
Get all of the recent policy activities for a given data source.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
offset
integer The number of results to skip (for paging).
No
size
integer The number of results to return per page.
No
Response schema
Attribute
Description
count
integer The number of results.
activities
array Includes details about the policy and the data source, including the policy and data source type, when the activity notification was triggered, and whether or not the policy change was triggered by a Global policy.
actionBy
array Details about who triggered the action.
targetUser
array Information about the user who received the notification.
Request example
The following request gets all of the recent policy activities for a given data source.
{"fullCount":1,"id":8,"state":"pending","type":"columnAdded","reason":"Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"","targetNames": ["Katie"],"targetEmails": ["katie@example.com"],"requester": {"name":"Immuta System Account","id":1 },"dataSource": {"id":11,"name":"Public Customer" },"createdAt":"2021-10-12T19:28:04.999Z"}
Delete a blob
DELETE/dataSource/{dataSourceId}/blob/{blobId*}
Delete a blob.
Query parameters
Attribute
Description
Required
dataSourceId
integer The data source ID.
Yes
blobId
string The blob ID.
Yes
Response schema
When the blob is successfully deleted, there will be no response.
integer The ID of the schema to run the schema monitoring job on. This will run on all tables associated with the specified ID. The schema ID can be found in the response body of. This parameter cannot be included in the payload if dataSourceIds or any combination of hostname, database, port, or table is included.
string If Databricks are configured, provide the alternative HTTP path to trigger schema monitoring on that ephemeral cluster.
