Subscribe to and Manage Data Sources
Data Source API reference guide
This page describes the dataSource endpoint, through which users can subscribe to data sources, make unmasking requests, and manage data source tasks. To create data sources, see the specific handler endpoints.
Data source workflow
Search data sources and data source details
GET
Get parent and child relationship records for derived data sources using a specified data source ID.
GET
Get all users with the provided access level for this data source.
GET
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
GET
Retrieves a summary of total records, total visibilities, and visibilities a given user has access to.
GET
Retrieves a summary of total records, total visibilities, and visibilities the current user has access to for a specified data source.
Search for data sources
GET /dataSource
Search for data sources that you own or are to see.
Query parameters
blobHandlerType
array[string] Describes the type of underlying blob handler that will be used with this data source (e.g., Custom, MS SQL).
No
subscription
array[string] The requesting user's subscription status: pending, owner, subscribed, not_subscribed, expert, or ingest.
No
status
array[string] The data source status: passed or failed.
No
tag
array[string] Filters data sources by tags associated with the data sources.
No
searchText
string Searches for data source names using the provided string.
No
column
array[string] Searches for data source column names.
No
connectionString
array[string] Searches by connection string.
No
schema
string Searches for data source schema.
No
nameOnly
boolean When true, searchText will only search data source names. Default is false.
No
idOnly
boolean When true, only returns the ID Of the data source and the user's subscription status.
No
dataSourceIds
array[integer] Searches for the provided data source IDs.
No
selectFields
array[string] This field accepts the values id, name, and columnEvolutionEnabled. When id or name are provided, the request will return only the ID or name of the data source and the subscription status. If columnEvolutionEnabled is provided, the response will also include information about the policies, policy conflicts, and workspaces associated with the data sources.
No
offset
integer Used in combination with size to fetch pages. Default is 0.
No
size
integer The number of results to return per page. Default is 10.
No
sortField
string Used to sort results by field, which must be createdAt, name, blobHandlerType, subscriptionStatus, recordCount, status, policy, or editable.
No
sortOrder
string Sorts results by order, which must be asc or desc.
No
excludedProjects
array[integer] Filter out any data sources that belong to the specified projects.
No
ephemeral
boolean When true, returns ephemeral data sources.
No
clusterName
string The name of the remote cluster the data source is connected to.
No
mode
integer Specifies the query mode, which must be 0 (FULL), 1 (COUNT), 4 (TAG), 5 (MIN_MAX), or 6 (STATUS).
No
globalPolicy
string Filter by data sources that have this Global Policy applied.
No
hostname
string Searches data sources by hostname.
No
determinePolicyContflicts
boolean When true, filters results to return the data sources with policy conflicts.
No
collectionId
string Filter by data sources in the domain with this ID.
No
collectionExclude
string When true, filters results to return the data sources in the domain of the provided ID.
No
sddTemplateName
string Filter by data sources that have the specified SDD identification framework applied.
No
excludeSddTemplateName
string Filter by data sources that do not have the specified SDD identification framework applied.
No
Response schema
The response will return data sources you own or are to see.
name
string Data source name.
id
integer Data source ID.
deleted
boolean If true the data source is a deleted data source.
description
string The data source description.
createdAt
timestamp The date and time the data source was created.
subscriptionPolicy
array Details the type of Subscription Policy applied to the data source.
schemaEvolutionId
integer The schema evolution ID.
recordCount
integer The record count.
status
array[string] Accepted statuses are passed or failed.
subscriptionStatus
array[string] Accepted statuses are subscribed or unsubscribed.
blobHandlerType
array[string] Describes the type of underlying blob handler of this data source (e.g., Custom, MS SQL).
subscriptionType
string The type of subscription policy on the project. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
connectionString
string The connection string information.
sqlSchemaName
string The schema name.
policy
string When this value is none, there are no data policies applied to the data source. Otherwise, this field indicates whether or not there are policy conflicts among the data policies applied to the data source.
policyHandlerType
string The policy handler type, such as None or Builder.
Request example
The following request returns 2 data sources.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource?size=2Response example
{
"name": "Public Barfoo",
"id": 22,
"recordFormat": "Not Provided",
"deleted": false,
"description": null,
"createdAt": "2021-07-22T14:11:55.539Z",
"subscriptionPolicy": {
"type": "subscription",
"automaticSubscription": true
},
"schemaEvolutionId": 1,
"recordCount": 0,
"status": "passed",
"subscriptionStatus": "subscribed",
"blobHandlerType": "Snowflake",
"subscriptionType": "automatic",
"connectionString": "[email protected]:###/test",
"sqlSchemaName": "public",
"policy": "None",
"policyHandlerType": "None",
"native": null,
"workspace": null
},
{
"name": "Public Aaa Tpc",
"id": 39,
"recordFormat": "Not Provided",
"deleted": false,
"description": null,
"createdAt": "2023-08-21T10:39:00.250Z",
"subscriptionPolicy": {
"type": "subscription",
"exceptions": {
"operator": "and",
"conditions": [
{
"type": "groups",
"group": {
"name": "alpha"
}
}
]
},
"allowDiscovery": true,
"automaticSubscription": true
},
"schemaEvolutionId": 1,
"recordCount": 0,
"blobHandlerType": "Snowflake",
"subscriptionType": "policy",
"sqlSchemaName": "public",
"status": "passed",
"subscriptionStatus": "owner",
"connectionString": "[email protected]:###/test",
"remoteTable": "tpc",
"remoteSchema": "public",
"domainId": null,
"domainName": null,
"policy": "None",
"policyHandlerType": "None",
"native": null,
"workspace": null
}Get a data source by ID
GET /dataSource/{dataSourceId}
Get a data source based on the ID.
Query parameters
dataSourceId
integer The data source ID .
Yes
Response Schema
name
string The data source name.
recordFormat
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the ID 22.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/22Response example
{
"name": "Public Barfoo",
"recordFormat": "Not Provided",
"description": null,
"policyHandler": null,
"sqlSchemaName": "public",
"sqlTableName": "barfoo",
"blobHandler": {
"url": "https://your-url/snowflake/handler/22",
"ca": {
"name": "Certificate Authority Bundle"
},
"manualDictionary": false
},
"createdBy": 2,
"deleted": false,
"type": "queryable",
"recordCount": 0,
"rowCount": 6,
"documentation": "# Public Barfoo",
"statsExpiration": "2021-08-27T16:34:47.846Z",
"id": 22,
"blobHandlerType": "Snowflake",
"policyHandlerType": "None",
"subscriptionType": "automatic",
"subscriptionPolicy": {
"type": "subscription",
"automaticSubscription": true
},
"globalPolicies": null,
"status": "passed",
"statusInfo": {
"sql": {
"status": "passed",
"message": "Passed"
}
}
}Get data source by name
GET /dataSource/name/{dataSourceName}
Get a data source based on the name.
Query parameters
dataSourceName
string The data source name.
Yes
Response schema
name
string The data source name.
recordFormat
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the name Public Barfoo.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/name/Public%20BarfooResponse example
{
"name": "Public Barfoo",
"recordFormat": "Not Provided",
"description": null,
"policyHandler": null,
"sqlSchemaName": "public",
"sqlTableName": "barfoo",
"blobHandler": {
"url": "https://your-url/snowflake/handler/22",
"ca": {
"name": "Certificate Authority Bundle"
},
"manualDictionary": false
},
"createdBy": 2,
"deleted": false,
"type": "queryable",
"recordCount": 0,
"rowCount": 6,
"documentation": "# Public Barfoo",
"statsExpiration": "2021-08-27T16:34:47.846Z",
"id": 22,
"blobHandlerType": "Snowflake",
"policyHandlerType": "None",
"subscriptionType": "automatic",
"subscriptionPolicy": {
"type": "subscription",
"automaticSubscription": true
},
"globalPolicies": null,
"status": "passed",
"statusInfo": {
"sql": {
"status": "passed",
"message": "Passed"
}
}
}Get a data source by the short name
GET /dataSource/sqlTableName/{shortName}
Get a data source based on the SQL table name.
Query parameters
shortName
string The data source SQL table name.
Yes
Response schema
name
string The data source name.
recordFormat
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName
string A string that represents this data source's schema name in Immuta.
sqlTableName
string The SQL table name in Immuta.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy
integer The ID of the profile creating the data source.
deleted
boolean If true, the data source was deleted.
type
string The data source type, such as queryable or ingested.
rowCount
integer The number of rows.
documentation
string Documentation associated with the data source.
id
integer The data source ID.
policyHandlerType
string The type of policy handler applied to the data source: Builder.
subscriptionType
string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy
array Details about the Subscription Policy applied to the data source.
globalPolicies
string Details about the Global Policies applied to the data source.
status
string The data source health status.
Request example
The following request gets a data source based on the SQL table name customer_data.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/sqlTableName/customer_dataResponse example
{
"name": "Dbo Customer Data",
"recordFormat": "Not Provided",
"description": null,
"policyHandler": {
"visibilitySchema": {
"fields": [],
"version": "2021-10-06T19:39:39.145Z"
},
"handlerId": 55,
"dataSourceId": 26
},
"sqlSchemaName": "dbo",
"sqlTableName": "customer_data",
"blobHandler": {
"url": "https://1.1.1.1.1/snowflake/testhandler/26",
"ca": {
"name": "Certificate Authority Bundle"
},
"manualDictionary": false
},
"createdBy": 2,
"deleted": false,
"type": "queryable",
"recordCount": 0,
"rowCount": 1000,
"documentation": "# Dbo Customer Data",
"statsExpiration": "2021-11-05T19:37:43.270Z",
"id": 26,
"blobHandlerType": "Snowflake",
"policyHandlerType": "Builder",
"subscriptionType": "automatic",
"subscriptionPolicy": {
"type": "subscription",
"automaticSubscription": false
},
"globalPolicies": null,
"status": "passed",
"statusInfo": {
"sql": {
"status": "passed",
"message": "Passed"
},
"stats": {
"status": "passed",
"lastAttempted": "2021-10-06T19:37:43.337Z"
},
"lastAttempt": {
"date": "2021-10-06T19:39:39.821Z"
},
"highCardinality": {
"status": "passed",
"lastAttempted": "2021-10-06T19:37:43.337Z"
}
},
"expiration": null,
"catalogMetadata": null,
"workspace": null,
"seeded": false,
"schemaEvolutionId": 4,
"columnEvolutionEnabled": true,
"createdAt": "2021-10-01T14:23:27.225Z",
"updatedAt": "2021-10-06T19:39:39.145Z",
"subscribedAsUser": true,
"subscriptionId": 45,
"acknowledgeRequired": false,
"subscriptionStatus": "owner",
"requestedState": "owner",
"approved": true,
"subscriptionExpiration": null,
"filterId": null
}Get data source relationships
GET /dataSource/{dataSourceId}/lineage/{type}
Get parent and child relationship records for derived data sources using a specified data source ID.
Query parameters
type
string The type of lineage records to return. Options include: parents, children, and all.
Yes
dataSourceId
integer The target data source ID.
Yes
Response schema
children
array Details of the child data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.
parents
array Details of the parent data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.
Request example
The following request gets the parent relationship records for the derived data source with the data source ID 4.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/4/lineage/parentsResponse example
{
"parents": [{
"dataSourceId": 3,
"dataSourceName": "Public Healthcare Data",
"deleted": false,
"createdAt": "2022-08-17T13:41:38.381Z",
"projectId": 2,
"projectName": "Derived Data Source",
"createdBy": "Your Username",
"policyHandlerDiff": {
"dsType": "queryable",
"currentHandlerId": null,
"previousHandlerId": null
}
}]
}Retrieve a Blob
GET /dataSource/{dataSourceId}/blob/{blobid*}
Retrieve a blob.
Query parameters
dataSourceId
integer The data source ID.
Yes
blobId
string The blob ID.
Yes
Response schema
The response will download the blobs in a file you specify.
Request example
The following request retrieves a blob.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--output ./the-blobs-will-be-saved-here
https://your-immuta-url.com/dataSource/22/blob/22Response example
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 215k 100 215k 0 0 541k 0 --:--:-- --:--:-- --:--:-- 541kGet users by access level
GET /dataSource/{dataSourceId}/access
Get all users with the provided access level for this data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
states
Array[string] The status levels to include when querying for user access.
No
approved
boolean Denotes whether the returned access objects should be approved.
No
searchText
string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.
No
size
integer The number of results to return.
No
offset
integer The number of results to skip (for paging).
No
sortField
string The field on which to sort the result set.
No
sortOrder
string The order in which to sort the results.
No
expandGroups
boolean If true will return individual members of any group subscribed.
No
ignoreSystemGenerated
boolean If true, will not return system generated accounts.
No
filterBySchemaEvolution
boolean If true, will only return users who have the specified level of access across ALL data sources within the same schema evolution group as this one.
No
Response schema
count
integer The number of users with access to the data source.
users
string The metadata regarding the users with access to the data source.
Request example
The following request gets all users with the provided access level for this data source.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/22/access?sortOrder=descResponse example
{
"count": 2,
"users": [
{
"profile": 2,
"name": "First Last",
"iamid": "bim",
"userid": "[email protected]",
"email": "[email protected]",
"type": "user",
"admin": "First Last",
"approved": true,
"state": "owner",
"systemGenerated": false,
"lastExternalRefresh": "2021-10-06T14:58:46.983Z",
"subscriptionId": 586,
"createdAt": "2021-10-05T14:33:01.518Z",
"updatedAt": "2021-10-05T14:33:01.518Z",
"approvals": [
{
"requiredPermission": "OWNER",
"state": "approved",
"approverId": null,
"ownerModelId": null,
"approver": "First Last",
"ownerModelName": null
}
],
"currentUserCanApprove": false
},
{
"profile": 3,
"name": "Tommy Test",
"iamid": "bim",
"userid": "[email protected]",
"email": "[email protected]",
"type": "user",
"admin": "First Last",
"approved": true,
"state": "subscribed",
"systemGenerated": false,
"lastExternalRefresh": "2021-09-07T16:16:29.957Z",
"subscriptionId": 649,
"createdAt": "2021-10-06T14:58:31.366Z",
"updatedAt": "2021-10-06T14:58:31.366Z",
"approvals": [
{
"requiredPermission": "OWNER",
"state": "approved",
"approverId": null,
"ownerModelId": null,
"approver": "First Last",
"ownerModelName": null
}
],
"currentUserCanApprove": false
}
]
}Get user access info for a data source
GET /dataSource/{dataSourceId}/users/{profileId}/policyInfo
Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
profileId
integer The profile ID of the user.
Yes
projectId
integer The project ID. If provided, this project will be used when evaluating the user's visibilities.
No
Response schema
visibilities
array Details of the user's visibilities, including anyKey.
visibilityRuleApplies
boolean If true, a visibility rule exists and the user is not excepted from it.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
Request example
The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/16/users/2/policyInfoResponse example
{
"visibilities": [],
"visibilityRuleApplies": false,
"masked": [
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_SOLD_DATE_SK",
"actionType": "Nullify"
},
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_BILL_CUSTOMER_SK",
"actionType": "Nullify"
}
],
"additionalFilters": {}
}Get user visibility info for a data source
GET /dataSource/{dataSourceId}/users/{profileId}/visibilityReport
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.
Query parameters
dataSourceId
integer The data source ID.
Yes
profileId
integer The profile ID of the user.
Yes
informationOnly
boolean If true, the query will just return information for the UI and will skip running some queries for ephemeral data sources.
No
includeNestedColumns
boolean If true, the query will return just information for the dictionary page, including the masking policies for nested columns.
No
Response schema
noVisibilities
boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer The number of visibilities the current user can see for the given data source.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
dataSource
integer The data source ID.
dataSourceName
string The data source name.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins
boolean If true the data source allows masked joins.
policySet
array Details about the policies on the data source.
Request example
The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/16/users/2/visibilityReportResponse example
[
{
"noVisibilities": true,
"dataSourceVisibilitiesCount": 0,
"userVisibilitiesCount": 0,
"masked": [
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_SOLD_DATE_SK"
},
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_BILL_CUSTOMER_SK"
}
],
"dataSource": 16,
"dataSourceName": "Web Sales",
"additionalFilters": {},
"allowMaskedJoins": false,
"policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}
]Get current user visibility info
GET /dataSource/{dataSourceId}/visibilityReport
Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
Response schema
noVisibilities
boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount
integer The total number of possible visibilities the given data source has.
userVisibilitiesCount
integer The number of visibilities the current user can see for the given data source.
denialReason
string Reason the user was denied visibility.
masked
array Masking information for the data source, including metadata, name, type, and actionType.
dataSource
integer The data source ID.
dataSourceName
string The data source name.
additionalFilters
array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins
boolean If true the data source allows masked joins.
policySet
array Details about the policies on the data source.
Request example
The following request gets all of the visibility information for the current user on the data source with the data source ID 16.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/16/visibilityReportResponse example
{
"noVisibilities": true,
"dataSourceVisibilitiesCount": 0,
"userVisibilitiesCount": 0,
"masked": [
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_SOLD_DATE_SK"
},
{
"type": "Consistent Value",
"metadata": {
"constant": null
},
"name": "WS_BILL_CUSTOMER_SK"
}
],
"dataSource": 16,
"dataSourceName": "Web Sales",
"additionalFilters": {},
"allowMaskedJoins": false,
"policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}Access data sources and make data source requests
Subscribe to a data source
POST /dataSource/subscribe
Subscribe to a data source.
Query parameters
dataSourceId
integer Data source ID number.
Yes
Payload parameters
dataSourceIds
array The ID of the data source the user is subscribing to.
No
approvals
array Includes details about the Subscription policy on the data source: requiredPermissions, specificApproverRequired, specificApprover, and ownerModelId.
No
Response schema
body
array Contains details about the data source, including the data source ID, subscription status of the user, the profile ID of the user, and the dates the data source was created and updated.
Request example
The following request subscribes to the data source with ID 22.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/subscribe?dataSourceId=22Payload example
{
"dataSourceIds": [
22
],
"metadata": {},
"approvals": [
{
"requiredPermission": "Owner",
"specificApproverRequired": false,
"specificApprover": 2,
"ownerModelId": 23
}
],
"groupId": 12
}Response example
{
"inError": [],
"success": [{
"id": 64,
"modelId": "22",
"modelType": "datasource",
"state": "subscribed",
"metadata": {},
"admin": null,
"denialReasoning": null,
"profile": 2,
"group": null,
"expiration": null,
"acknowledgeRequired": false,
"createdAt": "2021-08-26T16:36:09.587Z",
"updatedAt": "2021-08-26T16:36:09.587Z",
"approved": true
}]
}Request to unmask values
POST /dataSource/{dataSourceId}/reverseMask
Makes a request for values to be unmasked.
Query parameters
dataSourceId
integer The data source ID.
Yes
Payload parameters
column
string The column to unmask.
Yes
unmaskingReason
string The reason the values need to be unmasked.
Yes
unmaskingUsers
array[integer] The profile ID of the users who can unmask the values for the requestor.
Yes
projectId
integer The ID of the associated project.
No
dataSourceId
integer The data source ID.
No
Response schema
id
integer The ID of the request.
requestingUserProfile
integer The requesting user profile ID.
dataSourceId
integer The data source ID.
reason
string The reason for the unmasking request.
metadata
string Metadata regarding the masking, such as the column, values, and maskingConfig.
type
string The type of request.
state
string The state of the task, such as pending.
createdAt
timestamp The date and time the task was created.
updatedAt
timestamp The date and time the task was updated.
Request example
The following requests for values to be unmasked.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/23/reverseMaskRequest payload example
{
"column": "cc_county",
"values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
"unmaskReasoning": "Marketing project",
"unmaskingUsers": [1]
}Response example
{
"id": 1,
"requestingUserProfile": 13,
"dataSourceId": 12,
"reason": "Marketing Campaign",
"metadata": {
"salt": "**********87",
"column": "cc_county",
"values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
"maskingConfig": {
"type": "Reversible",
"metadata": {}
}
},
"type": "unmask",
"state": "pending",
"createdAt": "2021-10-27T20:35:48.253Z",
"updatedAt": "2021-10-27T20:35:48.253Z"
}Add a user to a data source
POST /dataSource/{dataSourceId}/access
Add a user to a specific data source. Requestors cannot add themselves to a data source. To request access to a data source, use the /dataSource/subscribe endpoint.
Query parameters
dataSourceId
integer The data source ID.
Yes
Payload parameters
state
string The status of the user: subscribed, owner, expert, or ingest.
Yes
profileId
integer The profile ID of the user being added to the data source.
Yes
groupId
integer The ID of the group being added to the data source.
No
approvals
array Details about the user approving access: requiredPermission, specificApproverRequired, and specificApprover.
No
expiration
date The date the user's data source subscription ends.
No
Response schema
id
integer The user's subscription ID.
modelId
integer The model ID.
modelType
string The Immuta component to add the user to: datasource or project.
state
string The user's data source role, such as subscribed.
denialReasoning
string If the user was denied access, the reason for denial.
profile
integer The user's profile ID.
group
integer If a group was added, the group ID.
expiration
date The date the user's subscription to the data source will expire.
acknowledgeRequired
boolean If the data source is associated with a project, this value will be true if the user needs to confirm they have read the project acknowledgment.
createdAt
timestamp The date and time of creation.
updatedAt
timestamp The date and time of update.
approved
boolean When true, the user's request has been approved.
Request example
The following request adds a user (saved in example-payload.json) to this data source.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/22/accessRequest payload example
{
"profileId": 3,
"state": "subscribed"
}Response example
{
"id": 19,
"modelId": "1",
"modelType": "datasource",
"state": "subscribed",
"metadata": null,
"admin": 2,
"denialReasoning": null,
"profile": 3,
"group": null,
"expiration": null,
"acknowledgeRequired": false,
"createdAt": "2021-09-21T14:24:12.528Z",
"updatedAt": "2021-09-21T14:24:12.528Z",
"approved": true
}Manage data source requests
GET
Get all pending tasks for this user and pending tasks this user has created.
GET
Returns all tasks the user has made/can approve or deny for this data source.
Get pending tasks by user
GET /dataSource/tasks/pending
Get all pending tasks for this user and pending tasks this user has created.
Query parameters
searchText
string If specified, will filter results using the specified string.
No
searchModel
string Will filter the results by model type: dataSource or schemaEvolution.
No
offset
integer The number of results to skip (for paging).
No
size
integer The number of results to return per page.
No
schemaEvolutionConnectionString
string The schema evolution connection string to filter by.
No
countBySchemaEvolution
boolean Iftrue, will only return the number of tasks, grouped by schema evolution.
No
countByDataSource
boolean Iftrue, will only return the number of tasks, grouped by data source.
No
countOnly
boolean When true, will only return a count of the pending tasks.
No
groupByDataSource
boolean If true, will return the results as an array of { dataSourceId: , rows: }.
No
types
Array[string] Filters the results by the type of task: unmask, dataSourceCreated, columnAdded, columnDeleted, or columnTypeChanged.
No
Response schema
outgoing
array Includes details of the tasks or requests created by the user, such as the count, type, and targetEmails.
incoming
array Includes details about the tasks received by the user, such as the count, type, and targetEmails.
Request example
The following request gets all pending tasks for a user and pending tasks the user has created.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/tasks/pendingResponse example
{
"outgoing": [],
"incoming": [{
"fullCount": 1,
"dataSource": {
"id": 11,
"name": "Public Customer"
},
"isOutgoing": false,
"rows": [{
"id": 1,
"state": "pending",
"type": "columnAdded",
"reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
"targetNames": [
"Katie"
],
"targetEmails": [
"[email protected]"
],
"requester": {
"name": "Immuta System Account",
"id": 1,
"email": null
},
"dataSource": {
"id": 11,
"name": "Public Customer"
},
"createdAt": "2024-08-06T14:37:09.279+00:00",
"metadata": {
"colName": "org",
"colType": "VARCHAR(16777216)"
},
"isOutgoing": false
}]
}]
}Mark tasks as complete
GET /dataSource/tasks/{taskId}
Handles the given task and marks it as complete.
Query parameters
taskId
integer The task ID.
Yes
Response schema
result
array Includes details about the task.
Request example
The following request handles the given task and marks it as complete.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/tasks/7Response example
{
"result": {
"unmaskPairs": [
{
"masked": "TnpjNFpXSXdOR1pqWkdNeE5EYzJPQT09OktaWjJEVldXZVluTmQ2SUVQdW1MajJtVTdqL2ZBT1JlaGFUUHJidmhkWmM9",
"unmasked": "[email protected]"
}
],
"column": "email"
}
}Return tasks for a data source
GET /dataSource/{dataSourceId}/tasks
Returns all tasks the user has made/can approve or deny for this data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
states
Array[string] The state of the tasks: pending or completed.
No
targetProfileId
integer Only returns tasks where the target user has this profile ID.
No
requestingUserProfileId
integer Only returns tasks where the requesting user has this profile ID.
No
profileId
integer Returns tasks where either the target or requesting user has this profile ID.
No
searchText
string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.
No
searchModel
string A string used to determine how results should be filtered using searchText.
No
types
Array[string] The type of task: unmask, dataSourceCreated, columnAdded, columnDeleted, or columnTypeChanged.
No
size
integer The number of results to return.
No
offset
integer The number of results to skip (for paging).
No
sortField
string The field by which to sort the result set.
No
sortOrder
string The order in which to sort the results. The default is desc.
No
countOnly
boolean If true, will only return the number of tasks.
No
Response schema
hits
array Includes details about each task, such as the id, state, type, and requestor.
count
integer The total number of tasks.
Request example
The following request returns all tasks the user has made/can approve or deny for this data source.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/tasks?sortOrder=descResponse example
{
"hits": [
{
"fullCount": 2,
"id": 6,
"state": "completed",
"type": "columnAdded",
"reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
"targetNames": [
"John"
],
"targetEmails": [
"[email protected]"
],
"requester": {
"name": "Immuta System Account",
"id": 1,
},
"dataSource": {
"id": 11,
"name": "Public Customer"
},
"createdAt": "2021-10-12T15:48:23.095Z"
},
{
"fullCount": 2,
"id": 7,
"state": "completed",
"type": "columnAdded",
"reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Fake Medical Claims 2017\"",
"targetNames": [
"John"
],
"targetEmails": [
"[email protected]"
],
"requester": {
"name": "Immuta System Account",
"id": 1
},
"dataSource": {
"id": 3,
"name": "Public Fake Medical Claims 2017"
},
"createdAt": "2021-10-12T18:56:22.954Z"
}
],
"count": 2
}Change user status
PUT /dataSource/{dataSourceId}/access/{subscriptionId}
Change user status for a specific data source. Requestors cannot update their own status for a data source.
Query parameters
dataSourceId
Integer The data source ID.
Yes
subscriptionId
Integer The data source member's subscription ID.
Yes
Payload parameters
state
string The new status for the user: denied, subscribed, owner, expert or ingest.
Yes
Response schema
id
integer The data source member's subscription ID.
modelId
integer The model ID.
modelType
array The Immuta component the user's subscription was updated for: datasource or project.
state
array The current state of the user's role: denied, subscribed, owner, expert, or ingest.
profile
integer The profile ID.
group
integer If a group's status is being updated, this is the group ID.
expiration
timestamp The date the user will no longer have access to the data source.
acknowledgeRequired
boolean This attribute is specific to projects. When true the user needs to confirm they have read the project acknowledgement statement.
createdAt
timestamp The date and time created.
updatedAt
timestamp The date and time updated.
originalState
array The user's previous status for the data source.
approved
boolean If true, the status is approved.
Request example
The following request changes the user status to subscribed for the specified data source.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/23/access/95Payload example
{
"state": "subscribed"
}Response example
{
"id": 95,
"modelId": "3",
"modelType": "datasource",
"state": "subscribed",
"metadata": {},
"admin": 2,
"profile": 3,
"group": null,
"expiration": null,
"acknowledgeRequired": false,
"createdAt": "2021-10-12T15:40:13.878Z",
"updatedAt": "2021-10-12T16:10:46.801Z",
"originalState": "expert",
"approved": true
}Update data sources
POST
Save blob metadata to Immuta and store raw content in local blob store.
PUT
Trigger the schema monitoring job for the specified detection group, or all groups if no ID is given.
Update a data source
PUT /dataSource/{dataSourceId}
Update a data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
Payload parameters
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
No
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
No
recordFormat
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
No
type
string The type of data source: ingested (metadata will exist in Immuta) or queryable (metadata is dynamically queried).
No
name
string The name of the data source. It must be unique within the Immuta tenant.
No
sqlTableName
string A string that represents this data source's table in Immuta.
No
organization
string The organization that owns the data source.
No
category
string The category of the data source.
No
description
string The description of the data source.
No
hasExamples
boolean When true, the data source contains examples.
No
Response schema
private
boolean When false, the data source will be publicly available in the Immuta UI.
blobHandler
array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType
string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
recordFormat
string The data format of blobs in the data source, such as json, xml, html, or jpeg.
type
string The type of data source: ingested (metadata will exist in Immuta) or queryable (metadata is dynamically queried).
name
string The name of the data source. It must be unique within the Immuta tenant.
sqlTableName
string A string that represents this data source's table in Immuta.
organization
string The organization that owns the data source.
description
string The description of the data source.
policyHandler
array The ID of the policy handler and details about the data policies enforced on the data source.
subscriptionPolicy
array Details about the subscription policy enforced on the data source, including the type of policy and exceptions.
Request example
The following request updates the data source's documentation (saved in example-payload.json).
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/22Request payload example
{
"documentation": "# Public Credit Accounts\nThis request updates the data source documentation."
}Response example
{
"name": "Public Credit Accounts",
"recordFormat": "Not Provided",
"description": null,
"policyHandler": {
"handlerId": 21,
"visibilitySchema": {
"fields": [],
"version": "2021-09-15T17:44:20.678Z"
},
"previousHandlerId": 20,
"dataSourceId": 1
},
"sqlSchemaName": "public",
"sqlTableName": "credit_accounts",
"blobHandler": {
"url": "https://1.1.1.1:1111/snowflake/handler/1",
"ca": {
"name": "Certificate Authority Bundle"
},
"manualDictionary": false
},
"createdBy": 2,
"deleted": false,
"type": "queryable",
"recordCount": 0,
"rowCount": "100000",
"documentation": "# Public Credit Accounts\nThis request updates the data source documentation.",
"statsExpiration": "2021-09-22T13:51:46.646Z",
"id": 1,
"blobHandlerType": "Snowflake",
"policyHandlerType": "Builder",
"subscriptionType": "approval",
"subscriptionPolicy": {
"type": "subscription",
"approvals": [{
"requiredPermission": "OWNER",
"specificApproverRequired": false
}]
},
"globalPolicies": null,
"status": "failed",
"statusInfo": {
"sql": {
"status": "passed",
"message": "Passed"
},
"stats": {
"status": "passed",
"lastAttempted": "2021-09-21T13:51:46.674Z"
},
"fingerprint": {
"status": "passed",
"lastAttempted": "2021-09-09T14:12:25.177Z"
},
"lastAttempt": {
"date": "2021-09-20T19:35:21.929Z"
},
"globalPolicy": {
"status": "passed",
"lastAttempted": "2021-09-17T19:07:38.092Z"
},
"highCardinality": {
"status": "failed",
"message": "Error could not connect to remote database",
"lastAttempted": "2021-09-20T16:43:19.426Z"
}
},
"expiration": null,
"catalogMetadata": null,
"workspace": null,
"seeded": false,
"schemaEvolutionId": 1,
"columnEvolutionEnabled": true,
"createdAt": "2021-09-09T14:12:09.511Z",
"updatedAt": "2021-09-21T13:52:42.908Z"
}Update multiple data sources
PUT /dataSource/bulk/{type}
Update data sources.
Query parameters
type
string The action to perform on the data sources: add-users, disable, restore, delete, or tags.
Yes
Payload parameters
ids
array[integer] The IDs of the data sources to update.
Yes
update
array[object] Only required for add-users (includes metadata about the users' profiles: id and state) and tags (includes metadata about the tags: name and source) types.
No
Response schema
bulkId
string The ID of the bulk data source update.
jobsCreated
integer The number of jobs created.
Request example
The following request adds the Address.email tag to two data sources.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/bulk/tagsPayload example
{
"ids": [49, 50],
"update": {
"tags": [{
"name": "Address.email",
"source": "curated"
}]
}
}Response example
{
"bulkId": "bulk_ds_update_4896d300e04c4a8f89493ebf125c5c6b",
"jobsCreated": 2
}Refresh views
POST /dataSource/bulkRefreshViews
Refresh views.
Payload parameters
dataSourceIds
array[integer] The IDs of the data sources of the native views to update.
Yes
Request example
The following request with the payload below refreshes the view for the data source with the ID 202.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer <TOKEN>" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/bulkRefreshViewsPayload example
{
"dataSourceIds": [202]
}Save blob metadata to Immuta
POST /dataSource/{dataSourceId}/blobs
Save blob metadata to Immuta.
Query parameters
dataSourceId
integer The data source ID.
Yes
Payload parameters
blobId
string The unique ID used to identify this blob within its data source.
Yes
file
string The binary file to add to the data source.
Yes
filename
string The name that will display in the filesystem.
No
tags
array[string] Tags to apply to the blob.
No
date
data A date that corresponds to a date within the record itself.
No
filesize
integer The size of the file in bytes.
No
Response schema
blobsWithoutIds
integer The number of blobs added without IDs.
blobsInError
array The blobs that were not added because of an error.
blobsInserted
array The blobs added to the data source.
tags
array[string] Tags applied to the blobs.
Request example
The following request saves blob metadata to Immuta.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/blobsPayload example
[
{
"blobId": "testblob",
"filename": "testfile",
"tags": [
"update"
],
"visibility": {},
"metadata": {},
"date": "2021-10-21",
"filesize": 2
}
]Response example
{
"blobsWithoutIds": 0,
"blobsInError": [],
"blobsInserted": [
"string"
],
"tags": [
"string"
]
}Store blob metadata locally
POST /dataSource/{dataSourceId}/persistBlob
Save blob metadata to Immuta and store raw content in local blob store.
Query parameters
dataSourceId
integer The data source ID.
Yes
Payload parameters
blobId
string The unique ID used to identify this blob within its data source.
Yes
file
string The binary file to add to the data source.
Yes
filename
string The name that will display in the filesystem.
No
tags
array[string] Tags to apply to the blob.
No
date
data A date that corresponds to a date within the record itself.
No
filesize
integer The size of the file in bytes.
No
Response schema
blobsWithoutIds
integer The number of blobs added without IDs.
blobsInError
array The blobs that were not added because of an error.
blobsInserted
array The blobs added to the data source.
tags
array[string] Tags applied to the blobs.
Request example
The following request saves blob metadata to Immuta and stores raw content in local blob stores.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json
https://your-immuta-url.com/dataSource/23/persistBlobPayload example
{
"blobId": "test/pov-queries-2.dbc",
"filename": "pov-queries",
"date": "2021-10-26",
"file": "pov-queries.dbc"
}Response example
{
"blobsWithoutIds": 0,
"blobsInError": [],
"blobsInserted": ["test/pov-queries.dbc"],
"tags": []
}Trigger schema monitoring jobs
PUT /dataSource/detectRemoteChanges
Trigger the schema monitoring job for the specified detection group, or all groups if no payload parameters are given.
Payload parameters
dataSourceIds
array[integer] The data source IDs to run the column detection job on. Leave empty to run this job globally on all data sources. This parameter cannot be included in the payload if schemaEvolutionId or any combination of hostname, database, port, or table is included.
No
hostname
string The hostname of the data sources. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
port
integer The port used to connect the data sources to Immuta. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
database
string The database name. This runs schema monitoring on the database provided. If data sources were initially registered via the V2 API, including this parameter will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, including this parameter will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
table
string The table name. This will run column detection to just update the columns in this table. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.
No
schemaEvolutionId
integer The ID of the schema to run the schema monitoring job on. This will run on all tables associated with the specified ID. The schema ID can be found in the response body of /dataSource/{dataSourceId}. This parameter cannot be included in the payload if dataSourceIds or any combination of hostname, database, port, or table is included.
No
skipColumnDetection
boolean When true, Immuta will only pull new tables from the source server. This parameter can only be paired with schemaEvolutionId.
No
overrides.httpPath
string If Databricks ephemeral overrides are configured, provide the alternative HTTP path to trigger schema monitoring on that ephemeral cluster.
No
Response schema
schemaDetection
object Includes details about the resulting schema detection jobs.
columnDetection
object Includes details about the resulting column detection jobs.
bulkId
string The unique identifier of the jobs running schema monitoring and column detection.
Responses may include bulkId, schemaDetection, or columnDetection objects, depending on the payload.
Request example
The following request triggers the schema monitoring job for the specified detection group.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/dataSource/detectRemoteChangesPayload example
The tabs below illustrate payloads for triggering schema monitoring on a host, database, or table. The request will run schema monitoring for all databases registered under the hostname provided in the payload.
The request will run schema monitoring for all databases registered under the hostname provided in the payload.
{
"hostname": "organization.us-east-1.snowflakecomputing.com"
}The request will run schema monitoring on the database provided in the payload. If data sources were initially registered via the V2 API, this request will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, this request will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas.
{
"database": "public"
}The request will run column detection and update the columns on the table specified in the payload.
{
"database": "public",
"table": "healthcare"
}Response examples
The tabs below illustrate the example response for each example payload provided above.
{
"bulkId": "31ab4312-b90a-49a6-baf8-70f87cd92a89"
}{
"bulkId": "5d129011-6254-413d-a365-6e394c06e277"
}{
"bulkId": "5d129011-6254-413d-a365-6e394c06e277",
"schemaDetection": {
"jobs": ["7d129033-6254-413d-a345-9o3242c06f242"]
},
"columnDetection" : {
"jobs": ["8d129033-6254-413d-a345-9o3123c06f123"]
}
}View and Review Data Sources
GET
Get all of the recent policy activities for a given data source.
GET
Return the users who can unmask the given column.
Run a data source health check
GET /dataSource/{dataSourceId}/test
Run a health check on the data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
Response schema
blob
object Indicates whether or not the blob was successfully crawled.
columnEvolution
object Indicates whether or not the job run to check for columns added or removed from the data source passed and when it was last run.
externalCatalog
object Indicates whether or not the external catalog was successfully linked to the data source.
fingerprint
object Indicates whether or not the fingerprint job was successful (passed) and when it was last run. The fingerprint captures summary statistics of the data source.
framework
object Indicates whether or not the classification was successfully run on the data source to determine its sensitivity.
globalPolicy
object Indicates whether or not global policies were successfully applied to the data source.
highCardinality
object Indicates whether or not the job run to calculate the data source's high cardinality column passed and when it was last run.
schemaEvolution
object Indicates whether or not the job run to check if a new table had been added in the remote database passed and when it was last run. If a new table was added, Immuta automatically creates a new data source. Correspondingly, if a remote table is removed, that data source will be disabled in the console.
sdd
object Indicates whether or not sensitive data discovery was successfully run on the data source.
sql
object Indicates whether or not the SQL query run to check the data source's health passed and when it was last run.
stats
object Indicates whether or not the job run to calculate the number of rows in the data source passed and when it was last run.
Request example
The following request tests a data source.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/testResponse example
{
"sql": {
"status": "passed",
"message": "Passed"
},
"stats": {
"status": "passed",
"lastAttempted": "2021-09-09T13:43:43.948Z"
},
"fingerprint": {
"status": "passed",
"lastAttempted": "2021-07-22T14:12:01.525Z"
},
"lastAttempt": {
"date": "2021-09-09T16:47:05.173Z"
},
"columnEvolution": {
"status": "passed",
"lastAttempted": "2021-09-08T16:36:05.557Z"
},
"highCardinality": {
"status": "passed",
"lastAttempted": "2021-07-22T14:11:58.439Z"
},
"schemaEvolution": {
"status": "passed",
"lastAttempted": "2021-09-08T16:35:57.867Z"
},
"status": "passed"
}Retrieve blob handlers
GET /dataSource/blobHandlerTypes
Retrieve all blob handlers the current user is allowed to create.
Response schema
name
string The name of the blob handler.
baseUrl
string The base URL for the data source.
config
array Includes information about the connection configuration.
port
integer The port number.
driver
string The name of the driver.
Request example
The following request retrieves all blob handlers the current user is allowed to create.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/blobHandlerTypesResponse example
{
"name": "Azure Synapse Analytics",
"pluginType": "odbcHandler",
"clientUrl": "/asa",
"baseUrl": "https://0.0.0.0:8823/asa",
"config": {
"port": 1433,
"allowSSLToggle": false
},
"displayOrder": 41,
"requiresHashPhraseForDP": true,
"driver": "Azure Synapse Analytics"
}Get data sources by purpose
GET /dataSource/byPurposes
Get data sources that match a set of purposes.
Query parameters
purposes
array[string] The purposes to filter the data sources by.
Yes
excludedProjects
array[integer] Excludes data sources associated with specified project IDs.
No
Response schema
id
integer The data source ID.
name
array The name of the data source.
policyId
integer The policy ID.
restrictions
array Details regarding the operator (and or or) and purposes.
Request example
The following request gets data sources that match a set of purposes.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/byPurposes?purposes=Data+AnalysisResponse example
[{
"id": 42,
"name": "Army Army Records",
"policyId": 56,
"restrictions": [{
"operator": "and",
"purposes": ["Data Analysis"]
}]
}]Retrieve data sources by user
GET /dataSource/rpc/mine
Retrieves all the data sources the current user has access to.
Response schema
id
integer The data source ID.
name
string The data source name.
type
string The type of data source, such as ingested.
sqlTableName
string The name of the table in Immuta.
sqlSchemaName
string The name of the schema in Immuta.
blobHandlerType
string The type of handler, such as Snowflake.
sparkUseJDBC
boolean When true, uses a JDBC driver.
Request example
The following request retrieves all the data sources the current user has access to.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/rpc/mineResponse example
{
"id": 23,
"name": "Public Record",
"type": "queryable",
"sqlTableName": "record",
"sqlSchemaName": "public",
"blobHandlerType": "Snowflake",
"sparkUseJDBC": true
}Get recent policy activities for a data source
GET /dataSource/{dataSourceId}/activities
Get all of the recent policy activities for a given data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
offset
integer The number of results to skip (for paging).
No
size
integer The number of results to return per page.
No
Response schema
count
integer The number of results.
activities
array Includes details about the policy and the data source, including the policy and data source type, when the activity notification was triggered, and whether or not the policy change was triggered by a Global policy.
actionBy
array Details about who triggered the action.
targetUser
array Information about the user who received the notification.
Request example
The following request gets all of the recent policy activities for a given data source.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/activitiesResponse example
{
"count": 3,
"activities": [{
"modelType": "datasource",
"modelId": "23",
"createdAt": "2021-09-07T16:13:25.197Z",
"notificationType": "policyUpdated",
"metadata": {
"dataSourceName": "Public Foobar",
"triggeredByGlobal": false,
"conflictCount": 0,
"policyType": "data",
"handlerId": 3,
"previousHandlerId": 2,
"dataSourceType": "queryable"
},
"read": false,
"id": 256,
"updatedAt": "2021-09-07T16:13:25.197Z",
"actionBy": {
"id": 2,
"name": "first last",
"email": "[email protected]"
},
"targetUser": {
"id": 2,
"name": "first last",
"email": "[email protected]"
}
}]
}Get profiles for data source owners and experts
GET /dataSource/{dataSourceId}/contacts
Gets the profiles for the data source owners and experts.
Query parameters
dataSourceId
integer The data source ID.
Yes
Response schema
id
integer The data source ID.
state
string The user's data source role, such as owner or subscribed.
name
string The user's name.
string The user's email.
profile
integer The user's profile ID.
Request example
The following request gets all the profiles for the data source owners and experts.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/contactsResponse example
{
"type": "profile",
"id": 23,
"state": "owner",
"name": "first last",
"email": "[email protected]",
"profile": 2
},
{
"type": "profile",
"id": 23,
"state": "owner",
"name": "Tommy Test",
"email": "[email protected]",
"profile": 3
}Get tags by data source
GET /dataSource/{dataSourceId}/tags
Get the tags for a data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
blobId
string Returns the tags for the specified blob.
No
blobTagsOnly
boolean When true, will only display blob tags associated with a data source.
No
Response schema
tags
array Includes details about the tags, such as the name, source, and the profile ID of the user who added the tag.
Request example
The following request gets the tags for data source 4.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/4/tagsResponse example
{
"tags": [
{
"name": "Discovered.Entity.Medicare Number",
"source": "curated",
"modelType": "datasource",
"modelId": "4",
"addedBy": 1,
"deleted": false
}
]
}Get users who can unmask columns
GET /dataSource/{dataSourceId}/{columnName}/unmaskUsers
Return the users who can unmask the given column.
Query parameters
dataSourceId
integer The data source ID.
Yes
columnName
string The name of the column to unmask.
Yes
Response schema
name
array The name of the user who can unmask the value.
profileId
integer The profile ID of the user who can unmask the value.
iamid
string The IAM ID of the user who can unmask the value.
Request example
The following request returns the users who can unmask the given column.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/47/gender/unmaskUsersResponse example
[
{
"name": "Katie",
"profileId": 2,
"iamid": "bim"
}
]Delete Data Sources and More
DELETE
Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.
Delete a data source
DELETE /dataSource/{dataSourceId}
Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.
Query parameters
dataSourceId
integer The data source ID.
Yes
Response schema
success
boolean If true, the request to disable or delete the data source was successful.
id
integer The data source ID.
schemaEvolutionId
integer The schema evolution ID.
name
string The data source name.
disabled
boolean If true, the data source is disabled.
handlerDeleteErrorMessage
string The delete error message.
Request example
The following request deletes the data source 23.
curl \
--request DELETE \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23Response example
{
"success": true,
"id": 23,
"schemaEvolutionId": 1,
"name": "Public Foobar",
"disabled": true,
"handlerDeleteErrorMessage": null
}Delete a task
DELETE /dataSource/tasks/{taskId}
Delete the specified task.
Query parameters
taskId
integer Target task ID.
Yes
Response schema
id
integer The deleted task ID.
state
array The state of the deleted task, such as pending.
type
array The type of deleted task, such as columnAdded.
targetNames
string The name of the user who received the request.
targetEmails
string The email of the user who received the request.
requester
metadata Details regarding the requesting profile.
dataSource
metadata details regarding the data source.
metadata
array Details about the deleted task.
Request example
The following request deletes a specified task.
curl \
--request DELETE \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/tasks/8Response example
{
"fullCount": 1,
"id": 8,
"state": "pending",
"type": "columnAdded",
"reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
"targetNames": ["Katie"],
"targetEmails": ["[email protected]"],
"requester": {
"name": "Immuta System Account",
"id": 1
},
"dataSource": {
"id": 11,
"name": "Public Customer"
},
"createdAt": "2021-10-12T19:28:04.999Z"
}Delete a blob
DELETE /dataSource/{dataSourceId}/blob/{blobId*}
Delete a blob.
Query parameters
dataSourceId
integer The data source ID.
Yes
blobId
string The blob ID.
Yes
Response schema
When the blob is successfully deleted, there will be no response.
Request example
The following request deletes a blob.
curl \
--request DELETE \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/blob/1Unsubscribe from a data source
DELETE /dataSource/{dataSourceId}/unsubscribe
Unsubscribe from a data source.
Query parameters
dataSourceId
integer The data source ID.
Yes
Response schema
status
boolean If true, the requesting user is unsubscribed from the data source.
Request example
The following request unsubscribes the user from data source 23.
curl \
--request DELETE \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/dataSource/23/unsubscribeResponse example
{
"success": true
}Last updated
Was this helpful?