Subscribe to and Manage Data Sources

Data Source API reference guide

This page describes the dataSource endpoint, through which users can subscribe to data sources and manage data source tasks. To create data sources, see the specific handler endpoints.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Data source workflow

1. Search data sources and data source details.

2. Access data sources and make data source requests.

3. Manage data source requests.

4. Update data sources.

5. Delete a data source.

Search data sources and data source details

Method	Path	Purpose
GET	/dataSource	Search for data sources.
GET	/dataSource/{dataSourceId}	Get a data source based on the ID.
GET	/dataSource/name/{dataSourceName}	Get data source based on the name.
GET	/dataSource/sqlTableName/{shortName}	Get data source based on the short name.
GET	/dataSource/{dataSourceId}/lineage/{type}	Get parent and child relationship records for derived data sources using a specified data source ID.
GET	/dataSource/{dataSourceId}/blob/{blobId*}	Retrieve a blob.
GET	/dataSource/{dataSourceId}/access	Get all users with the provided access level for this data source.
GET	/dataSource/{dataSourceId}/users/{profileId}/policyInfo	Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.
GET	/dataSource/{dataSourceId}/users/{profileId}/visibilityReport	Retrieves a summary of total records, total visibilities, and visibilities a given user has access to.
GET	/dataSource/{dataSourceId}/visibilityReport	Retrieves a summary of total records, total visibilities, and visibilities the current user has access to for a specified data source.

Search for data sources

GET /dataSource

Search for data sources that you own or are authorized to see.

Required: To appear in the response, you must own the data source, the data source must be publicly visible, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
blobHandlerType	array[string] Describes the type of underlying blob handler that will be used with this data source (e.g., Custom, MS SQL).	No
subscription	array[string] The requesting user's subscription status: pending, owner, subscribed, not_subscribed, expert, or ingest.	No
status	array[string] The data source status: passed or failed.	No
tag	array[string] Filters data sources by tags associated with the data sources.	No
searchText	string Searches for data source names using the provided string.	No
column	array[string] Searches for data source column names.	No
connectionString	array[string] Searches by connection string.	No
schema	string Searches for data source schema.	No
nameOnly	boolean When true, searchText will only search data source names. Default is false.	No
idOnly	boolean When true, only returns the ID Of the data source and the user's subscription status.	No
dataSourceIds	array[integer] Searches for the provided data source IDs.	No
selectFields	array[string] This field accepts the values id, name, and columnEvolutionEnabled. When id or name are provided, the request will return only the ID or name of the data source and the subscription status. If columnEvolutionEnabled is provided, the response will also include information about the policies, policy conflicts, and workspaces associated with the data sources.	No
offset	integer Used in combination with size to fetch pages. Default is 0.	No
size	integer The number of results to return per page. Default is 10.	No
sortField	string Used to sort results by field, which must be createdAt, name, blobHandlerType, subscriptionStatus, recordCount, status, policy, or editable.	No
sortOrder	string Sorts results by order, which must be asc or desc.	No
excludedProjects	array[integer] Filter out any data sources that belong to the specified projects.	No
ephemeral	boolean When true, returns ephemeral data sources.	No
clusterName	string The name of the remote cluster the data source is connected to.	No
mode	integer Specifies the query mode, which must be 0 (FULL), 1 (COUNT), 4 (TAG), 5 (MIN_MAX), or 6 (STATUS).	No
globalPolicy	string Filter by data sources that have this Global Policy applied.	No
hostname	string Searches data sources by hostname.	No
determinePolicyContflicts	boolean When true, filters results to return the data sources with policy conflicts.	No
collectionId	string Filter by data sources in the domain with this ID.	No
collectionExclude	string When true, filters results to return the data sources in the domain of the provided ID.	No
sddTemplateName	string Filter by data sources that have the specified SDD identification framework applied.	No
excludeSddTemplateName	string Filter by data sources that do not have the specified SDD identification framework applied.	No

Response schema

The response will return data sources you own or are to see.

Attribute	Description
name	string Data source name.
id	integer Data source ID.
deleted	boolean If true the data source is a deleted data source.
description	string The data source description.
createdAt	timestamp The date and time the data source was created.
subscriptionPolicy	array Details the type of Subscription Policy applied to the data source.
schemaEvolutionId	integer The schema evolution ID.
recordCount	integer The record count.
status	array[string] Accepted statuses are passed or failed.
subscriptionStatus	array[string] Accepted statuses are subscribed or unsubscribed.
blobHandlerType	array[string] Describes the type of underlying blob handler of this data source (e.g., Custom, MS SQL).
subscriptionType	string The type of subscription policy on the project. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
connectionString	string The connection string information.
sqlSchemaName	string The schema name.
policy	string When this value is none, there are no data policies applied to the data source. Otherwise, this field indicates whether or not there are policy conflicts among the data policies applied to the data source.
policyHandlerType	string The policy handler type, such as None or Builder.

Request example

The following request returns 2 data sources.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource?size=2

Response example

{
  "name": "Public Barfoo",
  "id": 22,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2021-07-22T14:11:55.539Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "status": "passed",
  "subscriptionStatus": "subscribed",
  "blobHandlerType": "Snowflake",
  "subscriptionType": "automatic",
  "connectionString": "[e-mail]:###/test",
  "sqlSchemaName": "public",
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
},
{
  "name": "Public Aaa Tpc",
  "id": 39,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2023-08-21T10:39:00.250Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "exceptions": {
          "operator": "and",
          "conditions": [
              {
                  "type": "groups",
                  "group": {
                      "name": "alpha"
                  }
              }
          ]
      },
      "allowDiscovery": true,
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "blobHandlerType": "Snowflake",
  "subscriptionType": "policy",
  "sqlSchemaName": "public",
  "status": "passed",
  "subscriptionStatus": "owner",
  "connectionString": "[e-mail]:###/test",
  "remoteTable": "tpc",
  "remoteSchema": "public",
  "domainId": null,
  "domainName": null,
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
}

Get a data source by ID

GET /dataSource/{dataSourceId}

Get a data source based on the ID.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID .	Yes

Response Schema

Attribute	Description
name	string The data source name.
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description	string The description of the data source.
policyHandler	array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName	string A string that represents this data source's schema name in Immuta.
sqlTableName	string The SQL table name in Immuta.
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy	integer The ID of the profile creating the data source.
deleted	boolean If true, the data source was deleted.
type	string The data source type: queryable.
rowCount	integer The number of rows.
documentation	string Documentation associated with the data source.
id	integer The data source ID.
policyHandlerType	string The type of policy handler applied to the data source: Builder.
subscriptionType	string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy	array Details about the Subscription Policy applied to the data source.
globalPolicies	string Details about the Global Policies applied to the data source.
status	string The data source health status.

Request example

The following request gets a data source based on the ID 22.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
  "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
  "sql": {
      "status": "passed",
      "message": "Passed"
   }
  }
}

Get data source by name

GET /dataSource/name/{dataSourceName}

Get a data source based on the name.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceName	string The data source name.	Yes

Response schema

Attribute	Description
name	string The data source name.
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description	string The description of the data source.
policyHandler	array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName	string A string that represents this data source's schema name in Immuta.
sqlTableName	string The SQL table name in Immuta.
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy	integer The ID of the profile creating the data source.
deleted	boolean If true, the data source was deleted.
type	string The data source type:queryable.
rowCount	integer The number of rows.
documentation	string Documentation associated with the data source.
id	integer The data source ID.
policyHandlerType	string The type of policy handler applied to the data source: Builder.
subscriptionType	string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy	array Details about the Subscription Policy applied to the data source.
globalPolicies	string Details about the Global Policies applied to the data source.
status	string The data source health status.

Request example

The following request gets a data source based on the name Public Barfoo.

curl \
   --request GET \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/name/Public%20Barfoo

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
      },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
      "sql": {
      "status": "passed",
      "message": "Passed"
      }
  }
}

Get a data source by the short name

GET /dataSource/sqlTableName/{shortName}

Get a data source based on the SQL table name.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
shortName	string The data source SQL table name.	Yes

Response schema

Attribute	Description
name	string The data source name.
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.
description	string The description of the data source.
policyHandler	array The ID of the policy handler and details about the data policies enforced on the data source.
sqlSchemaName	string A string that represents this data source's schema name in Immuta.
sqlTableName	string The SQL table name in Immuta.
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
createdBy	integer The ID of the profile creating the data source.
deleted	boolean If true, the data source was deleted.
type	string The data source type: queryable.
rowCount	integer The number of rows.
documentation	string Documentation associated with the data source.
id	integer The data source ID.
policyHandlerType	string The type of policy handler applied to the data source: Builder.
subscriptionType	string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).
subscriptionPolicy	array Details about the Subscription Policy applied to the data source.
globalPolicies	string Details about the Global Policies applied to the data source.
status	string The data source health status.

Request example

The following request gets a data source based on the SQL table name customer_data.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/sqlTableName/customer_data

Response example

    {
  "name": "Dbo Customer Data",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "visibilitySchema": {
      "fields": [],
      "version": "2021-10-06T19:39:39.145Z"
    },
    "handlerId": 55,
    "dataSourceId": 26
  },
  "sqlSchemaName": "dbo",
  "sqlTableName": "customer_data",
  "blobHandler": {
    "url": "https://1.1.1.1.1/snowflake/testhandler/26",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 1000,
  "documentation": "# Dbo Customer Data",
  "statsExpiration": "2021-11-05T19:37:43.270Z",
  "id": 26,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
    "type": "subscription",
    "automaticSubscription": false
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
    },
    "lastAttempt": {
      "date": "2021-10-06T19:39:39.821Z"
    },
    "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
      }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 4,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-10-01T14:23:27.225Z",
  "updatedAt": "2021-10-06T19:39:39.145Z",
  "subscribedAsUser": true,
  "subscriptionId": 45,
  "acknowledgeRequired": false,
  "subscriptionStatus": "owner",
  "requestedState": "owner",
  "approved": true,
  "subscriptionExpiration": null,
  "filterId": null
}

Get data source relationships

GET /dataSource/{dataSourceId}/lineage/{type}

Get parent and child relationship records for derived data sources using a specified data source ID.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
type	string The type of lineage records to return. Options include: parents, children, and all.	Yes
dataSourceId	integer The target data source ID.	Yes

Response schema

Attribute	Description
children	array Details of the child data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.
parents	array Details of the parent data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.

Request example

The following request gets the parent relationship records for the derived data source with the data source ID 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/lineage/parents

Response example

{
  "parents": [{
    "dataSourceId": 3,
    "dataSourceName": "Public Healthcare Data",
    "deleted": false,
    "createdAt": "2022-08-17T13:41:38.381Z",
    "projectId": 2,
    "projectName": "Derived Data Source",
    "createdBy": "Your Username",
    "policyHandlerDiff": {
      "dsType": "queryable",
      "currentHandlerId": null,
      "previousHandlerId": null
    }
  }]
}

Retrieve a blob

GET /dataSource/{dataSourceId}/blob/{blobid*}

Retrieve a blob.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
blobId	string The blob ID.	Yes

Response schema

The response will download the blobs in a file you specify.

Request example

The following request retrieves a blob.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --output ./the-blobs-will-be-saved-here
    https://your-immuta-url.com/dataSource/22/blob/22

Response example

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  215k  100  215k    0     0   541k      0 --:--:-- --:--:-- --:--:--  541k

Get users by access level

GET /dataSource/{dataSourceId}/access

Get all users with the provided access level for this data source.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
states	Array[string] The status levels to include when querying for user access.	No
approved	boolean Denotes whether the returned access objects should be approved.	No
searchText	string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.	No
size	integer The number of results to return.	No
offset	integer The number of results to skip (for paging).	No
sortField	string The field on which to sort the result set.	No
sortOrder	string The order in which to sort the results.	No
expandGroups	boolean If true will return individual members of any group subscribed.	No
ignoreSystemGenerated	boolean If true, will not return system generated accounts.	No
filterBySchemaEvolution	boolean If true, will only return users who have the specified level of access across ALL data sources within the same schema evolution group as this one.	No

Response schema

Attribute	Description
count	integer The number of users with access to the data source.
users	string The metadata regarding the users with access to the data source.

Request example

The following request gets all users with the provided access level for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22/access?sortOrder=desc

Response example

{
  "count": 2,
  "users": [
    {
      "profile": 2,
      "name": "First Last",
      "iamid": "bim",
      "userid": "[e-mail]",
      "email": "[e-mail]",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "owner",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-10-06T14:58:46.983Z",
      "subscriptionId": 586,
      "createdAt": "2021-10-05T14:33:01.518Z",
      "updatedAt": "2021-10-05T14:33:01.518Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    },
    {
      "profile": 3,
      "name": "Tommy Test",
      "iamid": "bim",
      "userid": "[e-mail]",
      "email": "[e-mail]",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "subscribed",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-09-07T16:16:29.957Z",
      "subscriptionId": 649,
      "createdAt": "2021-10-06T14:58:31.366Z",
      "updatedAt": "2021-10-06T14:58:31.366Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    }
  ]
}

Get user access info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/policyInfo

Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.

Required Immuta permission: FETCH_POLICY_INFO

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
profileId	integer The profile ID of the user.	Yes
projectId	integer The project ID. If provided, this project will be used when evaluating the user's visibilities.	No

Response schema

Attribute	Description
visibilities	array Details of the user's visibilities, including anyKey.
visibilityRuleApplies	boolean If true, a visibility rule exists and the user is not excepted from it.
masked	array Masking information for the data source, including metadata, name, type, and actionType.
additionalFilters	array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.

Request example

The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/policyInfo

Response example

{
  "visibilities": [],
  "visibilityRuleApplies": false,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK",
      "actionType": "Nullify"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK",
      "actionType": "Nullify"
    }
  ],
  "additionalFilters": {}
}

Get user visibility info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
profileId	integer The profile ID of the user.	Yes
informationOnly	boolean If true, the query will just return information for the UI and will skip running some queries for ephemeral data sources.	No
includeNestedColumns	boolean If true, the query will return just information for the dictionary page, including the masking policies for nested columns.	No

Response schema

Attribute	Description
noVisibilities	boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount	integer The total number of possible visibilities the given data source has.
userVisibilitiesCount	integer The number of visibilities the current user can see for the given data source.
masked	array Masking information for the data source, including metadata, name, type, and actionType.
dataSource	integer The data source ID.
dataSourceName	string The data source name.
additionalFilters	array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins	boolean If true the data source allows masked joins.
policySet	array Details about the policies on the data source.

Request example

The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/visibilityReport

Response example

[
  {
    "noVisibilities": true,
    "dataSourceVisibilitiesCount": 0,
    "userVisibilitiesCount": 0,
    "masked": [
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_SOLD_DATE_SK"
      },
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_BILL_CUSTOMER_SK"
      }
    ],
    "dataSource": 16,
    "dataSourceName": "Web Sales",
    "additionalFilters": {},
    "allowMaskedJoins": false,
    "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
  }
]

Get current user visibility info

GET /dataSource/{dataSourceId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
noVisibilities	boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.
dataSourceVisibilitiesCount	integer The total number of possible visibilities the given data source has.
userVisibilitiesCount	integer The number of visibilities the current user can see for the given data source.
denialReason	string Reason the user was denied visibility.
masked	array Masking information for the data source, including metadata, name, type, and actionType.
dataSource	integer The data source ID.
dataSourceName	string The data source name.
additionalFilters	array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.
allowMaskedJoins	boolean If true the data source allows masked joins.
policySet	array Details about the policies on the data source.

Request example

The following request gets all of the visibility information for the current user on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/visibilityReport

Response example

{
  "noVisibilities": true,
  "dataSourceVisibilitiesCount": 0,
  "userVisibilitiesCount": 0,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK"
    }
  ],
  "dataSource": 16,
  "dataSourceName": "Web Sales",
  "additionalFilters": {},
  "allowMaskedJoins": false,
  "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}

Access data sources and make data source requests

Method	Path	Purpose
POST	/dataSource/subscribe	Subscribe to a data source.
POST	/dataSource/{dataSourceId}/reverseMask	Make a request for values to be unmasked.
POST	/dataSource/{dataSourceId}/access	Add a user to a specific data source.

Subscribe to a data source

POST /dataSource/subscribe

Subscribe to a data source.

Query parameters

Attribute	Description	Required
dataSourceId	integer Data source ID number.	Yes

Payload parameters

Attribute	Description	Required
dataSourceIds	array The ID of the data source the user is subscribing to.	No
approvals	array Includes details about the Subscription policy on the data source: requiredPermissions, specificApproverRequired, specificApprover, and ownerModelId.	No

Response schema

Attribute	Description
body	array Contains details about the data source, including the data source ID, subscription status of the user, the profile ID of the user, and the dates the data source was created and updated.

Request example

The following request subscribes to the data source with ID 22.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/subscribe?dataSourceId=22

Payload example

{
  "dataSourceIds": [
    22
  ],
  "metadata": {},
  "approvals": [
    {
      "requiredPermission": "Owner",
      "specificApproverRequired": false,
      "specificApprover": 2,
      "ownerModelId": 23
    }
  ],
  "groupId": 12
}

Response example

{
  "inError": [],
  "success": [{
    "id": 64,
    "modelId": "22",
    "modelType": "datasource",
    "state": "subscribed",
    "metadata": {},
    "admin": null,
    "denialReasoning": null,
    "profile": 2,
    "group": null,
    "expiration": null,
    "acknowledgeRequired": false,
    "createdAt": "2021-08-26T16:36:09.587Z",
    "updatedAt": "2021-08-26T16:36:09.587Z",
    "approved": true
  }]
}

Request to unmask values

Deprecation notice

Support for unmask requests has been deprecated.

POST /dataSource/{dataSourceId}/reverseMask

Makes a request for values to be unmasked.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Payload parameters

Attribute	Description	Required
column	string The column to unmask.	Yes
unmaskingReason	string The reason the values need to be unmasked.	Yes
unmaskingUsers	array[integer] The profile ID of the users who can unmask the values for the requestor.	Yes
projectId	integer The ID of the associated project.	No
dataSourceId	integer The data source ID.	No

Response schema

Attribute	Description
id	integer The ID of the request.
requestingUserProfile	integer The requesting user profile ID.
dataSourceId	integer The data source ID.
reason	string The reason for the unmasking request.
metadata	string Metadata regarding the masking, such as the column, values, and maskingConfig.
type	string The type of request.
state	string The state of the task, such as pending.
createdAt	timestamp The date and time the task was created.
updatedAt	timestamp The date and time the task was updated.

Request example

The following requests for values to be unmasked.

curl \
   --request POST \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
   --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/reverseMask

Request payload example

{
  "column": "cc_county",
  "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
  "unmaskReasoning": "Marketing project",
  "unmaskingUsers": [1]
}

Response example

{
  "id": 1,
  "requestingUserProfile": 13,
  "dataSourceId": 12,
  "reason": "Marketing Campaign",
  "metadata": {
    "salt": "**********87",
    "column": "cc_county",
    "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
    "maskingConfig": {
      "type": "Reversible",
      "metadata": {}
    }
  },
  "type": "unmask",
  "state": "pending",
  "createdAt": "2021-10-27T20:35:48.253Z",
  "updatedAt": "2021-10-27T20:35:48.253Z"
}

Add a user to a data source

POST /dataSource/{dataSourceId}/access

Add a user to a specific data source. Requestors cannot add themselves to a data source. To request access to a data source, use the /dataSource/subscribe endpoint.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Payload parameters

Attribute	Description	Required
state	string The status of the user: subscribed, owner, expert, or ingest.	Yes
profileId	integer The profile ID of the user being added to the data source.	Yes
groupId	integer The ID of the group being added to the data source.	No
approvals	array Details about the user approving access: requiredPermission, specificApproverRequired, and specificApprover.	No
expiration	date The date the user's data source subscription ends.	No

Response schema

Attribute	Description
id	integer The user's subscription ID.
modelId	integer The model ID.
modelType	string The Immuta component to add the user to: datasource or project.
state	string The user's data source role, such as subscribed.
denialReasoning	string If the user was denied access, the reason for denial.
profile	integer The user's profile ID.
group	integer If a group was added, the group ID.
expiration	date The date the user's subscription to the data source will expire.
acknowledgeRequired	boolean If the data source is associated with a project, this value will be true if the user needs to confirm they have read the project acknowledgment.
createdAt	timestamp The date and time of creation.
updatedAt	timestamp The date and time of update.
approved	boolean When true, the user's request has been approved.

Request example

The following request adds a user (saved in example-payload.json) to this data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/22/access

Request payload example

{
  "profileId": 3,
  "state": "subscribed"
}

Response example

{
  "id": 19,
  "modelId": "1",
  "modelType": "datasource",
  "state": "subscribed",
  "metadata": null,
  "admin": 2,
  "denialReasoning": null,
  "profile": 3,
  "group": null,
  "expiration": null,
  "acknowledgeRequired": false,
  "createdAt": "2021-09-21T14:24:12.528Z",
  "updatedAt": "2021-09-21T14:24:12.528Z",
  "approved": true
}

Manage data source requests

Method	Path	Purpose
GET	/dataSource/tasks/pending	Get all pending tasks for this user and pending tasks this user has created.
GET	/dataSource/tasks/{taskId}	Handles the given task and marks it as complete.
GET	/dataSource/{dataSourceId}/tasks	Returns all tasks the user has made, can approve or deny, or validate for this data source.
PUT	/dataSource/{dataSourceId}/access/{id}	Change user status for a specific data source.

Get pending tasks by user

GET /dataSource/tasks/pending

Get all pending tasks for this user and pending tasks this user has created.

Query parameters

Attribute	Description	Required
searchText	string If specified, will filter results using the specified string.	No
searchModel	string Will filter the results by model type: dataSource or schemaEvolution.	No
offset	integer The number of results to skip (for paging).	No
size	integer The number of results to return per page.	No
schemaEvolutionConnectionString	string The schema evolution connection string to filter by.	No
countBySchemaEvolution	boolean Iftrue, will only return the number of tasks, grouped by schema evolution.	No
countByDataSource	boolean Iftrue, will only return the number of tasks, grouped by data source.	No
countOnly	boolean When true, will only return a count of the pending tasks.	No
groupByDataSource	boolean If true, will return the results as an array of { dataSourceId: , rows: }.	No
types	Array[string] Filters the results by the type of task: unmask, dataSourceCreated, columnAdded, columnDeleted, or columnTypeChanged. The dataSourceCreated, columnAdded, columnDeleted, and columnTypeChanged tasks are only created if a policy that contains the New tag is active and schema monitoring or object sync identifies a change in the remote platform.	No

Response schema

Attribute	Description
outgoing	array Includes details of the tasks or requests created by the user, such as the count, type, and targetEmails.
incoming	array Includes details about the tasks received by the user, such as the count, type, and targetEmails.

Request example

The following request gets all pending tasks for a user and pending tasks the user has created.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/pending

Response example

{
  "outgoing": [],
  "incoming": [{
    "fullCount": 1,
    "dataSource": {
      "id": 11,
      "name": "Public Customer"
    },
    "isOutgoing": false,
    "rows": [{
      "id": 1,
      "state": "pending",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
      "targetNames": [
        "Katie"
      ],
      "targetEmails": [
        "[e-mail]"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1,
        "email": null
      },
      "dataSource": {
        "id": 11,
        "name": "Public Customer"
      },
      "createdAt": "2024-08-06T14:37:09.279+00:00",
      "metadata": {
        "colName": "org",
        "colType": "VARCHAR(16777216)"
      },
      "isOutgoing": false
    }]
  }]
}

Mark tasks as complete

GET /dataSource/tasks/{taskId}

Handles the given task and marks it as complete.

Query parameters

Attribute	Description	Required
taskId	integer The task ID.	Yes

Response schema

Attribute	Description
result	array Includes details about the task.

Request example

The following request handles the given task and marks it as complete.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/7

Response example

{
  "result": {
    "unmaskPairs": [
      {
        "masked": "TnpjNFpXSXdOR1pqWkdNeE5EYzJPQT09OktaWjJEVldXZVluTmQ2SUVQdW1MajJtVTdqL2ZBT1JlaGFUUHJidmhkWmM9",
        "unmasked": "[e-mail]"
      }
    ],
    "column": "email"
  }
}

Return tasks for a data source

GET /dataSource/{dataSourceId}/tasks

Returns all tasks the user has made, can approve or deny, or validate for this data source.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
states	Array[string] The state of the tasks: pending or completed.	No
targetProfileId	integer Only returns tasks where the target user has this profile ID.	No
requestingUserProfileId	integer Only returns tasks where the requesting user has this profile ID.	No
profileId	integer Returns tasks where either the target or requesting user has this profile ID.	No
searchText	string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.	No
searchModel	string A string used to determine how results should be filtered using searchText.	No
types	Array[string] The type of task: unmask, dataSourceCreated, columnAdded, columnDeleted, or columnTypeChanged. The dataSourceCreated, columnAdded, columnDeleted, and columnTypeChanged tasks are only created if a policy that contains the New tag is active and schema monitoring or a object sync identifies a change in the remote platform.	No
size	integer The number of results to return.	No
offset	integer The number of results to skip (for paging).	No
sortField	string The field by which to sort the result set.	No
sortOrder	string The order in which to sort the results. The default is desc.	No
countOnly	boolean If true, will only return the number of tasks.	No

Response schema

Attribute	Description
hits	array Includes details about each task, such as the id, state, type, and requestor.
count	integer The total number of tasks.

Request example

The following request returns all tasks the user has made, can approve or deny, or validate for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/tasks?sortOrder=desc

Response example

{
  "hits": [
    {
      "fullCount": 2,
      "id": 6,
      "state": "completed",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
      "targetNames": [
        "John"
      ],
      "targetEmails": [
        "[e-mail]"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1,
      },
      "dataSource": {
        "id": 11,
        "name": "Public Customer"
      },
      "createdAt": "2021-10-12T15:48:23.095Z"
    },
    {
      "fullCount": 2,
      "id": 7,
      "state": "completed",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Fake Medical Claims 2017\"",
      "targetNames": [
        "John"
      ],
      "targetEmails": [
        "[e-mail]"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1
      },
      "dataSource": {
        "id": 3,
        "name": "Public Fake Medical Claims 2017"
      },
      "createdAt": "2021-10-12T18:56:22.954Z"
    }
  ],
  "count": 2
}

Change user status

PUT /dataSource/{dataSourceId}/access/{subscriptionId}

Change user status for a specific data source. Requestors cannot update their own status for a data source.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	Integer The data source ID.	Yes
subscriptionId	Integer The data source member's subscription ID.	Yes

Payload parameters

Attribute	Description	Required
state	string The new status for the user: subscribed, owner, expert or ingest.	Yes

Response schema

Attribute	Description
id	integer The data source member's subscription ID.
modelId	integer The model ID.
modelType	array The Immuta component the user's subscription is updated for: datasource or project.
state	array The current state of the user's role: subscribed, owner, expert, or ingest.
profile	integer The profile ID.
group	integer If a group's status is being updated, this is the group ID.
expiration	timestamp The date the user will no longer have access to the data source.
acknowledgeRequired	boolean This attribute is specific to projects. When true the user needs to confirm they have read the project acknowledgement statement.
createdAt	timestamp The date and time created.
updatedAt	timestamp The date and time updated.
originalState	array The user's previous status for the data source.
approved	boolean If true, the status is approved.

Request example

The following request changes the user status to subscribed for the specified data source.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/access/95

Payload example

{
  "state": "subscribed"
}

Response example

{
  "id": 95,
  "modelId": "3",
  "modelType": "datasource",
  "state": "subscribed",
  "metadata": {},
  "admin": 2,
  "profile": 3,
  "group": null,
  "expiration": null,
  "acknowledgeRequired": false,
  "createdAt": "2021-10-12T15:40:13.878Z",
  "updatedAt": "2021-10-12T16:10:46.801Z",
  "originalState": "expert",
  "approved": true
}

Update data sources

Method	Path	Purpose
PUT	/dataSource/{dataSourceId}	Update a data source.
PUT	/dataSource/bulk/{type}	Update multiple data sources.
POST	/dataSource/bulkRefreshViews	Refresh views.
POST	/dataSource/{dataSourceId}/blobs	Save blob metadata to Immuta.
POST	/dataSource/{dataSourceId}/persistBlob	Save blob metadata to Immuta and store raw content in local blob store.
PUT	/dataSource/detectRemoteChanges	Trigger the schema monitoring job for the specified detection group, or all groups if no ID is given.

Update a data source

PUT /dataSource/{dataSourceId}

Update a data source.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Payload parameters

Attribute	Description	Required
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.	No
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).	No
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.	No
type	string The type of data source: queryable (metadata is dynamically queried).	No
name	string The name of the data source. It must be unique within the Immuta tenant.	No
sqlTableName	string A string that represents this data source's table in Immuta.	No
organization	string The organization that owns the data source.	No
category	string The category of the data source.	No
description	string The description of the data source.	No
hasExamples	boolean When true, the data source contains examples.	No

Response schema

Attribute	Description
private	boolean When false, the data source will be publicly available in the Immuta UI.
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.
type	string The type of data source: queryable (metadata is dynamically queried).
name	string The name of the data source. It must be unique within the Immuta tenant.
sqlTableName	string A string that represents this data source's table in Immuta.
organization	string The organization that owns the data source.
description	string The description of the data source.
policyHandler	array The ID of the policy handler and details about the data policies enforced on the data source.
subscriptionPolicy	array Details about the subscription policy enforced on the data source, including the type of policy and exceptions.

Request example

The following request updates the data source's documentation (saved in example-payload.json).

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/22

Request payload example

{
  "documentation": "# Public Credit Accounts\nThis request updates the data source documentation."
}

Response example

{
  "name": "Public Credit Accounts",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "handlerId": 21,
    "visibilitySchema": {
      "fields": [],
      "version": "2021-09-15T17:44:20.678Z"
    },
    "previousHandlerId": 20,
    "dataSourceId": 1
  },
  "sqlSchemaName": "public",
  "sqlTableName": "credit_accounts",
  "blobHandler": {
    "url": "https://1.1.1.1:1111/snowflake/handler/1",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": "100000",
  "documentation": "# Public Credit Accounts\nThis request updates the data source documentation.",
  "statsExpiration": "2021-09-22T13:51:46.646Z",
  "id": 1,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "approval",
  "subscriptionPolicy": {
    "type": "subscription",
    "approvals": [{
      "requiredPermission": "OWNER",
      "specificApproverRequired": false
    }]
  },
  "globalPolicies": null,
  "status": "failed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-09-21T13:51:46.674Z"
    },
    "fingerprint": {
      "status": "passed",
      "lastAttempted": "2021-09-09T14:12:25.177Z"
    },
    "lastAttempt": {
      "date": "2021-09-20T19:35:21.929Z"
    },
    "globalPolicy": {
      "status": "passed",
      "lastAttempted": "2021-09-17T19:07:38.092Z"
    },
    "highCardinality": {
      "status": "failed",
      "message": "Error could not connect to remote database",
      "lastAttempted": "2021-09-20T16:43:19.426Z"
    }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 1,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-09-09T14:12:09.511Z",
  "updatedAt": "2021-09-21T13:52:42.908Z"
}

Update multiple data sources

PUT /dataSource/bulk/{type}

Update data sources.

Required: The global GOVERNANCE permission or be the data source owner for all data sources included in the payload

Query parameters

Attribute	Description	Required
type	string The action to perform on the data sources: add-users, disable, restore, delete, or tags.	Yes

Payload parameters

Attribute	Description	Required
ids	array[integer] The IDs of the data sources to update.	Yes
update	array[object] Only required for add-users (includes metadata about the users' profiles: id and state) and tags (includes metadata about the tags: name and source) types.	No

Response schema

Attribute	Description
bulkId	string The ID of the bulk data source update.
jobsCreated	integer The number of jobs created.

Request example

The following request adds the Address.email tag to two data sources.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/bulk/tags

Payload example

{
  "ids": [49, 50],
  "update": {
    "tags": [{
      "name": "Address.email",
      "source": "curated"
    }]
  }
}

Response example

{
  "bulkId": "bulk_ds_update_4896d300e04c4a8f89493ebf125c5c6b",
  "jobsCreated": 2
}

Refresh views

POST /dataSource/bulkRefreshViews

Refresh views.

Required: The global GOVERNANCE permission or be the data source owner for all data sources included in the payload

Payload parameters

Attribute	Description	Required
dataSourceIds	array[integer] The IDs of the data sources of the views to update.	Yes

Request example

The following request with the payload below refreshes the view for the data source with the ID 202.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer <TOKEN>" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/bulkRefreshViews

Payload example

{
  "dataSourceIds": [202]
}

Trigger schema monitoring jobs

PUT /dataSource/detectRemoteChanges

Trigger the schema monitoring job for the specified detection group, or all groups if no payload parameters are given.

Required: The global GOVERNANCE permission or be the data source owner for all data sources included in the payload

Payload parameters

Attribute	Description	Required
dataSourceIds	array[integer] The data source IDs to run the column detection job on. Leave empty to run this job globally on all data sources. This parameter cannot be included in the payload if schemaEvolutionId or any combination of hostname, database, port, or table is included.	No
hostname	string The hostname of the data sources. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.	No
port	integer The port used to connect the data sources to Immuta. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.	No
database	string The database name. This runs schema monitoring on the database provided. If data sources were initially registered via the V2 API, including this parameter will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, including this parameter will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.	No
table	string The table name. This will run column detection to just update the columns in this table. This parameter cannot be included in the payload if dataSourceIds or schemaEvolutionId is included.	No
schemaEvolutionId	integer The ID of the schema to run the schema monitoring job on. This will run on all tables associated with the specified ID. The schema ID can be found in the response body of /dataSource/{dataSourceId}. This parameter cannot be included in the payload if dataSourceIds or any combination of hostname, database, port, or table is included.	No
skipColumnDetection	boolean When true, Immuta will only pull new tables from the source server. This parameter can only be paired with schemaEvolutionId.	No
overrides.httpPath	string If Databricks ephemeral overrides are configured, provide the alternative HTTP path to trigger schema monitoring on that ephemeral cluster.	No

Response schema

Attribute	Description
schemaDetection	object Includes details about the resulting schema detection jobs.
columnDetection	object Includes details about the resulting column detection jobs.
bulkId	string The unique identifier of the jobs running schema monitoring and column detection.

Responses may include bulkId, schemaDetection, or columnDetection objects, depending on the payload.

Request example

The following request triggers the schema monitoring job for the specified detection group.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/detectRemoteChanges

Payload example

The tabs below illustrate payloads for triggering schema monitoring on a host, database, or table. The request will run schema monitoring for all databases registered under the hostname provided in the payload.

Host

The request will run schema monitoring for all databases registered under the hostname provided in the payload.

{
  "hostname": "organization.us-east-1.snowflakecomputing.com"
}

Database

The request will run schema monitoring on the database provided in the payload. If data sources were initially registered via the V2 API, this request will locate new schemas that contain tables Immuta has the ability to access, and Immuta will create a new schema project associated with these newly discovered schemas and create data sources for each table located. If data sources were initially registered via the V1 API, this request will only update the columns and tables of registered schema and tables of the specified database; it will not register any new schemas.

{
  "database": "public"
}

Table

The request will run column detection and update the columns on the table specified in the payload.

{
  "database": "public",
  "table": "healthcare"
}

Response examples

The tabs below illustrate the example response for each example payload provided above.

Host

{
  "bulkId": "31ab4312-b90a-49a6-baf8-70f87cd92a89"
}

Database

{
  "bulkId": "5d129011-6254-413d-a365-6e394c06e277"
}

Table

{
  "bulkId": "5d129011-6254-413d-a365-6e394c06e277",
  "schemaDetection": {
    "jobs": ["7d129033-6254-413d-a345-9o3242c06f242"]
  },
  "columnDetection" : {
    "jobs": ["8d129033-6254-413d-a345-9o3123c06f123"]
  }
}

View and Review Data Sources

Method	Path	Purpose
GET	/dataSource/{dataSourceId}/test	Refresh tags from an external catalog on a data source. The external catalog must be linked to the data source.
GET	/dataSource/blobHandlerTypes	Retrieve all blob handlers the current user is allowed to create.
GET	/dataSource/byPurposes	Get data sources that match a set of purposes.
GET	/dataSource/rpc/mine	Retrieve all the data sources the current user has access to.
GET	/dataSource/{dataSourceId}/activities	Get all of the recent policy activities for a given data source.
GET	/dataSource/{dataSourceId}/contacts	Get the profiles for the data source owners and experts.
GET	/dataSource/{dataSourceId}/tags	Get the tags for a data source.
GET	/dataSource/{dataSourceId}/{columnName}/unmaskUsers	Return the users who can unmask the given column.

Refresh data source health checks

GET /dataSource/{dataSourceId}/test

Refreshes the data source health checks.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
blob	object Indicates whether or not the blob was successfully crawled.
columnEvolution	object Indicates whether or not the job run to check for columns added or removed from the data source passed and when it was last run.
externalCatalog	object Indicates whether or not the external catalog was successfully linked to the data source.
fingerprint	object Indicates whether or not the fingerprint job was successful (passed) and when it was last run. The fingerprint captures summary statistics of a Snowflake data source.
framework	object Indicates whether or not the classification was successfully run on the data source to determine its sensitivity.
globalPolicy	object Indicates whether or not global policies were successfully applied to the data source.
highCardinality	object Indicates whether or not the job run to calculate the data source's high cardinality column passed and when it was last run.
schemaEvolution	object Indicates whether or not the job run to check if a new table had been added in the remote database passed and when it was last run. If a new table was added, Immuta automatically creates a new data source. Correspondingly, if a remote table is removed, that data source will be disabled in the console.
sdd	object Indicates whether or not sensitive data discovery was successfully run on the data source.
sql	object Indicates whether or not the SQL query run to check the data source's health passed and when it was last run.
stats	object Indicates whether or not the job run to calculate the number of rows in the data source passed and when it was last run.

Request example

The following request refreshes external catalog tags on the data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/test

Response example

{
  "sql": {
      "status": "passed",
      "message": "Passed"
  },
  "stats": {
      "status": "passed",
      "lastAttempted": "2021-09-09T13:43:43.948Z"
  },
  "fingerprint": {
      "status": "passed",
      "lastAttempted": "2021-07-22T14:12:01.525Z"
  },
  "lastAttempt": {
      "date": "2021-09-09T16:47:05.173Z"
  },
  "columnEvolution": {
      "status": "passed",
      "lastAttempted": "2021-09-08T16:36:05.557Z"
  },
  "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-07-22T14:11:58.439Z"
  },
  "schemaEvolution": {
      "status": "passed",
      "lastAttempted": "2021-09-08T16:35:57.867Z"
  },
  "status": "passed"
}

Retrieve blob handlers

GET /dataSource/blobHandlerTypes

Retrieve all blob handlers the current user is allowed to create.

Response schema

Attribute	Description
name	string The name of the blob handler.
baseUrl	string The base URL for the data source.
config	array Includes information about the connection configuration.
port	integer The port number.
driver	string The name of the driver.

Request example

The following request retrieves all blob handlers the current user is allowed to create.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/blobHandlerTypes

Response example

{
  "name": "Azure Synapse Analytics",
  "pluginType": "queryable",
  "clientUrl": "/asa",
  "baseUrl": "https://0.0.0.0:8823/asa",
  "config": {
  "port": 1433,
  "allowSSLToggle": false
  },
  "displayOrder": 41,
  "requiresHashPhraseForDP": true,
  "driver": "Azure Synapse Analytics"
}

Get data sources by purpose

GET /dataSource/byPurposes

Get data sources that match a set of purposes.

Query parameters

Attribute	Description	Required
purposes	array[string] The purposes to filter the data sources by.	Yes
excludedProjects	array[integer] Excludes data sources associated with specified project IDs.	No

Response schema

Attribute	Description
id	integer The data source ID.
name	array The name of the data source.
policyId	integer The policy ID.
restrictions	array Details regarding the operator (and or or) and purposes.

Request example

The following request gets data sources that match a set of purposes.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/byPurposes?purposes=Data+Analysis

Response example

[{
  "id": 42,
  "name": "Army Army Records",
  "policyId": 56,
  "restrictions": [{
    "operator": "and",
    "purposes": ["Data Analysis"]
  }]
}]

Retrieve data sources by user

GET /dataSource/rpc/mine

Retrieves all the data sources the current user has access to.

Response schema

Attribute	Description
id	integer The data source ID.
name	string The data source name.
type	string The type of data source: queryable.
sqlTableName	string The name of the table in Immuta.
sqlSchemaName	string The name of the schema in Immuta.
blobHandlerType	string The type of handler, such as Snowflake.
sparkUseJDBC	boolean When true, uses a JDBC driver.

Request example

The following request retrieves all the data sources the current user has access to.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/rpc/mine

Response example

{
  "id": 23,
  "name": "Public Record",
  "type": "queryable",
  "sqlTableName": "record",
  "sqlSchemaName": "public",
  "blobHandlerType": "Snowflake",
  "sparkUseJDBC": true
}

Get recent policy activities for a data source

GET /dataSource/{dataSourceId}/activities

Get all of the recent policy activities for a given data source.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
offset	integer The number of results to skip (for paging).	No
size	integer The number of results to return per page.	No

Response schema

Attribute	Description
count	integer The number of results.
activities	array Includes details about the policy and the data source, including the policy and data source type, when the activity notification was triggered, and whether or not the policy change was triggered by a Global policy.
actionBy	array Details about who triggered the action.
targetUser	array Information about the user who received the notification.

Request example

The following request gets all of the recent policy activities for a given data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/activities

Response example

{
  "count": 3,
  "activities": [{
    "modelType": "datasource",
    "modelId": "23",
    "createdAt": "2021-09-07T16:13:25.197Z",
    "notificationType": "policyUpdated",
    "metadata": {
      "dataSourceName": "Public Foobar",
      "triggeredByGlobal": false,
      "conflictCount": 0,
      "policyType": "data",
      "handlerId": 3,
      "previousHandlerId": 2,
      "dataSourceType": "queryable"
    },
    "read": false,
    "id": 256,
    "updatedAt": "2021-09-07T16:13:25.197Z",
    "actionBy": {
      "id": 2,
      "name": "first last",
      "email": "[e-mail]"
    },
    "targetUser": {
      "id": 2,
      "name": "first last",
      "email": "[e-mail]"
    }
  }]
}

Get profiles for data source owners and experts

GET /dataSource/{dataSourceId}/contacts

Gets the profiles for the data source owners and experts.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
id	integer The data source ID.
state	string The user's data source role, such as owner or subscribed.
name	string The user's name.
email	string The user's email.
profile	integer The user's profile ID.

Request example

The following request gets all the profiles for the data source owners and experts.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/contacts

Response example

{
  "type": "profile",
  "id": 23,
  "state": "owner",
  "name": "first last",
  "email": "[e-mail]",
  "profile": 2
},
{
  "type": "profile",
  "id": 23,
  "state": "owner",
  "name": "Tommy Test",
  "email": "[e-mail]",
  "profile": 3
}

Get tags by data source

GET /dataSource/{dataSourceId}/tags

Get the tags for a data source.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
blobId	string Returns the tags for the specified blob.	No
blobTagsOnly	boolean When true, will only display blob tags associated with a data source.	No

Response schema

Attribute	Description
tags	array Includes details about the tags, such as the name, source, and the profile ID of the user who added the tag.

Request example

The following request gets the tags for data source 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/tags

Response example

{
  "tags": [
    {
      "name": "Discovered.Entity.Medicare Number",
      "source": "curated",
      "modelType": "datasource",
      "modelId": "4",
      "addedBy": 1,
      "deleted": false
    }
  ]
}

Get users who can unmask columns

GET /dataSource/{dataSourceId}/{columnName}/unmaskUsers

Return the users who can unmask the given column.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes
columnName	string The name of the column to unmask.	Yes

Response schema

Attribute	Description
name	array The name of the user who can unmask the value.
profileId	integer The profile ID of the user who can unmask the value.
iamid	string The IAM ID of the user who can unmask the value.

Request example

The following request returns the users who can unmask the given column.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/47/gender/unmaskUsers

Response example

[
  {
    "name": "Katie",
    "profileId": 2,
    "iamid": "bim"
  }
]

Delete Data Sources and More

Method	Path	Purpose
DELETE	/dataSource/{dataSourceId}	Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.
DELETE	/dataSource/tasks/{taskId}	Delete the specified task.
DELETE	/dataSource/{dataSourceId}/blob/{blobId*}	Delete a blob.
DELETE	/dataSource/{dataSourceId}/unsubscribe	Unsubscribe from a data source.

Delete a data source

DELETE /dataSource/{dataSourceId}

Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
success	boolean If true, the request to disable or delete the data source was successful.
id	integer The data source ID.
schemaEvolutionId	integer The schema evolution ID.
name	string The data source name.
disabled	boolean If true, the data source is disabled.
handlerDeleteErrorMessage	string The delete error message.

Request example

The following request deletes the data source 23.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23

Response example

{
  "success": true,
  "id": 23,
  "schemaEvolutionId": 1,
  "name": "Public Foobar",
  "disabled": true,
  "handlerDeleteErrorMessage": null
}

Delete a task

DELETE /dataSource/tasks/{taskId}

Delete the specified task.

Query parameters

Attribute	Description	Required
taskId	integer Target task ID.	Yes

Response schema

Attribute	Description
id	integer The deleted task ID.
state	array The state of the deleted task, such as pending.
type	array The type of deleted task, such as columnAdded.
targetNames	string The name of the user who received the request.
targetEmails	string The email of the user who received the request.
requester	metadata Details regarding the requesting profile.
dataSource	metadata details regarding the data source.
metadata	array Details about the deleted task.

Request example

The following request deletes a specified task.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/8

Response example

{
  "fullCount": 1,
  "id": 8,
  "state": "pending",
  "type": "columnAdded",
  "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
  "targetNames": ["Katie"],
  "targetEmails": ["[e-mail]"],
  "requester": {
    "name": "Immuta System Account",
    "id": 1
  },
  "dataSource": {
    "id": 11,
    "name": "Public Customer"
  },
  "createdAt": "2021-10-12T19:28:04.999Z"
}

Unsubscribe from a data source

DELETE /dataSource/{dataSourceId}/unsubscribe

Unsubscribe from a data source.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
status	boolean If true, the requesting user is unsubscribed from the data source.

Request example

The following request unsubscribes the user from data source 23.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/unsubscribe

Response example

{
  "success": true
}