Subscribe to and Manage Data Sources

Data Source API reference guide

PreviousSearch Schemas NextManage Projects and Purposes

Last updated 1 month ago

Was this helpful?

This page describes the dataSource endpoint, through which users can subscribe to data sources and manage data source tasks. To create data sources, see the specific handler endpoints.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Data source workflow

Search data sources and data source details

Method

Path

Purpose

GET

Search for data sources.

GET

Get a data source based on the ID.

GET

Get data source based on the name.

GET

Get data source based on the short name.

GET

Get parent and child relationship records for derived data sources using a specified data source ID.

GET

Retrieve a blob.

GET

Get all users with the provided access level for this data source.

GET

Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.

GET

Retrieves a summary of total records, total visibilities, and visibilities a given user has access to.

GET

Retrieves a summary of total records, total visibilities, and visibilities the current user has access to for a specified data source.

Search for data sources

GET /dataSource

Search for data sources that you own or are authorized to see.

Required: To appear in the response, you must own the data source, the data source must be publicly visible, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

blobHandlerType

array[string] Describes the type of underlying blob handler that will be used with this data source (e.g., Custom, MS SQL).

subscription

array[string] The requesting user's subscription status: pending, owner, subscribed, not_subscribed, expert, or ingest.

status

array[string] The data source status: passed or failed.

tag

array[string] Filters data sources by tags associated with the data sources.

searchText

string Searches for data source names using the provided string.

column

array[string] Searches for data source column names.

connectionString

array[string] Searches by connection string.

schema

string Searches for data source schema.

nameOnly

boolean When true, searchText will only search data source names. Default is false.

idOnly

boolean When true, only returns the ID Of the data source and the user's subscription status.

dataSourceIds

array[integer] Searches for the provided data source IDs.

selectFields

array[string] This field accepts the values id, name, and columnEvolutionEnabled. When id or name are provided, the request will return only the ID or name of the data source and the subscription status. If columnEvolutionEnabled is provided, the response will also include information about the policies, policy conflicts, and workspaces associated with the data sources.

offset

integer Used in combination with size to fetch pages. Default is 0.

size

integer The number of results to return per page. Default is 10.

sortField

string Used to sort results by field, which must be createdAt, name, blobHandlerType, subscriptionStatus, recordCount, status, policy, or editable.

sortOrder

string Sorts results by order, which must be asc or desc.

excludedProjects

array[integer] Filter out any data sources that belong to the specified projects.

ephemeral

boolean When true, returns ephemeral data sources.

clusterName

string The name of the remote cluster the data source is connected to.

mode

integer Specifies the query mode, which must be 0 (FULL), 1 (COUNT), 4 (TAG), 5 (MIN_MAX), or 6 (STATUS).

globalPolicy

string Filter by data sources that have this Global Policy applied.

hostname

string Searches data sources by hostname.

determinePolicyContflicts

boolean When true, filters results to return the data sources with policy conflicts.

collectionId

string Filter by data sources in the domain with this ID.

collectionExclude

string When true, filters results to return the data sources in the domain of the provided ID.

sddTemplateName

string Filter by data sources that have the specified SDD identification framework applied.

excludeSddTemplateName

string Filter by data sources that do not have the specified SDD identification framework applied.

Response schema

The response will return data sources you own or are to see.

Attribute

Description

name

string Data source name.

integer Data source ID.

deleted

boolean If true the data source is a deleted data source.

description

string The data source description.

createdAt

timestamp The date and time the data source was created.

subscriptionPolicy

array Details the type of Subscription Policy applied to the data source.

schemaEvolutionId

integer The schema evolution ID.

recordCount

integer The record count.

status

array[string] Accepted statuses are passed or failed.

subscriptionStatus

array[string] Accepted statuses are subscribed or unsubscribed.

blobHandlerType

array[string] Describes the type of underlying blob handler of this data source (e.g., Custom, MS SQL).

subscriptionType

string The type of subscription policy on the project. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).

connectionString

string The connection string information.

sqlSchemaName

string The schema name.

policy

string When this value is none, there are no data policies applied to the data source. Otherwise, this field indicates whether or not there are policy conflicts among the data policies applied to the data source.

policyHandlerType

string The policy handler type, such as None or Builder.

Request example

The following request returns 2 data sources.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource?size=2

Response example

{
  "name": "Public Barfoo",
  "id": 22,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2021-07-22T14:11:55.539Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "status": "passed",
  "subscriptionStatus": "subscribed",
  "blobHandlerType": "Snowflake",
  "subscriptionType": "automatic",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "sqlSchemaName": "public",
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
},
{
  "name": "Public Aaa Tpc",
  "id": 39,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2023-08-21T10:39:00.250Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "exceptions": {
          "operator": "and",
          "conditions": [
              {
                  "type": "groups",
                  "group": {
                      "name": "alpha"
                  }
              }
          ]
      },
      "allowDiscovery": true,
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "blobHandlerType": "Snowflake",
  "subscriptionType": "policy",
  "sqlSchemaName": "public",
  "status": "passed",
  "subscriptionStatus": "owner",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "remoteTable": "tpc",
  "remoteSchema": "public",
  "domainId": null,
  "domainName": null,
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
}

Get a data source by ID

GET /dataSource/{dataSourceId}

Get a data source based on the ID.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID .

Yes

Response Schema

Attribute

Description

name

string The data source name.

recordFormat

string The data format of blobs in the data source, such as json, xml, html, or jpeg.

description

string The description of the data source.

policyHandler

array The ID of the policy handler and details about the data policies enforced on the data source.

sqlSchemaName

string A string that represents this data source's schema name in Immuta.

sqlTableName

string The SQL table name in Immuta.

blobHandler

array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.

blobHandlerType

string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).

createdBy

integer The ID of the profile creating the data source.

deleted

boolean If true, the data source was deleted.

type

string The data source type: queryable.

rowCount

integer The number of rows.

documentation

string Documentation associated with the data source.

integer The data source ID.

policyHandlerType

string The type of policy handler applied to the data source: Builder.

subscriptionType

string The type of subscription policy on the data source. The type can be automatic (which allows anyone to subscribe), approval (which requires the subscriber to be manually approved), policy (which only allows users with specific groups or attributes to subscribe), or manual (which requires users to be manually added).

subscriptionPolicy

array Details about the Subscription Policy applied to the data source.

globalPolicies

string Details about the Global Policies applied to the data source.

status

string The data source health status.

Request example

The following request gets a data source based on the ID 22.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
  "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
  "sql": {
      "status": "passed",
      "message": "Passed"
   }
  }
}

Get data source by name

GET /dataSource/name/{dataSourceName}

Get a data source based on the name.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

dataSourceName

string The data source name.

Yes

Response schema

Attribute

Description

name

string The data source name.

recordFormat

string The data format of blobs in the data source, such as json, xml, html, or jpeg.

description

string The description of the data source.

policyHandler

array The ID of the policy handler and details about the data policies enforced on the data source.

sqlSchemaName

string A string that represents this data source's schema name in Immuta.

sqlTableName

string The SQL table name in Immuta.

blobHandler

array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.

blobHandlerType

string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).

createdBy

integer The ID of the profile creating the data source.

deleted

boolean If true, the data source was deleted.

type

string The data source type:queryable.

rowCount

integer The number of rows.

documentation

string Documentation associated with the data source.

integer The data source ID.

policyHandlerType

string The type of policy handler applied to the data source: Builder.

subscriptionType

subscriptionPolicy

array Details about the Subscription Policy applied to the data source.

globalPolicies

string Details about the Global Policies applied to the data source.

status

string The data source health status.

Request example

The following request gets a data source based on the name Public Barfoo.

curl \
   --request GET \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/name/Public%20Barfoo

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
      },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
      "sql": {
      "status": "passed",
      "message": "Passed"
      }
  }
}

Get a data source by the short name

GET /dataSource/sqlTableName/{shortName}

Get a data source based on the SQL table name.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

shortName

string The data source SQL table name.

Yes

Response schema

Attribute

Description

name

string The data source name.

recordFormat

string The data format of blobs in the data source, such as json, xml, html, or jpeg.

description

string The description of the data source.

policyHandler

array The ID of the policy handler and details about the data policies enforced on the data source.

sqlSchemaName

string A string that represents this data source's schema name in Immuta.

sqlTableName

string The SQL table name in Immuta.

blobHandler

array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.

blobHandlerType

string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).

createdBy

integer The ID of the profile creating the data source.

deleted

boolean If true, the data source was deleted.

type

string The data source type: queryable.

rowCount

integer The number of rows.

documentation

string Documentation associated with the data source.

integer The data source ID.

policyHandlerType

string The type of policy handler applied to the data source: Builder.

subscriptionType

subscriptionPolicy

array Details about the Subscription Policy applied to the data source.

globalPolicies

string Details about the Global Policies applied to the data source.

status

string The data source health status.

Request example

The following request gets a data source based on the SQL table name customer_data.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/sqlTableName/customer_data

Response example

    {
  "name": "Dbo Customer Data",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "visibilitySchema": {
      "fields": [],
      "version": "2021-10-06T19:39:39.145Z"
    },
    "handlerId": 55,
    "dataSourceId": 26
  },
  "sqlSchemaName": "dbo",
  "sqlTableName": "customer_data",
  "blobHandler": {
    "url": "https://1.1.1.1.1/snowflake/testhandler/26",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 1000,
  "documentation": "# Dbo Customer Data",
  "statsExpiration": "2021-11-05T19:37:43.270Z",
  "id": 26,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
    "type": "subscription",
    "automaticSubscription": false
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
    },
    "lastAttempt": {
      "date": "2021-10-06T19:39:39.821Z"
    },
    "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
      }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 4,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-10-01T14:23:27.225Z",
  "updatedAt": "2021-10-06T19:39:39.145Z",
  "subscribedAsUser": true,
  "subscriptionId": 45,
  "acknowledgeRequired": false,
  "subscriptionStatus": "owner",
  "requestedState": "owner",
  "approved": true,
  "subscriptionExpiration": null,
  "filterId": null
}

Get data source relationships

GET /dataSource/{dataSourceId}/lineage/{type}

Get parent and child relationship records for derived data sources using a specified data source ID.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

type

string The type of lineage records to return. Options include: parents, children, and all.

Yes

dataSourceId

integer The target data source ID.

Yes

Response schema

Attribute

Description

children

array Details of the child data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.

parents

array Details of the parent data source, including dataSourceId, dataSourceName, projectId, policyHandlerDiff, deleted, createdBy, and createdAt.

Request example

The following request gets the parent relationship records for the derived data source with the data source ID 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/lineage/parents

Response example

{
  "parents": [{
    "dataSourceId": 3,
    "dataSourceName": "Public Healthcare Data",
    "deleted": false,
    "createdAt": "2022-08-17T13:41:38.381Z",
    "projectId": 2,
    "projectName": "Derived Data Source",
    "createdBy": "Your Username",
    "policyHandlerDiff": {
      "dsType": "queryable",
      "currentHandlerId": null,
      "previousHandlerId": null
    }
  }]
}

Retrieve a blob

GET /dataSource/{dataSourceId}/blob/{blobid*}

Retrieve a blob.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

blobId

string The blob ID.

Yes

Response schema

The response will download the blobs in a file you specify.

Request example

The following request retrieves a blob.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --output ./the-blobs-will-be-saved-here
    https://your-immuta-url.com/dataSource/22/blob/22

Response example

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  215k  100  215k    0     0   541k      0 --:--:-- --:--:-- --:--:--  541k

Get users by access level

GET /dataSource/{dataSourceId}/access

Get all users with the provided access level for this data source.

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

states

Array[string] The status levels to include when querying for user access.

approved

boolean Denotes whether the returned access objects should be approved.

searchText

string A string used to filter returned users. The query is executed with a wildcard prefix and suffix.

size

integer The number of results to return.

offset

integer The number of results to skip (for paging).

sortField

string The field on which to sort the result set.

sortOrder

string The order in which to sort the results.

expandGroups

boolean If true will return individual members of any group subscribed.

ignoreSystemGenerated

boolean If true, will not return system generated accounts.

filterBySchemaEvolution

boolean If true, will only return users who have the specified level of access across ALL data sources within the same schema evolution group as this one.

Response schema

Attribute

Description

count

integer The number of users with access to the data source.

users

string The metadata regarding the users with access to the data source.

Request example

The following request gets all users with the provided access level for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22/access?sortOrder=desc

Response example

{
  "count": 2,
  "users": [
    {
      "profile": 2,
      "name": "First Last",
      "iamid": "bim",
      "userid": "first.last@immuta.com",
      "email": "first.last@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "owner",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-10-06T14:58:46.983Z",
      "subscriptionId": 586,
      "createdAt": "2021-10-05T14:33:01.518Z",
      "updatedAt": "2021-10-05T14:33:01.518Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    },
    {
      "profile": 3,
      "name": "Tommy Test",
      "iamid": "bim",
      "userid": "tommy.test@immuta.com",
      "email": "tommy.test@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "subscribed",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-09-07T16:16:29.957Z",
      "subscriptionId": 649,
      "createdAt": "2021-10-06T14:58:31.366Z",
      "updatedAt": "2021-10-06T14:58:31.366Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    }
  ]
}

Get user access info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/policyInfo

Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.

Required Immuta permission: FETCH_POLICY_INFO

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

profileId

integer The profile ID of the user.

Yes

projectId

integer The project ID. If provided, this project will be used when evaluating the user's visibilities.

Response schema

Attribute

Description

visibilities

array Details of the user's visibilities, including anyKey.

visibilityRuleApplies

boolean If true, a visibility rule exists and the user is not excepted from it.

masked

array Masking information for the data source, including metadata, name, type, and actionType.

additionalFilters

array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.

Request example

The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/policyInfo

Response example

{
  "visibilities": [],
  "visibilityRuleApplies": false,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK",
      "actionType": "Nullify"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK",
      "actionType": "Nullify"
    }
  ],
  "additionalFilters": {}
}

Get user visibility info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

profileId

integer The profile ID of the user.

Yes

informationOnly

boolean If true, the query will just return information for the UI and will skip running some queries for ephemeral data sources.

includeNestedColumns

boolean If true, the query will return just information for the dictionary page, including the masking policies for nested columns.

Response schema

Attribute

Description

noVisibilities

boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.

dataSourceVisibilitiesCount

integer The total number of possible visibilities the given data source has.

userVisibilitiesCount

integer The number of visibilities the current user can see for the given data source.

masked

array Masking information for the data source, including metadata, name, type, and actionType.

dataSource

integer The data source ID.

dataSourceName

string The data source name.

additionalFilters

array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.

allowMaskedJoins

boolean If true the data source allows masked joins.

policySet

array Details about the policies on the data source.

Request example

The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/visibilityReport

Response example

[
  {
    "noVisibilities": true,
    "dataSourceVisibilitiesCount": 0,
    "userVisibilitiesCount": 0,
    "masked": [
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_SOLD_DATE_SK"
      },
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_BILL_CUSTOMER_SK"
      }
    ],
    "dataSource": 16,
    "dataSourceName": "Web Sales",
    "additionalFilters": {},
    "allowMaskedJoins": false,
    "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
  }
]

Get current user visibility info

GET /dataSource/{dataSourceId}/visibilityReport

Required: The data source must be publicly visible, you must own the data source, or you must have access to see the data source through the GOVERNANCE permission or a subscription policy

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

Response schema

Attribute

Description

noVisibilities

boolean If true, the data source has no row-level security or purpose-based restriction policies applied to it.

dataSourceVisibilitiesCount

integer The total number of possible visibilities the given data source has.

userVisibilitiesCount

integer The number of visibilities the current user can see for the given data source.

denialReason

string Reason the user was denied visibility.

masked

array Masking information for the data source, including metadata, name, type, and actionType.

dataSource

integer The data source ID.

dataSourceName

string The data source name.

additionalFilters

array Policy information for the data source, including customWhere, differentialPrivacy, eventTimeColumn, minimization, time, filterSeconds, and isOlderOrNewer.

allowMaskedJoins

boolean If true the data source allows masked joins.

policySet

array Details about the policies on the data source.

Request example

The following request gets all of the visibility information for the current user on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/visibilityReport

Response example

{
  "noVisibilities": true,
  "dataSourceVisibilitiesCount": 0,
  "userVisibilitiesCount": 0,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK"
    }
  ],
  "dataSource": 16,
  "dataSourceName": "Web Sales",
  "additionalFilters": {},
  "allowMaskedJoins": false,
  "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}

Access data sources and make data source requests

Method

Path

Purpose

POST

Subscribe to a data source.

POST

Make a request for values to be unmasked.

POST

Add a user to a specific data source.

POST /dataSource/subscribe

Subscribe to a data source.

Query parameters

Attribute

Description

Required

dataSourceId

integer Data source ID number.

Yes

Payload parameters

Attribute

Description

Required

dataSourceIds

array The ID of the data source the user is subscribing to.

approvals

array Includes details about the Subscription policy on the data source: requiredPermissions, specificApproverRequired, specificApprover, and ownerModelId.

Response schema

Attribute

Description

body

array Contains details about the data source, including the data source ID, subscription status of the user, the profile ID of the user, and the dates the data source was created and updated.

Request example

The following request subscribes to the data source with ID 22.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/subscribe?dataSourceId=22

Payload example

{
  "dataSourceIds": [
    22
  ],
  "metadata": {},
  "approvals": [
    {
      "requiredPermission": "Owner",
      "specificApproverRequired": false,
      "specificApprover": 2,
      "ownerModelId": 23
    }
  ],
  "groupId": 12
}

Response example

{
  "inError": [],
  "success": [{
    "id": 64,
    "modelId": "22",
    "modelType": "datasource",
    "state": "subscribed",
    "metadata": {},
    "admin": null,
    "denialReasoning": null,
    "profile": 2,
    "group": null,
    "expiration": null,
    "acknowledgeRequired": false,
    "createdAt": "2021-08-26T16:36:09.587Z",
    "updatedAt": "2021-08-26T16:36:09.587Z",
    "approved": true
  }]
}

Request to unmask values

Deprecation notice

Support for unmask requests has been deprecated.

POST /dataSource/{dataSourceId}/reverseMask

Makes a request for values to be unmasked.

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

Payload parameters

Attribute

Description

Required

column

string The column to unmask.

Yes

unmaskingReason

string The reason the values need to be unmasked.

Yes

unmaskingUsers

array[integer] The profile ID of the users who can unmask the values for the requestor.

Yes

projectId

integer The ID of the associated project.

dataSourceId

integer The data source ID.

Response schema

Attribute

Description

integer The ID of the request.

requestingUserProfile

integer The requesting user profile ID.

dataSourceId

integer The data source ID.

reason

string The reason for the unmasking request.

metadata

string Metadata regarding the masking, such as the column, values, and maskingConfig.

type

string The type of request.

state

string The state of the task, such as pending.

createdAt

timestamp The date and time the task was created.

updatedAt

timestamp The date and time the task was updated.

Request example

The following requests for values to be unmasked.

curl \
   --request POST \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
   --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/reverseMask

Request payload example

{
  "column": "cc_county",
  "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
  "unmaskReasoning": "Marketing project",
  "unmaskingUsers": [1]
}

Response example

{
  "id": 1,
  "requestingUserProfile": 13,
  "dataSourceId": 12,
  "reason": "Marketing Campaign",
  "metadata": {
    "salt": "**********87",
    "column": "cc_county",
    "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
    "maskingConfig": {
      "type": "Reversible",
      "metadata": {}
    }
  },
  "type": "unmask",
  "state": "pending",
  "createdAt": "2021-10-27T20:35:48.253Z",
  "updatedAt": "2021-10-27T20:35:48.253Z"
}

Add a user to a data source

POST /dataSource/{dataSourceId}/access

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute

Description

Required

dataSourceId

integer The data source ID.

Yes

Payload parameters

Attribute

Description

Required

state

string The status of the user: subscribed, owner, expert, or ingest.

Yes

profileId

integer The profile ID of the user being added to the data source.

Yes

groupId

integer The ID of the group being added to the data source.

approvals

array Details about the user approving access: requiredPermission, specificApproverRequired, and specificApprover.

expiration

date The date the user's data source subscription ends.

Response schema

Attribute

Description

integer The user's subscription ID.

modelId

integer The model ID.

modelType

string The Immuta component to add the user to: datasource or project.

state

string The user's data source role, such as subscribed.

denialReasoning

string If the user was denied access, the reason for denial.

profile

integer The user's profile ID.

group

integer If a group was added, the group ID.

expiration

date The date the user's subscription to the data source will expire.

acknowledgeRequired

boolean If the data source is associated with a project, this value will be true if the user needs to confirm they have read the project acknowledgment.

createdAt

timestamp The date and time of creation.

updatedAt

timestamp The date and time of update.

approved

boolean When true, the user's request has been approved.

Request example

The following request adds a user (saved in example-payload.json) to this data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/22/access

Request payload example

{
  "profileId": 3,
  "state": "subscribed"
}

Response example

{
  "id": 19,
  "modelId": "1",
  "modelType": "datasource",
  "state": "subscribed",
  "metadata": null,
  "admin": 2,
  "denialReasoning": null,
  "profile": 3,
  "group": null,
  "expiration": null,
  "acknowledgeRequired": false,
  "createdAt": "2021-09-21T14:24:12.528Z",
  "updatedAt": "2021-09-21T14:24:12.528Z",
  "approved": true
}

Manage data source requests

Method

Path

Purpose

GET

Get all pending tasks for this user and pending tasks this user has created.

GET

Handles the given task and marks it as complete.

GET

Returns all tasks the user has made, can approve or deny, or validate for this data source.

PUT

Change user status for a specific data source.

Get pending tasks by user

GET /dataSource/tasks/pending

Get all pending tasks for this user and pending tasks this user has created.

Query parameters

Attribute

Description

Required

searchText

string If specified, will filter results using the specified string.

searchModel

string Will filter the results by model type: dataSource or schemaEvolution.

offset

integer The number of results to skip (for paging).

size

integer The number of results to return per page.

schemaEvolutionConnectionString

string The schema evolution connection string to filter by.

countBySchemaEvolution

boolean Iftrue, will only return the number of tasks, grouped by schema evolution.

countByDataSource

boolean Iftrue, will only return the number of tasks, grouped by data source.

countOnly

boolean When true, will only return a count of the pending tasks.

groupByDataSource

boolean If true, will return the results as an array of { dataSourceId: , rows: }.

types

Response schema

Attribute

Description

outgoing

array Includes details of the tasks or requests created by the user, such as the count, type, and targetEmails.

incoming