2024.3
Search powered by AI

Subscribe to and Manage Data Sources

Data Source API reference guide

This page describes the dataSource endpoint, through which users can subscribe to data sources and manage data source tasks. To create data sources, see the specific handler endpoints.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Data source workflow

  1. Search data sources and data source details.

  2. Access data sources and make data source requests.

  3. Manage data source requests.

  4. Update data sources.

  5. Delete a data source.

Search data sources and data source details

Search for data sources

GET /dataSource

Search for data sources.

Query parameters

Response schema

Request example

The following request returns 2 data sources.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource?size=2

Response example

{
  "name": "Public Barfoo",
  "id": 22,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2021-07-22T14:11:55.539Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "status": "passed",
  "subscriptionStatus": "subscribed",
  "blobHandlerType": "Snowflake",
  "subscriptionType": "automatic",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "sqlSchemaName": "public",
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
},
{
  "name": "Public Aaa Tpc",
  "id": 39,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2023-08-21T10:39:00.250Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "exceptions": {
          "operator": "and",
          "conditions": [
              {
                  "type": "groups",
                  "group": {
                      "name": "alpha"
                  }
              }
          ]
      },
      "allowDiscovery": true,
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "blobHandlerType": "Snowflake",
  "subscriptionType": "policy",
  "sqlSchemaName": "public",
  "status": "passed",
  "subscriptionStatus": "owner",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "remoteTable": "tpc",
  "remoteSchema": "public",
  "domainId": null,
  "domainName": null,
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
}

Get a data source by ID

GET /dataSource/{dataSourceId}

Get a data source based on the ID.

Query parameters

Response Schema

Request example

The following request gets a data source based on the ID 22.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
  "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
  "sql": {
      "status": "passed",
      "message": "Passed"
   }
  }
}

Get data source by name

GET /dataSource/name/{dataSourceName}

Get a data source based on the name.

Query parameters

Response schema

Request example

The following request gets a data source based on the name Public Barfoo.

curl \
   --request GET \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/name/Public%20Barfoo

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
      },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
      "sql": {
      "status": "passed",
      "message": "Passed"
      }
  }
}

Get a data source by the short name

GET /dataSource/sqlTableName/{shortName}

Get a data source based on the SQL table name.

Query parameters

Response schema

Request example

The following request gets a data source based on the SQL table name customer_data.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/sqlTableName/customer_data

Response example

    {
  "name": "Dbo Customer Data",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "visibilitySchema": {
      "fields": [],
      "version": "2021-10-06T19:39:39.145Z"
    },
    "handlerId": 55,
    "dataSourceId": 26
  },
  "sqlSchemaName": "dbo",
  "sqlTableName": "customer_data",
  "blobHandler": {
    "url": "https://1.1.1.1.1/snowflake/testhandler/26",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 1000,
  "documentation": "# Dbo Customer Data",
  "statsExpiration": "2021-11-05T19:37:43.270Z",
  "id": 26,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
    "type": "subscription",
    "automaticSubscription": false
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
    },
    "lastAttempt": {
      "date": "2021-10-06T19:39:39.821Z"
    },
    "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
      }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 4,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-10-01T14:23:27.225Z",
  "updatedAt": "2021-10-06T19:39:39.145Z",
  "subscribedAsUser": true,
  "subscriptionId": 45,
  "acknowledgeRequired": false,
  "subscriptionStatus": "owner",
  "requestedState": "owner",
  "approved": true,
  "subscriptionExpiration": null,
  "filterId": null
}

Get data source relationships

GET /dataSource/{dataSourceId}/lineage/{type}

Get parent and child relationship records for derived data sources using a specified data source ID.

Query parameters

Response schema

Request example

The following request gets the parent relationship records for the derived data source with the data source ID 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/lineage/parents

Response example

{
  "parents": [{
    "dataSourceId": 3,
    "dataSourceName": "Public Healthcare Data",
    "deleted": false,
    "createdAt": "2022-08-17T13:41:38.381Z",
    "projectId": 2,
    "projectName": "Derived Data Source",
    "createdBy": "Your Username",
    "policyHandlerDiff": {
      "dsType": "queryable",
      "currentHandlerId": null,
      "previousHandlerId": null
    }
  }]
}

Retrieve a Blob

GET /dataSource/{dataSourceId}/blob/{blobid*}

Retrieve a blob.

Query parameters

Response schema

The response will download the blobs in a file you specify.

Request example

The following request retrieves a blob.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --output ./the-blobs-will-be-saved-here
    https://your-immuta-url.com/dataSource/22/blob/22

Response example

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  215k  100  215k    0     0   541k      0 --:--:-- --:--:-- --:--:--  541k

Get users by access level

GET /dataSource/{dataSourceId}/access

Get all users with the provided access level for this data source.

Query parameters

Response schema

Request example

The following request gets all users with the provided access level for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22/access?sortOrder=desc

Response example

{
  "count": 2,
  "users": [
    {
      "profile": 2,
      "name": "First Last",
      "iamid": "bim",
      "userid": "first.last@immuta.com",
      "email": "first.last@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "owner",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-10-06T14:58:46.983Z",
      "subscriptionId": 586,
      "createdAt": "2021-10-05T14:33:01.518Z",
      "updatedAt": "2021-10-05T14:33:01.518Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    },
    {
      "profile": 3,
      "name": "Tommy Test",
      "iamid": "bim",
      "userid": "tommy.test@immuta.com",
      "email": "tommy.test@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "subscribed",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-09-07T16:16:29.957Z",
      "subscriptionId": 649,
      "createdAt": "2021-10-06T14:58:31.366Z",
      "updatedAt": "2021-10-06T14:58:31.366Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    }
  ]
}

Get user access info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/policyInfo

Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.

Query parameters

Response schema

Request example

The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/policyInfo

Response example

{
  "visibilities": [],
  "visibilityRuleApplies": false,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK",
      "actionType": "Nullify"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK",
      "actionType": "Nullify"
    }
  ],
  "additionalFilters": {}
}

Get user visibility info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.

Query parameters

Response schema

Request example

The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/visibilityReport

Response example

[
  {
    "noVisibilities": true,
    "dataSourceVisibilitiesCount": 0,
    "userVisibilitiesCount": 0,
    "masked": [
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_SOLD_DATE_SK"
      },
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_BILL_CUSTOMER_SK"
      }
    ],
    "dataSource": 16,
    "dataSourceName": "Web Sales",
    "additionalFilters": {},
    "allowMaskedJoins": false,
    "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
  }
]

Get current user visibility info

GET /dataSource/{dataSourceId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.

Query parameters

Response schema

Request example

The following request gets all of the visibility information for the current user on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/visibilityReport

Response example

{
  "noVisibilities": true,
  "dataSourceVisibilitiesCount": 0,
  "userVisibilitiesCount": 0,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK"
    }
  ],
  "dataSource": 16,
  "dataSourceName": "Web Sales",
  "additionalFilters": {},
  "allowMaskedJoins": false,
  "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}

Access data sources and make data source requests

Subscribe to a data source

POST /dataSource/subscribe

Subscribe to a data source.

Query parameters

Payload parameters

Response schema

Request example

The following request subscribes to the data source with ID 22.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/subscribe?dataSourceId=22

Payload example

{
  "dataSourceIds": [
    22
  ],
  "metadata": {},
  "approvals": [
    {
      "requiredPermission": "Owner",
      "specificApproverRequired": false,
      "specificApprover": 2,
      "ownerModelId": 23
    }
  ],
  "groupId": 12
}

Response example

{
  "inError": [],
  "success": [{
    "id": 64,
    "modelId": "22",
    "modelType": "datasource",
    "state": "subscribed",
    "metadata": {},
    "admin": null,
    "denialReasoning": null,
    "profile": 2,
    "group": null,
    "expiration": null,
    "acknowledgeRequired": false,
    "createdAt": "2021-08-26T16:36:09.587Z",
    "updatedAt": "2021-08-26T16:36:09.587Z",
    "approved": true
  }]
}

Request to unmask values

Deprecation notice

Support for unmask requests has been deprecated.

POST /dataSource/{dataSourceId}/reverseMask

Makes a request for values to be unmasked.

Query parameters

Payload parameters

Response schema

Request example

The following requests for values to be unmasked.

curl \
   --request POST \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
   --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/reverseMask

Request payload example

{
  "column": "cc_county",
  "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
  "unmaskReasoning": "Marketing project",
  "unmaskingUsers": [1]
}

Response example

{
  "id": 1,
  "requestingUserProfile": 13,
  "dataSourceId": 12,
  "reason": "Marketing Campaign",