Subscribe to and Manage Data Sources

Data Source API reference guide

This page describes the dataSource endpoint, through which users can subscribe to data sources and manage data source tasks. To create data sources, see the specific handler endpoints.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Data source workflow

Search data sources and data source details

Search for data sources

GET /dataSource

Search for data sources.

Query parameters

Response schema

Request example

The following request returns 2 data sources.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource?size=2

Response example

{
  "name": "Public Barfoo",
  "id": 22,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2021-07-22T14:11:55.539Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "status": "passed",
  "subscriptionStatus": "subscribed",
  "blobHandlerType": "Snowflake",
  "subscriptionType": "automatic",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "sqlSchemaName": "public",
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
},
{
  "name": "Public Aaa Tpc",
  "id": 39,
  "recordFormat": "Not Provided",
  "deleted": false,
  "description": null,
  "createdAt": "2023-08-21T10:39:00.250Z",
  "subscriptionPolicy": {
      "type": "subscription",
      "exceptions": {
          "operator": "and",
          "conditions": [
              {
                  "type": "groups",
                  "group": {
                      "name": "alpha"
                  }
              }
          ]
      },
      "allowDiscovery": true,
      "automaticSubscription": true
  },
  "schemaEvolutionId": 1,
  "recordCount": 0,
  "blobHandlerType": "Snowflake",
  "subscriptionType": "policy",
  "sqlSchemaName": "public",
  "status": "passed",
  "subscriptionStatus": "owner",
  "connectionString": "test@immuta.connection.source.com:###/test",
  "remoteTable": "tpc",
  "remoteSchema": "public",
  "domainId": null,
  "domainName": null,
  "policy": "None",
  "policyHandlerType": "None",
  "native": null,
  "workspace": null
}

Get a data source by ID

GET /dataSource/{dataSourceId}

Get a data source based on the ID.

Query parameters

Response Schema

Request example

The following request gets a data source based on the ID 22.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
  "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
  "sql": {
      "status": "passed",
      "message": "Passed"
   }
  }
}

Get data source by name

GET /dataSource/name/{dataSourceName}

Get a data source based on the name.

Query parameters

Response schema

Request example

The following request gets a data source based on the name Public Barfoo.

curl \
   --request GET \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/name/Public%20Barfoo

Response example

{
  "name": "Public Barfoo",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": null,
  "sqlSchemaName": "public",
  "sqlTableName": "barfoo",
  "blobHandler": {
      "url": "https://your-url/snowflake/handler/22",
      "ca": {
          "name": "Certificate Authority Bundle"
  },
      "manualDictionary": false
      },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 6,
  "documentation": "# Public Barfoo",
  "statsExpiration": "2021-08-27T16:34:47.846Z",
  "id": 22,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "None",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
      "type": "subscription",
      "automaticSubscription": true
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
      "sql": {
      "status": "passed",
      "message": "Passed"
      }
  }
}

Get a data source by the short name

GET /dataSource/sqlTableName/{shortName}

Get a data source based on the SQL table name.

Query parameters

Response schema

Request example

The following request gets a data source based on the SQL table name customer_data.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/sqlTableName/customer_data

Response example

    {
  "name": "Dbo Customer Data",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "visibilitySchema": {
      "fields": [],
      "version": "2021-10-06T19:39:39.145Z"
    },
    "handlerId": 55,
    "dataSourceId": 26
  },
  "sqlSchemaName": "dbo",
  "sqlTableName": "customer_data",
  "blobHandler": {
    "url": "https://1.1.1.1.1/snowflake/testhandler/26",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": 1000,
  "documentation": "# Dbo Customer Data",
  "statsExpiration": "2021-11-05T19:37:43.270Z",
  "id": 26,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "automatic",
  "subscriptionPolicy": {
    "type": "subscription",
    "automaticSubscription": false
  },
  "globalPolicies": null,
  "status": "passed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
    },
    "lastAttempt": {
      "date": "2021-10-06T19:39:39.821Z"
    },
    "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-10-06T19:37:43.337Z"
      }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 4,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-10-01T14:23:27.225Z",
  "updatedAt": "2021-10-06T19:39:39.145Z",
  "subscribedAsUser": true,
  "subscriptionId": 45,
  "acknowledgeRequired": false,
  "subscriptionStatus": "owner",
  "requestedState": "owner",
  "approved": true,
  "subscriptionExpiration": null,
  "filterId": null
}

Get data source relationships

GET /dataSource/{dataSourceId}/lineage/{type}

Get parent and child relationship records for derived data sources using a specified data source ID.

Query parameters

Response schema

Request example

The following request gets the parent relationship records for the derived data source with the data source ID 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/lineage/parents

Response example

{
  "parents": [{
    "dataSourceId": 3,
    "dataSourceName": "Public Healthcare Data",
    "deleted": false,
    "createdAt": "2022-08-17T13:41:38.381Z",
    "projectId": 2,
    "projectName": "Derived Data Source",
    "createdBy": "Your Username",
    "policyHandlerDiff": {
      "dsType": "queryable",
      "currentHandlerId": null,
      "previousHandlerId": null
    }
  }]
}

Retrieve a Blob

GET /dataSource/{dataSourceId}/blob/{blobid*}

Retrieve a blob.

Query parameters

Response schema

The response will download the blobs in a file you specify.

Request example

The following request retrieves a blob.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --output ./the-blobs-will-be-saved-here
    https://your-immuta-url.com/dataSource/22/blob/22

Response example

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  215k  100  215k    0     0   541k      0 --:--:-- --:--:-- --:--:--  541k

Get users by access level

GET /dataSource/{dataSourceId}/access

Get all users with the provided access level for this data source.

Query parameters

Response schema

Request example

The following request gets all users with the provided access level for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/22/access?sortOrder=desc

Response example

{
  "count": 2,
  "users": [
    {
      "profile": 2,
      "name": "First Last",
      "iamid": "bim",
      "userid": "first.last@immuta.com",
      "email": "first.last@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "owner",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-10-06T14:58:46.983Z",
      "subscriptionId": 586,
      "createdAt": "2021-10-05T14:33:01.518Z",
      "updatedAt": "2021-10-05T14:33:01.518Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    },
    {
      "profile": 3,
      "name": "Tommy Test",
      "iamid": "bim",
      "userid": "tommy.test@immuta.com",
      "email": "tommy.test@immuta.com",
      "type": "user",
      "admin": "First Last",
      "approved": true,
      "state": "subscribed",
      "systemGenerated": false,
      "lastExternalRefresh": "2021-09-07T16:16:29.957Z",
      "subscriptionId": 649,
      "createdAt": "2021-10-06T14:58:31.366Z",
      "updatedAt": "2021-10-06T14:58:31.366Z",
      "approvals": [
        {
          "requiredPermission": "OWNER",
          "state": "approved",
          "approverId": null,
          "ownerModelId": null,
          "approver": "First Last",
          "ownerModelName": null
        }
      ],
      "currentUserCanApprove": false
    }
  ]
}

Get user access info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/policyInfo

Retrieves the visibilities, masking information, and filters that the passed in user has access to in the specified data source.

Query parameters

Response schema

Request example

The following request gets the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/policyInfo

Response example

{
  "visibilities": [],
  "visibilityRuleApplies": false,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK",
      "actionType": "Nullify"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK",
      "actionType": "Nullify"
    }
  ],
  "additionalFilters": {}
}

Get user visibility info for a data source

GET /dataSource/{dataSourceId}/users/{profileId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities a given user has access to.

Query parameters

Response schema

Request example

The following request gets all of the visibility information for the user with the profile ID 2 on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/users/2/visibilityReport

Response example

[
  {
    "noVisibilities": true,
    "dataSourceVisibilitiesCount": 0,
    "userVisibilitiesCount": 0,
    "masked": [
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_SOLD_DATE_SK"
      },
      {
        "type": "Consistent Value",
        "metadata": {
          "constant": null
        },
        "name": "WS_BILL_CUSTOMER_SK"
      }
    ],
    "dataSource": 16,
    "dataSourceName": "Web Sales",
    "additionalFilters": {},
    "allowMaskedJoins": false,
    "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
  }
]

Get current user visibility info

GET /dataSource/{dataSourceId}/visibilityReport

Retrieves a summary of total records, total visibilities (the unique values contained in a column protected by a row-level security policy that allow Immuta to determine whether or not a user can see a given row if they possess an attribute that matches the visibility of that row), and visibilities the current user has access to for a specified data source.

Query parameters

Response schema

Request example

The following request gets all of the visibility information for the current user on the data source with the data source ID 16.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/16/visibilityReport

Response example

{
  "noVisibilities": true,
  "dataSourceVisibilitiesCount": 0,
  "userVisibilitiesCount": 0,
  "masked": [
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_SOLD_DATE_SK"
    },
    {
      "type": "Consistent Value",
      "metadata": {
        "constant": null
      },
      "name": "WS_BILL_CUSTOMER_SK"
    }
  ],
  "dataSource": 16,
  "dataSourceName": "Web Sales",
  "additionalFilters": {},
  "allowMaskedJoins": false,
  "policySet": "[{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_BILL_CUSTOMER_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_bill_customer_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"3a2faac13e332cca1829ed773afa298a5455ac5bb54e68c53ae00991575d2a4b\"},{\"type\":\"masking\",\"rules\":[{\"type\":\"masking\",\"config\":{\"fields\":[\"WS_SOLD_DATE_SK\"],\"maskingConfig\":{\"type\":\"Consistent Value\",\"metadata\":{\"constant\":null}}},\"exceptions\":null,\"ruleAppliedForUser\":true}],\"global\":{\"id\":9,\"tag\":\"Confidential\",\"live\":true,\"name\":\"Data Policy\",\"reason\":null,\"staged\":false,\"deleted\":false,\"conflict\":null,\"disabled\":false,\"metadata\":null,\"template\":false,\"createdAt\":\"2022-08-09T17:23:02.634Z\",\"createdBy\":\"jane\",\"policyKey\":\"Data Policy\",\"updatedAt\":\"2022-08-09T17:23:02.634Z\",\"clonedFrom\":null,\"certification\":null,\"createdByName\":\"jane\",\"changedOnApply\":[\"dataTypeMismatch\"],\"parentPolicyId\":null,\"systemGenerated\":false,\"originalPolicyId\":\"masking(^_^)data policy(^_^)ws_sold_date_sk(^_^)consistent value(^_^)(^_^)(^_^)\",\"ownerRestrictions\":null},\"description\":null,\"policyHash\":\"2877a1ace4cfa6427370fd39b254ce0ea75dc22cb024a2f857e033c82a987f9a\"}]"
}

Access data sources and make data source requests

Subscribe to a data source

POST /dataSource/subscribe

Subscribe to a data source.

Query parameters

Payload parameters

Response schema

Request example

The following request subscribes to the data source with ID 22.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/subscribe?dataSourceId=22

Payload example

{
  "dataSourceIds": [
    22
  ],
  "metadata": {},
  "approvals": [
    {
      "requiredPermission": "Owner",
      "specificApproverRequired": false,
      "specificApprover": 2,
      "ownerModelId": 23
    }
  ],
  "groupId": 12
}

Response example

{
  "inError": [],
  "success": [{
    "id": 64,
    "modelId": "22",
    "modelType": "datasource",
    "state": "subscribed",
    "metadata": {},
    "admin": null,
    "denialReasoning": null,
    "profile": 2,
    "group": null,
    "expiration": null,
    "acknowledgeRequired": false,
    "createdAt": "2021-08-26T16:36:09.587Z",
    "updatedAt": "2021-08-26T16:36:09.587Z",
    "approved": true
  }]
}

Request to unmask values

Deprecation notice

Support for unmask requests has been deprecated.

POST /dataSource/{dataSourceId}/reverseMask

Makes a request for values to be unmasked.

Query parameters

Payload parameters

Response schema

Request example

The following requests for values to be unmasked.

curl \
   --request POST \
   --header "Content-Type: application/json" \
   --header "Authorization: Bearer dea464c07bd07300095caa8" \
   --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/reverseMask

Request payload example

{
  "column": "cc_county",
  "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
  "unmaskReasoning": "Marketing project",
  "unmaskingUsers": [1]
}

Response example

{
  "id": 1,
  "requestingUserProfile": 13,
  "dataSourceId": 12,
  "reason": "Marketing Campaign",
  "metadata": {
    "salt": "**********87",
    "column": "cc_county",
    "values": ["WlRObU9ERTVNVE0yTVdabU9XVXdPQT09OktYN2lkNEZqZjlaWUluck8xTnVHOGlSN25wWmdudVZjbnZES1ArUkxhMGc9"],
    "maskingConfig": {
      "type": "Reversible",
      "metadata": {}
    }
  },
  "type": "unmask",
  "state": "pending",
  "createdAt": "2021-10-27T20:35:48.253Z",
  "updatedAt": "2021-10-27T20:35:48.253Z"
}

Add a user to a data source

POST /dataSource/{dataSourceId}/access

Add a user to a specific data source.

Query parameters

Payload parameters

Response schema

Request example

The following request adds a user (saved in example-payload.json) to this data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/22/access

Request payload example

{
  "profileId": 3,
  "state": "subscribed"
}

Response example

{
  "id": 19,
  "modelId": "1",
  "modelType": "datasource",
  "state": "subscribed",
  "metadata": null,
  "admin": 2,
  "denialReasoning": null,
  "profile": 3,
  "group": null,
  "expiration": null,
  "acknowledgeRequired": false,
  "createdAt": "2021-09-21T14:24:12.528Z",
  "updatedAt": "2021-09-21T14:24:12.528Z",
  "approved": true
}

Manage data source requests

Get pending tasks by user

GET /dataSource/tasks/pending

Get all pending tasks for this user and pending tasks this user has created.

Query parameters

Response schema

Request example

The following request gets all pending tasks for a user and pending tasks the user has created.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/pending

Response example

{
  "outgoing": [],
  "incoming": [{
    "fullCount": 1,
    "dataSource": {
      "id": 11,
      "name": "Public Customer"
    },
    "isOutgoing": false,
    "rows": [{
      "id": 1,
      "state": "pending",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
      "targetNames": [
        "Katie"
      ],
      "targetEmails": [
        "katie@immuta.com"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1,
        "email": null
      },
      "dataSource": {
        "id": 11,
        "name": "Public Customer"
      },
      "createdAt": "2024-08-06T14:37:09.279+00:00",
      "metadata": {
        "colName": "org",
        "colType": "VARCHAR(16777216)"
      },
      "isOutgoing": false
    }]
  }]
}

Mark tasks as complete

GET /dataSource/tasks/{taskId}

Handles the given task and marks it as complete.

Query parameters

Response schema

Request example

The following request handles the given task and marks it as complete.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/7

Response example

{
  "result": {
    "unmaskPairs": [
      {
        "masked": "TnpjNFpXSXdOR1pqWkdNeE5EYzJPQT09OktaWjJEVldXZVluTmQ2SUVQdW1MajJtVTdqL2ZBT1JlaGFUUHJidmhkWmM9",
        "unmasked": "jalekseev2@phoca.cz"
      }
    ],
    "column": "email"
  }
}

Return tasks for a data source

GET /dataSource/{dataSourceId}/tasks

Returns all tasks the user has made, can approve or deny, or validate for this data source.

Query parameters

Response schema

Request example

The following request returns all tasks the user has made, can approve or deny, or validate for this data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/tasks?sortOrder=desc

Response example

{
  "hits": [
    {
      "fullCount": 2,
      "id": 6,
      "state": "completed",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
      "targetNames": [
        "John"
      ],
      "targetEmails": [
        "john@example.com"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1,
      },
      "dataSource": {
        "id": 11,
        "name": "Public Customer"
      },
      "createdAt": "2021-10-12T15:48:23.095Z"
    },
    {
      "fullCount": 2,
      "id": 7,
      "state": "completed",
      "type": "columnAdded",
      "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Fake Medical Claims 2017\"",
      "targetNames": [
        "John"
      ],
      "targetEmails": [
        "john@example.com"
      ],
      "requester": {
        "name": "Immuta System Account",
        "id": 1
      },
      "dataSource": {
        "id": 3,
        "name": "Public Fake Medical Claims 2017"
      },
      "createdAt": "2021-10-12T18:56:22.954Z"
    }
  ],
  "count": 2
}

Change user status

PUT /dataSource/{dataSourceId}/access/{subscriptionId}

Change user status for a specific data source.

Query parameters

Payload parameters

Response schema

Request example

The following request changes the user status to subscribed for the specified data source.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/23/access/95

Payload example

{
  "state": "subscribed"
}

Response example

{
  "id": 95,
  "modelId": "3",
  "modelType": "datasource",
  "state": "subscribed",
  "metadata": {},
  "admin": 2,
  "profile": 3,
  "group": null,
  "expiration": null,
  "acknowledgeRequired": false,
  "createdAt": "2021-10-12T15:40:13.878Z",
  "updatedAt": "2021-10-12T16:10:46.801Z",
  "originalState": "expert",
  "approved": true
}

Update data sources

Update a data source

PUT /dataSource/{dataSourceId}

Update a data source.

Query parameters

Payload parameters

Response schema

Request example

The following request updates the data source's documentation (saved in example-payload.json).

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/22

Request payload example

{
  "documentation": "# Public Credit Accounts\nThis request updates the data source documentation."
}

Response example

{
  "name": "Public Credit Accounts",
  "recordFormat": "Not Provided",
  "description": null,
  "policyHandler": {
    "handlerId": 21,
    "visibilitySchema": {
      "fields": [],
      "version": "2021-09-15T17:44:20.678Z"
    },
    "previousHandlerId": 20,
    "dataSourceId": 1
  },
  "sqlSchemaName": "public",
  "sqlTableName": "credit_accounts",
  "blobHandler": {
    "url": "https://1.1.1.1:1111/snowflake/handler/1",
    "ca": {
      "name": "Certificate Authority Bundle"
    },
    "manualDictionary": false
  },
  "createdBy": 2,
  "deleted": false,
  "type": "queryable",
  "recordCount": 0,
  "rowCount": "100000",
  "documentation": "# Public Credit Accounts\nThis request updates the data source documentation.",
  "statsExpiration": "2021-09-22T13:51:46.646Z",
  "id": 1,
  "blobHandlerType": "Snowflake",
  "policyHandlerType": "Builder",
  "subscriptionType": "approval",
  "subscriptionPolicy": {
    "type": "subscription",
    "approvals": [{
      "requiredPermission": "OWNER",
      "specificApproverRequired": false
    }]
  },
  "globalPolicies": null,
  "status": "failed",
  "statusInfo": {
    "sql": {
      "status": "passed",
      "message": "Passed"
    },
    "stats": {
      "status": "passed",
      "lastAttempted": "2021-09-21T13:51:46.674Z"
    },
    "fingerprint": {
      "status": "passed",
      "lastAttempted": "2021-09-09T14:12:25.177Z"
    },
    "lastAttempt": {
      "date": "2021-09-20T19:35:21.929Z"
    },
    "globalPolicy": {
      "status": "passed",
      "lastAttempted": "2021-09-17T19:07:38.092Z"
    },
    "highCardinality": {
      "status": "failed",
      "message": "Error could not connect to remote database",
      "lastAttempted": "2021-09-20T16:43:19.426Z"
    }
  },
  "expiration": null,
  "catalogMetadata": null,
  "workspace": null,
  "seeded": false,
  "schemaEvolutionId": 1,
  "columnEvolutionEnabled": true,
  "createdAt": "2021-09-09T14:12:09.511Z",
  "updatedAt": "2021-09-21T13:52:42.908Z"
}

Update multiple data sources

PUT /dataSource/bulk/{type}

Update data sources.

Query parameters

Payload parameters

Response schema

Request example

The following request adds the Address.email tag to two data sources.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/bulk/tags

Payload example

{
  "ids": [49, 50],
  "update": {
    "tags": [{
      "name": "Address.email",
      "source": "curated"
    }]
  }
}

Response example

{
  "bulkId": "bulk_ds_update_4896d300e04c4a8f89493ebf125c5c6b",
  "jobsCreated": 2
}

Refresh native views

POST /dataSource/bulkRefreshViews

Refresh native views.

Payload parameters

Request example

The following request with the payload below refreshes the view for the data source with the ID 202.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer <TOKEN>" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/bulkRefreshViews

Payload example

{
  "dataSourceIds": [202]
}

Save blob metadata to Immuta

POST /dataSource/{dataSourceId}/blobs

Save blob metadata to Immuta.

Query parameters

Payload parameters

Response schema

Request example

The following request saves blob metadata to Immuta.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/blobs

Payload example

[
  {
    "blobId": "testblob",
    "filename": "testfile",
    "tags": [
      "update"
    ],
    "visibility": {},
    "metadata": {},
    "date": "2021-10-21",
    "filesize": 2
  }
]

Response example

{
  "blobsWithoutIds": 0,
  "blobsInError": [],
  "blobsInserted": [
      "string"
  ],
  "tags": [
      "string"
  ]
}

Store blob metadata locally

POST /dataSource/{dataSourceId}/persistBlob

Save blob metadata to Immuta and store raw content in local blob store.

Query parameters

Payload parameters

Response schema

Request example

The following request saves blob metadata to Immuta and stores raw content in local blob stores.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json
    https://your-immuta-url.com/dataSource/23/persistBlob

Payload example

{
  "blobId": "test/pov-queries-2.dbc",
  "filename": "pov-queries",
  "date": "2021-10-26",
  "file": "pov-queries.dbc"
}

Response example

{
  "blobsWithoutIds": 0,
  "blobsInError": [],
  "blobsInserted": ["test/pov-queries.dbc"],
  "tags": []
}

Trigger schema monitoring jobs

PUT /dataSource/detectRemoteChanges

Trigger the schema monitoring job for the specified detection group, or all groups if no payload parameters are given.

Payload parameters

Response schema

Responses will include either

  • bulkId, schemaDetection, and columnDetection (when table or dataSourceIds is provided in payload) or

  • bulkId (payloads that do not include dataSourceIds or table)

Request example

The following request triggers the schema monitoring job for the specified detection group.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.com/dataSource/detectRemoteChanges

Payload example

The tabs below illustrate payloads for triggering schema monitoring on a host, database, or table. The request will run schema monitoring for all databases registered under the hostname provided in the payload.

The request will run schema monitoring for all databases registered under the hostname provided in the payload.

{
  "hostname": "organization.us-east-1.snowflakecomputing.com"
}

Response examples

The tabs below illustrate the example response for each example payload provided above.

{
  "bulkId": "31ab4312-b90a-49a6-baf8-70f87cd92a89"
}

View and Review Data Sources

Refresh external catalog tags on a data source

GET /dataSource/{dataSourceId}/test

Refreshes tags from an external catalog on a data source. The external catalog must be linked to the data source.

Query parameters

Response schema

Request example

The following request refreshes external catalog tags on the data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/test

Response example

{
  "sql": {
      "status": "passed",
      "message": "Passed"
  },
  "stats": {
      "status": "passed",
      "lastAttempted": "2021-09-09T13:43:43.948Z"
  },
  "fingerprint": {
      "status": "passed",
      "lastAttempted": "2021-07-22T14:12:01.525Z"
  },
  "lastAttempt": {
      "date": "2021-09-09T16:47:05.173Z"
  },
  "columnEvolution": {
      "status": "passed",
      "lastAttempted": "2021-09-08T16:36:05.557Z"
  },
  "highCardinality": {
      "status": "passed",
      "lastAttempted": "2021-07-22T14:11:58.439Z"
  },
  "schemaEvolution": {
      "status": "passed",
      "lastAttempted": "2021-09-08T16:35:57.867Z"
  },
  "status": "passed"
}

Retrieve blob handlers

GET /dataSource/blobHandlerTypes

Retrieve all blob handlers the current user is allowed to create.

Response schema

Request example

The following request retrieves all blob handlers the current user is allowed to create.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/blobHandlerTypes

Response example

{
  "name": "Azure Synapse Analytics",
  "pluginType": "odbcHandler",
  "clientUrl": "/asa",
  "baseUrl": "https://0.0.0.0:8823/asa",
  "config": {
  "port": 1433,
  "allowSSLToggle": false
  },
  "displayOrder": 41,
  "requiresHashPhraseForDP": true,
  "driver": "Azure Synapse Analytics"
}

Get data sources by purpose

GET /dataSource/byPurposes

Get data sources that match a set of purposes.

Query parameters

Response schema

Request example

The following request gets data sources that match a set of purposes.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/byPurposes?purposes=Data+Analysis

Response example

[{
  "id": 42,
  "name": "Army Army Records",
  "policyId": 56,
  "restrictions": [{
    "operator": "and",
    "purposes": ["Data Analysis"]
  }]
}]

Retrieve data sources by user

GET /dataSource/rpc/mine

Retrieves all the data sources the current user has access to.

Response schema

Request example

The following request retrieves all the data sources the current user has access to.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/rpc/mine

Response example

{
  "id": 23,
  "name": "Public Record",
  "type": "queryable",
  "sqlTableName": "record",
  "sqlSchemaName": "public",
  "blobHandlerType": "Snowflake",
  "sparkUseJDBC": true
}

Get recent policy activities for a data source

GET /dataSource/{dataSourceId}/activities

Get all of the recent policy activities for a given data source.

Query parameters

Response schema

Request example

The following request gets all of the recent policy activities for a given data source.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/activities

Response example

{
  "count": 3,
  "activities": [{
    "modelType": "datasource",
    "modelId": "23",
    "createdAt": "2021-09-07T16:13:25.197Z",
    "notificationType": "policyUpdated",
    "metadata": {
      "dataSourceName": "Public Foobar",
      "triggeredByGlobal": false,
      "conflictCount": 0,
      "policyType": "data",
      "handlerId": 3,
      "previousHandlerId": 2,
      "dataSourceType": "queryable"
    },
    "read": false,
    "id": 256,
    "updatedAt": "2021-09-07T16:13:25.197Z",
    "actionBy": {
      "id": 2,
      "name": "first last",
      "email": "first.last@immuta.com"
    },
    "targetUser": {
      "id": 2,
      "name": "first last",
      "email": "first.last@immuta.com"
    }
  }]
}

Get profiles for data source owners and experts

GET /dataSource/{dataSourceId}/contacts

Gets the profiles for the data source owners and experts.

Query parameters

Response schema

Request example

The following request gets all the profiles for the data source owners and experts.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/contacts

Response example

{
  "type": "profile",
  "id": 23,
  "state": "owner",
  "name": "first last",
  "email": "first.last@immuta.com",
  "profile": 2
},
{
  "type": "profile",
  "id": 23,
  "state": "owner",
  "name": "Tommy Test",
  "email": "tommy.test@immuta.com",
  "profile": 3
}

Get tags by data source

GET /dataSource/{dataSourceId}/tags

Get the tags for a data source.

Query parameters

Response schema

Request example

The following request gets the tags for data source 4.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/4/tags

Response example

{
  "tags": [
    {
      "name": "Discovered.Entity.Medicare Number",
      "source": "curated",
      "modelType": "datasource",
      "modelId": "4",
      "addedBy": 1,
      "deleted": false
    }
  ]
}

Get users who can unmask columns

GET /dataSource/{dataSourceId}/{columnName}/unmaskUsers

Return the users who can unmask the given column.

Query parameters

Response schema

Request example

The following request returns the users who can unmask the given column.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/47/gender/unmaskUsers

Response example

[
  {
    "name": "Katie",
    "profileId": 2,
    "iamid": "bim"
  }
]

Delete Data Sources and More

Delete a data source

DELETE /dataSource/{dataSourceId}

Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.

Query parameters

Response schema

Request example

The following request deletes the data source 23.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23

Response example

{
  "success": true,
  "id": 23,
  "schemaEvolutionId": 1,
  "name": "Public Foobar",
  "disabled": true,
  "handlerDeleteErrorMessage": null
}

Delete a task

DELETE /dataSource/tasks/{taskId}

Delete the specified task.

Query parameters

Response schema

Request example

The following request deletes a specified task.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/tasks/8

Response example

{
  "fullCount": 1,
  "id": 8,
  "state": "pending",
  "type": "columnAdded",
  "reason": "Immuta has detected a new column \"org\" in the remote table and has added it to \"Public Customer\"",
  "targetNames": ["Katie"],
  "targetEmails": ["katie@example.com"],
  "requester": {
    "name": "Immuta System Account",
    "id": 1
  },
  "dataSource": {
    "id": 11,
    "name": "Public Customer"
  },
  "createdAt": "2021-10-12T19:28:04.999Z"
}

Delete a blob

DELETE /dataSource/{dataSourceId}/blob/{blobId*}

Delete a blob.

Query parameters

Response schema

When the blob is successfully deleted, there will be no response.

Request example

The following request deletes a blob.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/blob/1

Unsubscribe from a data source

DELETE /dataSource/{dataSourceId}/unsubscribe

Unsubscribe from a data source.

Query parameters

Response schema

Request example

The following request unsubscribes the user from data source 23.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/23/unsubscribe

Response example

{
  "success": true
}

Last updated

Self-managed versions

2024.32024.22024.12023.4

Copyright © 2014-2024 Immuta Inc. All rights reserved.

#141: DSIA API Updates

Change request updated