1 of 1

Databricks Unity Catalog Query Audit Logs

Query audit for Databricks Unity Catalog captures user data access within Unity Catalog and presents them in a universal format as Immuta audit logs. Multiple access options are supported for audit:

Cluster queries with the following supported languages: SQL, Scala, Python, and R.
SQL warehouse queries

Immuta audits the activity of all Unity Catalog users and tables regardless of whether they are registered in Immuta.

Requirements

A Databricks deployment with capabilities

Immuta collects audit records once an hour by default, and the frequency can be . The frequency is a global setting based on integration type, so organizations with multiple Databricks Unity Catalog integrations will have the same audit frequency for all of them. The more frequent the audit records are ingested, the more current the audit records. However, there could be performance and cost impacts from the frequent jobs because Immuta will start a Databricks cluster to complete the audit ingest job if one is not already running.

To manually prompt the query audit, click Load Audit Events on the Immuta audit page.

Immuta audits all data sources and users in Unity Catalog. An administrator can configure the integration to just ingest specific workspaces when . The audit is intended to ingest user queries for data access, so queries with

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Property

Description

Example

Enrichment of audit logs with Immuta entitlements information is not supported. While you will see these entitlements in the Databricks Spark audit logs, the following will not be in the query audit for Unity Catalog:
- Immuta policies information
- User attributes

Databricks Unity Catalog Query Audit Logs

Query audit for Databricks Unity Catalog captures user data access within Unity Catalog and presents them in a universal format as Immuta audit logs. Multiple access options are supported for audit:

Cluster queries with the following supported languages: SQL, Scala, Python, and R.
SQL warehouse queries

Immuta audits the activity of all Unity Catalog users and tables regardless of whether they are registered in Immuta.

Requirements

A Databricks deployment with capabilities

To manually prompt the query audit, click Load Audit Events on the Immuta audit page.

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Property

Description

Example

Enrichment of audit logs with Immuta entitlements information is not supported. While you will see these entitlements in the Databricks Spark audit logs, the following will not be in the query audit for Unity Catalog:
- Immuta policies information
- User attributes

The Immuta user ID of the actor who made the query. When the actor is not registered with Immuta, the type, id, and name fields will be unknown.

actor.name

The Immuta name of the user who made the query. When the user is not registered with Immuta, the type, id, and name fields will be unknown.

Taylor

actionStatus

Indicates whether or not the user was granted access to the data. Possible values are UNAUTHORIZED, FAILURE, or SUCCESS.

SUCCESS

auditPayload.errorCode

The errorCode for the denied query.

null if actionStatus is SUCCESS

auditPayload.technologyContext.service

Where in Unity Catalog the query was made. Possible values are SQL for SQL warehouses and NOTEBOOK for notebooks.

SQL

auditPayload.technologyContext.queryLanguage

The queryLanguage corresponds to the programming language used: SQL, Python, Scala, or R. Audited JDBC queries will indicate that it came from JDBC here.

python

The Immuta user type of the actor who made the query. When the actor is not registered with Immuta, the type, id, and name fields will be unknown.

{
  "action": "QUERY",
  "actor": {
    "type": "USER_ACTOR",
    "id": "taylor@immuta.com",
    "name": "Taylor",
    "identityProvider": "bim",
    "profileId": "10"
  },
  "sessionId": "01ee14d9-cab3-1ef6-9cc4-f0c315a53788",
  "requestId": "504b8fd9-38c1-4a90-966e-7445a6675f79",
  "actionStatus": "SUCCESS",
  "actionStatusReason": null,
  "eventTimestamp": "2023-06-27T11:03:59.000Z",
  "id": "01ee14da-517a-1670-afce-0c3e0fdcf7d4",
  "tenantId": "your-immuta.com",
  "userAgent": "",
  "targetType": "DATASOURCE",
  "targets": [
      {
      "type": "DATASOURCE",
      "id": "2034",
      "name": "University Art Gallery Exhibition",
      "technology": "DATABRICKS"
    }
  ],
  "relatedResources": [],
  "auditPayload": {
    "type": "QueryAuditPayload",
    "queryId": "01ee14da-517a-1670-afce-0c3e0fdcf7d4",
    "query": "SELECT VERSION AS `version` FROM `sample-data`.`__immuta_version`",
    "startTime": "2023-06-27T11:03:59.000Z",
    "duration": 23.568,
    "errorCode": null,
    "technologyContext": {
      "type": "DatabricksContext",
      "clusterId": null,
      "workspaceId": "3841033049363283",
      "service": "SQL",
      "warehouseId": "559483c6eac0359f",
      "notebookId": null,
      "account": {
        "id": "52e863bc-ea7f-46a9-8e17-6aed7541832d",
        "username": "taylor@databricks.com"
      },
      "host": "deployment-name.cloud.databricks.com",
      "clientIp": "0.0.0.0"
    },
    "objectsAccessed": [],
    "securityProfile": {
      "sensitivity": {
        "score": "INDETERMINATE"
      }
    },
    "version": 1
  },
  "receivedTimestamp": "2023-06-27T15:18:22.314Z"
}

{
  "action": "QUERY",
  "actor": {
    "type": "USER_ACTOR",
    "id": "taylor@immuta.com",
    "name": "Taylor",
    "identityProvider": "bim",
    "profileId": "10"
  },
  "sessionId": "01ee14d9-cab3-1ef6-9cc4-f0c315a53788",
  "requestId": "504b8fd9-38c1-4a90-966e-7445a6675f79",
  "actionStatus": "SUCCESS",
  "actionStatusReason": null,
  "eventTimestamp": "2023-06-27T11:03:59.000Z",
  "id": "01ee14da-517a-1670-afce-0c3e0fdcf7d4",
  "tenantId": "your-immuta.com",
  "userAgent": "",
  "targetType": "DATASOURCE",
  "targets": [
      {
      "type": "DATASOURCE",
      "id": "2034",
      "name": "University Art Gallery Exhibition",
      "technology": "DATABRICKS"
    }
  ],
  "relatedResources": [],
  "auditPayload": {
    "type": "QueryAuditPayload",
    "queryId": "01ee14da-517a-1670-afce-0c3e0fdcf7d4",
    "query": "SELECT VERSION AS `version` FROM `sample-data`.`__immuta_version`",
    "startTime": "2023-06-27T11:03:59.000Z",
    "duration": 23.568,
    "errorCode": null,
    "technologyContext": {
      "type": "DatabricksContext",
      "clusterId": null,
      "workspaceId": "3841033049363283",
      "service": "SQL",
      "warehouseId": "559483c6eac0359f",
      "notebookId": null,
      "account": {
        "id": "52e863bc-ea7f-46a9-8e17-6aed7541832d",
        "username": "taylor@databricks.com"
      },
      "host": "deployment-name.cloud.databricks.com",
      "clientIp": "0.0.0.0"
    },
    "objectsAccessed": [],
    "securityProfile": {
      "sensitivity": {
        "score": "INDETERMINATE"
      }
    },
    "version": 1
  },
  "receivedTimestamp": "2023-06-27T15:18:22.314Z"
}

Databricks Unity Catalog Query Audit Logs

Requirements

Databricks Unity Catalog Query Audit Logs

Requirements

Audit frequency

Audit scope

Audit schema

Example audit record

Limitations