Immuta reports allow data governors to use a natural language builder to instantly create reports that delineate user activity across Immuta. These reports can be based on various entity types, including users, groups, projects, data sources, purposes, policy types, or connection types.

Generate reports in the Immuta UI or using the Immuta API.

User reports options

User reports can be run for all users or for individual users who have been registered in Immuta. Non-registered users' activity will not appear in reports.

All users report option

• Data sources subscribed to. This report lists data sources each user is subscribed to and includes user roles, subscription types, when users last subscribed, who approved the users' subscriptions to the data sources, when the subscriptions expire, what attributes the users possess, and the groups the users belong to.

• Status of all users. This report lists account information of all users in the system, including the users' full names, usernames, IAMs, HDFS principals, and last login dates.

Individual user report options

• Groups the user belongs to. This report lists the names of the groups the user belongs to and the dates that groups were joined.

• Data sources the user subscribes to. This report details the data source names, the user's roles, when the user last subscribed, who approved the subscriptions, when the subscriptions expire (if applicable), and the reasons for subscribing (if applicable).

• Projects the user is currently a member of. This report lists the project names, whether the projects are public or private, the user's roles in the projects, the creator of the projects, when the projects were created, and when the user joined the projects.

• All data sources ever accessed by the user. This report lists the data source names, when the data sources were first accessed by the user (or "read date"), and when the data sources were last accessed by the user. By default, this report only displays the last month of results. (You can download the full report by clicking Export to CSV.) The time period can be configured in the date field at the top of report's page.

• Attributes the user has. This report lists the current attributes a user has and the values assigned to each attribute.

• Purposes for accessing data. This report lists all purposes under which the user has accessed data sources. By default, this report only displays the last month of results. (The full report can be downloaded by clicking Export to CSV.) The time period can be configured in the date field at the top of the report's page.

Group reports options

Group Reports can be run for all groups or for individual groups.

All groups report option

• Data sources that members of this group are subscribed to. This report lists the data source names, the group's role, when the group last subscribed to the data sources, who approved the subscriptions, and the expiration dates (if applicable), and reasons (if applicable) for the subscriptions.

Individual group report options

• Users who belong to the group. This report lists the names of users and the dates the users joined the group.

• Data sources that members of this group are subscribed to. This report lists the data source names, the group's role, when the group last subscribed to the data sources, who approved the subscriptions, and the expiration dates (if applicable), and reasons (if applicable) for the subscriptions.

• Projects that users in this group are members of. This report includes the names of the projects, whether the projects are public or private, the group's role in the projects, the names of the project creators, when the projects were created, and when the group joined the projects.

• Attributes of the group. This report includes the names of the attributes assigned to this group.

Project reports options

• Users and groups who are members of the project. This report includes usernames, email addresses, user roles in the project, when the users joined, and the subscription types. The subscription types may be "Individual User," indicating that the user joined the project directly, or it might be "Group," in which case the name of the group will be stated. Group subscriptions occur when an entire group is added to a project.

• Data sources that are part of the project. This report lists the data source names, the reasons given when added to the project (if applicable), the users who added the data sources, and when the data sources were added to the project.

• Purpose of the project. This report includes the purpose name, the user who added the purpose, and when the purpose was added to the project.

Data source reports options

Data source reports can be run for all data sources or for individual data sources that are registered in Immuta. Activity to non-registered tables will not appear in the reports.

All data sources report option

• Users and groups subscribed to data sources. This report lists all users and groups subscribed to every data source and includes usernames, email addresses, subscription types, user roles, subscription dates, who approved the subscriptions, expiration dates, and user attributes.

Individual data source reports options

• Users and groups subscribed to the data source. This report lists the names of users, reasons for accessing the data sources (if applicable), user roles, email addresses, when users last subscribed, who approved the subscriptions, when the subscriptions expire (if applicable), and the subscription types. A subscription type may be "Individual User," indicating that the user subscribed to the data sources directly, or it might be "Group," in which case the name of the group will be stated. Group subscriptions occur when an entire group is added to a data source.

• Projects that contain the data source. This report lists the project names, the users who added the data source to projects, when the data source was added to projects, the reasons for adding the data sources (if applicable), whether the projects are public or private, who created the projects, and when the projects were created.

• Purposes of all projects that contain the data source. This report states the purpose names, the users who assigned the purposes to the projects, the dates the purposes were assigned, the names of the projects, the reasons the purposes were added (if applicable), whether the projects are public or private, who created the projects, and when the projects were created.

• All users who have accessed the data source. This report lists usernames, email addresses, each user's latest query, and the date of the last access. By default, this report only displays the last month of results. (The full report can be downloaded by clicking Export to CSV.) The time period can be configured in the date field at the top of report's page.

• All purposes for data source access. This report lists users who have accessed the data source and the purposes under which they were working. By default, this report only displays the last month of results. (The full report can be downloaded by clicking Export to CSV.) The time period can be configured in the date field at the top of report's page.

• All users who have subscribed to the data source. This report lists users or groups, email addresses, when users subscribed, reasons for subscriptions (if applicable), who approved the subscriptions, when the subscriptions expire, and the dates and reasons users unsubscribed (if applicable). By default, this report only displays the last month of results. (The full report can be downloaded by clicking Export to CSV.)

• All identifiers for the columns of the data source. This report lists all the identifiers that matched to a column of the data source through sensitive data discovery. It includes information about the column name, the hit percentage, and the number of rows sampled.

Purpose reports options

• Users who are members of projects with this purpose. This report lists usernames, email addresses, their roles in the project, the names of the projects, whether the projects are public or private, the creators of the projects, when the projects were created, when users joined, and their subscription types (individual or group).

• Data sources that are part of projects with this purpose. This report lists the names of the data sources, who created the data sources, the project names, whether the projects are public or private, the creators of the projects, whether the projects have other purposes, and when the projects were created. Note that whether projects have other purposes will be assigned as "True" or "False."

• Whether any other purposes have been combined with this purpose. This report lists the names of the other purposes combined with the purpose you select, the project name where they are combined, the users who added each purpose, the project creator, whether the project is public or private, and the date the project was created.

• Projects that have this purpose. This report lists the names of the projects, the users who added the purpose, whether the projects are public or private, creators of the projects, whether the projects have other purposes, and when the projects were created.

• Data sources that have been accessed for this purpose. This report lists the names of the data sources, the users who accessed data sources for this purpose, the project names, and whether projects have other purposes. By default, this report only displays the last month of results, but the time period can be configured in the date field at the top of this report's page.

Policy type reports option

• Data sources with this policy type. Immuta supports a range of policy types, such as masking, WHERE clauses, purpose restrictions, and more. This report lists every data source with this policy type, including when they were created, who created the data sources, who created the policy, and when the policy was created.

Global policy reports options

Global policy reports can be run for all global policies or for individual global policies.

All global policies report options

• Global policies that have been disabled. This report details the names of the policies, the policies themselves, the policy types, the data sources from which the policies were disabled, who disabled the policies, when they were disabled, the justifications the users provided for disabling the policies, who created the policies, when the policies were created, and how the policies were associated with the data sources.

• Global policies that cannot currently be applied. This report details the names of the policies, the policies themselves, the policy types, the names of the data sources the policies cannot be applied to, when the data sources were created, when the policies were created, the reasons the policies cannot be applied, who created the policies, and how the policies are associated with the data sources.

Individual global policy reports option

• Data sources impacted by the policy. This report lists the data sources, when the data sources were created, and whether or not the policy is fully applied to the data sources.

• Data sources impacted by the policy that have not been certified. This report lists the data sources that have not been certified, when the global policy was applied, and the data owner.

• Data sources impacted by the policy that have been certified. This report lists the data sources that have been certified, the user that certified it, when the global policy was applied, and when it was certified.

Connection reports option

• Data sources with this connection type. This report lists the data sources, each data source's creator, the creation date, and the tables or queries used by the connection selected.

Tag reports options

Tag reports can be run for all tags or for individual tags.

All tag reports options

• Data sources this tag has been assigned to. This report generates a list of data sources associated with that tag and includes the columns tagged, the value types of the data tagged, who tagged the data sources, when the data sources were tagged, and when the data sources were created.

• Purposes associated with data sources containing this tag. This report generates a list of purposes under which users have accessed data sources containing this tag. By default, this report only displays the last month of results. (The full report can be downloaded by clicking Export to CSV.) The time period can be configured in the date field at the top of the report's page.

• Users who have accessed data sources containing this tag. This report lists users who have accessed data sources with this tag, their email addresses, when they queried the data, and when the data sources were created.

• Projects that contain data with this tag. This report details the projects associated with this tag, whether or not the projects are public or private, when the projects were created, the data sources in the projects, and when the data sources were created.

Individual tag options

• Users that have subscribed to data sources with any tag. This report lists users, their subscription type, and all of the tags in Immuta with information of whether or not users are subscribed to at least one data source where that tag is applied.

• Data sources any tag has been applied to. This report lists data sources with the tags applied to them and the columns they are applied to.

• Projects that contain a data source with any tag. This report lists projects and the data sources assigned to them with the tag they have applied.

Sensitive data discovery reports options

• Columns with SDD tags applied. This report generates a list of all Discovered tags that have been applied to data sources by sensitive data discovery. It includes information about the column it is applied to within each data source and active policies that use the tag.

• Columns with legacy SDD tags. This report generates a list of all Discovered tags applied by legacy SDD and provides context if native SDD also found those tags. It includes information about the data sources, columns, and active policies that use the tag.

Immuta’s universal audit model (UAM) provides audit logs with a consistent structure for query, authentication, policy, project, and tag events from your Immuta users and data sources. You can view the information in these UAM audit logs on the Detect dashboards or export the full audit logs to S3 and ADLS for long-term backup and processing with log data processors and tools. This capability fosters convenient integrations with log monitoring services and data pipelines.

You can specify an S3 bucket destination where Immuta will periodically export audit logs when using S3. When using ADLS, you can specify the container destination where Immuta will export audit logs. If desired, users can configure both export options to export their audit logs to S3 and ADLS simultaneously.

The events captured are events relevant to user and system actions that affect Immuta or the integrated data platforms, such as creating policies or data sources and running queries.

See a list of the events captured and example schemas on the UAM schema reference guide.

Immuta audit service

The Immuta audit service is an independent microservice that captures audit events from Immuta and queries run against your Snowflake, Databricks Spark, or Unity Catalog integration.

Immuta stores the export endpoints you provide during configuration, retrieves the audit records pushed to the audit service by your integration, and manages the audit exports based on an export schedule you define. These audit records are also stored to support future reporting and user interface enhancements that will allow you to search based on keywords and facets easily across the entire body of audit events.

Audit export workflow

1. When you configure the audit export using the CLI for S3 and ADLS, the audit service stores the export endpoint you provided.

2. After the integration endpoint has been configured, the export scheduler will run on the schedule you defined in your configuration.

3. When users query data and the event is audited, the audit service receives events from your Snowflake, Databricks Spark, Databricks Unity Catalog, or Starburst (Trino) integration.

4. Immuta exports the audit logs to your configured S3 bucket or ADLS container.

Audit support for platform queries

The table below outlines what information is included in the query audit logs for each integration where query audit is supported.

Legend:

Limitations

• The audit service does not capture system-level logging and debugging information, such as 404 errors.

Snowflake query audit limitations

• Snowflake query audit events from a query using cached results will show 0 for the rowsProduced field.

Unity Catalog query audit limitations

• Enrichment of audit logs with Immuta entitlements information is not supported. While you will see these entitlements in the Databricks Spark audit logs, the following will not be in the Databricks Unity Catalog audit logs:

  • Immuta policies information

  • User attributes

  • Groups

• Immuta determines unauthorized events based on error messages within Unity Catalog records. When the error messages contain expected language, unauthorized events will be available for Databricks Unity Catalog audit logs, in other cases it is not possible to determine the cause of an error.

• Unauthorized logs for cluster queries are not marked as unauthorized; they always will be a failure.

• Data source information will be provided when available:

  • For some queries, Databricks Unity Catalog does not report the target data source for the data access operation. In these cases the activity is audited, yet the audit record in Immuta will not include the target data source information.

  • The target data source information is not available for unauthorized queries and events.

• The column affected by the query is not currently supported.

• The cluster for the Unity Catalog integration must always be running for Immuta to audit activity and present audit logs.

Starburst (Trino) limitations

• Audit for the columns accessed in the query is not currently supported for Starburst, but is coming soon.

• Audit for unauthorized access is not currently supported.

• Audit including the user’s entitlements is not currently supported.

Unity Catalog native query audit brings in audit information for all tables and data sources, so some audit logs are created from activity by users not registered in Immuta. These audit records will appear in Immuta, providing valuable information of activity, with the username Unknown. This can be seen on the audit page or in user and data activity dashboards.

Identify users

While the Immuta user is unknown, the user's Databricks Unity Catalog username can be found within the audit log. To view the user's data platform username:

1. Navigate to the event page.

2. Select View JSON.

3. The username can be found in the auditPayload.technologyContext.account.username field.

Register users

To improve your future audit records, ensure these users are properly registered and can be named in the logs:

1. If you have not registered any users, pull in users from your IAM.

2. If you have registered users but this user was missed, manually create the Immuta user.

3. If this user is in Immuta but not appearing in the audit record, map the user's Databricks username into Immuta.

Use these audit export configuration commands to manage exporting your audit logs to S3 and ADLS Gen2. To configure an audit export see the Export to S3 or Export to ADLS guides.

Disable a configuration

To disable a configuration, use the disableExportConfiguration mutation:

mutation {
  disableExportConfiguration(
    id: "<export configuration ID>"
  )
  {
    id
  }
}

"data": {
  "disableExportConfiguration": {
    "id": "<export configuration ID>"
  }
}

Enable a configuration

To enable a disabled configuration, use the enableExportConfiguration mutation:

mutation {
  enableExportConfiguration(
    id: "<export configuration ID>"
  )
  {
    id
  }
}

"data": {
  "enableExportConfiguration": {
    "id": "<export configuration ID>"
  }
}

Delete a configuration

To delete a configuration, use the deleteExportConfiguration mutation:

mutation {
  deleteExportConfiguration(
    id: "<export configuration ID>"
  )
  {
    id
  }
}

"data": {
  "deleteExportConfiguration": {
    "id": "<export configuration ID>"
  }
}

In addition to the executed Spark plan, the tables, and the tables' underlying paths for every audited Spark job, Immuta captures the code or query that triggers the Spark plan. Immuta audits the activity of Immuta users on Immuta data sources.

Requirements

• Databricks Spark integration with native query audit enabled

• Databricks tables registered as Immuta data sources

• Databricks users registered as Immuta users: Note that the users' Databricks usernames must be mapped to Immuta. Without this, Immuta will not know the users are Immuta users and will not collect audit events for their data access activity.

Audit schema

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Example queryText

Below is an example of the queryText, which contains the full notebook cell (since the query was the result of a notebook). If the query had been from a JDBC connection, the queryText would contain the full SQL query.

testTable = 'default.crime_data_delta'
testDb = 'test'

df = spark.table(testTable)
df.limit(1).collect()

filteredDf = df.filter('victim_age > 20')

filteredDf.write.saveAsTable('{}.audit_cell'.format(testDb))
spark.table('{}.audit_cell'.format(testDb)).limit(1).collect()

spark.sql('DROP TABLE IF EXISTS {}.audit_cell'.format(testDb))

This notebook cell had multiple audit records associated with it.

Example audit record

{
  "action": "QUERY",
  "actor": {
    "type": "USER_ACTOR",
    "name": "Taylor",
    "id": "taylor@immuta.com",
    "identityProvider": "okta",
    "impersonatedBy": null
  },
  "sessionId": "abc123456589",
  "actionStatus": "SUCCESS",
  "actionStatusReason": null,
  "actorIp": "1.2.3.4",
  "eventTimestamp": "2022-10-13T20:03:41.013Z",
  "id": "abc123",
  "customerId": "abc123",
  "targetType": "DATASOURCE",
  "targets": [{
    "id": "4",
    "name": "Movies",
    "technology": "DATABRICKS"
  }],
  "auditPayload": {
    "type": "QueryAuditPayload",
    "queryId": "81fe4385-1329-444a-b6d9-b26bce5c8dc7",
    "query": "Project [director#778904]\n+- Filter ((YEAR#778903L = 1999) OR (YEAR#778903L = 2000))\n   +- Relation[movie_id#778901L,Title#778902,Year#778903L,Director#778904,Budget_million#778905,Gross_worldwide#778906L] parquet\n",
    "startTime": "2022-10-13T20:03:41.013Z",
    "endTime": null,
    "duration": null,
    "accessControls": {
      "entitlements": {
        "groups": [],
        "attributes": []
      },
      "policySet": [{
        "type": "SUBSCRIPTION",
        "global": false,
        "subscriptionPolicyType": "MANUAL",
        "ruleAppliedForUser": true
      }]
    },
    "technologyContext": {
      "type": "DatabricksContext",
      "clusterId": "1006-194110-8j0shd5d",
      "clusterName": "databricks-cluster-name",
      "workspaceId": "123456789",
      "pathUris": [
        "dbfs:/user/hive/warehouse/your_database.db/movies"
      ],
      "metastoreTables": ["your_database.movies"],
      "queryLanguage": "python",
      "queryText": "query_success = []\nnum_queries_run = 0\nimpersonate_probability = .20\nspark.sql(\"set immuta.impersonate.user=\")\n\ndef make_fail_query(query):\n  try:\n    spark.sql(\"set immuta.impersonate.user=taylor@databricks.com\")\n    spark.sql(query).toPandas()\n  except: \n    pass\n  \nfor index, query in enumerate(new_queries.values):\n  if(num_queries_run % 100 == 0):\n    print(f\"Queries Successfully Ran: {num_queries_run}/2000, out of total queries ran: {index+1}\")\n  to_impersonate = random.randrange(100)\n  if to_impersonate < impersonate_probability * 100:\n    make_fail_query(query)\n    spark.sql(\"set immuta.impersonate.user=\")\n    num_queries_run += 1\n  else:\n    try:\n      spark.sql(query).toPandas()\n      query_success.append((query, True))\n      num_queries_run += 1\n      if num_queries_run == 2000:\n        break\n    except Exception as e:\n      query_success.append((query, False))\n      \n    ",
      "immutaPluginVersion": "2022.3.0-spark-3.1.1"
    }
  },
  "receivedTimestamp": "2022-10-13T20:03:41.044Z"
}

Enriched Databricks audit logs

Beyond raw audit events (such as “John Doe queried Table X in Databricks"), the Databricks audit records include the policy information enforced during the query execution, even if a query was denied.

Queries will be denied if at least one of the conditions below is true:

• User does not meet policy conditions.

• User is not subscribed to the data source.

• Data source is not in the user's current project.

• Data source is in the user's current project, but the user is not subscribed to the data source.

• Data source is not registered in Immuta.

User entitlements

The user's entitlements represent the state at the time of the query. This includes the following fields:

Policy information

The policySet includes the following fields:

Starburst (Trino) query audit logs is a feature that audits queries that users run natively in Starburst (Trino) and presents them in a universal format as Immuta audit logs. Users can view audit records for queries made in Starburst (Trino) against Immuta data sources on the audit page. Immuta audits the activity of Immuta users on Immuta data sources.

Requirements

• Starburst (Trino) integration with the Starburst or Trino plugin version 443 or newer, or Trino 435 with the Immuta Trino 435.1 plugin

• Starburst (Trino) tables registered as Immuta data sources

• Starburst (Trino) users registered as Immuta users: Note that the users' Starburst (Trino) usernames must be mapped to Immuta. Without this, Immuta will not know the users are Immuta users and will not collect audit events for their data access activity.

Audit schema

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Example audit record

{
  "id": "20240221_200952_00200_qhadw",
  "action": "QUERY",
  "actionStatus": "SUCCESS",
  "actor": {
    "name": "Taylor",
    "id": "taylor@starburst.com",
    "identityProvider": "bim",
    "profileId": "13",
    "type": "USER_ACTOR"
  },
  "tenantId": "your-immuta.com",
  "targetType": "DATASOURCE",
  "targets": [
    {
      "type": "DATASOURCE",
      "id": "35",
      "name": "Tiny Lineitem",
      "technology": "STARBURST_TRINO"
    },
    {
      "type": "DATASOURCE",
      "id": "33",
      "name": "Tiny Orders",
      "technology": "STARBURST_TRINO"
    }
  ],
  "relatedResources": [],
  "auditPayload": {
    "type": "QueryAuditPayload",
    "version": 1,
    "queryId": "20240417_193813_00005_67mum",
    "query": "select c.name, o.clerk from tpch.tiny.customer c join tpch.tiny.orders o on c.custkey = o.custkey limit 10",
    "startTime": "2024-04-17T19:38:13.651Z",
    "endTime": "2024-04-17T19:38:14.068Z",
    "duration": 417.00005531311035,
    "objectsAccessed": [
      {
        "name": "\"tpch\".\"tiny\".\"customer\"",
        "datasourceId": "17",
        "databaseName": "tpch",
        "schemaName": "tiny",
        "type": "LOGICAL_TABLE",
        "columns": [
          {
            "name": "custkey",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "NONSENSITIVE"
              }
            },
            "inferred": true
          },
          {
            "name": "name",
            "tags": [
              {
                "type": "TAG",
                "name": "Discovered.Entity.Person Name",
                "id": "104",
                "source": "curated",
                "context": "manual",
                "deleted": false,
                "transient": false
              },
              {
                "type": "TAG",
                "name": "DSF.Control.Personal",
                "id": "116",
                "source": "curated",
                "context": "manual",
                "deleted": false,
                "transient": false
              },
              {
                "type": "TAG",
                "name": "RAF.Confidentiality.Medium",
                "id": "21882",
                "source": "curated",
                "context": "framework",
                "deleted": false,
                "transient": false,
                "framework": {
                  "id": "9144acdd-f92e-4324-bcd4-d22e49dd895e",
                  "version": "d37671bc-58ef-906f-19b0-daf7ce32e878",
                  "name": "Risk Assessment Framework",
                  "measures": {
                    "sensitivity": "SENSITIVE"
                  }
                }
              }
            ],
            "securityProfile": {
              "sensitivity": {
                "score": "SENSITIVE"
              }
            },
            "inferred": true
          }
        ],
        "tags": [],
        "securityProfile": {
          "sensitivity": {
            "score": "SENSITIVE"
          }
        },
        "directlyReferenced": true
      },
      {
        "name": "\"tpch\".\"tiny\".\"orders\"",
        "datasourceId": "13",
        "databaseName": "tpch",
        "schemaName": "tiny",
        "type": "LOGICAL_TABLE",
        "columns": [
          {
            "name": "clerk",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "NONSENSITIVE"
              }
            },
            "inferred": true
          },
          {
            "name": "custkey",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "NONSENSITIVE"
              }
            },
            "inferred": true
          }
        ],
        "tags": [],
        "securityProfile": {
          "sensitivity": {
            "score": "NONSENSITIVE"
          }
        },
        "directlyReferenced": true
      }
    ],
    "technologyContext": {
      "trinoUsername": "taylor@immuta.com",
      "immutaPluginVersion": "444",
      "rowsProduced": 10,
      "type": "TrinoContext"
    }
  },
  "eventTimestamp": "2024-02-21T20:09:52.000Z",
  "receivedTimestamp": "2024-02-21T20:09:53.599Z"
}

Limitations

• objectsAccessed is not available with Hive or Iceberg views.

• columnsAccessed will include columns related to the query that were not actually accessed in some cases:

  • For row access policies that rely on a column in the queried table, even if that column was not a part of the query, it will be included in the columnsAccessed.

  • For conditional masking, if the policy protects a column accessed, then the conditional column will be included in the columnsAccessed.

Snowflake query audit logs is a feature that audits queries that users run natively in Snowflake and presents them in a universal format as Immuta audit logs. Immuta uses the Snowflake QUERY_HISTORY and ACCESS_HISTORY tables and translates them into the audit logs that can be viewed at query events in the Immuta UI or exported to S3. Immuta audits the activity of Immuta users on Immuta data sources.

Requirements

• Snowflake Enterprise Edition or higher

• Snowflake integration with native query audit enabled

• Snowflake tables registered as Immuta data sources

• Snowflake users registered as Immuta users: Note that the users' Snowflake usernames must be mapped to Immuta. Without this, Immuta will not know the users are Immuta users and will not collect audit events for their data access activity.

Audit frequency

Immuta collects audit records at the frequency configured when enabling the integration, which is between 1 and 24 hours. The frequency is a global setting based on integration type, so organizations with multiple Snowflake integrations will have the same audit frequency for all of them. The more frequent the audit records are ingested, the more current the audit records; however, there could be performance and cost impacts from the frequent jobs.

To manually request native query audit ingestion, click Load Audit Events on the Immuta audit page.

Audit schema

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Example audit record

{
  "action": "QUERY",
  "actor": {
    "type": "USER_ACTOR",
    "name": "Taylor",
    "id": "taylor@snowflake.com",
    "identityProvider": "bim",
    "impersonatedBy": null
  },
  "sessionId": "18245308848957358",
  "actionStatus": "SUCCESS",
  "actionStatusReason": null,
  "eventTimestamp": "2023-03-21T13:39:45.040598-04:00",
  "id": "abcd",
  "userAgent": "Snowflake Web App",
  "tenantId": "your-immuta.com",
  "targetType": "DATASOURCE",
  "targets": [
    {
      "type": "DATASOURCE",
      "id": "2034",
      "name": "University Art Gallery Exhibition",
      "technology": "SNOWFLAKE"
    }
  ],
  "relatedResources": [],
  "auditPayload": {
    "type": "QueryAuditPayload",
    "queryId": "01a9c8f5-0602-eeb3-0040-d203014c166a",
    "query": "SELECT T2.theme ,  T1.date ,  T1.attendance FROM \"DETECT_DEMO_DB\".\"UNIVERSITY_ART_GALLERY\".\"EXHIBITION_RECORD\" AS T1 JOIN \"DETECT_DEMO_DB\".\"UNIVERSITY_ART_GALLERY\".\"EXHIBITION\" AS T2 ON T1.exhibition_id  =  T2.exhibition_id WHERE T2.year  =  2004",
    "startTime": "2023-03-21T13:39:45.040598-04:00",
    "endTime": "2023-03-21T13:05:07.040598-04:00",
    "duration": 163,
    "errorCode": null,
    "technologyContext": {
      "type": "SnowflakeContext",
      "host": "example.snowflakecomputing.com",
      "clientIp": "xxx.xx.xx.xx",
      "snowflakeUsername": "taylor@snowflake.com",
      "rowsProduced": 3,
      "roleName": "ACCOUNTADMIN",
      "warehouseId": "3",
      "warehouseName": "DETECT_WH",
      "clusterNumber": 1
    },
    "objectsAccessed": [
      {
        "name": "DB.PUBLIC.CASE",
        "datasourceId": "3",
        "databaseName": "DB",
        "schemaName": "PUBLIC",
        "type": "TABLE",
        "columns": [
          {
            "name": "DOB",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "COUNTRY",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "DEPARTMENT",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "FIRSTNAME",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "SSN",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "ID",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "COMPANY",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          },
          {
            "name": "LASTNAME",
            "tags": [],
            "securityProfile": {
              "sensitivity": {
                "score": "INDETERMINATE"
              }
            },
            "inferred": false
          }
        ],
        "tags": [],
        "securityProfile": {
          "sensitivity": {
            "score": "INDETERMINATE"
          }
        }
      }
    ],
    "securityProfile": {
      "sensitivity": {
        "score": "NONSENSITIVE"
      }
    }
  },
  "receivedTimestamp": "2023-03-22T13:22:04.979644-04:00"
}

Native query audit for Databricks Unity Catalog captures user data access within Unity Catalog and presents them in a universal format as Immuta audit logs. Multiple access options are supported for audit:

• Cluster queries with the following supported languages: SQL, Scala, Python, and R.

• SQL warehouse queries

Immuta audits the activity of all Unity Catalog users and tables.

Requirement

Audit frequency

Immuta collects audit records at the frequency , which is between 1 and 24 hours. The frequency is a global setting based on integration type, so organizations with multiple Unity Catalog integrations will have the same audit frequency for all of them. The more frequent the audit records are ingested, the more current the audit records; however, there could be performance and cost impacts from the frequent jobs. Immuta will start a Databricks cluster to complete the audit ingest job if one is not already running.

To manually prompt the native query audit, click Load Audit Events on the Immuta audit page.

Audit schema

Each audit message from the Immuta platform will be a one-line JSON object containing the properties listed below.

Example audit record

Limitations

• Enrichment of audit logs with Immuta entitlements information is not supported. While you will see these entitlements in the Databricks Spark audit logs, the following will not be in the native query audit for Unity Catalog:

  • Immuta policies information

  • User attributes

  • Groups

• Immuta determines unauthorized events based on error messages within Unity Catalog records. When the error messages contain expected language, unauthorized events will be available for native query audit for Unity Catalog. In other cases, it is not possible to determine the cause of an error.

• Audit for cluster queries do not support UNAUTHORIZED status. If a cluster query is unauthorized, it will show FAILURE.

• Data source information will be provided when available:

  • For some queries, Databricks Unity Catalog does not report the target data source for the data access operation. In these cases the activity is audited, yet the audit record in Immuta will not include the target data source information.

  • Data source information is not available for unauthorized queries and events.

• Column information from the query is not currently supported.

Universal audit model (UAM) is Immuta's consistent structure for all Immuta system and user query audit logs. This reference guide provides example schemas of all the UAM events available in Immuta.

There are some parameter details throughout to help better understand the UAM schemas. But there are two important parameters to each event:

• targetType: Informs the Immuta object that's the target of the action being audited. This will specify if it was a user, project, policy, etc. being affected by the action.

• action: Informs the base action being performed on the target. This will specify if something was created, deleted, updated, etc.

To learn more about Immuta's audit, see the UAM reference page or view the examples below.

Events and descriptions

ApiKeyCreated event

• Event: ApiKeyCreated

• Legacy event: apiKey

• Description: An audit event for when an API key is created on the Immuta app settings page or from an Immuta user's profile page.

{
    "relatedResources": [],
    "auditPayload": {
        "type": "ApiKeyCreatedAuditPayload",
        "apiKeyId": "1112158",
        "name": "T1",
        "version": 1
    },
    "action": "CREATE",
    "tenantId": "your-immuta-tenant.com",
    "actionStatus": "SUCCESS",
    "eventTimestamp": "2024-01-25T18:04:58.368Z",
    "actor": {
        "id": "taylor@immuta.com",
        "identityProvider": "bim",
        "profileId": "1",
        "type": "USER_ACTOR",
        "name": "Taylor Smith"
    },
    "targetType": "APIKEY",
    "id": "d9dc3cee-98d0-47d6-ba81-e0b38f9f4014",
    "receivedTimestamp": "2024-01-25T18:04:58.505Z",
    "actorIp": "xxx.xx.xx.xx",
    "targets": [],
    "requestId": "60c68659-ac83-5299-bf3e-14856178a0de",
    "sessionId": "9c553d7ace0aa3ee735fd3c14f737bc6"
}

ApiKeyDeleted event

• Event: ApiKeyDeleted

• Legacy event: apiKey

• Description: An audit event for when an API key is deleted on the Immuta app settings page or from an Immuta user's profile page.

{
    "id": "fe47ddd1-fd22-4dcf-9648-83950a10ef34",
    "targetType": "APIKEY",
    "receivedTimestamp": "2024-01-25T18:08:28.891Z",
    "actor": {
        "type": "USER_ACTOR",
        "identityProvider": "bim",
        "id": "taylor@immuta.com",
        "profileId": "1",
        "name": "Taylor Smith"
    },
    "requestId": "220c7faa-7f56-5d8c-aa31-8ac2212b7707",
    "sessionId": "9c553d7ace0aa3ee735fd3c14f737bc6",
    "targets": [],
    "actorIp": "xxx.xx.xx.xx",
    "relatedResources": [],
    "auditPayload": {
        "type": "ApiKeyDeletedAuditPayload",
        "apiKeyId": "1112158",
        "version": 1
    },
    "eventTimestamp": "2024-01-25T18:08:21.319Z",
    "action": "DELETE",
    "actionStatus": "SUCCESS",
    "tenantId": "your-immuta-tenant.com"
}

AttributeApplied event

• Event: AttributeApplied

• Legacy events: accessUser and accessGroup

• Description: An audit event for an attribute applied to a group or user.

• Additional parameter details: targetType will specify whether the attribute was added to a USER or GROUP.

{
    "actorIp": "xxx.xx.xx.xx",
    "targets": [
        {
            "name": "Deepu Murty",
            "id": "deepu@immuta.com",
            "identityProvider": "bim",
            "profileId": "20",
            "type": "USER"
        }
    ],
    "requestId": "e45652cd-4d00-5def-8e8c-aca696822fc2",
    "sessionId": "cdbffff8804103418350947c6586712c",
    "actor": {
        "type": "USER_ACTOR",
        "profileId": "1",
        "id": "taylor@immuta.com",
        "identityProvider": "bim",
        "name": "Taylor Smith"
    },
    "id": "63f111a7-0835-4696-8fdb-188130c44fac",
    "targetType": "USER",
    "receivedTimestamp": "2023-09-13T14:36:02.926Z",
    "action": "ATTRIBUTE_APPLY",
    "tenantId": "your-immuta-tenant.com",
    "actionStatus": "SUCCESS",
    "eventTimestamp": "2023-09-13T14:36:02.688Z",
    "relatedResources": [
        {
            "values": ["Product"],
            "name": "Department",
            "id": "department",
            "type": "ATTRIBUTE"
        }
    ],
    "auditPayload": {
        "entityType": "USER",
        "type": "AttributeAppliedAuditPayload",
        "version": 1,
        "attributes": [
            {
                "values": ["Product"],
                "attribute": "Department"
            }
        ],
        "entityIdProvider": "bim",
        "entityId": "deepu@immuta.com"
    }
}