Universal Audit Model (UAM)

Immuta’s universal audit model (UAM) provides audit logs with a consistent structure for query, authentication, policy, project, and tag events from your Immuta users and data sources. You can view the information in these UAM audit logs on the audit dashboards or export the full audit logs to S3 and ADLS for long-term backup and processing with log data processors and tools. This capability fosters convenient integrations with log monitoring services and data pipelines.

You can specify an S3 bucket destination where Immuta will periodically export audit logs when using S3. When using ADLS, you can specify the container destination where Immuta will export audit logs. If desired, users can configure both export options to export their audit logs to S3 and ADLS simultaneously.

The events captured are events relevant to user and system actions that affect Immuta or the integrated data platforms, such as creating policies or data sources and running queries.

See a list of the events captured and example schemas on the UAM schema reference guide.

Immuta audit service

The Immuta audit service is an independent microservice that captures audit events from Immuta and queries run against your Snowflake, Databricks Spark, Databricks Unity Catalog, or Starburst (Trino) integration.

Immuta stores the export endpoints you provide during configuration, retrieves the audit records pushed to the audit service by your integration, and manages the audit exports based on an export schedule you define. These audit records are also stored to support future reporting and user interface enhancements that will allow you to search based on keywords and facets easily across the entire body of audit events.

Audit export workflow

Public preview: This feature is available to all accounts.

  1. When you configure the audit export using the CLI for S3 and ADLS, the audit service stores the export endpoint you provided.

  2. After the integration endpoint has been configured, the export scheduler will run on the schedule you defined in your configuration.

  3. When users query data and the event is audited, the audit service receives events from your Snowflake, Databricks Spark, Databricks Unity Catalog, or Starburst (Trino) integration.

  4. Immuta exports the audit logs to your configured S3 bucket or ADLS container.

Audit support for platform queries

The table below outlines what information is included in the query audit logs for each integration where query audit is supported.

Snowflake
Databricks Spark
Databricks Unity Catalog
Starburst (Trino)

Table and user coverage

Registered data sources and users

Registered data sources and users

All tables and users

Registered data sources and users

Object queried

Columns returned

Rows returned

Query text

Unauthorized information

Legend:

  • This is available and the information is included in audit logs.

  • This is not available and the information is not included in audit logs.

Limitations

The audit service does not capture system-level logging and debugging information, such as 404 errors.

Snowflake query audit limitation

Snowflake query audit events from a query using cached results will show 0 for the rowsProduced field.

Unity Catalog query audit limitations

  • Immuta determines unauthorized events based on error messages within Unity Catalog records. When the error messages contain expected language, unauthorized events will be available for Databricks Unity Catalog audit logs. In other cases, it is not possible to determine the cause of an error.

  • Immuta audit records include unregistered data sources and users; however, activity from them will not appear in any governance reports.

  • Databricks queries where the response is being served from cache will not be represented in the system.access.column_lineage table which means the query audit records will be unmapped in Immuta.

Starburst (Trino) limitation

  • Audit for unauthorized access is not currently supported.

  • objectsAccessed is not available with Hive or Iceberg views.

  • columnsAccessed will include columns related to the query that were not actually accessed in some cases:

    • For row access policies that rely on a column in the queried table, even if that column was not a part of the query, it will be included in the columnsAccessed.

    • For conditional masking, if the policy protects a column accessed, then the conditional column will be included in the columnsAccessed.

Last updated

Was this helpful?