Skip to content

Detect Overview

Immuta Detect is a tool that monitors your data environment and provides analytic dashboards in the Immuta UI based on your data use. These dashboards offer visualizations of audit events, including user queries and when Discover classification is enabled, the sensitivity of those queries, data sources, and columns. It can work within your current Immuta integration.

Immuta Detect continually monitors your data environment to help answer questions about your most active data users, the most accessed data, and the events happening within your data environment. Detect can provide even more value with Discover classification enabled to answer questions about the sensitive data accessed by your users and the tables that contain sensitive data. Because of this information, your organization can do the following:

  • Meet compliance requirements more effectively
  • Quickly decide what data access is allowed for what purposes
  • Reduce the effort and time to respond to auditors about data access in your company
  • Reduce the effort of classifying data within the scope of security or regulatory compliance frameworks

Detect Data Sources Dashboard

Recommended: Use Discover classification when using a Snowflake integration.

You have the option to use Immuta Detect on its own or, if you are using a Snowflake integration, to enable Discover to classify your data. There are benefits to both, but for the fullest functionality, greatest value, and best experience, it is recommended to enable and tune classification.

Detect with Discover

Only available with Snowflake integrations.

Benefits

  • Dashboards with data activity patterns for data sources and users
  • Dynamic query sensitivity on joined tables calculate sensitivity based on the columns queried and their toxicity when joined
  • Dashboards to help users find the most recently accessed data sources and active columns

Architecture

Immuta Detect uses several features of the Immuta platform to create user-friendly dashboards that are always available in the UI and do not need to be generated like Immuta reports. These dashboards are created by combining Snowflake audit events from registered users and the sensitivity of your data. Audit information and events are gathered from the Snowflake ACCOUNT_USAGE views into Immuta Detect. Additionally, Immuta Discover calculates the sensitivity of your data using Immuta built-in frameworks: the Data Security Framework and Risk Assessment Framework, which find sensitive data on a column-by-column basis using tags applied by SDD. Once Immuta does this work behind the scenes, users with the AUDIT permission will see dashboards that show the sensitive data within your organization’s data environment and what users are accessing that data.

Sensitivity

With Discover classification enabled, Immuta qualifies both columns and queries as the following sensitivity types in the dashboards:

  • Highly sensitive: Includes data that can cause severe harm or loss with inappropriate access or misuse.
  • Sensitive: Includes personal data and data that could cause harm or loss with inappropriate access or misuse.
  • Non-sensitive: Includes publicly available information or data that would not typically cause harm or loss if disclosed.
  • Indeterminate: The sensitivity of the data is unknown. Immuta deems sensitivity indeterminate because of an error in the query or because the sensitive data discovery (SDD) or classification has not completed processing at the time the query was run.

How does Immuta determine column sensitivity?

Column sensitivity is determined by the classification tags applied to the columns by the frameworks. The classification tags contain sensitivity metadata.

How does Immuta determine query sensitivity?

For queries that read from a single table, query sensitivity is determined by the column with the highest sensitivity in the query

For a query that joins tables, Immuta uses the same classification rules applied to tables and applies those rules to columns of the query. Immuta applies a new set of classification tags to the query columns and calculates sensitivity for the query event in the audit record. These query classification tags are not included on the tables' data dictionary.

Detect without Discover classification

Benefits

  • Quicker and easier onboarding experience
  • Dashboards with data activity patterns for data sources and users
  • Dashboards to help users find the most recently accessed data sources and active columns

Architecture

Immuta Detect uses several features of the Immuta platform to create user-friendly dashboards that are always available in the UI and do not need to be generated like Immuta reports. These dashboards are created from audit information and events gathered from Snowflake, Databricks Spark, and Databricks Unity Catalog into Immuta Detect. Immuta pulls audit information from Snowflake and Databricks Spark for data sources and users registered in Immuta; for Databricks Unity Catalog, Immuta pulls in audit information for all users and tables. Users with the AUDIT permission will see dashboards that show the data events within your organization’s data environment and what users are accessing that data.

Detect features

User, data, and audit dashboards

Immuta Detect provides at-a-glance dashboards to monitor change in user activity, data access, and security posture.

Historical audit ingestion

Only available with Snowflake integrations.

Immuta will start ingesting audit events when any integration is configured. For Snowflake specifically, Immuta supports historical audit ingestion which adds additional query audit events.

The Immuta Detect historical audit ingestion is a feature that pulls in past query audit information from Snowflake. You can ingest up to a year's worth of historical audit information, but the time is configurable. Once you confirm classification tags are working well against your organization’s data, security, and compliance needs, you and your Immuta representative may enable the historical audit collection of Snowflake query events. The tags applied at the time of the audit will be applied to all of the historical queries that are synced. This historical audit data cannot be changed after, so ensure that the tags applied at the time are accurate.

If you are not interested in the sensitivity of your data, you can ingest the historical audit information without any classification tags. However, this action will make the historical information permanently unclassified. If you decide to enable classification and start viewing sensitivity in the future, it will only be available from that point forward; the historical information cannot be updated.

Tag mutability for historical audit ingestion

While you can migrate between classification on and off, your audit data will always have the classification from when it is ingested. Once you enable classification, all your data from that day forward will have the classification tags, but any audit data from before will remain unclassified. Conversely, if you decide to turn off classification, the classification tags will stay on the past audit data, but moving forward the data will be unclassified.

Limitations

  • Organizations with multiple integrations should configure and run the historical audit for them at the same time. If you run historical audit for just one of your integrations, then try to run it for another separately, there will be a gap of historical data for the months you pulled in for your first integration.

Universal audit model (UAM)

Immuta’s universal audit model (UAM) and export features allow you to export your audit logs to S3 to monitor services and data pipelines and process structured audit logs in your log data processors and tools. This capability fosters convenient integrations with log monitoring services and data pipelines.

With UAM, you can specify an S3 bucket destination where Immuta will periodically export audit logs. The events captured are only events relevant to user and system actions that affect Immuta or the integrated data platforms, such as creating data sources and running queries.

Limitations

  • The Detect dashboard shows near real-time events for Immuta events, such as login, policy changes, and data platform policy changes. Query events are ingested from Snowflake and Databricks once a day, but you can manually trigger an immediate query retrieval by using the ↻Native Query Audit button on the Audit page or the Load Audit Events button on the Audit page. To update your automatic query retrieval, edit your integration.

    The most recent query history that is available to Immuta Detect depends on the underlying data platform latency. For example, there is 15 minutes of latency between an executed query and recording the event on the Snowflake data platform side.

  • Detect does not support any policy push integrations, such as the Snowflake integration without governance features.

  • Detect with Databricks Spark and Databricks Unity Catalog does not support using Discover classification to determine query sensitivity at this time.