Public preview
This feature is available to all accounts. Contact your Immuta representative to enable this feature.
Requirements:
Immuta permission AUDIT
Monitors feature enabled
Navigate to Detect in the navigation menu.
Click Create Monitor.
Enter a Name for the monitor.
Choose what to monitor in the dropdown menu:
When User Accessed Any Data Source: This monitors user activity for all data sources in Immuta.
When User Accessed Data Source in Schema: This monitors user activity for just the data sources within the schema you enter.
When User Accessed Specific Data Source: This monitors user activity for the specific data source you enter.
Create conditions for the monitor to further scope user activities:
Query Duration: This scopes the monitor to consider the duration of the query. Enter the Query Duration in seconds.
Tag: This scopes the monitor to consider queries whose event contexts include all of the selected tags. The query must be associated with all specified tags in any combination of queried column tags, queried classification tags, and queried table tags. For more information, see the query event context concept.
Query Outcome: This scopes the monitor to consider the queries' results as successful, unauthorized, or failure. You can select Unauthorized or Failed to create a monitor that can notify you when a registered Immuta user has exceeded the configurable threshold for unauthorized or failed queries. This condition only works with the User Query Count metric scoped to When User Accessed Any Data Source.
Sensitivity: This scopes the monitors to only consider queries that are classified as sensitive or highly sensitive. This condition should only be used if classification has been configured.
All conditions must be satisfied for the query to be considered by the monitor.
Select Next to configure rules.
Select the Timeframe from the dropdown menu to specify the time range the threshold cannot be exceeded within.
Choose what kind of user activity metric to monitor in the metric dropdown menu:
Number of Rows Accessed: This monitors for the quantity of rows the user accessed and can be combined with additional conditions on tags and sensitivity. The exact number of rows is configured in the severity thresholds.
User Query Count: This monitors the number of queries the user made and can be combined with additional conditions on tags, sensitivity, and query outcome. The exact number of queries is configured in the severity thresholds.
Select at least one of the Severity Thresholds to set thresholds for the configured user activity metric. An observation will be created and assigned the matching severity when the metric exceeds the threshold.
Click Next to show the notifications configuration.
Choose the frequency of the notifications to webhooks when an observation is created:
Never: You can review observations in the Immuta application, and Immuta will not send webhook notifications when observations are made.
Notify each time an Observation is generated: Every time the monitor creates an observation, a webhook notification will be sent.
Notify the first time an Observation is generated for each user: Every time the monitor creates an observation, a webhook notification will be sent for the first observation about a user. You will not receive notifications for observations from the monitor again for previously notified observations about the same user. New observations about users that were previously notified can be reviewed in the Immuta UI.
Select a webhook from the dropdown menu or opt to create a new webhook.
Choose the severity you want notifications for. This will send out webhook notifications only for the severity threshold that you select.
Click Next and review the monitor selections.
Click Create Monitor.
Public preview
This feature is available to all accounts. Contact your Immuta representative to enable this feature.
Monitors allow you to gain awareness of when your users' behavior changes, maintain data availability through data platform policy changes, ensure access patterns remain consistent for your data controls to remain effective, and be notified about anomalies.
This guide illustrates how to create a monitor to generate user activity metrics based on query events.
This guide explains the design and requirements for monitors and the contexts in which to use them.
Public preview
This feature is available to all accounts. Contact your Immuta representative to enable this feature.
Immuta Detect monitors automates manual aggregation and calculation of user activity metrics based on query events. It can then notify you when the metrics exceed your intended operating thresholds. When a user's query activity metric exceeds a configured threshold within a timeframe, Immuta creates an observation that links the user's query audit events that contributed to the breached activity metric. You can specify monitor conditions that work with the information in the query event context to compute each user's activity metrics. Monitors can be configured with multiple thresholds to assign an observation severity which allows you to control the severity of observations Immuta Detect sends a webhook notification for.
Create monitors to gain awareness of when your users' behavior changes, maintain data availability through data platform policy changes, ensure access patterns remain consistent for your data controls to remain effective, and be notified about anomalies. See the following example use cases:
Change management: Configure a monitor to notify you when unauthorized or failed queries to production data sources exceed a baseline threshold, indicating potential data outage from a recent subscription policy change.
Know when a new user has gained access to sensitive data: Configure a monitor to notify the first time a user has accessed any data source that's classified sensitive or highly-sensitive and tagged specifically within a controlled set of data sources.
Find high-volume data activity: Use Detect to review typical query count over a supported timeframe, and configure a monitor for when a user has issued more than the typical number of queries.
Surface forbidden data combinations: For data sources or columns that must never be joined in a query, configure a monitor with query tag conditions to notify you when forbidden queries may have occurred.
Track data platform cost and experience deviations: Configure a monitor to watch for long-running queries that dominate resource or need optimization.
Immuta permission AUDIT
Detect monitors can be created for data sources from the following integrations:
The query event context includes a set of metadata that is used by Detect monitors to compute each user's query activity metrics. Query audit events in Immuta can include metadata such as the sensitivity classification, query execution outcome, queried column tag, queried table tag, data source name, and schema. The sensitivity classification is set if Immuta Discover was configured at the time of query audit processing. For example, the sensitivity measures emitted by Discover's Data Security Framework rules that inspect adjacent columns work in Detect to help you assess sensitivity of each query.
You can review the full context of a query audit event in the Query detail page in Detect.
Monitors are organization-defined thresholds for access to your organization's data by users. Within monitors, an organization can define what user actions should trigger an observation. If a user activity metric exceeds the configured severity thresholds in the specified timeframe, Immuta creates an observation.
Monitors can be scoped to individual data sources, all the data sources within a schema, or all registered data sources in Immuta.
Additionally, monitors can selectively assess user activity based on matching conditions in the query event context. You can create monitors for
when a user accesses more than a set number of rows of data with specific tags or sensitivity.
when a user has made more than a set number of queries of data with specific tags or sensitivity.
when a user has made more than a set number of queries, each over a specific query duration.
when a user has more than a set number of unauthorized or failed queries against any data source in Immuta.
All specified conditions must be satisfied for the query event to contribute to the user activity metric. For example, you can specify both highly sensitivity and a specific tag for the event to count towards the selected user activity metric.
Query duration details
The query duration option for monitors provides notifications for long-running queries:
When monitoring the user query count metric, each query must meet the query duration condition to be included in a user's query count. When the user query count metric crosses the monitor thresholds, an observation will be created.
When monitoring rows accessed, each query must meet the query duration condition for that query row count to be included in the user's rows accessed metric. When the user's rows accessed metric crosses the monitor thresholds, an observation will be created.
Best practice: Ensure the audit frequency for each integration is set to a lowest value possible for your organization when using monitors. For Snowflake and Databricks Unity Catalog integrations, frequent audit record ingestion will lead to highly current audit records; however, there could be performance and cost impacts from the frequent jobs.
Monitors create observations when a user's query activity metrics exceed the specified threshold within the configured timeframe. Evaluating whether the user activity metrics have exceeded monitor thresholds is performed using a sliding window algorithm in increments of 10 minutes aligned to query events' timestamps.
Only completed queries are evaluated for the monitors. Once they are audited, they then fall into the timeframe window for the time that they complete.
An observation is based on the results of monitors and a user's query activity metric. Each observation can be marked open or acknowledged for review tracking purposes.
Observations surface the user query activity being monitored once the acceptable threshold has been exceeded. When creating a monitor, you can choose to be notified about the observations in one of three ways:
Never: Every time the monitor is exceeded, an observation will be created in Immuta, but you will not receive any webhook events from them.
Notify each time an Observation is generated: Every time the monitor is exceeded, it will create an observation, and a webhook event will notify you of that creation.
Notify the first time an Observation is generated for each user: Every time the monitor is exceeded, it will create an observation, and a webhook event will notify you when the first observation is created for a user. You will not receive notifications for observations from that monitor again for that specific user.
Observations contain the following information:
Open or acknowledged label: A quick visual of the state of the observation.
Severity label: This label aligns with the severity specified by the threshold within the monitor.
The user name: The user who exceeded the severity thresholds of the monitor.
Message: An Immuta-generated message describing what created the observation.
Monitor type: The type of monitor that created the observation.
Related events: A list of audit events that created the observation with links to the event details page for each.
Users can use the webhooks configured when creating a monitor to send notifications to outside applications. Applications like Slack and Teams can receive these webhooks and surface them as alerts in the app. The notification contents include the Immuta profile ID for the user that caused the observation, the severity of the observation, a short message about what prompted the observation, and a link to look at the observation in Immuta for additional details. Note that the notification does not show any personally identifiable information about your users. The Immuta profile ID is an integer that represents a user profile in Immuta, but any additional information about the user must be accessed in the Immuta UI.
Due to a Snowflake limitation, a monitor for unauthorized query outcome cannot be combined with specific tags or sensitivities. It must be created for all data sources and cannot be scoped to a schema or specific data source. Successful query outcomes can be combined with specific tags or sensitivities and scoped to a specific schema or data source.