Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Immuta allows you to automate discovering and tagging data across your data platform. Tagging is critical for two reasons:
It allows you to define data sensitivity, which in turn allows you to monitor where you have potential data security issues and gaps in your security posture.
It allows you to abstract your physical structure from your access policy logic. For example, you can build access policies like mask all columns tagged Person Name
(where Person Name
was auto-tagged by Discover) rather than much less scalable policies that must be knowledgeable of your physical layers like mask column x in database y in data platform z
.
Today’s sensitive data discovery tools give you a shallow overview of your data corpus across a long list of platforms. They give you pointers on where you have sensitive data without the granularity to drive your column- or row-level access controls. They help you understand what data you possess according to a regulatory framework, like HIPAA or PCI but without the details needed to automate your audits or compliance reporting. Knowing that you need to drive east to west on a road map from New York to California is helpful but ultimately insufficient to get you from a specific location to another.
Existing tools promise a high degree of automation, yet their many false positives result in painful manual work that never stops. Although data gets scanned automatically, performance breaks down at scale, or you manually need to fine-tune the computing resources of the scanners. Last but not least, your security team objects to the agent-based processing that requires taking data out of your data platform, and the associated data residency concerns may give you pause.
At Immuta, we believe that data security should not be painful. We believe that you can innovate and move quickly, while at the same time protecting your data and adhering to your internal policies and external regulations. Technology and automation allow you to make the right trade-off decisions quickly. It all starts with highly accurate and actionable metadata. If you trust your metadata and if it’s actionable, you can leverage it to automatically grant access to data, mask sensitive information, and automate your audit reporting.
Immuta Discover was built to tackle those challenges and address them through a unique architecture that was designed in collaboration with the largest financial institutions, healthcare companies, and government agencies in the world. The cloud and AI paradigm requires a fundamentally different approach. You must assume that your data is dynamic, unique, and collected in a multitude of different geographies and legal jurisdictions. Immuta Discover is built for this new world and its specific demands.
Identifying and classifying data requires analyzing and looking at the data - there’s no way around it. Immuta Discover does all the analysis and processing inside the remote technology. It takes advantage of those platforms’ inherent scalability to enable you to analyze large amounts of data quickly, efficiently, and without the need for separate resource optimization for containers or virtual machines.
By processing data directly inside the data platform, Immuta Discover automatically adheres to data residency and locality requirements. If you run your data warehouse or lake globally - across North America, the European Union, and Asia - Immuta processes the data in the region where your data is stored. No data ever leaves the data platform, and it will never move across different cloud regions.
In-platform processing greatly reduces risk and improves your data security posture. Provisioning agents, whether they’re in a container, virtual machine, or Amazon Machine Image (AMI), create complexity and an unnecessary security risk. Not only can those agents become compromised, but their misconfiguration might lead to data leaks to other parts of your cloud infrastructure. An agentless approach can better leverage data platform optimizations to process data instead of transferring it out to re-optimize and analyze. This simplifies operations and increases efficiency for your infrastructure teams.
The advantages of in-platform processing are abundant, but implementing it across a multitude of platforms is challenging. Immuta helps bypass the obstacles by doing all the heavy lifting for you and building in specific implementations for each technology. Although all those implementations are ultimately different, Immuta abstracts the results to one standardized taxonomy, so you can have consistently accurate and granular metadata across all your data stores.
Immuta Discover classifies data on a column level and instantaneously identifies schema changes. Only with that level of granularity and automation can you adhere to your audit requirements and understand what actions have been taken on your data. For example, if non-sensitive data is joined with sensitive data at query time, Immuta Discover will monitor and record that for your review. Continuous schema monitoring ensures schema changes never result in holes in your access controls and data security posture.
Trust in your metadata is critical for data security.
To unblock your data consumers, you need to automate your data access controls; this requires trusting that your classification and metadata are accurate and actionable. Immuta Discover provides you with highly accurate metadata and tags out-of-the-box and assists you in fine-tuning the classification mechanism to deal with false positives quickly. That enables you to build policies that dynamically grant or restrict access to protected data (like PHI or PII) depending on who is accessing it and what protections you want to apply.
Immuta Discover works in three phases: identification, categorization, and classification.
Identification: In this first phase, data is identified by its kind – for example, a name or an age. This identification can be manually performed, externally provided by a catalog, or automatically determined by Immuta Discover through column-level analysis of patterns.
Categorization: In the second phase, data is categorized in the context of where it appears, subject to your active frameworks. For example, a record occurring in a clinical context containing both a name and individual health data is protected health information (PHI) under HIPAA.
This categorization of data helps to understand the context it is in, including information like whether or not a record pertains to an individual, the composition and kinds of identifiers present, the data subject, whether the data belongs to any controlled data categories under certain legislation, etc.
Classification: In the third and final phase, data is classified according to its sensitivity level (e.g., Customer Financial Data is Highly Sensitive) and the risk associated to the data subject. Detect dashboards support 3 sensitivity levels. However, customers are free to customize the sensitivity names for the tags as needed.
Immuta can scan your data sources and apply relevant tags when data is recognized. This eliminates a manual tagging process for your data, saving you time and providing standard taxonomy across all your data sources.
Registered Snowflake, Databricks, Redshift, or Starburst (Trino) data sources
Immuta permission GOVERNANCE
Sensitive data discovery (SDD) is an Immuta Discover feature that identifies your data sources and applies relevant tags when data is recognized. This eliminates a manual tagging process for your data, saving you time and providing standard taxonomy across all your data sources.
To learn more, see the Data discovery page.
Enable sensitive data discovery on your tenant. Opt to have SDD run automatically for new data sources by setting a global framework, or run SDD granularly by applying data sources to specific frameworks.
For additional control, create your own identifiers to recognize the data that matters to you. Add these identifiers to new frameworks and specify the data sources that need this framework. This fine-level control creates automatic tagging that is relevant and accurate to your data, requiring fewer manual adjustments to the resulting tags.
Customize SDD for your data:
If you have any tags that are applied to your data sources by SDD that you don't want, you can easily disable these tags for each data source. This ensures that they will not be applied to the data source again if identification is re-run.
Reference pages:
Immuta comes with a default framework containing built-in Discovered tags and built-in identifiers. These identifiers and tags can be used in your own frameworks.
Classification is an Immuta Discover feature that categorizes your data based on the content and the associated risk the data poses. This increases your understanding of your data and allows you to make faster decisions about it.
To create or manage a framework using the Immuta API, see the Frameworks API reference page.
If you have any tags that are applied to your data sources by classification that you don't want, you can easily disable these tags for each data source. This ensures that they will not be applied to the data source again when classification is re-run.
Sensitive data discovery (SDD) is an Immuta feature that uses data patterns to determine what type of data your column represents. Using identification frameworks and identifiers, Immuta evaluates your data and can assign the appropriate tags to your data dictionary based on what it finds. This saves the time of identifying your data manually and provides the benefit of a standard taxonomy across all your data sources in Immuta.
Sensitive data discovery is supported for from the following technologies:
or
: Sensitive data discovery for Starburst (Trino) is currently in public preview and available to all accounts. Reach out to your Immuta representative to enable it on your tenant.
: Sensitive data discovery for Redshift is currently in private preview and available to all accounts. Reach out to your Immuta representative to enable it on your tenant.
To evaluate your data, SDD generates a SQL query using the identification framework's identifiers; the Immuta system account then executes that query in the remote technology. Immuta receives the query result, containing the column name and the matching identifiers but no raw data values. These results are then used to apply the resulting tags to the appropriate columns.
This evaluating and tagging process occurs when identification runs and happens automatically from the following events, if a global framework is set:
A new data source is created.
Schema monitoring is enabled, and a new data source is detected.
The following actions will also trigger identification:
Column detection is enabled, and new columns are detected. Here, SDD will only run on new columns, and no existing tags will be removed or changed. Note, this will use the identification framework that already ran on the data source.
A user manually triggers it from the data source health check menu. Note, this will use the identification framework that already applies to the data source or the global framework, if set.
A user manually triggers it from the identification frameworks page.
A user manually triggers it through the API.
An identification framework is a group of identifiers that will look for particular criteria and tag any columns where those conditions are met.
Each organization can set a global framework to apply to all the data sources in Immuta by default unless they have a different framework assigned. It is labeled on the frameworks page with a globe icon. If a global framework is set, identification will run on all new data sources. If a global framework is not set, identification will only run on data sources manually applied to an identification framework.
An identifier is a criteria and the tags to apply to data that matches the criteria. When Immuta recognizes that criteria, it can tag the data to describe the type.
Improved identifiers
If you are interested in these improved identifiers, reach out to your Immuta support professional.
Criteria are the conditions that need to be met for resulting tags to be applied to data.
SDD only supports regular expressions (regex) written in RE2 syntax.
Regex: This criteria contains a case-insensitive regular expression that searches for matches against column values.
Dictionary: This criteria contains a list of words and phrases to match against column values.
Column name: This criteria includes a case-insensitive regular expression matched against column names, not against the values in the column. The identifier's tags will be applied to the column where the name is found. Multiple column name identifiers can match a column and be applied.
The amount of time it takes to run identification on a data source depends on several factors:
Columns: The time to run identification grows nearly linearly with the number of text columns in the data source.
Row count: Performance of identification may vary depending on the sampling method used by each technology. For Snowflake, the number of rows has little impact on the time because data sampling has near-constant performance.
Views: Performance on views is limited by the performance of the query that defines the view.
*Two built-in patterns support and match based on additional data types:
DATE
: Columns will match this identifier if they are string and the regex matches or if the data type is date, date+time, or timestamp.
TIME
: Columns will match this identifier if they are string and the regex matches or if the data type is time. Note that if the date is included in the data, it will not match this identifier.
Immuta compiles dictionary patterns into a regex that is sent in the body of a query.
Redshift Spectrum is not supported with SDD.
The Redshift cluster must be up and running for SDD to successfully run.
The username and password auth method is fully supported with SDD.
AWS access key is supported with limitations with SDD:
redshift-data:BatchExecuteStatement
redshift-data:CancelStatement
redshift-data:DescribeStatement
redshift-data:ExecuteStatement
redshift-data:GetStatementResult
redshift-data:ListStatements
The AWS access key used to register the data source must have redshift:GetClusterCredentials
for the cluster, user, and database that they onboard their data sources with.
Redshift Serverless data sources are not supported for native SDD with the AWS access key authentication method.
These limitations are only relevant to users who have previously enabled and run Immuta SDD.
Immuta has improved the performance and behavior of sensitive data discovery (SDD), so references to two types of SDD can be found in the product:
Legacy SDD was available before October 2023. It is no longer available, but some users may still see the term "legacy SDD" in the context of their data tags.
Native SDD was released to Snowflake and Databricks in May 2023. It was released to Starburst (Trino) and Redshift in April 2024. Native SDD is the only type of SDD available. It is now just referred to as SDD.
If you had legacy SDD enabled, running native SDD can result in different tags being applied because native SDD is more accurate and has fewer false positives than legacy SDD. Running a new SDD scan against a table will change the context of the resulting tags, but no Discovered tags previously applied by legacy SDD will be removed.
Private preview: This feature is only available to select accounts.
Identifiers in domains allows you to use the same domains you already organize your data in to hold identifiers and run sensitive data discovery (SDD) without having to use identification frameworks. See the for more information about the feature and limitations.
Identifiers can be added and SDD can be run in any of your current domains. However, if you are not already using domains, set up a domain specifically to run SDD:
.
.
.
.
Navigate to the Identifiers tab of your domain.
Click Get Started.
Add reference identifiers to your domain that are relevant to your data by clicking the checkboxes. Note: When added to your domain, the identifier is a point-in-time copy of the reference identifier. It has the same name, pattern, and tags.
Click Add Identifiers.