Identifiers in Domains

Sensitive data discovery (SDD) runs identifiers to discover data. These identifiers are grouped into domains with data sources. Each identifier contains a single criteria and the tags that will be applied when the criteria's conditions have been met.

Identifier

There are two types of identifiers in Immuta:

1. Reference identifiers: These identifiers are a library of the identifiers that can be added to domains. When added to a domain, reference identifiers are copied over and become domain-specific identifiers.

   1. Immuta comes with built-in identifiers to discover common categories of data. These cannot be modified or deleted.

   2. Data governors can create their own reference identifiers for use within your organization.

2. Domain-specific identifiers: These identifiers only exist within a specific domain and are checked against the data sources in that domain when SDD runs.

   1. Users with the Manage Identifiers permission can create these identifiers or add them to a domain from a reference identifier.

   2. If a domain-specific identifier was copied over from a reference identifier, there is no lineage and any edits to the reference identifier will not be reflected in the domain-specific copy.

Criteria

Criteria are the conditions in an identifier that need to be met for resulting tags to be applied to data.

SDD only supports regular expressions (regex) written in RE2 syntax.

Supported criteria types for identifiers

• Competitive criteria analysis: This criteria is a process that will review all the regex and dictionary criteria within the identifiers of the domain and search for the identifier with the best fit. In this review, each competitive criteria analysis identifier in the domain competes against each other to find the best and most specific identifier that fits the data. The resulting tags for the best identifier are then applied to the column. Only one competitive criteria analysis identifier for each domain will apply per column. Competitive criteria identifiers, both built-in and custom, must match at least 90% of the data sampled. To learn more about the competitive nature, see the How competitive criteria analysis works guide.

  • Regex: This criteria contains a case-insensitive regular expression that searches for matches against column values.

  • Dictionary: This criteria contains a list of words and phrases to match against column values.

• Column name: This criteria includes a case-insensitive regular expression matched against column names, not against the values in the column. The identifier's tags will be applied to the column where the name is found. Multiple column name identifiers can match a column and be applied.

Create a new identifier in the Immuta UI or with the sdd/classifier endpoint.

What has changed with SDD when using this new feature?

If you used SDD prior to this feature release in January 2025, there are some differences:

1. There are now two types of identifiers:

   1. Reference identifiers

   2. Domain identifiers

   See information about these in the Identifiers section.

2. There is a new permission to manage identifiers within domains: Manage Identifiers. The permission allows you to do the following:

   1. Create an identifier within your domain

   2. View the reference identifiers in Immuta

   3. Add, edit, and delete identifiers within your domain

3. The following have been removed:

   1. Identification frameworks: Previously, all identifiers had to be contained within a framework and that framework had to be assigned to a data source to run. Now, identifiers are added to domains with data sources.

   2. Global framework: Previously, a global framework could be set to run SDD automatically on all new data sources. This behavior cannot be achieved with identifiers in domains.

Limitations

See the table below for information on when SDD runs with the SDD feature before vs after with identifiers in domains.