When following this use case, you need to decide which path you fall in. This decision drives how you will manage user and data metadata as well as policies, so it's a critical decision.

If you aren’t sure, you should strive for ABAC. While it may seem more complicated to get started, in the long run it will provide you powerful flexibility and scalability of policy management. In this method, you tag your users and data with facts and prescribe policies that leverage those facts to make real-time decisions.

Orchestrated RBAC puts more strain on managing access decisions outside of your access logic (Immuta) because you need all access decisions in a single attribute on the user. Because of this, it more closely resembles the role explosion problem, and if you incorrectly select this path you will end up there over time. Orchestrated RBAC is tag-orchestrated RBAC and is supported by Immuta (in fact, many customers stick to this because of the benefits of the tag-orchestration).

Orchestrated RBAC: one-to-one

Do you have many permutations of static access with no overlap?

This method is for organizations whose access decision typically depends on a single variable:

If you have x, you have access to everything tagged y.

Furthermore, the access decision to objects tagged y never strays beyond x. In other words, there’s only ever one way to get access to objects tagged y - a 1:1 relationship. A good real-world example of orchestrated RBAC is

You must have signed data use agreement x to have access to data y.

ABAC: many-to-many

Do you have a lot of variables at play to determine access?

This method is for organizations that may have many differing x’s for access to objects y. Furthermore, x and y may actually be many different variables, such as

If you have a, b, and c you get access to objects tagged x, y, and z.

or

If you have d, you get access to objects tagged x.

Notice in this example that access to objects tagged x can happen through different decisions.

ABAC is also important when you have federated governance in play, where many different people are making policy logic decisions and that logic may stack on the same objects. A good real world example of ABAC is

you must reside in the US and be a full time employee to see data tagged US and Highly Sensitive.

Immuta's Secure module allows you to secure your data through various access control policies you configure. These policies can mitigate the compliance and security findings from Immuta Discover and Detect.

Getting started

This section provides use cases to guide you through implementing Immuta Secure.

Introduction

This section provides a conceptual overview of Immuta Secure and its benefits.

Authoring policies in Secure

This section provides how-to and references guides for authoring policies to enforce data access controls.

Domains

The how-to and references guides in this section illustrate how to use domains. Domains are containers of data sources that allow you to assign data ownership and access management to specific business units, subject matter experts, or teams at the nexus of cross-functional groups.

Projects and purpose-based access control

Projects combine users and data sources under a common purpose, which can then be used to restrict access to data and streamline collaboration.

Data consumers

The guides in this section illustrate how to subscribe to data sources in Immuta, run health jobs, and complete other actions as a data source subscriber.

Immuta's Secure module allows you to secure your data through various access control policies you configure. These policies can mitigate the compliance and security findings from Immuta Discover and Detect, so it is recommended that you follow the Immuta Detect getting started guide before diving into Immuta Secure.

Prerequisites

• Data platform integration configured in Immuta

• Users and data sources have been registered in Immuta

Use cases

Immuta Secure allows you to build complex access control policies in a simple, scalable manner. However, there are many different ways customers think about access control, so Secure is extremely flexible. Because of this, Immuta has broken up Secure into common use cases that customers follow to speed up their onboarding process. Choose the use case below that best fits your goals. If none of them fit, contact your customer support representative for a more personalized onboarding experience:

• Automate data access control decisions: This is the most common use case. It walks you through how to build table access control policies in a scalable manner and clarifies how to think about table access and adjust existing paradigms.

• Compliantly open more sensitive data for ML and analytics: This is an approach where every user has access to every table, yet you mask the sensitive columns appropriately using data policies. You can apply some of the concepts from the automate data access control decisions use case to how you think about masking policy rules.

• Federated governance for data mesh and self-serve data access: Data mesh is a concept where you delegate operational control of data products to data producers across your organization. This requires delegation of policy management as well, which is the goal of this use case. This use case also covers a generic data shopping experience where users can discover and subscribe to data.

The use cases are a starting point for your Secure journey - you don’t necessarily need to follow them exactly, but they should give you the conceptual framework for thinking through your own use cases, so it is encouraged to read them all.

Who is this for?

This guide is intended for users who want to build table access control policies in a scalable manner using Immuta.

Prerequisite

It's best to start with the Monitor and secure sensitive data platform query activity use case prior to this use case because it takes you through the basic configuration of Immuta.

Goals

This use case is the most common across customers. With it, you solve the problem of entitlement of users to data based on metadata about the user (attributes and groups) and metadata about the data (tags).

This use case is unique to Immuta, because rather than coupling an access decision with a role, you are able to instead decouple access decisions from user and data metadata. This is powerful because it means when metadata about the users or metadata about the data changes, the access for a user or set of users may also change - it is a dynamic decision.

Decoupling access decisions from metadata also eliminates the classic problem of role explosion. In a world where policy decisions are coupled to a role, you must manage a new role for every permutation of access, causing an explosion of roles to manage. Instead, with this use case you decouple that logic from the user metadata and data metadata, so real-time, dynamic decisions are possible. Immuta’s method of building policies allows for these many permutations in a clear, concise manner through the decoupling of policy logic from the user and data metadata.

This use case also eliminates the need to have a human in-the-loop for approval to data access. If you can describe clearly why the approver would approve an access request - that can instead be expressed as an Immuta subscription policy, as you’ll see below. Removing humans from this process increases the speed to access data, makes access decisions consistent, and removes error and favoritism.

Want to learn more? Check out this ABAC 101 blog on the methodology.

Table vs column access

Lastly, this use case is primarily focused on automating table grants, termed subscription policies in Immuta. We recommend you also read the Compliantly open more sensitive data for ML and analytics use case to learn about column masking, which will allow you to enforce more granular controls so that you can open more data. It's common to see customers mix the automate data access control decisions use case with the compliantly open more sensitive data for ML and analytics use case, so it's recommended to read both.

Business value

Following this use case will reap huge operational cost savings. You will have to manage far fewer data policies; those policies will be more granular, accurate, and easily authored; and you will be able to prove compliance more easily. Furthermore, instead of an explosion of incomprehensible roles, the users and data will have meaningful metadata matched with clear policies.

Quantified benefits:

• 75x fewer policy changes required to accomplish the same use cases

• 70% reduction in dedicated resources to data access management and policy administration

• Onboarding new employees two weeks faster and provisioning new data one week faster

• 5% to 15% increase in client contract values to improve revenue

Unquantified benefits:

• Improved compliance standard and security posture

• Enhanced employee satisfaction

• Better customer experiences

More details on the business value can be found in these reports:

• GIGAOM: ABAC vs RBAC: The Advantage of Attribute-Based Access Control over Role-Based Access Control (Immuta is an OT-ABAC approach)

• Forrester: The Total Economic Impact Of Immuta

Configuration steps

Follow these steps to learn more about and start using Immuta Secure to automate data access control decisions:

1. Complete the Monitor and secure sensitive data platform query activity use case.

2. Read the overview of the two access control paths: orchestrated RBAC and ABAC. These are the two different approaches (or mix) you can take to managing policy in Secure and their tradeoffs.

3. Manage user metadata. This step is critical to building scalable policy and understanding the considerations around how and what to capture. Tag your users with attributes and groups that are meaningful for Immuta global policies.

4. Manage data metadata. This is the final setup step you must complete before authoring policy. Tag your columns with tags that are meaningful.

5. Author policy. In this step, you will define your global data policy logic. Optionally test and deploy policy.

Who is this for?

This guide is intended for users who want to open more data for access by creating more granular and powerful policies at the data layer.

Prerequisite

It's best to start with the use case prior to this use case because it takes you through the basic configuration of Immuta.

Goals

Firstly, it's crucial to remember that just because a subscription policy, as described in the , grants a user access to data, it doesn’t mean that they should have access to all of that data. Often, organizations stop at just granting access without considering the nuances of what specific columns or rows should be accessible to different users. It's important to see the process all the way through by masking sensitive values that are not necessary for a user's role. This ensures that while users have the data access they need, sensitive information is appropriately protected.

Secondly, when considering in the context of , an interesting perspective emerges. A subscription policy could essentially be seen as mirroring the functionality of a global masking policy. This is because, like a global masking policy, a subscription policy can be used to mask or redact the entirety of a table. This interpretation underscores the potential of global data policies for comprehensive data protection.

One of the primary advantages is an easy and maintainable way to manage data leak risks without impeding data access, which means more data for ML and analytics. By focusing on global data policies, organizations can ensure that sensitive data, down to the row and column level, is appropriately protected, regardless of who has access to it. This means that while data remains broadly accessible for business operations and decision-making, the risk of data leaks is significantly reduced. This is because you can

• be more specific with your policies as described above and

• mask using advanced that allow you to get utility from data in a column while still preserving privacy in that same column.

However, it's important to note that this approach does not mean that you should never create subscription policies. Subscription policies still have their place in data governance. The key point here is that the primary focus shifts away from subscription policies and towards global data policies, which offer a more comprehensive and effective approach to data protection. This shift in focus allows for more nuanced control over data access, enhancing both data security and compliance.

When is this appropriate?

This use case is particularly suitable in scenarios where you already have a process for granting access to tables. If your organization has established procedures for table access that are working effectively, introducing can enhance your data governance without disrupting existing workflows.

It's also fitting when you grant access to tables to everyone. In such cases, the focus is less on who has access and more on what they can access. Global data policies can help ensure that while data is broadly accessible, sensitive information is appropriately masked or redacted, maintaining compliance and security.

In essence, this use case is appropriate when you want to maintain or improve data accessibility while ensuring robust data protection, regardless of your current table grants.

When is this not appropriate?

Data sensitivity

If the existence of certain tables, schemas, or columns is considered sensitive information within your organization, this solution pattern may not be appropriate. Revealing the existence of certain data, even without granting access to the actual data, can pose a security risk in some contexts. In such cases, a more restrictive strategy may be required.

Data navigation

With this use case, users might have to navigate through a large number of tables to find the data they need. This could potentially hinder user experience, especially in large organizations with extensive data environments.

Configuration steps

Follow these steps to learn more about and start using Immuta Secure to compliantly open more sensitive data for ML and analytics:

You may have read the already. If so you are aware of the you must choose between: orchestrated-RBAC vs ABAC. To manage user metadata with this particular use case, you should use ABAC.

This is because you must know the contents and sensitivity of every column in your data ecosystem to follow this use case. With orchestrated RBAC, you tag your columns with access logic baked in. ABAC means you tag your columns with facts: what is in the column. It is feasible to do the latter, extremely hard to do the former (unless you use described in the next topic), especially in a data ecosystem with constant change. This means that your users will need to have facts about them that drive policy decisions (ABAC) rather than single variables that drive access (as in orchestrated-RBAC).

Understanding that, please read the ABAC section in the automate data access control decisions use case's guide.

Prerequisites

using either of the Detect use cases:

• use case (Snowflake only)

• use case (if not using Snowflake)

Data tags for Detect

You should have already done some data tagging while configuring Immuta in the . That guide focuses on understanding where compliance issues may exist in your data platform and may not have fully covered all tags required for policy. Please read on to see if there's more work to be done with data tagging.

Considerations

• Orchestrated RBAC method: tag data sources at the table level

• ABAC method: tag data at the table and column level

While it is possible to target policies using both table- and column-level tags, for ABAC it’s more common to target column tags because they represent more granularly what is in the table. Just like user metadata needs to be facts about your users, the data metadata must be facts about the data. The tags on your tables should not contain any policy logic.

Fact-based column tags are descriptive (recommended):

• Column ssn has column tag social security number

• Column f_name has column tag name

• Column dob has column tags date and date of birth

Logic-based column tags requires subjective decisions (not recommended):

• Column ssn has column tag PII

• Column f_name has column tag sensitive

• Column dob has column tag indirect identifier

Entity tags are facts about the contents of individual columns in isolation. Entity tags are what we listed above: social security number, name, date, and data of birth. Entity tags do not attempt to contextualize column contents with neighboring columns' contents. Instead, categorization and classification tags describe the sensitive contents of a table with the context of all its columns, which is what is listed in the logic-based tags above, things like PII, sensitive, and indirect identifier.

For example, under the HIPAA framework a list of procedures a doctor performed is only considered protected health information (PHI) if it can be associated with the identity of patients. Since entity tagging operates on a single column-by-column basis, it can’t reason whether or not a column containing procedure codes merits classification as PHI. Therefore, entity tagging will not tag procedure codes as PHI. But categorization tagging will tag it PHI if it detects patient identity information in the other columns of the table.

Additionally, entity tagging does not indicate how sensitive the data is, but categorization tags carry a sensitivity level, the classification tag. For example, an entity tag may identify a column that contains telephone numbers, but the entity tag alone cannot say that the column is sensitive. A phone number associated with a person may be classified as sensitive, while the publicly-listed phone number of a company might not be considered sensitive.

Contextual tags are really what you should target with policy where possible. This provides a way to create higher level objects for more scalable and generic policy. Rather than building a policy like “allow access to tables with columns tagged person name and phone number,” it would be much easier to build it like “allow access to tables with columns tagged PII.”

In short, you must tag your entities, and then rely on a classification framework (provided by Immuta or customized by you) to provide the higher level context, also as tags. Remember, the owners of the tables (those who created them) can tag the data with facts about what is in the columns without having to understand the higher level implications of those tags (categorization and classification). This allows better separation of duty.

For orchestrated-RBAC, the data tags are no longer facts about your data, they are instead a single variable that determines access. As such, they should be table-level tags (which also improves the amount of processing Immuta must do).

Applying data tags

Data tag hierarchy

As a quick example, it is possible to tag your data with Cars and then also tag that same data with more specific tags (in the hierarchy) such as Cars.Nissan.Xterra. Then, when you build policies, you could allow access to tables tagged Cars to administrators, but only those tagged Cars.Nissan.Xterra to suv_inspectors. This will result in two separate policies landing on the same table, and the beauty of Immuta is that it will handle the conflict of those two separate policies. This provides a large amount of scalability because you have to manage far fewer policies.

Imagine if you didn’t have this capability? You would have to include administrators access to every policy you created for the different vehicle makes - and if that policy needed to evolve, such as adding more than administrators to all cars, it would be an enormous effort to make that change. With Immuta, it’s one policy change.

No matter if you choose orchestrated RBAC or ABAC as described in the , you must have metadata on your users. This can be done , via your , or other .

Doing so allows you to build scalable policies that do not reference individual users and instead use metadata about them to target them with access decisions. In Immuta, user metadata is termed user attributes and groups.

Considerations

Referring back to our triangle of access decision, user metadata is the first point of that triangle. User metadata is made up of the following elements:

• Attributes: key:value pairs tied to your users

• Groups: a single value tied to your users - you can think of groups as containing users. Groups can also have their own attributes, a shortcut for assigning attributes to all members of a group. In either case, multiple users can be attached to the same attribute or group.

Fact-based user metadata (ABAC) allows you to decouple policy logic from user metadata, allowing highly scalable ABAC policy authoring:

• Steve has attribute: country:USA

• Sara has attribute: role:administrator

• Stephanie has attribute: sales_region: Ohio, Michigan, Indiana

• Sam is in group: developers

• Group developers has attribute: organization:engineering

Logic-based user metadata (orchestrated-RBAC) couples user metadata with access logic:

• Steve has attribute: access_to:USA

• Sara has attribute: role:admin_functions

• Stephanie has groups: Ohio_sales, Michigan_sales, Indiana_sales

• Sam is in group: all_developer_data

• Group developer has attribute: access_to:AWS_all

The one exception where you have the above scenario but may want to use orchestrated RBAC is if these multiple paths for access to the same objects are a true hierarchy. For example, data is tagged with Strictly Confidential, Confidential, Internal, and Public:

• users assigned group Strictly Confidential can see data tagged Strictly Confidential, Confidential, Internal, and Public

• users assigned group Confidential can see data tagged Confidential, Internal, and Public;

Orchestrated RBAC: user attributes and groups

Without getting into the details of how the policy is authored (yet), it is possible to leverage a user attribute hierarchy to contribute to access decisions through hierarchical matching. For example, users with attributes that are either a hierarchical parent or an exact match to a given data tag can be subscribed to the data source via a hierarchical match. To better illustrate this behavior, see the matrix below:

So you should take care organizing your user attributes with a hierarchy that will support the hierarchical matching.

Going back to our hierarchical example of Strictly Confidential, Confidential, Internal, and Public above, you would want to ensure that user attributes follow the same hierarchy. For example,

• A user with access to all data: Classification: Strictly Confidential

• A user with access to only Internal and Public: Classification: Strictly Confidential.Confidential.Internal

Remembering the last row in Figure 3, hierarchical matching only happens in a single direction → user attribute contains the data source tag. Since the user attribute is more specific (Strictly Confidential.Confidential.Internal) than data tagged Strictly Confidential and Strictly Confidential.Confidential, the user would only gain access to data tagged Strictly Confidential.Confidential.Internal and Strictly Confidential.Confidential.Internal.Public.

ABAC: user attributes and groups

To use ABAC, you need to divorce your mind from thinking that every permutation of access requires a new role. (We use the word role interchangeably with groups in this document, because many identity and access systems call groups roles, especially when they contain policy logic.)

Instead you need to focus on the facts about your users they must possess in order to grant them access to data. Ask yourself, why should someone have access to this data? It’s easiest to come at this from the compliance perspective and literally ask yourself that question for all the compliance rules that exist in your organization.

Why should someone have access to my PHI data? Example answers might be

• If they are an analyst

• If they have taken privacy training

So, at a minimum, you are going to need to attach attributes or groups to users representing if they are an analyst and if they have taken privacy training. Note you do not have to (nor should you) negate these attributes, meaning nobody should have a group not_analyst and no_privacy_training.

Repeat this process until you have a good starting point for the user attributes you need. This is a lot of upfront work and may lead to the perception that ABAC also suffers from attribute explosion. However, when attributes are well enumerated, there’s a natural limit to how many different relevant attributes will be present in an organization. This differs substantially from the world where every permutation of access is represented with a role, where role growth starts off slow, but increases linearly over time.

As an illustration, well-defined subject attributes tend to follow the red curve over time, while roles and poorly defined subject attributes tend to follow the blue and quickly become unwieldy.

Applying user metadata

Where do those attributes and groups come from? Really, wherever you want - Immuta has several interfaces to make this work, and you can combine them:

Domains are containers of data sources that allow you to assign data ownership and access management to specific business units, subject matter experts, or teams at the nexus of cross-functional groups. Instead of centralizing your data governance and giving users too much governance over all your data, you delegate and control their power over data sources they own by granting them permission within domains in Immuta.

Domains are typically defined by your organization’s data governance board and closely aligned with business departments or units, such as Sales, Customer Service, or Finance. Domains can also be defined by value stream, such as Patient Health Records, Fraud Detection, or Quality Control.

Who will do this?

The task of setting up domains in Immuta can be done by any member of the team with the GOVERNANCE permission and assigning domain permissions can be done by any member of the team with the USER_ADMIN permission.

Consider users' jobs in a domain

When delegating control to your data product owners, it's possible to further segment that control within domains. If you want to give the user control to build policies on the data sources (the data products) in the domain, that is managed by giving them the Manage Policies permission in the domain.

Related guides

• Creating domains

• Assigning domain permissions

Immuta Secure is the final piece of the puzzle: Now that you understand where sensitive data lives (via Discover) and can monitor activity against that data (via Detect), you can now mitigate risk using Immuta Secure.

In short, Immuta Secure enables the management and delivery of trusted data at scale.

Challenge and goals

Managing access control in your data platform typically starts off easy, but over time becomes a house of cards. This concept is termed role explosion and is a result of having to keep up with every permutation of access across your organization. Once this occurs, it becomes difficult to evolve policies for fear of breaking existing access or because of a lack of understanding across your extensive role list.

Secure allows you to apply engineering principles to how you manage data access, giving your team the agility to lower time-to-data across your organization while meeting your stringent and granular compliance requirements. Immuta allows massively scalable, evolvable, and understandable automation around data policies; creates stability and repeatability around how those policies are maintained; allows distributed stewardship across your organization, but provides consistency of enforcement across your data ecosystem no matter your compute or data warehouse; and fosters more availability of data through the use of highly granular data controls.

How does it work?

Each of the guides below explains Secure principles in detail:

• Scalability and Evolvability: A scalable and evolvable data management system allows you to make changes that impact thousands of tables at once, accurately. It also allows you to evolve your policies over time with minor changes (or no changes at all) through policy logic.

• Understandability: Immuta can present policies in a natural language form that is easily understood and provide an audit history of changes to create a trust and verify environments. This allows you to prove policy is being implemented correctly to business leaders concerned with compliance and risk, and your business can meet audit obligations to external parties or customers.

• Distributed Stewardship: Immuta enables fine-grained data ownership and controls over organizational domains, allowing a data mesh environment for sharing data - embracing the ubiquity of your organization. You can enable different parts of your organization to manage their data policies in a self-serve manner without involving you in every step, and you can make data available across the organization without the need to centralize both the data and authority over the data. This frees your organization to share more data more quickly.

• Consistency: With inconsistency comes complexity, both for your team and the downstream analysts trying to read data. That complexity from inconsistency removes all value of separating policy from compute. Immuta provides complete consistency so that you can build a policy once, in a single location, and have it enforced scalably and consistently across all your data platforms.

• Availability (of data): Because of these highly granular decisions at the access control level, you can increase data access by over 50% in some cases when using Immuta because friction between compliance and data access is reduced.

Prerequisites

Your schema metadata is registered using either of the Detect use cases:

• Monitor and secure sensitive data platform query activity use case (Snowflake or Databricks)

• General Immuta configuration use case (if not using Snowflake nor Databricks).

Enriching your data metadata with tags

You may have read the Automate data access control decisions use case already. If so, you are aware of the two paths you must choose between: orchestrated-RBAC vs ABAC. To manage data metadata with this particular use case, you should use ABAC.

This is because you must know the contents and sensitivity of every column in your data ecosystem to follow this use case. With orchestrated RBAC, you tag your columns with access logic baked in. ABAC means you tag your columns with facts: what is in the column. It is feasible to do the latter, extremely hard to do the former (unless you use tag lineage, described below), especially in an data ecosystem with constant change.

Understanding that, read the Managing data metadata guide.

However, there are some considerations specific to this use case with regard to data metadata.

Automated data tagging

Automated data tagging with Immuta sensitive data discovery (SDD) is recommended with this use case because it significantly reduces the manual effort involved in classifying and managing data. It ensures that data is consistently and accurately tagged, which is crucial for implementing effective data policies. Moreover, it allows for real-time tagging of data, ensuring that new or updated data is immediately protected by the appropriate policies. This is critical when all columns need to be considered for policy immediately, which is the case with this use case.

Schema monitoring enabled

While not directly related to data tagging, schema monitoring is a feature in Immuta that allows organizations to keep track of changes in their data environments. When enabled, Immuta actively monitors the data platform to find when new tables or columns are created or deleted. It then automatically registers or disables these tables in Immuta and updates the tags. This feature ensures that any global policies set in Immuta are applied to these newly created or updated data sources. It is assumed you are using schema monitoring when also using SDD. Without this feature, data will remain in data downtime while waiting for the policies to update.

Data tags are facts

As discussed above, with ABAC your data tags need to be facts. Otherwise, a human must be involved to bake access logic into your tags. As an example, it's easy for SDD to find an address, but it's harder for a human to decide if that address should be sensitive and who should have access to it - all defined with a single tag - for every column!

If you focus on tagging with facts, and use Immuta's frameworks to build higher level logic on those tags, then you are set up to build data policies in a scalable and error proof manner with limited data downtime.

Tag lineage

There is one way you can accomplish this use case using orchestrated RBAC: lineage. Immuta's lineage feature, currently in private preview and for Snowflake only, is able to propagate tags based on transform lineage. What this means is that columns that existed in tables for which the downstream table was derived, those upstream table's tags will get carried through to that new downstream table's columns. This is powerful because you can tag the root table(s) once - with your policy logic tags, as described in Managing data metadata - and they will carry through to all downstream tables' columns without any work. Please be aware that tag lineage only works for tables; for views, the tags will not be propagated. However, policies on backing tables will be enforced on the views (which is why they are not propagated). Also be aware that if a table exists after some series of views, the tags will propagate to that table. As an example,

Now we are ready to focus on the third point of the triangle for data access decisions: access control policies, and, more specifically, how you can author them with Immuta now that you have the foundation with your data and user metadata in place.

It’s important to understand that you only need a starting point to begin onboarding data access control use cases - you do not have to have every user perfectly associated with metadata for all use cases, nor do you need all data tagged perfectly for all use cases. Start small on a focused access control use case, and grow from there.

The policy authoring approaches are broken into the two paths you should choose between discussed previously.

Path 1: Orchestrated RBAC policy authoring

With orchestrated RBAC you have established one-to-one relationships with how your users are tagged (attribute or group) and what that single tag explicitly gives them access to. Remember, the user and data metadata should be facts and not reflect policy decisions veiled in a single fact; otherwise, you are setting yourself up for role explosion and should be using ABAC instead.

Since orchestrated RBAC is all about one-to-one matching of user metadata to data metadata, Immuta has special functions in our subscription policies for managing this:

• @hasTagAsAttribute('Attribute Name', 'dataSource' | 'column')

• @hasTagAsGroup('dataSource' | 'column')

The first argument for @hasTagAsAttribute is the attribute key whose value tags will be matched against (or for @hasTagAsGroup simply the group name), and the second argument specifies whether the attribute values are matched against data source tags or column tags.

Understanding this, let’s use a simple example before we get into a hierarchical example. For this simple example, let’s say there are Strictly Confidential tables and Public Tables. Everyone has access to Public, and only special people have access to Strictly Confidential. Let’s also pretend there aren’t other decisions involved with someone getting access to Strictly Confidential (there probably is, but we’ll talk more about this in the ABAC path). It is just a single fact: you do or you don’t have that fact associated with you.

Then we could tag our users with their access under the Access attribute key:

• Bob: Access: Strictly Confidential

• Steve: Access: Public

Then we could also tag our tables with the same metadata:

• Table 1: Strictly Confidential

• Table 2: Public

Now we build a policy like so:

And ensure we target all tables with the policy in the builder:

Here is the policy definition payload YAML for the above policy using the v2 api:

actions:
  advanced: '@hasTagAsAttribute(''Access'', ''dataSource'')'
  allowDiscovery: false
  automaticSubscription: true
  shareResponsibility: false
  type: entitlements
name: Access
policyKey: Access
staged: false
template: false
type: subscription

Since this policy applies to all data sources, you only need a single policy in Immuta. When you change user metadata or data metadata, the access to those tables will automatically be updated by Immuta through the matching algorithm (user metadata matches the data metadata).

We could also use column tags instead, by tagging the columns appropriately:

• Table 1 --> Column 2: Strictly Confidential

  This tag change would require the following alteration to the function: @hasTagAsAttribute('Access', 'column') instead of @hasTagAsAttribute('Access', 'dataSource').

Finally, we could use groups instead of attributes:

• Bob: Strictly Confidential

• Steve: Public

  This change would require the following alteration to the function: @hasTagAsGroup('Access', 'column') instead of @hasTagAsAttribute('Access', 'column').

In the above case, we are using column tags again, but it could (and probably should) be data source tags. The reason is that orchestrated RBAC is not using facts about what is in your data source (the table) to determine access. Instead, it's using the single variable as a tag, and because of that, the single variable is better represented as a table tag. This also reduces the amount of processing Immuta has to do behind the scenes.

We can get more complicated with a hierarchy as well, but the approach remains the same: you need a single policy and you use the hierarchy to solve who should gain access to what by moving from most restrictive to least restrictive in your hierarchy tree on both the users and data metadata:

Groups approach:

• Bob: Strictly Confidential

• Steve: Strictly Confidential.Confidential.Internal.Public

In this case, Steve would only gain access to tables or tables with columns tagged Strictly Confidential.Confidential.Internal.Public, but Bob would have access to tables or tables with columns tagged Strictly Confidential, Strictly Confidential.Confidential, Strictly Confidential.Confidential.Internal, and Strictly Confidential.Confidential.Internal.Public.

Orchestrated RBAC one-to-one matching can be powerful, but can also be a slippery slope to role explosion. You can quickly fall into rolling-up many decisions into a single group or attribute tied to the user (like Strictly Confidential) instead of prescriptively describing the real root reason of why someone should have access to Strictly Confidential in your policy engine. This is the power behind ABAC, and using the ABAC method of access can give you ultimate scalability and evolvability of policy with ease.

Path 2: ABAC policy authoring

Let’s say that having access to Strictly Confidential data isn’t a single simple fact attached to a user like we described in the orchestrated RBAC section (it likely isn't). Instead, let’s try to boil down why you would give someone access to Strictly Confidential. Let’s say that to have access to Strictly Confidential, someone should be

• an employee (not contractor)

• in the US

• part of the Legal team

Those are clearly facts about users, more so than Strictly Confidential. If you can get to the root of the question like this, you should, because Immuta allows you to pull in that information about users to drive policy.

Why should you bother getting to the root?

Well, consider what we just learned about how to author policy with orchestrated RBAC above. Now let’s say your organization changes their mind and now wants the policy for access to Strictly Confidential to be this:

• be an employee (not contractor)

• be in the US or France

• be part of the Legal team

With orchestrated RBAC this would be a nightmare, unless you already have a process to handle it. You would have to manually figure out all the users that meet that new criteria (in France) and update them in and out of Strictly Confidential. This is why orchestrated RBAC should always use a single fact that gains a user access.

However, if you are on the ABAC path, it’s a single simple policy logic change because of our powerful triangle that decouples policy logic from user and metadata attributes.

Let’s walk through an example:

The best approach is to build individual policies for each segment of logic associated with why someone should have access to Strictly Confidential information. Coming back to the original logic above, that would mean 3 (much more understandable) separate policies:

Here is the policy definition payload YAML for the above policy using the v2 api:

actions:
  allowDiscovery: false
  automaticSubscription: true
  entitlements:
    groups:
      - Employees
    operator: all
  shareResponsibility: false
  type: entitlements
circumstanceOperator: any
circumstances:
  - columnTag: Strictly Confidential
    type: columnTags
name: Employee Access
policyKey: Employee Access
staged: false
template: false
type: subscription

Here is the policy definition payload YAML for the above policy using the v2 api:

actions:
  allowDiscovery: false
  automaticSubscription: true
  entitlements:
      attributes:
        - name: Country
          value: US
      operator: all
  shareResponsibility: false
  type: entitlements
circumstanceOperator: any
circumstances:
  - columnTag: Strictly Confidential
    type: columnTags
name: Country Access
policyKey: Country Access
staged: false
template: false
type: subscription

Here is the policy definition payload YAML for the above policy using the v2 api:

actions:
  allowDiscovery: false
  automaticSubscription: true
  entitlements:
    groups:
      - Legal Team
    operator: all
  shareResponsibility: false
  type: entitlements
circumstanceOperator: any
circumstances:
  - columnTag: Strictly Confidential
    type: columnTags
name: Legal Team Accesss
policyKey: Legal Team Accesss
staged: false
template: false
type: subscription

Each policy targets tables tagged Strictly Confidential (they could alternatively target columns tagged that):

When each of those policies converge on a single table tagged Strictly Confidential, they will be merged together appropriately by Immuta, demonstrating Immuta's conflict resolution:

And whether merged with an AND or an OR is prescribed by this setting in the policy builder for each of the 3 individual policies:

If either side of the policies are Always Required that will result in an AND; otherwise, if both sides agree to Share Responsibility it will result in an OR. In this case, each of the 3 policies chose Always Required, which is why the three policies were AND’ed together on merge. Note this merging behavior is relevant to how you can have different users manage policies without risk of conflicts.

Now let’s move into our nightmare (without Immuta) scenario where the business has changed the definition of who should have access to Strictly Confidential data by adding France along with the other existing three requirements.

With Immuta, all you’ve got to do is change that single Country Access subscription policy to include France, and voila, you’re done:

Here is the policy definition payload YAML for the above policy using the v2 api:

actions:
  allowDiscovery: false
  automaticSubscription: true
  entitlements:
    attributes:
      - name: Country
        value: US
      - name: Country
        value: France
    operator: any
  shareResponsibility: false
  type: entitlements
circumstanceOperator: any
circumstances:
  - columnTag: Strictly Confidential
    type: columnTags
name: Country Access Two
policyKey: Country Access
staged: false
template: false
type: subscription

Which results in this merged policy on all the Strictly Confidential tagged tables:

As you can see, ABAC is much more scalable and evolvable. Policy management becomes a breeze if you are able to capture facts about your users and facts about your data, and decouple policy decisions from it.

Also be aware, this auto-merging logic Immuta provides enables you to also combine orchestrated RBAC and ABAC policies.

Manual overrides

Last but not least are manual overrides. While this use case is all about automating access decisions, there may be cases where you want to give a user access even if they don’t meet the automated criteria.

This is possible through this setting in each policy:

In this scenario, users marked as Owner of the registered data source in Immuta have the power to approve this override request by the user. There are ways to specify different approvers as well, or multiple approvers.

Also notice the grayed out Allow Data Source Discovery: normally, if you do not meet the criteria of a policy, the data source is not even visible to you in the Immuta UI or the data platform. However, if you have this checked, it is visible in the Immuta UI even if you don’t meet the policy. This must be checked in order to request approval to access. Otherwise, users would never find the data source to ask for approval in the Immuta UI.

Be aware that all policies that land on the table must have this same setting turned on in order to allow the manual override.

Who is this for?

This guide is intended for users who wish to delegate control of data products within a data mesh without breaking their organization's security and compliance standards.

Prerequisite

It's best to start with the use case prior to this use case because it takes you through the basic configuration of Immuta.

Goals

This guide is designed for those interested in understanding how Immuta can effectively assist organizations in adopting core data mesh principles. By leveraging this document, readers will be empowered to implement the essential procedures for integrating Immuta within their organization's data mesh framework. It is recommended to read the ebook before starting with this use case.

Even if you are not interested in data mesh, this use case will explain how you can create a self-serve "data shopping experience" in Immuta.

Configuration steps

Follow these steps to learn more about and start using Immuta Detect and Secure to apply federated governance for data mesh and self-serve data access:

1. Complete the use case.

2. Opt to review the and use cases.

3. . This step is critical in order to delegate policy controls over certain segments of your data.

4. to allow your data product creators to compliantly release data products for consumption.

5. . This is the final setup step you must complete before authoring policy. Tag your columns with tags that are meaningful to ensure federated governance.

6. Work with your data product creators and global compliance users to . Enforce global standards across all data products while empowering data product owners to expand on those policies.

7. Allow your data consumers to self-serve access data through the Immuta data portal by .

You may have read the use case already. If so you are aware of the you must choose between: orchestrated-RBAC vs ABAC. To manage data metadata with this particular use case, you should use ABAC.

This is because you want your data product owners to tag data with facts - what they have intimate knowledge of because they built the data product - and not have to be knowledgeable about all policies across your organization. With orchestrated RBAC, you tag your columns with access logic baked in. ABAC means you tag your columns with facts, what is in the column, which is why ABAC makes much more sense.

Understanding that, read the automate data access control decisions use case's guide.

Tags in a federated governance world

It is important to distinguish between tag definition and tag application. While tag definition (e.g., a tag called “Business Unit” with the values “Finance”, “Marketing”, “Sales”) should be strongly governed to guarantee consistency and coherence, tag application can be fully decentralized, meaning every domain or data owner can apply tag values (from the centrally governed list) to their data. There needs to be a process in place for data owners to request the definition of a new tag in case they identify any gaps.

Monitoring data products

It is important to leverage Immuta Discover's sensitive data discovery (SDD) to monitor your data products. This allows you to uncover if and when sensitive data may be leaked unknowingly by a data product and mitigate that leak quickly.

The use case covers this in great detail and is highly recommended for a data mesh environment.

A data product is a valuable output derived from data. It is designed to provide insights, support decision-making, and drive business value. Data products are created to meet business needs or solve problems within a domain or organization. These products can take various forms. In most cases, it is a collection of tables and views, which is what is supported by Immuta.

Data product metadata registration

The use case covers how to register your data metadata, but it's worth discussing in more depth here because you have multiple data product owners.

As a first step, all the tables/views on the data platform(s) must be onboarded in Immuta. With the (which is the Immuta default), onboarding objects in Immuta will not make any changes to existing accesses already in place.

Immuta only starts taking control when the first policy has been applied. We suggest that one team registers all objects from your data platform(s). Alternatively, different domain owners can register their own domain data. It is recommended to use a system account to do the data registration.

Regardless of which user registered the data, it's critical that is enabled to ensure future tables and views are discovered by Immuta and automatically registered to the proper domain.

Be aware that registering your data product metadata with Immuta does not make it available to users. It simply makes Immuta aware of its existence. To allow users to see it and subscribe, you would manage policies on it, discussed in the guide next.

Who will do this?

Any team member that has the necessary user credentials in the underlying platform to read the data.

Related guide

Now that you have a sense of what policies you want to enforce, it is sometimes necessary to first test those policies before deploying them. This is important if you have existing users accessing the tables you are protecting and you want to understand the impact to those users before moving a policy to production. However, consider this step optional.

It's also important to remember that Immuta subscription policies are additive, meaning no existing SELECT grants on tables will be revoked by Immuta when a subscription policy is created. It is your responsibility to revoke all pre-Immuta SELECT grants once you are happy with Immuta's controls.

Use a single Immuta for testing policies

While it may seem wise to have a separate Immuta tenant for development and production mapped to separate development and production data platforms, that is not necessary nor recommended because there are too many variables to account for and keep up-to-date in your data platform and identity management system:

1. Many Immuta data policies are enforced with a heavy reliance on the actual data values. Take for example the following row-level policy: only show rows where user possesses an attribute in Work Location that matches the value in the column tagged Discovered.Entity.Location. This policy compares the user’s office location to the data in the column tagged Location, so if you were to test this policy against a development table with incorrect values, it is an invalid test that could lead to false positives or false negatives.

2. Similar to #1, if you are not using your real production users, just like having invalid data can get you an invalid test result, so could having invalid or mock user attributes or groups.

3. Policies can (and should) target tables using tags discovered by or (such as Alation or Collibra). For SDD, that means if using development data, the data needs to match the production data so it is discovered and tagged correctly. For external catalogs, that would mean you need your external catalog to have everything in development tagged exactly like it is in production.

4. Users can have attributes and groups from , so similar to #3, you would need to have that all synchronized correctly against your development user set as it is synchronized for your production user set.

5. Your development user set may also lack all relevant permutations of access that need to be tested (sets of attributes/groups relevant to the policy logic). These permutations are not knowable a priori because they are dependent on the policy logic. So you would have to create all permutations (or validate existing ones) every time you create a new policy.

6. Lastly, you have a development data environment to test data changes before moving to production. That means your development data environment needs the production policies in place. In other words, policies are part of what needs to be replicated consistently across development environments.

Be aware, this is not to suggest that you don’t need a development data platform environment; that could certainly be necessary for your transformation jobs/testing (which should include policy, per #6). However, for policy testing it is a bad approach to use non-prod data and non-prod users because of the complexity of replicating everything perfectly in development - by the time you’ve done all that, it matches what’s in production exactly.

Best practice for policy testing

Immuta recommends testing against clones of production data, in a separate database, using your production data platform and production user identities with a single Immuta tenant. If you believe you do have a perfectly matching development environment that covers 1-5 in the section above, you can use it (and should for #6), but we still recommend a single Immuta tenant because Immuta allows logical separation of development and production policy work without requiring physically separated Immuta tenants.

Logically separating your data platform

So how do you test policies without impacting production workloads if we are testing against production? You create a logical separation of development and production in your data platform. What this means is that rather than physically separating your data into completely separate accounts or workspaces (or whatever term your data warehouse may use), logically separate development from production using the mechanisms provided by the data platform - such as databases. This reduces the amount of replication management burden you need to undertake significantly, but does put more pressure on ensuring your policy controls are accurate - which is why you have Immuta!

Many of our customers already take this approach for development and testing in their data platform. Instead of having physically separate accounts or workspaces of their data platform, they create logical separation using databases and table/view clones within the same data platform instance. If you are already doing this, you can skip the rest of this section.

Follow the below recommendations on how to create logical separation of development data in your data platform using the following approaches:

• Redshift: Create new views that are backed by the tables in question to a different development database and register them with Immuta. Since Immuta enforces policies in Redshift using views, creating new views will not be impacted by any existing policy on the tables that back the views.

• Starburst (Trino): You should virtualize the tables as new tables to a different development catalog in Starburst (Trino) for testing the policy.

• Azure Synapse Analytics: You should virtualize the tables as new tables to a different development database in Synapse for testing the policy.

If you are managing creation of tables in an automated way, it may be prudent to automatically create the clones/views as described above as part of that process. That way you don’t have to pick and choose tables to logically separate every time you need to do policy work - they are always logically separated already (and presumably discovered and registered by Immuta because they are registered for monitoring as well).

Logically separating Immuta

If using a physically separate development data platform, ensure you have the Immuta data integration configured there as well.

If you are unable to leverage domains, we recommend adding an additional tag when the data is registered so it can be included when you describe where to target the policy. For example, tag the tables with policy 1 tag.

Ensuring development data is tagged correctly

If using SDD to tag data:

If using an external catalog:

If manually tagging:

Testing a new policy

Once the policy is activated (using either technique), it will only apply to the cloned development tables/views because the policy was scoped to that domain/tag, or because the user creating the policy is scoped to managing policies in that domain. At this point, you can invite users that will be impacted by the policy to test the change or your standard set of testers.

Once you have your testers set, you can give them all an attribute such as policy test: [policy in question] and use it to create a "subscription-testers" subscription policy. It should target only the tables/views in that domain per the steps described in the above paragraph in order to give them access to the development tables/views to test.

If you are testing a masking or row-level security policy, you don't need to do anything else; it's ready for your policy testers. However, if your ultimate goal is to test a subscription policy, you need to build it separately from the above "subscription-testers" subscription policy, but ensure that you select the Always Required option for "subscription-testers" subscription policy. That means when the "subscription-testers" subscription policy is merged with the real subscription policy you are testing, both policies will be considered and some of your test users may accurately be blocked from the table (which is part of the testing). For example, notice how these two policies (the "subscription-testers" and real policy being tested) were merged with an AND.

There's nothing stopping you from testing a set of changes (many subscription policies, many data policies) all together as well.

Once you’ve received feedback from your test users and are happy with the policy change, you can move on to the deployment of the policy. For example, add prod-domain, remove dev-domain, and then apply the policy. Or build the policy using a user that has manage policy in the production domain.

Testing an edit to an existing policy

The approach here is the same as above; however, before applying your new policy to the development domain, you should first locally disable the pre-existing global policy in the data source. This can be done by going into the development data source in the Immuta UI as the data owner and disabling the policy you plan to change in the policy tab of the data source. This must be done because the clone will have the pre-existing policy applied because it should have all matching tags.

Now you can create the new version of the policy in its place without any conflict.

Policy approvals

Sometimes you may have a requirement that multiple users approve a policy before it can be enabled for everyone. If you are in this situation, you may want to first consider doing this in git (or any source control tool) using our policy-as-code approach, so all approvals can happen in git.

In the use case, we covered at length. Subscription policies control table level access.

In this use case, we are focused one level deeper: columns and rows within a table that can be protected in a more granular manner with .

Global data policy: column masking

With Immuta, you are able to , or even mask cells within a given column per row using (also termed conditional masking).

This is important for this use case to granularly mask or grant unmasked access to specific columns to specific users. This granularity of policy allows more access to data because you no longer have to roll up coarse access decisions to the table level and can instead make them at the individual column, row, or cell level.

Using tags

The concept of masking columns has been mentioned a few times already, but as you already know per the guide, your masking policies will actually target tags instead of physical columns. This abstraction is powerful, because it allows you to build a single policy that may apply to many different columns across many different tables (or ).

Masking conflicts

If you build policy using tags, isn't there a good chance that multiple masking policies could land on the same column? Yes.

This can be avoided . Immuta supports tag hierarchy, so if the depth of a tag targeted by a policy is deeper than the conflicting policy, the deepest (the more specific one) wins. As an example, mask by making null columns tagged PII is less specific (depth of 1) than the policy mask using hasing columns tagged Discovered.name (depth of 2), so the hashing masking policy would apply to columns tagged Discovered.name and PII rather than the null one.

Principle of least privilege

Immuta meets principle of least privilege by following an . What this means is that if you mask a column, that mask will apply to everyone except [some list of exceptions]. This uni-directional approach avoids policy conflicts, makes change management easier, authoring policy less complex, and (most importantly) avoids data leaks.

Masking techniques

There are many different approaches you can take to masking a column. Some masks render the column completely useless to the querying user, such as nulling it, while other masking techniques can provide some level of utility from the column while at the same time maintaining a level of privacy and security. These advanced masking techniques are sometimes termed . Immuta provides a that allow for privacy-vs-utility trade-off decisions when authoring masking policies.

Dealing with data types

If you were to build masking policies natively in your data platform, they require that you build a masking policy per data type it could mask. This makes sense, because a varchar column type can't display numerics, or vice versa, for example. Furthermore, when building masking policies that target tags instead of physical columns, it is possible that policy may target many differing data types, or even target new unforeseen data types in the future when new columns appear.

Global data policy: row-level

You may hear this policy called row filter or row access policy. The idea is to redact rows at query time based on the user running the query.

Without this capability, you would need a transform process that segments your data across different tables and then manage access to those tables. This introduces extra compute costs and at some point, when dealing with large tables and many differing permutations of access, it may be impossible to maintain as those tables grow.

Using tags

Advanced functions

• @groupsContains('SQL Column or Expression') allows you to compare the value of a column to see if the user possesses any groups with a matching name (case sensitive).

• @attributeValuesContains('Attribute Name', 'SQL Column or Expression') allows you to compare the value of a column to see if the user possesses any attribute values under a specific key with a matching name (case sensitive).

• @purposesContains('SQL Column or Expression') allows you to compare the value of a column to see if the user is acting under a matching purpose (case sensitive).

• @columnTagged('Tag Name') allows you to target tags instead of physical columns when using one of the above functions.

Here's a simple example that targets a physical column COUNTRY comparing it to the querying user's country attribute key's values:

@attributeValuesContains('country', 'COUNTRY')

This could also be written to instead use the @columnTagged function instead of the physical column name:

@attributeValuesContains('country', @columnTagged('Discovered.Country'))

Allowing this policy to be reused across many different tables that may have the COUNTRY column name spelled differently.

Data policy temporary overrides

...for everyone except users acting under purpose [some legitimate purpose(s)]

• Data sources with the policies they want to be excluded from and

• Purposes

This can be made temporary by deleting the project once access is no longer needed or revoking approval for the project after the need for access is gone.

Subscription policies

After you've created global data policies as described above, how do you actually give access to the tables?

Data sources to users in the Immuta UI. This allows users to read the metadata associated with data sources (e.g., tags, descriptions, data dictionary, and contacts) and permits users to request access to the data itself.

This allows customers to use Immuta as a data catalog and marketplace. In case an organization uses an external marketplace, Immuta easily integrates with it via the API to bring in metadata or manage access controls for users that requested access via external marketplace.

While we have been talking about data mesh, using Immuta as a marketplace for a data shopping experience is not limited to data mesh use cases - it can be used this way for any use case.

User authentication

Immuta integrates with your identity manager in order to enforce controls in your data platform(s) - you likely configured this when completing the use case. This setup has another point, though: it also allows all your users to authenticate into Immuta with their normal credential to leverage it for this "shopping" experience.

Any action taken in Immuta will automatically be reflected in the data platform via Immuta's integration.

In the distributed domains of a data mesh architecture, data governance and access control must be applied vertically (locally) within specific domains or data products and horizontally (globally). Global policies should be authored and applied in line with the ecosystem’s most generic and all-encompassing principles, regardless of the data’s domain (e.g., mask all PII data). Localized domain- or product-level policies should be fine-grained and applicable to only context-specific purposes or use cases (e.g., only show rows in the Sales table where the country value matches the user's office location). In Immuta we distinguish between subscription policies and data policies.

Subscription policies

It is possible to build Immuta subscription policies across all data products (horizontal), as shown in the diagram above, and then have those policies merged with additional subscription policies authored at the domain level (vertical). When this occurs, those subscription policies are merged as prescribed by the two or more policies being merged.

Whether the requirements for access are merged with an AND or an OR is prescribed by this setting in the policy builder for each of the individual policies:

• Always Required = AND

• Share Responsibility = OR

Subscription policies for a data product "shopping" experience

When building subscription policies, it can impact what a user can discover and, if desired, "put in their shopping cart" to use.

• Allow Data Source Discovery: Normally, if a user does not meet the subscription policy, that data source is hidden from them in the Immuta UI. Should you check this option when building your subscription policy, the inverse is true: anyone can see this data source. This is important if you want users to understand if the data product exists, even if they don't have access.

• Require Manual Subscription: Even if the user does meet the policy, instead of automatically subscribing them, they would have to discover and subscribe themselves. If they meet the policy, they will automatically be subscribed with no intervention. This is important if you want the users to maintain the list of data products they see in the data platform rather than all data products they have access to.

• Request Approval to Access: This allows the user to request access, even if they don't meet the policy. Rules can determine what user manually overrides the policy to let them in.

Data policies

Once a user is subscribed to a data source via Immuta or has a pre-existing direct access on the underlying data platform, the data policies that are applied to that data source determine what data the user sees. Data policy types include masking, row-level, and other privacy-enhancing techniques.

An exemplary three-step approach to managing data policies would be

• Update the global data policies as new sensitive data is potentially released by data products and discovered using Immuta Detect.

Strategy

Data mesh is a higher level use case that pulls from concepts learned across the other use cases:

Example of anonymizing a column rather than blocking it

By having highly granular controls coupled with anonymization techniques, more data than ever can be at the fingertips of your analysts and data scientists (in some cases, up to 50% more).

Why is that?

Let’s start with a simple example and get more complex. Obviously, if you can’t do row- and column-level controls and are limited to only GRANTing access to tables, you are either over-sharing or under-sharing. In most cases, it’s under-sharing: there are rows and columns in that table the users can see, just not all of them, but they are blocked completely from the table.

That example was obvious, but it can get a little more complex. If you have column-level controls, now you can give them access to the table, but you can completely hide a column from a user by making all the values in it null, for example. Thus, they’ve lost all data/utility from that column, but at least they can get to the other columns.

That masked column can be more useful, though. If you hash the values in that column instead, utility is gained because the hash is consistent - you can track and group by the values, but can’t know exactly what they are.

But you can make that masked column even more useful! If you use something like k-anonymization instead of hashing, they can know many of the values, but not all of them, gaining almost complete utility from that column. As your anonymization techniques become more advanced, you gain utility from the data while preserving privacy. These are termed privacy enhancing technologies (PETs) and Immuta places them at your fingertips.

This is why advanced anonymization techniques can get significantly more data into your analysts' hands.

Using k-anonymization to mask columns

While columns like first_name, last_name, email, and social security number can certainly be directly identifying, something like gender and race, on the surface, seem like they may not be directly identifying, but it could be. Imagine if there are very few Tongan men in a data set...in fact, for the sake of this example, lets say there’s only one. So if I know of a Tongan man in that company, I can easily run a query like this and figure out that person’s salary without using their name, email, or social security number:

select salary from [table] where race = 'Tongan' and gender = 'Male';

This is the challenge with indirect identifiers. It comes down to how much your adversary, the person trying to break privacy, knows externally, which is unknowable to you. In this case, all they had to know was the person was Tongan and a man (and there happens to be only one of them in the data) to figure out their salary, sensitive information. Let's also pretend the result of that query was a salary of 106072. This is called a linkage attack and is specifically called out in privacy regulations as something you must contend with, for example, from GDPR:

Article 4(1): "Personal data" means any information relating to an identified or identifiable natural person ("data subject"); an identifiable person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that person.

Almost any useful column with many unique values will be a candidate for indirectly identifying an individual, but also be an important column for your analysis. So if you completely hide every possible indirectly identifying column, your data is left useless.

You can solve this problem with PETs. Take note of two things by querying the data:

• If you only search for “Tongan” alone (no Male), there are several Tongan women, so this linkage attack no longer works: select salary, gender from [table] where race = 'Tongan';

• There are no null values in the gender or race columns.

Now let's say you apply the k-anonymization masking policy using Immuta.

Then you run this query again to find the Tongan man's salary: select salary from immuta_fake_hr_data where race = 'Tongan' and gender = 'Male';

You get no results.

Now you run this query ignoring the gender: select salary, gender from immuta_fake_hr_data where race = 'Tongan';

Only the women are returned.

The linkage attack was successfully averted. Remember, from our queries prior to the policy, the salary was 106072, so let’s run a query with that: select race, gender from immuta_fake_hr_data where salary = 106072;

There he is! But race will be suppressed (NULL) so this linkage attack will not work. It was also smart enough to not suppress gender because that did not contribute to the attack; suppressing race alone averts the attack. This is the magic of k-anonymization: it provides as much utility as possible while preserving privacy by suppressing values that appear so infrequently (along with other values in that row) that they could lead to a linkage attack.

Cell-level security

Cell-level security is not exactly an advanced privacy enhancing technology (PET) as in the example above, but it does provide impressive granular controls within a column for common use cases.

What is cell-level security?

If you have values in a column that should sometimes be masked, but not always, that is masking at the cell-level, meaning the intersection of a row with a column. What drives whether that cell should be masked or not is some other value (or set of values) in the rest of the row shared with that column (or a joined row from another table).

For example, a user wants to mask the credit card numbers but only when the transaction amount is greater than $500. This allows you to drive masking in a highly granular manner based on other data in your tables.

This technique is also possible using Immuta, and you can leverage tags on columns to drive which column in the row should be looked at to mask the cell in question, providing further scalability.

Data mesh

If your goal is data mesh, please read the content below, but also refer to the Data mesh use case. It will help you understand how distributed stewardship aligns with additional data mesh strategies in Immuta.

Separation of policy building from data tagging

Separation of duties is a critical component of policy enforcement. An additional component to consider is also separation of understanding, where some people in your organization are much more knowledgeable about what policies must be enforced compared to the people in your organization that understand deeply what data is contained in certain tables, experts on data, so to speak.

Wouldn’t it be nice if you could rely on data experts to ensure that data is being tagged correctly, and rely on the compliance experts to ensure that policy is being authored appropriately based on requirements - separation of understanding? This is possible with Immuta.

You can have a set of users manage the tags on the data - those who know the data best - and a separate set of users to author the policies. When they author those policies, they reference tags, a semantic layer, rather than the physical tables and columns, which they don't understand.

The tags bridge the gap between the physical world and the logical world, allowing the compliance experts to build meaningful policy leveraging the knowledge of the physical world transferred into the tags.

Remember also, it is possible to automatically tag data through Immuta Discover, which further automates this process.

Data ownership vs data governance

The GOVERNANCE permission in Immuta is quite powerful, as described in our permissions section. It is for a situation where a select few users are the only ones that control all policies.

It is possible to instead delegate policy control to data owners without giving them governance permission. This allows them to write global policies just like governors, but they are restricted to only the data sources they own.

Note that this capability is further enhanced with the Immuta domains feature which is currently private preview.

Build anonymization against compute/platform 1 and n consistently

This is one of the largest challenges for organizations. Having multiple data platforms/compute, which is quite common, means that you must configure policies uniquely in each of them. For example, the way you build policies in Databricks is completely different from how you build policies in Snowflake. This becomes highly complex to manage, understand, and evolve (really hard to make changes).

Just like the big data era created the need to separate compute from storage, the privacy era requires you to separate policy from platform. Immuta does just that; it abstracts the policy definition from your many platforms, allowing you to define policy once and apply anywhere - consistently!

You should not build policies uniquely in each data platform/compute you use; this will create chaos, errors, and make the data platform team a bottleneck for getting data in the hands of analysts.

Subscription policies manage table-level access to data. The how-to and reference guides in this section detail how to implement subscription policies to meet your compliance requirements and describe the types of subscription policies available.

How-to guides

• Subscription policy: Author a subscription policy to manage access to data sources.

• ABAC subscription policy: Author a subscription policy that uses attribute-based access controls to automate data access decisions.

• Advanced DSL builder: Author a subscription policy using the advanced special functions.

• Restricted subscription policy: As a data owner, author a single subscription policy that can apply to multiple data sources you own.

• Clone, activate, or stage a global policy: Clone, activate, or stage a global policy.

Reference guides

• Subscription policies: This reference guide describes the design and behavior of Immuta subscriptions policies.

• Subscription policy access types: This reference guide describes read and write access policies.

• Advanced use of special functions: This reference guide defines the advanced functions available when building subscription policies.

Natural language represented policy

This is a pretty simple one: if you can’t show your work, you are in a situation of trust with no way to verify. Writing code to enforce policy (Snowflake, Databricks, etc.) or building complex policies in Ranger does show your work to a certain extent - but not enough for outsiders to easily understand the policy goals and verify their accuracy, and certainly not to the non-engineering teams that care that policy enforcement is done correctly.

With Immuta, policy is represented in natural language that is easily understood by all. This allows non-engineering users to verify that policy has been written correctly. Remember that when using global policies they leverage tags rather than physical table/column names, which further enhances understandability.

Lastly, and as covered in the scalability principle, with Immuta you are able to build far fewer policies, upwards of 75x fewer policies, which provides an enormous amount of understandability with it.

Certainly this does not mean you have to build every policy through our UI - data engineers can build automation through the Immuta API, if desired, and those policies are presented in a human readable form to the non-engineering teams that need to understand how policy is being enforced.

Policy history and changes

Understandability of policy is critically important. This should be further augmented by change history around policy, and being able to monitor and attribute change.

Immuta provides this capability through extensive audit logs and takes it a step further by providing history views and diffs in the user interface.

This is different from query activity in your data platform, as discovered and surfaced in Immuta Detect. In addition to that, actions taken in Immuta that alter policy decisions are audited and allowing the creation of compliance reports around that information.

Without Immuta, if you build policy based on tasking an engineer in an ad-hoc manner there is no history of the change, nor is it possible to see the difference between the old and new policies. That makes it impossible to take a historical look at change and understand where an issue may have arisen. If you have a standardized platform for making policy changes, like Immuta, then you are able to understand and inspect those changes over time.

This page details how users can create more complex policies using functions and variables in the Advanced DSL policy builder than the Subscription Policy builder allows.

For instructions on writing Global Subscription Policies, see the following tutorial.

Enabling Enhanced Subscription Policy Variables (Public Preview)

1. Navigate to the App Settings Page.

2. Click Advanced Settings in the left panel, and scroll to the Preview Features section.

3. Check the Enable Enhanced Subscription Policy Variables checkbox.

4. Click Save.

Create and Edit Global Subscription Policies Using Advanced DSL

1. Navigate to the Policies Page.

2. Select Subscription Policies and click + Add Subscription Policy.

3. Choose a name for your policy and select how the policy should grant access.

4. Select Create using Advanced DSL.

5. Select the rules for your policy from the Advanced DSL options. For example, creating a @hasTagsAsAttribute('Department', 'dataSource') would subscribe all users who have an attribute that matches a tag on a data source to that data source. So users with the attribute Department.Marketing would be subscribed to data sources with the tag Marketing.

6. Check the Require Manual Subscription checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

7. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Data Source Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

8. If you would like users to have the ability to request approval to the data source, even if they do not have the required attributes or traits, check the Request Approval to Access checkbox. This will require an approver with permissions to be set.

9. Select how you want Immuta to merge multiple global subscription policies that apply to a single data source.

   • Always Required: Users must meet all the conditions outlined in each policy to get access (i.e., the conditions of the policies are combined with AND).

   • Share Responsibility: Users need to meet the condition of at least one policy that applies (i.e., the conditions of the policies are combined with OR).

10. Select where this policy should be applied, On data sources, When selected by data owners, or On all data sources

    • If a user selects On data sources options include, with columns tagged, with columns spelled like, in server, and created between.

11. Click Create Policy.

Data owners who are not governors can write restricted subscription policies and data policies, which allow them to enforce policies on multiple data sources simultaneously, eliminating the need to write redundant local policies.

Unlike global policies, the application of these policies is restricted to the data sources owned by the users or groups specified in the policy and will change as users' ownerships change.

Write access policy requirements

• At least one of the following permissions is required to manage write policies:

  • CREATE_DATA_SOURCE Immuta permission (to create local write policies)

  • GOVERNANCE Immuta permission (to create local or global write policies)

  • MANAGE_POLICIES domain permission (to create global write policies)

• Databricks Unity Catalog, Snowflake, or Starburst (Trino) integration

• Snowflake table grants enabled (for Snowflake integrations)

Create a restricted subscription policy

1. Click the Policies in the left sidebar and select Subscription Policies.

2. Click Add Policy, complete the Enter Name field.

3. Select the access type:

   • Read Access: Control who can view the data source.

   • Write Access: Control who can view and modify data in the data source.

4. Select the level of access restriction you would like to apply to your data sources:

   • Allow anyone: Check the Require Manual Subscription checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

   • Allow anyone who asks (and is approved):

     1. Click Anyone or An individual selected by user from the first dropdown menu in the subscription policy builder.

        Note: If you choose An individual selected by user, when users request access to a data source they will be prompted to identify an approver with the permission specified in the policy and how they plan to use the data.

     2. Select the Owner (of the data source), USER_ADMIN, GOVERNANCE, or AUDIT permission from the subsequent dropdown menu.

        Note: You can add more than one approving party by selecting + Add Another Approver.

   • Allow users with specific groups/attributes:

     1. Choose the condition that will drive the policy: when user is a member of a group or possesses attribute. Note: To build more complex policies than the builder allows, follow the Advanced rules DSL policy guide.

     2. Use the subsequent dropdown to choose the group or attribute for your condition. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the subscription policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

     3. Check the Require Manual Subscription checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

     4. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Data Source Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

     5. If you would like users to have the ability to request approval to the data source, even if they do not have the required attributes or traits, check the Request Approval to Access checkbox. This will require an approver with permissions to be set.

     6. Select how you want Immuta to merge multiple global subscription policies that apply to a single data source.

        • Always Required: Users must meet all the conditions outlined in each policy to get access (i.e., the conditions of the policies are combined with AND).

        • Share Responsibility: Users need to meet the condition of at least one policy that applies (i.e., the conditions of the policies are combined with OR).

        Note: To make this option selected by default, see the app settings page.

   • Allow individually selected users

5. From the Where should this policy be applied dropdown menu, select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

6. Beneath Whose Data Sources should this policy be restricted to, add users or groups to the policy restriction by typing in the text fields and selecting from the dropdown menus that appear.

7. Opt to complete the Enter Rationale for Policy (Optional) field.

8. Click Create Policy, and then click Activate Policy or Stage Policy.

Write access policy requirements

• At least one of the following permissions is required to manage write policies:

  • CREATE_DATA_SOURCE Immuta permission (to create local write policies)

  • GOVERNANCE Immuta permission (to create local or global write policies)

  • MANAGE_POLICIES domain permission (to create global write policies)

• Databricks Unity Catalog, Snowflake, or Starburst (Trino) integration

• Snowflake table grants enabled (for Snowflake integrations)

Create a policy

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Subscription Policies tab. Click Add Subscription Policy and complete the Enter Name field.

   • Local policy: Navigate to a specific data source and click the Policies tab. Click Add Subscription Policy and select New Local Subscription Policy.

2. Select the access type:

   • Read Access: Control who can view the data source.

   • Write Access: Control who can view and modify data in the data source.

3. Select the level of access restriction you would like to apply:

   • Allow anyone: Check the Require Manual Subscription checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

   • Allow anyone who asks (and is approved):

     1. Click Anyone or An individual selected by user from the first dropdown menu in the subscription policy builder.

        Note: If you choose An individual selected by user, when users request access to a data source they will be prompted to identify an approver with the permission specified in the policy and how they plan to use the data.

     2. Select the Owner (of the data source), USER_ADMIN, GOVERNANCE, or AUDIT permission from the subsequent dropdown menu.

        Note: You can add more than one approving party by selecting + Add Another Approver.

   • Allow users with specific groups/attributes: See the ABAC subscription policy guide for instructions.

   • Allow individually selected users

4. For global policies: From the Where should this policy be applied dropdown menu, select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

5. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

Manually grant access

Read and write access can also be granted manually by a data owner. See the Manage data source members guide for details.

Additional global ABAC subscription policies

When you have multiple global ABAC subscription policies to enforce, create separate global ABAC subscription policies, and then Immuta will use boolean logic to merge all the relevant policies on the tables they map to.

Immuta policies restrict access to data and apply to data sources at the local or global level:

• Local policies are authored against specific tables, one at a time.

• Global policies target tags instead of specific tables, allowing you to build a single policy that impacts a large percentage of your data rather than building separate local policies for each table.

Consider the following local and global policy examples:

• Local policy example: Mask using hashing the values in the columns ssn, last_name, and home_address.

• Global policy example: Mask using hashing values in columns tagged PII on all data sources.

In this scenario, the local policy would mask the sensitive columns specified in the data policy on the single data source it was created for. If only using local policies, a data owner or governor would have to write that policy for every data source on which they wanted to mask sensitive data. The second policy would mask any column tagged PII on all data sources that had the PII tag applied to a column. Because this global policy automatically applies to those qualifying data sources, that policy only needs to be written once.

Consequently, global policies are the best practice for using Immuta: they provide the most scalability and manageability of access control.

For details about subscription and data policy types, see the subscription policies overview or data policies overview.

Policy authors

Global policies can be authored by users with GOVERNANCE permission or data owners. Data owners' global policies will only attach to data sources they own (also called restricted global policies), even if the tags their policies target go beyond their data sources.

Reference user metadata, not users

When building global policies at scale, never reference individual users. The goal is to instead reference metadata about users. This way, as users gain or lose that metadata, access logic is automatically updated. This future proofs policies.

User metadata are values connected to specific Immuta users and are used in policies to gain access to tables (with subscription policies) or data (with data policies). These attributes fall into three categories:

• attributes: Attributes are key/value pairs that can be assigned to users.

• groups: Groups are similar to roles in a data platform; users can be assigned to one or many groups.

• group attributes: It is possible to assign attributes to groups. This is a shortcut for assigning a specific attribute to a large set of users: all the users in the group will be assigned the attribute.

Policy states

The table below outlines the various states of global policies in Immuta.

Note: Policies that contain the circumstance When selected by data owners cannot be staged.

Restricted global policies

Data owners who are not governors can write restricted global policies for data sources that they own. With this feature, data owners have higher-level policy controls and can write and enforce policies on multiple data sources simultaneously, eliminating the need to write redundant local policies on data sources.

There is no special configuration necessary, once a user is a data owner, the global policy builder is available to them, and they can build global policies just like any other user with GOVERNANCE permission. The only difference is that global policy will be additionally restricted to only the data sources the user owns.

Data ownership can be assigned, but it is also automatically granted to the user that registered the data source with Immuta.

Data source policies tab

All data source subscribers can access a data source's policies on the policies tab of the data source, but only data owners and governors can edit it. Although not recommended, from there users can view existing policies or apply new local policies to the data source with the following features:

• Apply Existing Policies Button: By clicking this button in the top right corner of the policies tab, users can search for and apply existing policies to the data source from other data sources or global policies.

• Subscription Policy Builder: In this section, users can determine who may access the data source. If a subscription policy has already been set by a global policy, a notification and a Disable button appear at the bottom of this section. Users can click the Disable button to make changes to the subscription policy.

• Data Policy Builder: In this section, users can create policies to enforce privacy controls on the data source and see a list of data policies currently applied to the data source.

All changes made to policies by data owners or governors appear in a collapsible Activity panel on the right side of the screen.

The information recorded in the activity panel includes when the data source was created, the name and type of the policy, when the policy was applied or changed, and if the policy is in conflict on the data source. Additionally, global policy changes are identified by the governance icon; all other updates are labeled by the data sources icon.

When executing transforms in your data platform, new tables and views are constantly being created, columns added, data changed - transform DDL. This constant transformation can cause latency between the DDL and Immuta policies discovering, adapting, and attaching to those changes, which can result in data leaks. This policy latency is referred to as policy downtime.

The goal is to have as little policy downtime as possible. However, because Immuta is separate from data platforms and those data platforms do not currently have webhooks or eventing service, Immuta does not receive alerts of DDL events. This causes policy downtime.

This page describes the appropriate steps to minimize policy downtime as you execute transforms using dbt or any other transform tool and links to tutorials that will help you complete these steps.

Prerequisites

Required:

• Schema monitoring: This feature detects destructively recreated tables (from CREATE OR REPLACE statements) even if the table schema wasn’t changed. Enable schema monitoring when you register your data sources.

Recommended (if you are using Snowflake):

• Snowflake table grants enabled: This feature implements Immuta subscription policies as table GRANTS in Snowflake rather than Snowflake row access policies. Note this feature may not be automatically enabled if you were an Immuta customer before January 2023; see Enable Snowflake table grants to enable.

• Low row access mode enabled: This feature removes unnecessary Snowflake row access policies when Immuta project workspaces or impersonation are disabled, which improves the query performance for data consumers.

Step 1: Create global policies and prepare tags for data sources

To benefit from the scalability and manageability provided by Immuta, you should author all Immuta policies as global policies. Global policies are built at the semantic layer using tags rather than having to reference individual tables with policy. When using global policies, as soon as a new tag is discovered by Immuta, any applicable policy will automatically be applied. This is the most efficient approach for reducing policy downtime.

There are three different approaches for tagging in Immuta:

1. Auto-tagging (recommended): This approach uses sensitive data discovery (SDD) to automatically tag data.

2. Manually tagging with an external catalog: This approach pulls in the tags from an external catalog. Immuta supports Snowflake, Databricks Unity Catalog, Alation, and Collibra to pull in external tags.

3. Manually tagging in Immuta: This approach requires a user to create and manually apply tags to all data sources using the Immuta API or UI.

Note that there is added complexity with manually tagging new columns with Alation, Collibra, or Immuta. These listed catalogs can only tag columns that are registered in Immuta. If you have a new column, you must wait until after schema detection runs and detects that new column. Then that column must be manually tagged. This issue is not present when manually tagging with Snowflake or Databricks Unity catalog because they are already aware of the column or using SDD because it runs after schema monitoring.

Auto-tagging (recommended)

Using this approach, Immuta can automatically tag your data after it is registered by schema monitoring using sensitive data discovery (SDD). SDD is made of algorithms you can customize to discover and tag the data most important to you and your organization's policies. Once customized and deployed, any time Immuta discovers a new table or column through schema monitoring, SDD will run and automatically tag the new columns without the need for any manual intervention. This is the recommended option because once SDD is customized for your organization, it will eliminate the human error associated with manually tagging and is more proactive than manual tagging, further reducing policy downtime.

Native SDD should be enabled and customized before registering any data with Immuta. Contact your Immuta support professional to enable this feature flag for you before enabling SDD on the app settings page.

Manually tagging with an external catalog

Using this approach, you will rely on humans to tag. Those tags will be stored in the data platform (Snowflake, Databricks Unity Catalog) or catalog (Alation, Collibra). Then they will be synchronized back to Immuta. If using this option, Immuta recommends storing the tags in the data platform because the write of the tags to Snowflake or Databricks Unity Catalog can be managed and reused with SQL from tools like dbt, removing burden from manually tagging on every run. API calls to Alation and Collibra are also possible, but are not accessible over dbt. Manually tagging through the Alation or Collibra UI will negatively impact data downtime.

Manually tagging in Immuta

Using this approach, you will rely on humans to tag, but the tags will be stored directly in Immuta. This can be done using the Immuta API or through the Immuta UI However, manually tagging through the Immuta UI will negatively impact data downtime.

Step 2: Register your data in Immuta

When registering tables with Immuta, you must register each database or catalog with schema monitoring enabled. Schema monitoring means that you do not need to individually register tables but rather make Immuta aware of databases, and then Immuta will periodically scan that database for changes and register any new changes for you. You can also manually run schema monitoring using the Immuta API.

Step 3: Consider the result and user making transformations

Views vs tables

Views will have existing data policies (row-level security, masking) enforced on them that exist on the backing tables by nature of how views work (the query is passed down to the backing tables). So when you tag and register a view with Immuta, you are re-applying the same data policies on the view that already exist on the backing tables, assuming the tags that drive the data policies are the same on the view’s columns.

If you do not want this behavior or its possible negative performance consequences, then Immuta recommends the following based on how you are tagging data:

• For auto-tagging, place your incremental views in a separate database that is not being monitored by Immuta. Do not register them with Immuta, and schema monitoring will not detect them from the separate database.

• For either manually tagging option, do not tag view columns.

Using either option, the views will only be accessible to the person who created them. The views will not have any subscription policies applied to give other users access because the views are either not registered in Immuta or there are no tags. To give other users access to the data in the view, they should subscribe to the table at the end of the transform pipeline.

However, if you do want to share the views using subscription policies, you should ensure that the tags that drive the subscription policies exist on the view and that those tags are not shared with tags that drive your data policies. It is possible to target subscription policies on all tables or tables from a specific database rather than using tags.

Access level of job executioner

Policy is enforced on READ. Therefore, if you run a transform that creates a new table, the data in that new table will represent the policy-enforced data.

For example, if the credit_card_number column is masked for Steve, on read, the real credit card numbers will be dynamically masked. If Steve then copies them into a new table via the transform, he is physically loading masked credit card numbers into that table. Now if another user, Jane, is allowed to see credit card numbers and queries the table, her query will not show the credit card numbers. This is because credit card numbers are already masked in that table. This problem only exists for tables, not views, since tables have the data physically copied into them.

To address this situation, you can do one of the following:

• Use views for all transforms.

• Ensure the users who are executing the transforms always have a higher level of access than the users who will consume the results of the transforms. Or, if this is not possible,

• Set up a dev environment for creating the transformation code; then, when ready for production, have a promotion process to execute those production transformations using a system account free of all policies. Once the jobs execute as that system account, Immuta will discover, tag, and apply the appropriate policy.

Step 4: Force data downtime

Data downtime refers to the techniques you can use to hide data after transformations until Immuta policies have had a chance to synchronize. It makes data inaccessible; however, it is preferred to the data leaks that could occur while waiting for policies to sync.

Whenever DDL occurs, it can result in policy downtime, such as in the following examples:

• An existing table or view is recreated with the CREATE OR REPLACE statement. This will drop all policies resulting in users losing all access. There is one exception: with Databricks Unity, the grants on the table are not dropped, which means masking and row filtering policies are dropped but the table access is not, making policy downtime even more critical to manage.

• A new column is added to a table that needs to be masked from users that have access to that table (potential data leak).

• A new table is created in a space where other users have read access on future tables (potential data leak).

• A tag that drives a policy is updated, deleted, or newly added with no other changes to the schema or table. This is a limitation of how Immuta can discover changes - it is too inefficient to search for tag changes, so schema changes are what drives Immuta to take action.

Best practices for Snowflake

Immuta recommends all of the following best practices to ensure data downtime occurs during policy downtime:

• Do not COPY GRANTS when executing a CREATE OR REPLACE statement.

• Do not use GRANT SELECT ON FUTURE TABLES.

• Use CREATE OR REPLACE for all DDL, including altering tags, so that access is always revoked.

Without these best practices, you are making unintentional policy decision in Snowflake that may be in conflict with your organization's actual policies enforced by Immuta.

For example, if the CREATE OR REPLACE added a new column that contains sensitive data, and the user COPY GRANTS, it opens that column to users, causing a data leak. Instead, access must be blocked using the above data downtime techniques until Immuta has synchronized.

Best practices for Databricks Unity Catalog

Immuta recommends all of the following best practices to ensure data downtime occurs during policy downtime:

• Do not grant to catalogs or schemas, because those apply to current and future objects (future is the problematic piece).

• There is no way to stop Databricks Unity Catalog from carrying over table grants on CREATE OR REPLACE statements, so ensure you initiate policy uptime as quickly as possible if you have row filters or masking policies on that table.

• Use CREATE OR REPLACE for all DDL, including altering tags, so that access is always revoked.

Without these best practices, you are making unintentional policy decision in Unity Catalog that may be in conflict with your organization's actual policies enforced by Immuta.

For example, if you GRANT SELECT on a schema, and then someone writes a new table with sensitive data into that schema, it could cause a data leak. Instead, access must be blocked using the above data downtime techniques until Immuta has synchronized.

Step 5: Initiate policy uptime

As discussed above, data platforms do not currently have webhooks or eventing service, so Immuta does not receive alerts of DDL events. Schema monitoring will run every 24 hours by default to detect changes, but schema monitoring should also be run across your databases after you make changes to them. You can manually run schema monitoring using the Immuta API and the payload can be scoped down run schema monitoring on a specific database or schema or column detection on a specific table.

When schema monitoring is run globally, it will detect the following:

• Any new table

• Any new view

• Changes to the object type backing an Immuta data source (for example, when a TABLE changes to a VIEW); when an object type changes, Immuta reapplies existing policies to the data source.

• Any existing table destructively recreated through CREATE OR REPLACE (even if there are no schema changes)

• Any existing view destructively recreated through CREATE OR REPLACE (even if there are no schema changes)

• Any dropped table

• Any new column

• Any dropped column

• Any column type change (which can impact policy application)

• Any tag created, updated, or deleted (but only if the schema changed; otherwise tag changes alone are detected with Immuta’s health check)

Then, if any of the above is detected, for those tables or views, Immuta will complete the following:

1. Synchronize the existing policy back to the table or view to reduce data downtime

2. If SDD is enabled, execute SDD on any new columns or tables

3. If an external catalog is configured, execute a tag synchronization

4. Synchronize the final updated policy based on the SDD results and tag synchronization

5. Apply "New" tags to all tables and columns not previously registered in Immuta and lock them down with the "New Column Added" templated global policy

See the image below for an illustration of this process with Snowflake.

The two options for running schema monitoring are described in the sections below. You can implement them together or separately.

Alert Immuta through the API or a custom function

If the data platform supports custom UDFs and external functions, you can wrap the /dataSource/detectRemoteChanges endpoint with one. Then, as your transform jobs complete, you can use SQL to call this UDF or external function to tell Immuta to execute schema monitoring. The reason for wrapping in a UDF or external function is because dbt and transform jobs always compile to SQL, and the best way to make this happen immediately after the table is created (after the transform job completes) is to execute more SQL in the same job.

Consult your Immuta professional for a custom UDF compatible with Snowflake or Databricks Unity Catalog.

Periodic schema monitoring

The default schedule for Immuta to run schema monitoring is every night at 12:30 a.m. UTC. However, this schedule can be updated by changing some advanced configuration. The processing time for schema monitoring is dependent on the number of tables and columns changed in your data environment. If you want to change the schedule to run more frequently than daily, Immuta recommends you test the runtime (with a representative set of DDL changes) before making the configuration change.

Consult your Immuta professional to update the schema monitoring schedule, if desired.

Recommended Immuta policy types

There are some use cases where you want all users to have access to all tables, but want to mask sensitive data within those tables. While you could do this using just data policies, Immuta recommends you still utilize subscription policies to ensure users are granted access.

Subscription policies allow for Immuta to have a state to move table access into post-data-downtime to realize policy uptime. Without subscription policies, when Immuta synchronizes policy, users will continue to not have access to tables because there is no subscription policy granting them access. If you want all users to have access to all tables, use a global "Anyone" subscription policy in Immuta for all your tables. This will ensure users are granted access back to the tables after data downtime.

Immuta allows you to define policies at different levels of your data stack.

First are subscription policies, which are commonly termed table access grants or table-level access. Subscription policies control access to your tables. Immuta calls them subscription policies because they are not always an access grant but could also be the result of a data consumer finding the data, requesting access, and then being subscribed to it via Immuta policy you have in place.

Second are data policies, which control access more granularly inside a table. For example, Immuta can help you build policies to redact rows, mask columns, or even mask cells.

Authoring policy at scale

While it is possible to build policies one table at a time using Immuta, there isn't much value in doing so. These are termed local policies in Immuta.

To build policy at scale, you must use global policies. Global policies allow you to build policies that reference tags rather than physical tables or columns. So instead of building a policy like this mask column name in table customers, you can instead build a policy such as mask columns tagged name anywhere you see the name tag.

These global policies will then seek out the name tag, wherever found, and apply the policy, no matter the physical location of the tables that contain names. It's important to understand that Immuta supports tag-based global policies for more than just masking. Both subscription and row-level policies can be authored as global policies targeting tags instead of physical tables and columns.

How you get the tags on the tables and columns is outlined in the Automate data access control decisions use case.

Section contents

There are many guides found in this section, but an efficient approach to learning how to author secure policy would be to first read the two Immuta use cases specific to secure:

1. Automate data access control decisions

2. Compliantly open more sensitive data for ML and analytics

And then to focus on the complex topics around how applying policy at scale is managed in Immuta, specifically

• Overview on how to author policies at scale

• Overview of subscription policies and data policies

• Full reference guide for all data policies

• Details on how to minimize policy downtime if there's a large amount of change due to data engineering in your data platform(s)

• Details on how subscription policy conflicts and data policy conflicts are managed

ABAC vs RBAC

Do you find yourself spending too much time managing roles and defining permissions in your system? When there are new requests for data, or a policy change, does this cause you to spend an inordinate amount of time to make those changes? Scalability and evolvability will completely remove this burden. When you have a scalable and evolvable data policy management system, it allows you to make changes that impact hundreds if not thousands of tables at once, accurately. It also allows you to evolve your policies over time with minor changes or no changes at all, through future-proof policy logic.

Lack of scalability and evolvability are rooted in the fact that you are attempting to apply a coarse role-based access control (RBAC) model to your modern data architecture. Using Apache Ranger, a well known legacy RBAC system built for Hadoop, as an example, independent research has shown the explosion of management required to do the most basic of tasks with an RBAC system: Apache Ranger Evaluation for Cloud Migration and Adoption Readiness.

In a scalable solution such as Immuta, that count of policy changes required will remain extremely low, providing the scalability and evolvability. GigaOm researched this exactly, comparing Immuta’s ABAC model to what they called Ranger’s RBAC with Object Tagging (OT-RBAC) model and showed a 75 times increase in policy management with Ranger.

https://gigaom.com/report/cloud-data-security/

• Value to you: You have more time to spend on the complex tasks you should be spending time on and you don’t fear making a policy change.

• Value to the business: Policies can be easily enforced and evolved, allowing the business to be more agile and decrease time-to-data across your organization and avoid errors.

Separating policy definition from role definition

When building access control into our database platforms, the concept of role-based access control (RBAC) is familiar. Roles both define who is in them, but also determine what those roles get access to. A good way to think about this is roles conflate the who and what: who is in them and what they have access to (but lack the why).

In contrast, attribute-based access control (ABAC) allows you to decouple your roles from what they have access to, essentially separating the what and why from the who, which also allows you to explicitly explain the “why” in the policy. This gives you an incredible amount of scalability and understandability in policy building. Note this does not mean you have to throw away your roles necessarily, you can make them more powerful and likely scale them back significantly.

If you remember this picture and article from the start of this introduction, most of the Ranger, Snowflake, Databricks, etc. access control scalability issues are rooted in the fact that it’s an RBAC model vs ABAC model.

Example: Building row-level security with an ABAC model

Consider that you have a table which contains a transaction_country column and you have data localization needs which requires you to limit specific countries to specific users.

With a classic RBAC approach, you would need to create a role for every permutation of country access. Remember that it's not necessarily just a role per country, because some users may need access to more than one country. Every time a new permutation of country combination is required, a new role must be managed to represent that access.

With Immuta's ABAC approach, since Immuta is able to decouple policy logic from users, you can simply assign users countries and Immuta will filter appropriately on the fly. This can be done with a single policy in Immuta which references the user country metadata. If you add a new user with a never before seen combination of countries, in the RBAC model, you would have to remember to create a new role and policy for them to see data. In the ABAC model it will “just work” since everything is dynamic - future proofing your policies.

For more discussion about this model, see the Role-Based Access Control vs. Attribute-Based Access Control — Explained blog or the NIST article on ABAC, Guide to Attribute Based Access Control (ABAC) Definition and Considerations.

Policy boolean logic

The only way to support AND boolean logic with a role-based model (RBAC) is by creating a new role that conflates the two or more roles you want to AND together.

For example, a governor wants users to only see certain data if they have security awareness training and have consumer privacy training. It would be natural to assume you need both separately as metadata attached to users to drive the policy. However, when you build policies in a role based model, it assumes roles are either OR’ed together in the policy logic or you can only act under one role at a time, and because of this, you will have to create a single role to represent this combination of requirements “users with security awareness training AND consumer privacy training.” This is completely silly and unmanageable - you need to account for every possible combination relevant to a policy, and you have no way of knowing that ahead of time.

With Immuta and its ABAC model, you are able to keep user attributes as meaningful separate facts about the users and then use boolean logic to combine those facts in policy logic. As an example, consider the country filtering policy described in the prior section: you could build the filtering, as described, but additionally add an exception such as "do this filtering for everyone except members of group security awareness training and members of group consumer privacy training" without the need to create a new role that represents those combined.

Exception-based policy authoring

This next section draws on an analogy: Imagine you are planning your wedding reception. It’s a rather posh affair, so you have a bouncer checking people at the door.

Do you tell your bouncer who’s allowed in? (exception-based) Or, do you tell the bouncer who to keep out? (rejection-based)

The answer to that question should be obvious, but many policy engines allow both exception- and rejection-based policy authoring, which causes a conflict nightmare. Exception-based policy authoring in our wedding analogy means the bouncer has a list of who should be let into the reception. This will always be a shorter list of users/roles if following the principle of least privilege, which is the idea that any user, program, or process should have only the bare minimum privileges necessary to perform its function - you can’t go to the wedding unless invited. This aligns with the concept of privacy by design, the foundation of the CPRA and GDPR, which states “Privacy as the default setting.”

What this means in practice is that you should define what should be hidden from everyone, and then slowly peel back exceptions as needed.

How could your data leak if it wasn’t exception based?

What if you did two policies:

• Mask Person Name using hashing for everyone who possesses attribute Department HR.

• Mask Person Name using constant REDACTED for everyone who possesses attribute Department Analytics.

Now, some user comes along who is in Department Finance - guess what, they will see the Person Name columns in the clear because they were not accounted for, just like the bouncer would let them into your wedding because you didn’t think ahead of time to add them to your deny list.

There are two main issues with allowing bi-directional policies, which is why Immuta only allows exception-based policies, aligning to the industry standard of least privileged access:

1. Ripe for data leaks: Rejection-based policies are extremely dangerous and why Immuta does not allow them except with a catch-all OTHERWISE statement at the end. Again this is because if a new role/attribute comes along that you haven’t accounted for, that data will be leaked. It is impossible for you to anticipate every possible user/attribute/group that could possibly exist ahead of time just like it’s impossible for you to anticipate any person off the street that could try to enter your posh wedding that you would have to account for on your deny list.

2. Ripe for conflicts and confusion: Tools that specifically allow both rejection-based and exception-based policy building create a conflict disaster. Let’s walk through a simple example, noting this is very simple, imagine if you have hundreds of these policies:

   • Policy 1: mask name for everyone who is member of group A

   • Policy 2: mask name for everyone except members of group B

What happens if someone is in both groups A and B? The policy will have to fall back on policy ordering to avoid this conflict, which requires users to understand all other policies before building their policy and it is nearly impossible to understand what a single policy does without looking at all policies.

Hierarchical tag-based policy definitions

While many platforms support the concept of object tagging / sensitive data tagging, very few truly support hierarchical tag structures.

First, a quick overview of what hierarchical tag structure means:

This would be a flat tag structure:

• SUV

• Subaru

• Truck

• Jeep

• Gladiator

• Outback

Each tag stands on its own and is not associated with one another in any way; there’s no correlation between Jeep and Gladiator nor Subaru and Outback.

A hierarchical tagging structure establishes these relationships:

• SUV.Subaru.Outback

• Truck.Jeep.Gladiator

Support for a tagging hierarchy is more than just supporting the tag structure itself. More importantly, policy enforcement should respect the hierarchy as well. Let’s run through a quick contrived example; you want the following policies:

• Mask by making null any SUV data

• Mask using hashing any Outback data

With a flat structure, if you build those policies they will be in conflict with one another. To avoid that problem you would have to order which policies take precedence, which can get extremely complex when you have many policies.

Instead, if your policy engine truly supports a tagging hierarchy, like Immuta does, it will recognize that Outback is more specific than SUV, and have that policy take precedence.

• Mask by making null any SUV data

• Mask using hashing any SUV.Subaru.Outback data

Policies are applied correctly without any need for complex ordering of policies.

Yes, this does put some work on the business to correctly build specificity, or depth, into their tagging hierarchy. This is not necessarily easy; however, the logic will have to live somewhere, and having it in the tagging hierarchy rather than policy order again allows you to separate policy definition from data definition. This provides you scalability, evolvability, understandability, and, most importantly, correctness because policy conflicts can be caught at policy-authoring-time.

Subscription policies: benefits of attribute-based table GRANTs

There are a myriad of techniques and processes companies use to determine what users should have access to which tables. Some customers have had 7 people responding to an email chain for approval before a DBA runs a table GRANT statement, for example. Manual approvals are sometimes necessary, of course, but there’s a lot of power and consistency in establishing objective criteria for gaining access to a table rather than subjective human approvals.

Let’s take the “7 people approve with an email chain” example. Ask the question, “Why do any of you 7 say yes to the user gaining access?” If it’s objective criteria, you can completely automate this process. For example, if the approver says, “I approve them because they are in group x and work in the US,” that is user metadata that could allow the user to automatically gain access to the tables, either ahead of time or when requested. This removes a huge burden from your organization and avoids mistakes.

Being objective is always better than subjective: it increases accuracy, removes bias, eliminates errors, and proves compliance. If you can be objective and prescriptive about who should gain access to what tables - you should.

The anti-pattern is manual approvals. Although there are some regulatory requirements for this, if there’s any possible way to switch to objective approvals, you should do it. With subjective human-driven approvals, there is bias, larger chance for errors, and no consistency - this makes it very difficult to prove compliance and is simply passing the buck (and risk) to the approvers and wasting their valuable time.

One could argue that it’s subjective or biased to assign a user the Country.JP attribute. This is not true, because, remember, data policy is separated from user metadata. The act of giving a user the Country.JP attribute is simply defining that user - it is a fact about that user, there is no implied access given to the user from this act and that attribute will be objective - e.g., you know if they are in Japan or not.

The approach where an access decision is conflated with a role or group is common practice. So not only do you end up with manual approval flows, but you also end up with role explosion from so many roles to meet every combination of access.

There are several different that are available for building subscription policies. Some of these functions, listed below, are narrowly focused on orchestrated RBAC use cases. Orchestrated RBAC is when an organization has many roles that represent access, and rather than switching to using the ABAC model provided by Immuta, they use these special functions to orchestrate existing roles using Immuta.

Specifically, the functions to enable orchestrated-RBAC are:

• @hostname

• @database

• @schema

• @table

• @hasTagAsAttribute('Attribute Name', 'dataSource' or 'column')

• @hasTagAsGroup('dataSource' or 'column')

Example 1

Policy:

@hasAttribute('SpecialAccess', '@hostname.@database.*')

User:

has the attribute SpecialAccess with the value us-east-1-snowflake.default.*

The user would be subscribed to all the data sources in the default database. Note this has nothing to do with tags, it is based purely on the physical name of the host, database, schema, and table in the native data platform. Also note that the user attribute contains an asterisk * to denote everything under the default database hierarchy. Asterisks are supported only for the infrastructure special functions:

• @hostname

• @database

• @schema

• @table

This is because, since it's an infrastructure view, Immuta can assume a 4-level hierarchy (hostname.database.schema.table) and an asterisk can be placed between any two objects in that 4-level hierarchy to represent any object, such as us-east-1-snowflake.*.hr. That would give the user access to any schema named hr in host us-east-1-snowflake no matter the database.

However, that is not possible when using the tag-based special functions:

• @hasTagAsAttribute('Attribute Name', 'dataSource' or 'column')

• @hasTagAsGroup('dataSource' or 'column')

Lastly, the asterisk represents any object, but cannot be used for a concatenated wildcard like so: snowfl*.tpc.*.*

Example 2

Policy:

@hasTagAsAttribute('PersonalData', 'dataSource')

User:

has the attribute key PersonalData with the values

• Discovered.PII

• Discovered.Entity

Data source 1:

tagged:

• Discovered.Identifier Indirect

• Discovered.PHI

• Discovered.PII

Data source 2:

tagged:

• Discovered.Identifier Direct

• Discovered.PCI

• Discovered.Entity.Social Security Number

Data source 3:

tagged:

• Discovered.Identifier Direct

• Discovered.PHI

The user would be subscribed to data source 1 and 2, but the user would not be subscribed to data source 3. This is because access moves from left-to-right in the hierarchy based on what the user possesses (the wildcard asterisk is implied).

So if a user had a more specific attribute key PersonalData with the values Discovered.Entity.Social Security Number, they would only get access to hypothetical data source 2, because their attribute is further left or matches (in this case matches) Discovered.Entity.Social Security Number.

The below table provides more examples:

Merging special functions

This could be helpful for use cases with a policy like the following:

If user has the attribute “Allowed_Domain.Domain A” they get access to generic data that is part of domain A.

If user has the attribute “Badge_Allowed.Badge X” they should gain access to both “generic data + any additional data (only in domain A because they only have “Data Domain A General Access”) that has been tagged as “Badge X”.

In this case it can be two separate subscription policies, such as

Policy 1: @hasTagAsAttribute(Allowed_Domain, ‘datasource’) this would limit to the domains where they are allowed to see generic data.

Policy 2: @hasTagAsAttribute(Badge_Allowed, ‘datasource’) this would limit to the badges they are allowed to see.

Then, when the data sources are tagged with table tags that represent access, if the table only has the domain tag, only policy 1 will apply; however, if it has a domain tag and a badge tag, both policies will be applied and merged successfully by Immuta.

Use with caution

This guide demonstrates how to build subscription policies using the policy builder in the Immuta UI. To build more complex policies than the builder allows, follow the policy guide.

1. Determine your policy scope:

   • : Click the Policies page icon in the left sidebar and select the Subscription Policies tab. Click Add Subscription Policy and complete the Enter Name field.

   • : Navigate to a specific data source and click the Policies tab. Click Add Subscription Policy and select New Local Subscription Policy.

2. Select Allow users with specific groups/attributes.

3. Choose the condition that will drive the policy: when user is a member of a group or possesses attribute.

4. Use the subsequent dropdown to choose the group or attribute for your condition. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the subscription policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

5. Check the Require Manual Subscription checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

6. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Data Source Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

7. If you would like users to have the ability to request approval to the data source, even if they do not have the required attributes or traits, check the Request Approval to Access checkbox. This will require an approver with permissions to be set.

8. For global policies: Select how you want Immuta to merge multiple global subscription policies that apply to a single data source.

   • Always Required: Users must meet all the conditions outlined in each policy to get access (i.e., the conditions of the policies are combined with AND).

   • Share Responsibility: Users need to meet the condition of at least one policy that applies (i.e., the conditions of the policies are combined with OR).

   Note: To make this option selected by default, see .

9. For global policies: Click the dropdown menu beneath Where should this policy be applied and select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

10. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

Additional global ABAC subscription policies

Clone a policy

1. Click the Policies icon in the left sidebar and navigate to the Data Policies or Subscription Policies tab.

2. Click the dropdown menu in the Actions column of a policy and select Clone.

3. Open the dropdown menu and click Edit in the Global Policy Builder. Then make your changes using the dropdown menus.

4. Click Create Policy, select Activate Policy, and then click Confirm.

The policy will display as Pending until it is enforced on all data sources it applies to. See the for details.

Note: If a cloned policy contains custom certifications, the certifications will also be cloned.

Activate a templated policy

1. Click the Policies icon in the left sidebar and navigate to the Data Policies tab.

2. Click the dropdown menu in the Actions column of one of the templated policies and select Activate. Note: If data governors decide to stage an active policy, they select Stage from this dropdown menu.

The policy will display as Pending until it is enforced on all data sources it applies to. See the for details.

Stage the policy

1. Click the dropdown menu in the Actions column of a policy and select Stage. Note: If data governors decide to make a staged policy active, they select Activate from this dropdown menu.

2. Click Confirm in the dialog that appears.

Immuta offers two types of to manage read and write access in a single system:

• Read access policies manage who can read data.

• Write access policies manage who can modify data.

Both of these access types can be enforced at any of the restriction levels outlined in the .

The table below illustrates the access types supported by each integration.

To create a read or write access policy, see the .

Policy enforcement

Once a read or write access policy is enforced on an Immuta data source, it translates to the relevant privileges on the table, view, or object in the remote platform. The sections below detail how these access types are enforced for each integration.

Granting Snowflake privileges

The Snowflake integration supports read and write access subscription policies. However, when applying read and write access policies to Snowflake data sources, the privileges granted by Immuta vary depending on the object type. For example, users can register Snowflake views as Immuta data sources and apply read and write policies to them, but when a write policy is applied to a view only the SELECT privilege will take effect in Snowflake, as views are read-only objects.

Users can register any object stored in Snowflake’s information_schema.tables view as an Immuta data source. The table below outlines the Snowflake privileges Immuta issues when read and write policies are applied to various object types in Snowflake. Beyond the privileges listed, Immuta always grants the USAGE privilege on the parent schema and database for any object that access is granted to for a particular user.

Granting Databricks Unity Catalog privileges

The Databricks Unity Catalog integration supports read and write access subscription policies. When users create a subscription policy in Immuta, Immuta uses the Unity Catalog API to issue GRANTS or REVOKES against the catalog, schema, or table in Databricks for every user affected by that subscription policy.

Users can register any object stored in Databricks Unity Catalog’s information_schema.tables view as an Immuta data source. However, when applying read and write access policies to these data sources, the privileges granted by Immuta vary depending on the object type. For example, users can register federated tables as Immuta data sources and apply read and write policies to them, but only a read policy will take effect in Databricks and allow users to SELECT those tables. If a write policy is applied, Immuta will not issue SELECT or MODIFY privileges in Databricks.

The table below outlines the Databricks privileges Immuta issues when read and write policies are applied to various object types in Databricks Unity Catalog. Beyond the privileges listed, Immuta always grants the USAGE privilege on the parent schema and catalog for any object that access is granted to for a particular user.

Granting Databricks Spark privileges

The Databricks Spark integration supports read access subscription policies. When a read access policy is applied to a data source, Immuta modifies the logical plan that Spark builds when a user queries data to enforce policies that apply that user. If the user is subscribed to the data source, the user is granted SELECT on the object in Databricks. If the user does not have read access to the object, they are denied access.

Granting Starburst (Trino) privileges

The Starburst (Trino) integration supports read and write access subscription policies. In the Starburst (Trino) integration's default configuration, the following access values grant read and write access to Starburst (Trino) data when a user is granted access through a subscription policy:

• READ: When a user is granted read access to a data source, they can SELECT on tables or views and SHOW on tables, views, or columns in Starburst (Trino). This setting in enabled by default when you configure the Starburst (Trino) integration.

• WRITE: In its default setting, the Starburst (Trino) integration's write access value controls the authorization of SQL operations that perform data modification (such as INSERT, UPDATE, DELETE, MERGE, and TRUNCATE). When users are granted write access to a data source through a subscription policy, they can INSERT, UPDATE, DELETE, MERGE, and TRUNCATE on tables and REFRESH on materialized views. This setting is enabled by default when you configure the Starburst (Trino) integration.

Custom configuration

Because Starburst (Trino) can govern certain table modification operations (like ALTER) separately from data modification operations (like INSERT), Immuta allows users to specify what modification operations are permitted on data in Starburst (Trino). Administrators can allow table modification operations (such as ALTER and DROP tables) to be authorized as write operations through advanced configuration in the Immuta web service or Starburst (Trino) cluster with the following access values:

• OWN: When mapped via advanced configuration to Immuta write policies, users who are granted write access to Starburst (Trino) data can ALTER and DROP tables and SET comments and properties on a data source.

• CREATE: When this privilege is granted on Starburst (Trino) data, an Immuta user can create catalogs, schemas, tables, or views on a Starburst (Trino) cluster. CREATE is a Starburst (Trino) privilege that is not controlled by Immuta policies, and this property can only be set in the access-control.properties file on the Starburst (Trino) cluster.

Administrators can customize table and data modification settings in one or both of the following places; however, the access-control.properties overrides the settings configured in the Immuta web service:

Immuta web service access grants mapping

Customizing read and write access in the Immuta web service affects operations on all Starburst (Trino) data registered as Immuta data sources in that Immuta tenant. This configuration method should be used when all Starburst (Trino) data source operations should be affected identically across Starburst (Trino) clusters connected to the Immuta web service. Example configurations are provided below. Contact your Immuta representative to customize the mapping of read or write access policies for your Immuta tenant.

Starburst (Trino) cluster access grants mapping

The Starburst (Trino) integration can also be configured to allow read and write policies to apply to any data source (registered or unregistered in Immuta) on a specific Starburst (Trino) cluster.

Two properties customize the behavior of read or write access for all Immuta users on that Starburst (Trino) cluster:

• immuta.allowed.immuta.datasource.operations: This property governs objects (catalogs, schemas, tables, etc.) that are registered as data sources in Immuta. For these permissions to apply, the user must be subscribed in Immuta and not be an administrator (who gets all permissions).

• immuta.allowed.non.immuta.datasource.operations: This property governs objects (catalogs, schemas, tables, etc.) that are not registered as data sources in Immuta. This is the only property that allows the CREATE permission, since CREATE is enforced on new objects that do not exist in Starburst (Trino) or Immuta yet (such as a new table being created with CREATE TABLE).

Granting Redshift privileges

The Redshift integration supports read access subscription policies. Immuta grants the SELECT Redshift privilege to the PUBLIC role when the integration is configured, which allows all users who meet the conditions of a subscription policy to access the Immuta-managed view. When a data source is created, Immuta creates a corresponding dynamic view of the table with a join to a secure view that contains all Immuta users, their entitlements, their projects, and a list of the tables they have access to. When a read policy is created or updated (or when a user's entitlements change, they switch projects, or when their data source access is approved or revoked), Immuta updates the secure view to grant or revoke users' access to the data source. If a user is granted access to the data source, they can access the view. If a user does not have read access to the view, zero rows are returned when they attempt to query the view.

Granting Azure Synapse Analytics privileges

The Azure Synapse Analytics integration supports read access subscription policies. Immuta grants the SELECT privilege to the PUBLIC role when the integration is configured, which allows all users who meet the conditions of a subscription policy to access the Immuta-managed view. When a read policy is created or removed (or when a user's entitlements change, they switch projects, or when their data source access is approved or revoked), Immuta updates the view that contains the users' entitlements, projects, and a list of tables they have access to grant or revoke their access to the dynamic view. Users' read access is enforced through an access check function in each individual view. If a user is granted access to the data source, they can access the view. If a user does not have read access to the view, they receive an Access denied: you are not subscribed to the data source error when they attempt to query the view.

Granting Google BigQuery privileges

The Google BigQuery integration supports read access subscription policies. In this integration, Immuta creates views that contain all policy logic. Each view has a 1-to-1 relationship with the original table, and read access controls are applied in the view. After data sources are registered, Immuta uses the custom user and role, created before the integration is enabled, to push the Immuta data sources as views into a mirrored dataset of the original table. Immuta manages grants on the created view to ensure only users subscribed to the Immuta data source will see the data.

Granting Amazon S3 privileges

S3 Access Grants READ and READWRITE access levels

Write access policy limitations

• With the exception of the Starburst (Trino) integration, users can only modify existing data when they are granted write access to data; they cannot create new tables or delete tables.

• Write actions are not currently captured in audit logs.

Data policies determine what users see when they query data in a table they have access to.

This guide provides a general overview of data policies and their behavior.

How-to guides

• : Certify policies, exempt users from policies, and view policy diffs on a data source.

• : Set up Immuta to use the encryption or masking algorithm in your external masking service.

Reference guides

• : This guide describes all the data policies available in Immuta.

• : This guide describes the types of masking policies available and when to use each.

• : Row-level policies compare data values with user metadata at query-time to determine whether or not the querying user should have access to the individual rows of data.

• : This guide describes the custom functions you can use to extend the PostgreSQL WHERE syntax.

• : In some cases, two conflicting global masking policies apply to a single data source. This guide describes how Immuta handles those conflicts.

• : When building a global data policy, governors can create custom certifications that must be acknowledged by data owners when the policy is applied to data sources.

• : These policies reduce conflicts between masking policies that apply to a single column, allowing policies to scale more effectively across your organization.

Data policies manage what users see when they query data in a table they have access to.

There are three different ways to restrict data with data policies:

• : Filter rows from certain users at query time.

• : Mask values in a column at query time.

• : Mask specific cells in a column based on separate values in the same row at query time.

When applying a data policy, it will always be enforced for all users, following the principle of least privilege, unless optional exceptions are added to policies. Data policy exceptions are built using any of the following conditions, which can be mixed with boolean logic:

• If the user is a member of a group (or several groups)

• If the user possesses a particular attribute (or several attributes)

• If the (or several purposes) for which the data is allowed to be used

Data policy exceptions are very similar to from this perspective. With subscription policies, nobody has access to a newly created table until someone says otherwise with a subscription policy (as long as you follow for newly created tables and views). Similarly, when a masking policy is set on a column or a row-level policy on a table, it applies to everyone until someone says otherwise with an exception to the data policy.

Leveraging lookup tables

If user metadata is stored in a table in the same data platform where a policy is enforced, it is not necessary to move that user metadata in Immuta. Instead it can be referenced directly using functions in data policies.

Below is an example row-level policy that leverages a lookup table to dynamically drive access to rows in a table:

The final column in the table, ACCESS_LEVEL, defines who can see that row of data.

Now consider the following hierarchy:

In this diagram, there are 11 different access levels (AL) to data and the tree defines access. For example, if a user has Vegetables, they get access levels 2, 3, 4, 9, 10, and 11. If a user has Pear, they only get access level 8. In other words, a user with Vegetables would see the first row of the above table, a user with Pear would see the second row of the above table, and a user with Food would see both rows of the table.

Taking the example further, that hierarchy tree is represented as a table in the data platform that we wish to use to drive the row-level policy:

That hierarchy lookup table can be referenced in the row-level policy as user metadata like this:

@columnTagged('access_level') IN (SELECT ACCESS_LEVEL from [lookup table] where @attributeValuesContains('user_level', 'ROOT'))

Walking through the policy step-by-step:

• @columnTagged('access_level'): This allows us to target multiple tables with an ACCESS_LEVEL column that needs protecting with a single policy. Simply tag all the ACCESS_LEVEL columns with the access_level tag and this policy would apply to all of them.

• IN (SELECT ACCESS_LEVEL from [lookup table]: This is selecting the matching ACCESS_LEVEL from the lookup table to use as the IN clause for filtering the actual business table.

So, you can then add metadata to your users in Immuta, such as Vegetables or Pear and that will result in them seeing the appropriate rows in the business table in question.

The above example used a row-level policy, but it could instead do cell masking using the same technique:

Mask columns tagged Credit Card Number using hashing where @columnTagged('access_level') NOT IN (SELECT ACCESS_LEVEL from [lookup table] where @attributeValuesContains('user_level', 'ROOT'))

In this case, the credit card number will be masked if the access_level is not found for the user for that row.

Even if not using a lookup table, the power of the @columnTagged('tag name') function is apparent for applying your masking or row-level policies at scale.

Other topics of interest

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Data Policies tab. Click Add Policy and enter a name for your policy.

   • Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.

2. Select Only show data by time from the first dropdown.

3. Select where data is more recent than or older than from the next dropdown, and then enter the number of minutes, hours, days, or years that you would like to restrict the data source to. Note that unlike many other policies, there is no field to select a column to drive the policy. This type of policy will be driven by the data source's event-time column, which is selected at data source creation.

4. Choose for everyone, everyone except, or for everyone who to drive the policy. If you choose for everyone except, use the subsequent dropdown to choose the group, purpose, or attribute for your condition. If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.

5. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.

6. For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

7. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

Data owners who are not governors can write restricted subscription and data policies, which allow them to enforce policies on multiple data sources simultaneously, eliminating the need to write redundant local policies.

Unlike global policies, the application of these policies is restricted to the data sources owned by the users or groups specified in the policy and will change as users' ownerships change.

1. Click Policies in the left sidebar and select Data Policies.

2. Click Add Policy and complete the Enter Name field.

3. Select how the policy should protect the data. Click a link below for instructions on building that specific data policy:

   • Mask

   • Only show rows

   • Only show data by time

   • Limit usage to purpose(s)

   • Minimize data source

4. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.

5. From the Where should this policy be applied dropdown menu, select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

6. Beneath Whose Data Sources should this policy be restricted to, add users or groups to the policy restriction by typing in the text fields and selecting from the dropdown menus that appear.

7. Click Create Policy, and then click Activate Policy or Stage Policy.

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Data Policies tab. Click Add Policy and enter a name for your policy.

   • Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.

2. Select Mask from the first dropdown menu.

3. Select columns tagged, columns with any tag, columns with no tags, all columns, or columns with names spelled like.

4. Select a masking type:

   • using hashing

   • with reversibility

   • by making null

   • using a constant: Enter a constant in the field that appears next to the masking type dropdown.

   • using a regex:

     1. Enter a regular expression and replacement value in the fields that appear next to the masking type dropdown.

     2. From the next dropdown, choose to make the regex Case Insensitive and/or Global.

   • by rounding: Select the Bucket Type and then enter the bucket size.

   • with format preserving masking

   • with K-Anonymization: Select either using fingerprint or requiring group size of at least and enter a group size in the subsequent dropdown menu.

   • using randomized response

   • using the custom function: Enter the custom function native to the underlying database.

     Note: The function must be valid for the data type of the column. If it is not, the default masking type will be applied to the column.

5. Select everyone except, everyone, or everyone who to continue the condition.

   • everyone except: In the subsequent dropdown menus, choose is a member of group, possesses attribute, or is acting under purpose. Complete the condition with the subsequent dropdown menus.

   • for everyone who: Complete the Otherwise clause. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

6. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.

7. For global policies: Click the dropdown menu beneath Where should this policy be applied and select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

8. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

Create a custom certification for a global policy

This step is optional, but data governors can add certifications that outline acknowledgements or require approvals from data owners. For example, data governors could add a custom certification that states that data owners must verify that tags have been added correctly to their data sources before certifying the policy.

1. Click Add Certification in the data policy builder.

2. Enter a Certification Label and Certification Text in the corresponding fields of the dialog that appears.

3. Click Save.

Requirement and prerequisite:

• CREATE_DATA_SOURCE or GOVERNANCE Immuta permission

• A purpose has been created

Build the policy

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Data Policies tab. Click Add Policy and enter a name for your policy.

   • Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.

2. Select Limit usage to purpose(s) in the first dropdown menu.

3. In the next field, select a specific purpose that you would like to restrict usage of this data source to or ANY PURPOSE. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

4. Select for everyone or for everyone except. If you select for everyone except, you must select conditions that will drive the policy such as group, purpose, or attribute.

5. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.

6. For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

7. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

Related guides

How-to guides

• Create a project: To restrict access to data and associate your data source with a purpose, create a project and add the purpose and relevant data sources to the project.

• Manage project purposes

Reference guides

• Projects and purposes

• Purpose-based policy restrictions

Conceptual guide

Why use projects?

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Data Policies tab. Click Add Policy and enter a name for your policy.

   • Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.

2. Select Minimize data source from the first dropdown.

3. Complete the enter percentage field to limit the amount of data returned at query-time.

4. Select for everyone except from the next dropdown menu to continue the condition. Additional options include for everyone and for everyone who.

5. Use the next field to choose the attribute, group, or purpose that you will match values against.

   Notes:

   • If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.

   • You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

6. Opt to complete the Enter Rationale for Policy (Optional), and then click Add.

7. For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

8. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.

1. Determine your policy scope:

   • Global policy: Click the Policies page icon in the left sidebar and select the Data Policies tab. Click Add Policy and enter a name for your policy.

   • Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.

2. Select the Only show rows action from the first dropdown.

3. Choose one of the following policy conditions:

   • Where user

     1. Choose the condition that will drive the policy from the next dropdown: is a member of a group or possesses an attribute.

     2. Use the next field to choose the attribute, group, or purpose that you will match values against.

     3. Use the next dropdown menu to choose the tag that will drive this policy. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the far right of the policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

   • Where the value in the column tagged

     1. Select the tag from the next dropdown menu.

     2. From the subsequent dropdown, choose is or is not in the list, and then enter a list of comma-separated values.

   • Where

     1. Enter a valid SQL WHERE clause in the subsequent field. When you place your cursor in this field, a tooltip details valid input and the column names of your data source. See Custom WHERE Clause Functions for more information about specific functions.

   • Never

     The never condition blocks all access to the data source.

     1. Choose the condition that will drive the policy from the next dropdown: for everyone, for everyone except, or for everyone who.

     2. Select the condition that will further define the policy: is a member of group, is acting under a purpose, or possesses attribute.

     3. Use the next field to choose the group, purpose, or attribute that you will match values against.

4. Choose for everyone, everyone except, or for everyone who to drive the policy. If you choose for everyone except, use the subsequent dropdown to choose the group, purpose, or attribute for your condition. If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.

5. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.

6. For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:

   • tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

   • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

   • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

   • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

7. Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.