Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Data policies manage what users see when they query data in a table they have access to.
There are three different ways to restrict data with data policies:
Row-level: Filter rows from certain users at query time.
Column masking: Mask values in a column at query time.
Cell masking: Mask specific cells in a column based on separate values in the same row at query time.
When applying a data policy, it will always be enforced for all users, following the principle of least privilege, unless optional exceptions are added to policies. Data policy exceptions are built using any of the following conditions, which can be mixed with boolean logic:
If the user is a member of a group (or several groups)
If the user possesses a particular attribute (or several attributes)
If the user is acting under a purpose (or several purposes) for which the data is allowed to be used
Data policy exceptions are very similar to subscription policies from this perspective. With subscription policies, nobody has access to a newly created table until someone says otherwise with a subscription policy (as long as you follow best practices for newly created tables and views). Similarly, when a masking policy is set on a column or a row-level policy on a table, it applies to everyone until someone says otherwise with an exception to the data policy.
If user metadata is stored in a table in the same data platform where a policy is enforced, it is not necessary to move that user metadata in Immuta. Instead it can be referenced directly using custom WHERE functions in data policies.
Below is an example row-level policy that leverages a lookup table to dynamically drive access to rows in a table:
0123456789
Lewes, DE
00:07:34
4
9876543210
College Park, MD
09:16:08
8
The final column in the table, ACCESS_LEVEL, defines who can see that row of data.
Now consider the following hierarchy:
In this diagram, there are 11 different access levels (AL) to data and the tree defines access. For example, if a user has Vegetables
, they get access levels 2, 3, 4, 9, 10, and 11. If a user has Pear
, they only get access level 8. In other words, a user with Vegetables
would see the first row of the above table, a user with Pear
would see the second row of the above table, and a user with Food
would see both rows of the table.
Taking the example further, that hierarchy tree is represented as a table in the data platform that we wish to use to drive the row-level policy:
1
Food
2
Food
3
Food
4
Food
5
Food
6
Food
7
Food
8
Food
9
Food
10
Food
11
Food
2
Vegetables
3
Vegetables
4
Vegetables
9
Vegetables
10
Vegetables
11
Vegetables
5
Fruits
6
Fruits
7
Fruits
8
Fruits
4
Carrots
9
Leafy
10
Leafy
11
Leafy
5
Orange
6
Orange
7
Orange
8
Pear
10
Lettuce
11
Lettuce
That hierarchy lookup table can be referenced in the row-level policy as user metadata like this:
@columnTagged('access_level')
IN (SELECT ACCESS_LEVEL from [lookup table] where@attributeValuesContains('user_level', 'ROOT')
)
Walking through the policy step-by-step:
@columnTagged('access_level')
: This allows us to target multiple tables with an ACCESS_LEVEL column that needs protecting with a single policy. Simply tag all the ACCESS_LEVEL columns with the access_level
tag and this policy would apply to all of them.
IN (SELECT ACCESS_LEVEL from [lookup table]
: This is selecting the matching ACCESS_LEVEL from the lookup table to use as the IN clause for filtering the actual business table.
where @attributeValuesContains('user_level', 'ROOT')
: This is comparing the user's attribute user_level
to the value in the ROOT column, and if there's a match, that ACCESS_LEVEL is used for filtering in the previous step. See the custom WHERE documentation for more details on these functions.
So, you can then add metadata to your users in Immuta, such as Vegetables
or Pear
and that will result in them seeing the appropriate rows in the business table in question.
The above example used a row-level policy, but it could instead do cell masking using the same technique:
Mask columns tagged
Credit Card Number
using hashing where@columnTagged('access_level')
NOT IN (SELECT ACCESS_LEVEL from [lookup table] where@attributeValuesContains('user_level', 'ROOT')
)
In this case, the credit card number will be masked if the access_level is not found for the user for that row.
Even if not using a lookup table, the power of the @columnTagged('tag name')
function is apparent for applying your masking or row-level policies at scale.
There are several ways to mask data, and choosing the correct masking type has various tradeoffs. It's important to understand those tradeoffs when choosing masking types.
How data policy conflicts and intelligent fallbacks are managed
Data policies determine what users see when they query data in a table they have access to.
This guide provides a general overview of data policies and their behavior.
Policy certifications and diffs: Certify policies and view policy diffs on a data source.
All data policy types: This guide describes all the data policies available in Immuta.
Masking policies: This guide describes the types of masking policies available and when to use each.
Row-level policies: Row-level policies compare data values with user metadata at query-time to determine whether or not the querying user should have access to the individual rows of data.
Custom WHERE clause functions: This guide describes the custom functions you can use to extend the PostgreSQL WHERE syntax.
Data policy conflicts and fallback: In some cases, two conflicting global masking policies apply to a single data source. This guide describes how Immuta handles those conflicts.
Custom data policy certifications: When building a global data policy, governors can create custom certifications that must be acknowledged by data owners when the policy is applied to data sources.
Orchestrated masking policies: These policies reduce conflicts between masking policies that apply to a single column, allowing policies to scale more effectively across your organization.
Requirement and prerequisite:
CREATE_DATA_SOURCE
or GOVERNANCE
Immuta permission
Determine your policy scope:
Global policy: Click the Policies icon in the navigation menu and select the Data Policies tab. Click Add Policy and enter a name for your policy.
Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.
Select Limit usage to purpose(s) in the first dropdown menu.
In the next field, select a specific purpose that you would like to restrict usage of this data source to or ANY PURPOSE. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
Select for everyone or for everyone except. If you select for everyone except, you must select conditions that will drive the policy such as group, purpose, or attribute.
Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.
For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:
tagged: Select this option and then search for tags in the subsequent dropdown menu.
with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.
Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.
Create a project: To restrict access to data and associate your data source with a purpose, create a project and add the purpose and relevant data sources to the project.
Best practice: write global policies
Build global policies with tags instead of writing local policies to manage data access. This practice will prevent you from having to write or rewrite single policies for every data source added to Immuta.
Determine your policy scope:
Global policy: Click the Policies icon in the navigation menu and select the Data Policies tab. Click Add Policy and enter a name for your policy.
Local policy: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.
Select Mask from the first dropdown menu.
Select columns tagged, columns with any tag, columns with no tags, all columns, or columns with names spelled like.
Select a masking type (some of these types will only be available for Snowflake integrations):
using a constant: Enter a constant in the field that appears next to the masking type dropdown.
Enter a regular expression and replacement value in the fields that appear next to the masking type dropdown.
From the next dropdown, choose to make the regex Case Insensitive and/or Global.
by rounding: Select the Bucket Type and then enter the bucket size.
with K-Anonymization: Select either using fingerprint or requiring group size of at least and enter a group size in the subsequent dropdown menu.
using the custom function: Enter the custom function native to the underlying database.
Note: The function must be valid for the data type of the column. If it is not, the default masking type will be applied to the column.
Select everyone except, everyone, or everyone who to continue the condition.
everyone except: In the subsequent dropdown menus, choose is a member of group, possesses attribute, or is acting under purpose. Complete the condition with the subsequent dropdown menus.
for everyone who: Complete the Otherwise clause. You can add more than one condition by selecting + Add Another Condition. The dropdown menu in the policy builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.
For global policies: Click the dropdown menu beneath Where should this policy be applied and select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:
tagged: Select this option and then search for tags in the subsequent dropdown menu.
with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.
Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.
This step is optional, but data governors can add certifications that outline acknowledgements or require approvals from data owners. For example, data governors could add a custom certification that states that data owners must verify that tags have been added correctly to their data sources before certifying the policy.
Click Add Certification in the data policy builder.
Enter a Certification Label and Certification Text in the corresponding fields of the dialog that appears.
Click Save.
To manage and apply existing policies to data sources, a user must have either the CREATE_DATA_SOURCE Immuta permission or be manually assigned the owner role on a data source.
After a policy with a certification requirement is applied to a data source, data owners will receive a notification indicating that they need to certify the policy.
Navigate to the Policies tab of the affected data source, and review the policy in the Data Policies section.
Click Certify Policy.
In the Policy Certification modal, click Sign and Certify.
Once you have a data policy in effect, you can view the changes in your policies by clicking the Policy Diff button in the data policies section on a data source's policies tab.
The Policy Diff button displays previous policies and the current policy applied to the data source.
Data owners who are not governors can write restricted and data policies, which allow them to enforce policies on multiple data sources simultaneously, eliminating the need to write redundant .
Unlike global policies, the application of these policies is restricted to the data sources owned by the users or groups specified in the policy and will change as users' ownerships change.
Click the Policies icon in the navigation menu and select Data Policies.
Click Add Policy and complete the Enter Name field.
Select how the policy should protect the data. Click a link below for instructions on building that specific data policy:
Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.
From the Where should this policy be applied dropdown menu, select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:
tagged: Select this option and then search for tags in the subsequent dropdown menu.
with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.
Beneath Whose Data Sources should this policy be restricted to, add users or groups to the policy restriction by typing in the text fields and selecting from the dropdown menus that appear.
Click Create Policy, and then click Activate Policy or Stage Policy.
Determine your policy scope:
: Click the Policies icon in the navigation menu and select the Data Policies tab. Click Add Policy and enter a name for your policy.
: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.
Select the Only show rows action from the first dropdown.
Choose one of the following policy conditions:
Where user
Choose the condition that will drive the policy from the next dropdown: is a member of a group or possesses an attribute.
Use the next field to choose the attribute, group, or purpose that you will match values against.
Use the next dropdown menu to choose the tag that will drive this policy. You can add more than one condition by selecting + Add Another Condition. The dropdown menu then contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
Where the value in the column tagged
Select the tag from the next dropdown menu.
From the subsequent dropdown, choose is or is not in the list, and then enter a list of comma-separated values.
Where
Enter a valid SQL WHERE clause in the subsequent field. When you place your cursor in this field, a tooltip details valid input and the column names of your data source. See for more information about specific functions.
Never
The never condition blocks all access to the data source.
Choose the condition that will drive the policy from the next dropdown: for everyone, for everyone except, or for everyone who.
Select the condition that will further define the policy: is a member of group, is acting under a purpose, or possesses attribute.
Use the next field to choose the group, purpose, or attribute that you will match values against.
Choose for everyone, everyone except, or for everyone who to drive the policy. If you choose for everyone except, use the subsequent dropdown to choose the group, purpose, or attribute for your condition. If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.
Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.
For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:
tagged: Select this option and then search for tags in the subsequent dropdown menu.
with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.
Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.
Determine your policy scope:
: Click the Policies icon in the navigation menu and select the Data Policies tab. Click Add Policy and enter a name for your policy.
: Navigate to a specific data source and click the Policies tab. Scroll to the Data Policies section and click Add Policy.
Select Only show data by time from the first dropdown.
Select where data is more recent than or older than from the next dropdown, and then enter the number of minutes, hours, days, or years that you would like to restrict the data source to. Note that unlike many other policies, there is no field to select a column to drive the policy. This type of policy will be driven by the data source's event-time column, which is selected at data source creation.
Choose for everyone, everyone except, or for everyone who to drive the policy. If you choose for everyone except, use the subsequent dropdown to choose the group, purpose, or attribute for your condition. If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.
Opt to complete the Enter Rationale for Policy (Optional) field, and then click Add.
For global policies: Click the dropdown menu beneath Where should this policy be applied, and select On all data sources, On data sources, or When selected by data owners. If you select On data sources, finish the condition in one of the following ways:
tagged: Select this option and then search for tags in the subsequent dropdown menu.
with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.
Click Create Policy. If creating a global policy, you then need to click Activate Policy or Stage Policy.