Managing Data Sources Tutorial

Audience: Data Owners

Content Summary: As a Data Owner, you create the data sources and prescribe the policy controls around the data you're exposing to users. Data can be structured or unstructured and can be made accessible from anywhere. This guide details the basics of managing an Immuta data source, including editing your data source; managing users; creating documentation; creating and managing the Data Dictionary; managing public queries; and disabling, deleting, and restoring a data source.

See the Advanced Data Source Management guide for information on how to create highly customized solutions for your data sources.

Creating a Data Source

To create a new data source,

1. Click the (+) icon in the lower left corner of Immuta, and then click the data source icon.

   

2. Follow the instructions for query-backed data sources or object-backed data sources depending on your chosen storage technology.

Alternatively,

1. Navigate to the My Data Sources page.
2. Click the add data sources button.
3. Follow the instructions for query-backed data sources or object-backed data sources depending on your chosen storage technology.

Editing a Data Source

Data Owners have the ability to edit their data source settings, which include, but are not limited to, the folder/bucket/table/SQL statement used to build the data source, its output type, columns, column types, event time, directory structure and/or latency tolerance.

To change one or more of these items,

1. Navigate to the Overview tab.

2. Click the menu icon in the upper right corner of the page and select Edit Data Source.

   

3. This will drop the Data Owner inside the data source workflow with access to all of the current data source settings. Simply navigate through the workflow to the item that needs to be changed and change it.

   Note: Some settings cannot be changed once the data source has been created. In these cases, simply create a new data source with the new settings. 1. When completed, navigate to the end of the workflow and click Save.

   Note: Some data sources may require the Data Owner to reconnect to the remote database (from the Connection Information page) before any changes to the data source can be saved.

For more information on data source settings, see the tutorials for query-backed data sources and object-backed data sources.

Viewing User Access Requests

Requests are where Immuta keeps track of a user's outgoing and incoming requests to data sources. Outgoing requests are requests submitted by the user requesting access to other users' data sources. Incoming requests are requests received from another user requesting access to a data source. Similar to notifications, a red dot displays whenever there are pending requests.

Requests Icon

To view pending requests,

1. Click on the Requests icon in the upper right corner of the console window (cell phone icon).

   

2. Manage incoming requests in the resulting drop-down menu by clicking the Approve button or the Deny button.

Requests Tab

To view and manage pending requests,

1. Navigate to your Profile page, and then click the Requests tab. The names of the users who have submitted requests are displayed in the left pane. Once a user is selected, the corresponding pending requests are displayed on the right.

   

2. If you would like to view more information about the request, click the View Details button in the Actions column of the request you would like to view, and then click Close.

3. To deny a request, click the Deny button in the Actions column of the request you would like to deny.
4. To approve a request, click the Approve button in the Actions column of the request you would like to approve.

Bulk Approvals

To approve or deny multiple access requests simultaneously,

1. Navigate to your Profile page, and then click the Requests tab.

2. Select the checkbox next to each request you would like to approve or deny, and then click the Approve Selected or Deny Selected button.

   

Managing Data Source Users

1. Navigate to the Members tab.

From here, Data Owners can add users or groups, set user access expiration dates, modify user or group roles, view user or group subscription history, or remove users or groups.

Adding Users or Groups to a Data Source

1. Navigate to the Members tab, and then click Add Members in the top right of the center pane.

2. Fill in the User’s Name (or Group's Name) and the Role (as a subscriber, expert, alternative owner or ingest user) fields in the form that appears. You can also opt to specify an expiration date for when the user’s access should expire.

   Immuta will display a warning if users do not have access to any of the data because of policies applied to the data source.

   

3. When finished, click Add and the user or group will automatically be added to the data source.

Users and/or group members (as well as alternative Data Owners) will receive notifications that new users have been approved to access the data source.

Setting User Access Expiration Date for a Data Source

As a Data Owner, you can limit the amount of time a user or group has access to your data source by setting an access expiration date.

1. Navigate to the Members tab.

2. Increment or decrement the number of days under the Expires column for the user/group whose access you want to limit (the limit is counting from today, so users/groups with 0 days left means their access will be revoked by the end of today and users with 1 day left means their access will be revoked by the end of tomorrow).

   

3. Hit enter to save the changes.

4. To remove the limit (or set the limit to Never), delete the number from the field and hit enter.

Modifying User or Group Roles within a Data Source

1. Navigate to the Members tab.

2. Click the drop-down arrow under the Role column next to the user/group whose role you’d like to change.

   

3. Select another role (subscribed, expert, owner or ingest user, if applicable).

Notifications about the change will be sent to the affected users and groups (as well as alternative Owners).

Viewing User or Group Subscription History

1. Navigate to the Members tab.

2. Click the drop-down arrow under the Actions column next to the user or group whose history you’d like to review.

   

Removing Users or Groups from a Data Source

As a Data Owner, you can deny access to any users or groups at any time.

1. Navigate to the Members tab.

2. To remove a user or group from a data source, click the Deny button in the Access column next to the user or group you'd like to remove.

   

3. Complete the Deny Access form, including a reason for revoking the access.

   

4. This action will immediately update users' or groups' subscription status, and they will no longer have any access to the data source.

5. Notifications will be sent to the affected users (as well as alternative Data Owners) informing them of the change in subscription status.

This value can be updated at any time before the user’s access expires.

Running Queries in the Query Editor

The Query Editor allows users to write, modify, and execute queries against data sources they are subscribed to.

Previewing Sample Data

1. Click the Query Editor icon in the left sidebar.

   

2. Select a data source in the Tables list.

3. Click the dropdown menu icon next to the data source and select Preview Sample Data, or click Preview Sample Data in the Table Schema panel.

   

4. View data in the Results panel.

   • Filter results by clicking the overflow menu next to the column name.
   • Rearrange and resize columns by clicking and dragging.
   • Run and export full results or export current results to .csv by clicking one of the corresponding download buttons in the top right corner of the table.

Writing and Executing Queries

1. Click the Query Editor icon in the left sidebar.
2. Write your query in the Query Editor panel.

3. Execute your query by clicking the Run Query button. Note: Clicking this button will only run the currently highlighted query. Queries (or portions of queries) can be executed by manually highlighting the query (or portion of the query) and clicking Run Query.

   

4. View data in the Results panel.

   • Filter results by clicking the overflow menu next to the column name.
   • Rearrange and resize columns by clicking and dragging.
   • Export results to .csv by clicking the download button in the top right corner of the table.

Managing Public Queries

You will receive activity notifications when users run queries against policy-enforced data sources , and you have the ability to monitor public queries.

If a public query is too revealing or inappropriate to share, you can revert them to private.

To make public queries private,

1. Navigate to the Queries tab.

2. Click the Public queries view button in the top left of the page.

   

3. Click the slider icon located to the left of the query you want to make private again.

   

The page will automatically update and no longer display that query publicly. It will now only display under the original user’s My Queries page.

Managing the Data Dictionary

For query-backed data sources,

1. Navigate to the Data Dictionary tab.

   

2. To add or edit alternative column names and definitions, click the Edit icon in the Actions column next to the entry you'd like to change.

3. Complete the fields in the form that appears, and then click the Save Changes icon in the upper right corner of the form.

For object-backed data sources,

1. Navigate to the Data Dictionary tab. There will be no default entries in the Data Dictionary for an object-backed data source.

   

2. To add additional columns, click New Entry.

3. Complete the fields in the form that appears, and then click Save All.

   

4. To add or edit alternative column names and definitions, click the Edit icon in the Actions column next to the entry you'd like to change.

5. To add child definitions, click the plus (+) icon to the far right of the entry and fill out the nested column name, type, and definition.

   

6. To remove a column, click the Delete icon to the far right of the entry.

For details on Dictionary discussions, see Commenting on Dictionary Discussions.

Managing Tasks

If users make Debug Query or Unmask requests, a Tasks tab will appear on the Data Source Overview page for the user making the request and the user receiving the request. From this tab, users can view and manage two different task views:

• Your Created Tasks: This page lists the status and information of the Debug Query or Unmask requests you've submitted.
• Tasks For You: This page lists the status and information of the Debug Query or Unmask requests that have been submitted to you.

To complete a Query Debug task,

1. Navigate to the Tasks tab from the Data Source Overview page, and then click the toggle at the top of the page to Tasks For You.

2. Click the Debug Query icon in the Actions column of the relevant request. Note: You can view information about this request, including the reason for the request and the date is was created, by clicking the Task Info button in the Actions column.

   

To complete an Unmask task,

1. Navigate to the Tasks tab from the Data Source Overview page, and then click the toggle at the top of the page to Tasks For You.
2. Click the Unmask Values icon in the Actions column of the relevant task.
3. A dialog box will appear with the masked and unmasked value. Click Close after you have accessed the value. Note: You can view information about this request, including the reason for the request and the date is was created, by clicking the Task Info button in the Actions column.

To delete a task,

1. Navigate to the Tasks tab from the Data Source Overview page, and then click the toggle at the top of the page to Tasks For You.
2. Click the Delete icon in the Actions column of the relevant task.

Running Data Source Health Checks

Immuta data sources come equipped with health status indicators on the data source overview.

To determine the health of a data source, the following tests are run:

• List Files: Verifies that the data source's directory structure is functional (object-backed data sources only).
• Read Files: Verifies that blobs can be fetched from the data source (object-backed data sources only).
• Run SQL Statement: Verifies that the data source is accessible through the Immuta Query Engine.

The health of a data source can be one of three states:

• Healthy: All three health tests have run successfully.
• Not Healthy: One or more of the health tests failed.
• Health Unknown: Health tests could not be run. This can happen if the Data Owner does not have access to any data due to policy restrictions.

Health checks run automatically on the data source Overview tab. If you wish to re-run the health check, follow these instructions:

1. Navigate to the Overview tab.

2. Click the health status in the top right corner. The status will either be green (healthy), orange (health unknown), or red (not healthy).

   

3. Click Run Tests in the resulting drop-down menu. The health status of the data source will be updated.

   

Disabling a Data Source

Disabling a data source essentially hides it and its data from all users except the Data Owner. While in this state, the data source will display as disabled in the console for the Data Owner(s) and other users will not be able to see it at all. Data Owners can also continue to view their data through their SQL connection, but other users cannot; to them, it will appear as if the disabled data source no longer exists.

To disable a data source,

1. Navigate to the Overview tab.

2. Click on the menu icon in the upper right corner and select Disable.

   

A label will appear next to the data source indicating it is now disabled, and a notification will be sent to all users of the data source informing them that the data source has been disabled.

Restoring a Data Source

1. Navigate to the Overview tab.

2. Click on the menu icon in the upper right corner and select Restore.

   

A notification will be sent out to all users of the data source informing them that the data source has been restored.

Deleting a Data Source

Deleting a data source permanently removes it from Immuta. Data sources must first be disabled before they can be deleted.

1. Assuming you have already disabled the data source, navigate to the Overview tab.

2. Click on the menu icon in the upper right corner and select Delete.

   

3. Confirm that the data source should be deleted.

   

A notification will be sent out to all users of the data source informing them that the data source has been deleted.

Adding Tags to a Data Source

Data Owners in Immuta have the ability to apply tags to data sources and specific columns within data sources. To add a tag, that tag must have already been created by an Immuta Governor. For more details, see the Tags guide.

To add a tag to a data source,

1. Navigate to the Overview tab, scroll to the Tags section, and click Add Tags.

   

2. In the resulting pop-up window, search for tags that you want to add to the data source. You can apply multiple tags at once. When complete, Click Add at the bottom of the window.

   

   You can now view the tags that have been applied to this data source in the Overview tab.

   

To add a tag to a specific column within a data source,

1. Navigate to the Data Dictionary tab.

2. In the Data Dictionary tab, find the column(s) that you wish to tag, and click the tag icon for that column on the right side of the page.

   

3. In the resulting pop-up window, search for tags that you want to add to the column. You can apply multiple tags at once. When complete, Click Add at the bottom of the window.

   

   You can now view the tags that have been applied to this column in the Data Dictionary tab.