arrow-left

All pages
gitbookPowered by GitBook
1 of 10

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Query Data

Data Consumers

Data consumers are Immuta users that consume the data available through Immuta in their data platform as usual. Find how-to guides specific to data consumers listed below.

hashtag
How-to guides

  • Subscribe to data sources: Subscribe to data sources in Immuta, run health jobs, and complete other actions as a data source subscriber.

  • Query data: Query policy-protected data in your normal data platform.

  • : Subscribe to projects in Immuta to collaborate with others, work under a purpose, or write data to a workspace.

Querying Starburst (Trino) Data

Prerequisites:

  • Starburst integration configured with Immuta

  • Starburst tables registered as Immuta data sources

hashtag
Query data

  1. Use your tool of choice to connect to Starburst.

  2. Query the Immuta-protected data as you normally would:

Querying Databricks SQL Data

Prerequisites:

  • Databricks Unity Catalog integration configured with Immuta

  • Databricks SQL tables registered as Immuta data sources

hashtag
Query data

  1. Select SQL from the navigation menu in Databricks.

  2. Click Create → Query.

  3. Run your query as you normally would:

Query Databricks SQL

  • Query Snowflake

  • Query Starburst (Trino)

  • Query Amazon Redshift Spectrum
    Query Azure Synapse Analytics
    Query Databricks
    Subscribe to projects
    select * from "tpch"."sf1000"."customer" limit 100
    SELECT
      concat(pickup_zip, '-', dropoff_zip) as route,
      AVG(fare_amount) as average_fare
    FROM
      `samples`.`nyctaxi`.`trips`
    GROUP BY
      1
    ORDER BY
      2 DESC
    LIMIT 1000

    Querying Databricks Data

    Prerequisites:

    • Databricks Unity Catalog integration configured with Immuta

    • Databricks tables registered as Immuta data sources

    hashtag
    Query data with Python

    1. Create a new workspace.

    2. Query the Immuta-protected data as you normally would:

    hashtag
    Query data with SQL

    1. Create a new workspace.

    2. Query the Immuta-protected data as you normally would:

    df = spark.sql('select * from database.table_name')
    df.show()
    select * from database.table_name;

    Querying Snowflake Data

    Prerequisites:

    • Snowflake integration configured with Immuta

    • Snowflake tables registered as Immuta data sources

    hashtag
    Query data with Snowflake table grants

    1. Execute the USE SECONDARY ROLES ALL command or change your role to the .

    2. Query the data as you normally would in Snowflake.

    hashtag
    Query data without Snowflake table grants

    Prerequisite: Users have been granted SELECT privileges on all relevant Snowflake tables

    Query the data as you normally would in Snowflake:

    SELECT * FROM emp_basic LIMIT 100;
    table grants role
    SELECT * FROM emp_basic LIMIT 100;

    Subscribe to a Data Source

    hashtag
    View my data sources

    1. Click on the Data Source icon in the navigation menu.

    2. By default, all data sources will be displayed.

    3. Opt to filter the data sources by your access, the backing technology, or the health of the data sources using filters.

    4. To view the overview details of a data source, click on the arrow icon for the data source, or open the data source details page.

    hashtag
    Subscribe to a data source

    1. Click the Get Access button from either the data sources list page or the data source overview tab, which can be accessed by clicking on the Data Source.

    2. If prompted, select the access type you would like to request (READ or WRITE), and then fill out the set up by the system admin and click Request Access.

    3. A notification will be sent to the data owners informing them of your request.

    hashtag
    Bulk access requests

    To request access to multiple data sources simultaneously,

    1. Navigate to the data sources list page.

    2. Select the checkboxes for the data sources you want to subscribe to.

    3. Select More Actions.

    hashtag
    Manually run health jobs

    If a data source health check fails and needs to be re-generated,

    1. From the Data Source list page, select the data source you want to re-run the health check on.

    2. Click the health indicator on the data source. If the health check failed, it will be Not Healthy.

    3. Select Re-run on the job you want to run.

    Note: To generate a fingerprint, the data source must be a Snowflake data source and the row count must be up-to-date.

    hashtag
    View the data dictionary

    To view the data dictionary,

    1. Select the data source from the data source list page.

    2. Navigate to the Data Dictionary tab.

    3. The data dictionary will display and include the column’s name, type, and tags. Masked columns will display a symbol next to their names.

    hashtag
    View data source contact information

    Contact information for data owners is provided for each data source, which allows other users to ask them questions about accessibility and attributes required for viewing the data.

    To view this contact information, click the Contacts tab.

    hashtag
    Search for data sources

    Immuta's UI provides a list of all data sources visible to the current user. Users can search for data sources by keyword, tag, column, connection string, or global policy name.

    • To access a list of all data sources, click the Data Sources icon in the navigation menu.

    • To search for data sources by keyword, type a keyword into the search box, and hit enter. If the list of results is empty, then no data sources with names matching that keyword currently exist.

    • To search for data sources by tag, column, connection string, or global policy name, type the relevant name into the search box and select the tag, column, connection string

    hashtag
    Create a data source search filter

    Data source search filters allow users to codify complex, recurring search criteria into a shared asset that can be used by any other user. This eliminates the need for team members to repeatedly reconstruct identical search queries for frequent tasks inside Immuta.

    To create a new data source search filter,

    1. Type your search criteria into the Global Search bar and hit enter.

    2. Review and refine the resulting list of data sources from your search, and then click Save Search Filter.

    3. In the dialog box, type in a Name for your search filter and then confirm by clicking Save.

    To view existing data source search filters,

    1. Click into the Global Search bar.

    2. Any available search filters will be displayed in the dropdown.

    To delete a data source search filter,

    1. Click into the Global Search bar.

    2. Any search filters that the current user is allowed to delete will be displayed with an ✖️ in the dropdown. Click ✖️ next to the search filter you want to delete.

    3. Click Confirm.

    circle-info

    Users with APPLICATION_ADMIN or USER_ADMIN global permission can delete search filters created by any user. All other users can only delete search filters that they originally created.

    Once reviewed, you will receive a notification with a response indicating if your request was accepted or denied.

  • If accepted, the status displayed next to that data source will be updated to "Subscribed" and you will have access to the data source via your personal SQL connection. If not accepted, a reason will be provided in the notification details.

  • Click Request Access.
  • If prompted, fill out the set up by the system admin and click Request Access.

  • A notification will be sent to the data owners informing them of your request.

  • Once reviewed, you will receive a notification with a response indicating if your request was accepted or denied.

  • If accepted, the status displayed next to that data source will be updated to "Subscribed" and you will have access to the data source via your personal SQL connection. If not accepted, a reason will be provided in the notification details.

  • , or
    global policy
    from the list of auto-completed results. If a list does not display, then no results matching that search criteria currently exist.
    custom request form
    custom request form

    Querying Azure Synapse Analytics Data

    Prerequisites:

    • Azure Synapse Analytics integration configured with Immuta

    • Azure Synapse Analytics tables registered as Immuta data sources

    • REVOKE users' access to raw tables

    • GRANT users' access to the Immuta schema

    hashtag
    Query data

    1. Click the Data menu in Synapse Studio.

    2. Click the Workspace tab.

    3. Expand the databases, and you should see the dedicated pool you specified when .

    Expand the dedicated pool and you should see the Immuta schema you created when configuring the integration.
  • Select that schema.

  • Select New SQL script and then Empty script.

  • Run your query (note that Synapse does not support LIMIT and the SQL is case sensitive). It should look something like this:

  • configuring the integration
    SELECT TOP 100 * FROM immuta_schema.backing_database_backing_table;

    Subscribe to Projects

    hashtag
    Subscribe to a project

    1. Click Data in the navigation menu and then select Projects.

    Click Join Project from either the all projects view or the project overview tab, which can be accessed by clicking on the project.
  • Click Join to confirm you want to join the project.

  • After you have been granted access to the project, go to the project and click I Agree to acknowledge that you will only use the project for its specified purposes.

  • hashtag
    Change project contexts

    hashtag
    Switch projects in the Immuta UI

    1. Click the Project dropdown menu in the menu bar. If no project is currently selected, it is No Current Project.

    2. Select a project. Once selected, the current project will display at all times in the menu bar.

    If you unsubscribe from the project, this display will default to No Current Project.

    hashtag
    Switch projects using UDFs in Databricks

    1. View your available projects by running the following query in Spark: select * from immuta.list_projects. In the resulting table, note the values listed in the id column; this value will be used at the parameter in the following step.

    2. Run select immuta.set_current_project(<id>). This UDF must be called in its own notebook cell to ensure the changes take effect.

    Your project context will be switched, and that project's data sources and workspaces will now be visible. To set your project context to None, run select immuta.set_current_project() with no parameters.

    Note: Since the UDFs are not actually registered with the FunctionRegistry, if you call DESCRIBE FUNCTION immuta.set_current_project, you won't get back the documentation for the UDF. For a complete list of functions, see the Project UDFs reference guide.

    hashtag
    Create a project-based API key

    Any project member can create project-based API keys, which are used for authenticating external tools with Immuta.

    1. Navigate to the Project Overview tab.

    2. Click Get API Key in the project information section.

    3. A window will display with the API key. Store this key somewhere secure. If you misplace it, you will have to generate a new one and re-authenticate all services connected to Immuta via this key.

    4. Click Close.

    hashtag
    Leave a project

    1. Navigate to the Project Overview tab.

    2. Click the Leave Project button, and then click Confirm.

    hashtag
    Manage project data sources

    hashtag
    Add data sources to a project

    Any project member can add data sources to a project, unless the project equalization or allow masked joins features are enabled; in those cases only project owners can add data sources to the project.

    1. Select the project, and then navigate to the Project Overview tab.

    2. Click the Add Data Sources button in the Data Sources section in the center pane.

    3. Start typing the name of a data source you'd like to include in the project.

    4. Select the data source from the list of auto-completed options in the dropdown menu.

    5. Repeat this process to add additional data sources to the list. Click Remove to remove them.

    6. When complete, click the Add button.

    hashtag
    Add data sources by purpose

    You can automatically add all data sources to a project that contain a Limit usage to purpose policy that matches the purpose of that project.

    1. Select a Project, and click the Add Data Sources button on the Data Sources tab.

    2. Click Add By Purpose in the dialog.

    3. All data sources matching the project's purpose(s) will populate. Review this list, and then click Save.

    hashtag
    Remove data sources from a project

    As a project member, you can only delete data sources you've added to the project.

    1. Select a project, and then click the Data Sources tab.

    2. Click the Remove Data Source icon in the Actions column of the data source you want to remove.

    3. Click Confirm in the window that appears.

    hashtag
    Search for projects

    Immuta's UI provides a list of all projects, excluding those that have been set to private. Users can search for projects by keyword, tag, or connection string.

    • To access a list of all projects, click the Projects icon in the navigation menu.

    • To search for projects by keyword, type a keyword into the search box, and hit enter. If the list of results is empty, then no projects with names matching that keyword currently exist.

    • To search for projects by tag or connection string, type the relevant name into the search box and select the tag or connection string from the list of auto-completed results. If a list does not display, then no results matching that search criteria currently exist.

    hashtag
    Create a project search filter

    Project search filters allow users to codify complex, recurring search criteria into a shared asset that can be used by any other user. This eliminates the need for team members to repeatedly reconstruct identical search queries for frequent tasks inside Immuta.

    To create a new project search filter,

    1. Type your search criteria into the Global Search bar and hit enter.

    2. Review and refine the resulting list of projects from your search, and then click Save Search Filter.

    3. In the dialog box, type in a Name for your search filter and then confirm by clicking Save.

    To view existing project search filters,

    1. Click into the Global Search bar.

    2. Any available search filters will be displayed in the dropdown.

    To delete a project search filter,

    1. Click into the Global Search bar.

    2. Any search filters that the current user is allowed to delete will be displayed with an ✖️ in the dropdown. Click ✖️ next to the search filter you want to delete.

    3. Click Confirm.

    circle-info

    Users with APPLICATION_ADMIN or USER_ADMIN global permission can delete search filters created by any user. All other users can only delete search filters that they originally created.

    Querying Redshift Spectrum Data

    circle-info

    This guide is specific to querying data sources registered in the Amazon Redshift Spectrum integration. For instructions on how to query data registered in the Amazon Redshift integration, see the Accessing data page.

    Prerequisites:

    • Redshift Spectrum integration configured with Immuta

    • REVOKE users' access to raw tables

    hashtag
    Query data

    1. Use your tool of choice to connect to Redshift.

    2. Select the Immuta database name used when .

    3. Query the Immuta-protected data, which takes the form of immuta_database.backing_schema.table_name:

    Immuta Database: The Immuta database name used when configuring the integration.

  • Backing Schema: The schema that houses the backing tables of your Immuta data sources.

  • Table Name: The name of the table backing your Immuta data sources.

  • Run your query (it is recommended that you use the catalog in the query). It should look something like this:

  • Redshift tables registered as Immuta data sources
    configuring the integration
    select * from immuta_database.backing_schema.table_name limit 100