Create and Manage Databricks Spark Project Workspaces

After workspaces are configured, project owners can enable workspaces within their projects. This feature allows project members to write data to projects and share this data with other users as derived data sources.

Requirement: You must own the project

Prerequisites:

• Databricks Spark integration configured.

• Databricks workspace configured.

• Databricks tables registered in Immuta.

• External IDs have been mapped in for Databricks.

Create a Databricks Spark workspace

Databricks cluster configuration

Before creating a workspace, the cluster must send its configuration to Immuta; to do this, run a simple query on the cluster (i.e., show tables). Otherwise, an error message will occur when you attempt to create a workspace.

1. Navigate to the Policies tab and enable Project Equalization by clicking the Project Equalization slider to on.

2. Scroll to the Native Workspace section and click Create.

3. Select Databricks from the Workspace Configuration dropdown menu.

4. Opt to edit the sub-directory in the Workspace Directory field; this sub-directory auto-populates as the project name.

5. Enter the Workspace Database Name.

6. Click Create to enable the workspace.

Delete a workspace

1. Scroll to the Native Workspace section on the policies tab and click the toggle to disable the workspace.

2. Click Delete in the native workspace section.

3. Choose one of the following options in the modal:

   • Purge Generic Workspace Data: Permanently delete data, while the data used by derived data sources is preserved. Note: If you created a derived data source that references a view on top of a table in Snowflake that isn't a derived data source, that table will be deleted and break the derived data source.

   • Purge Everything & Delete Derived Data Sources: Permanently delete data and purge all derived data sources.

4. Click Delete.