Integrating with Existing Catalogs
Last updated
Was this helpful?
Last updated
Was this helpful?
If you would like data consumers to discover data products through your existing catalog rather than through the Immuta Marketplace app, this is possible, by redirecting requests for access from your catalog to the Marketplace app. However, we do feel .
Integration is made possible through deep link URLs provided by the Marketplace app that direct the user to the request access screen for any given data product. At a high level the workflow looks like this:
A data product is published in your catalog that the same data product in the Marketplace app
The consumer discovers data product in your catalog
The consumer from your catalog to the request access page for that mirrored data product in the Marketplace app
From this point forward, the works the same
In order to request access to a data product, your catalog must have a mirrored representation of the corresponding data product that exists in the Marketplace app. Depending how you configure the redirect, it's likely these need to be named exactly the same so that the works correctly and you don't confuse your data consumers.
Right now you must separately publish the data product in your catalog, but once Marketplace supports webhooks, it will be possible to listen to those to automate publishing in your catalog when published in the Marketplace app.
To support redirects, Marketplace exposes deep link URLs to the request access page for each data product. That URL can be built two ways:
https://app.immutacloud.com/marketplace/data-product/{data product name}/request-access
https://app.immutacloud.com/marketplace/data-product/{data product id}/request-access
https://app.immutacloud.com/marketplace/data-product/{data product name}/request-access?accountId={account identifier}
https://app.immutacloud.com/marketplace/data-product/{data product id}/request-access?accountId={account identifier}
Make sure that if you use the data product name URL option that if you change the data product name in the Marketplace app, you also change it in the URL/catalog. Note that in Marketplace all data product names must be unique.
These URLs will direct the user to the data product's access request screen or, if they are not authenticated, to the Marketplace login screen with a redirect back to the access request screen.
The following are only recommendations for popular catalogs, you can do this however you feel appropriate and can reuse some of these approaches in different catalogs or custom marketplace applications you may have built yourselves.
The first thing to consider when linking a catalog is how you appropriately represent a domain, data product, and data source in that catalog should it not have those object types. The most common approach we've seen users take with Collibra is the following:
Domain (which contains the data product(s)) = Collibra Collection
Data product (which contains the data source(s)) = Collibra Data Set
Data source = Collibra Data Asset
A Collibra custom workflow can be leveraged to redirect to the Marketplace app to request access. We have a sample workflow created you can download. This workflow was exported in a way that allows you to see its workflow "code" in the Collibra Workflow designer should you want to make changes.
This workflow is intended to be an example you can use and is not part of the Immuta software product nor SLAs.
Once you are happy with the workflow "code" you must deploy it (top right of the Collibra workflow designer) and then configure Collibra:
Uncheck "Show in global create"
This workflow "attaches" to a Data Set by configuring the "Applies To" in Collibra to a Data Set. You can change this to whatever object you represent data products with in Collibra.
Ensure the variables are empty
For the "Start Events" choose "Workflow Started"
Under "Other" ensure "Any user can start the workflow"
Save
The first thing to consider when linking a catalog is how you appropriately represent a domain, data product, and data source in that catalog should it not have those object types. There is not an obvious approach to this in Alation; the best we've seen is:
Domain (which contains the data product(s)) = Alation Glossary
Data product (which contains the data source(s)) = Alation Domain
Data source = Alation Data
We hope to work closely with Alation to make this smoother in the future - ask them about it!
The first thing to consider when linking a catalog is how you appropriately represent a domain, data product, and data source in that catalog should it not have those object types. Thankfully, Atlan's object mapping is very close to the marketplace app:
Domain (which contains the data product(s)) = Atlan Domain
Data product (which contains the data source(s)) = Atlan Data Product
Data source = Atlan Data Asset
We hope to work closely with Atlan to make this smoother in the future - ask them about it!
While you are able to create request and approval flows in some catalogs and even other tools, we do not recommend you do this. Doing data request and approval flows in Immuta has various benefits and future benefits:
When approved, the access is automatically provisioned in the data platform
Provides full audit and reporting (coming soon) of requests/approvals
In the future, we intend to provide even more details to make the access determination as accurate and meaningful as possible:
More details about the requestor (attributes and groups)
What is in the data
Existing queries and types of questions asked of the data
Existing access/denials and why
Request policies will be able to be dynamically configured based on the risk level of the data product (coming soon). For example, if the data product contains PII and the requestor is a contractor: require x approvers and y data use agreement. Conversely, if the data does not contain PII and the employee is an internal employee: grant automatic access, etc.
Access requests can be temporarily approved which can avoid recertifications (coming soon)
Access requests to data can be as granular as you need, even down to column and row (coming soon)
You should also . This will ensure the user being redirected to request access does not need to know the accountId in order to login:
The data product ID can be found by examining the data product details page's URL here: or by simply clicking the Request access link button found on the data product details screen, which will place the above URL in your clipboard, including the account identifier.
When , it is possible to include your own custom data product ID as long as that data product ID is a valid UUID (which is the case for Collibra, for example). Doing so would allow you to have matching data product IDs in your catalog and Immuta allowing you to easily use this more resilient redirect method.
Once complete, when you visit a Data Set in Collibra there should be a new action "Request Access through Immuta". When clicked, a modal will appear asking to request access, and when selected, will dynamically redirect to the appropriate data product in the Immuta Marketplace app using the . Note that although Collibra is supposed to URL encode special characters from this workflow, it only appears to do so for spaces and not other characters like parentheses, so limit special characters in your data product name.
Unlike Collibra, there is not a dynamic way to redirect to the Marketplace app to request access. Instead, we recommend that you include a Description in the Alation Domain which includes a hyperlink to request access using the .
Unlike Collibra, there is not a dynamic way to redirect to the Marketplace app to request access. Instead, we recommend that you include a READ ME in the Atlan data product which includes a hyperlink to request access using the .
The person making a determination on a request is presented with in order to make a meaningful access decision. Furthermore, users doing approvals to data are different from the users doing approvals to applications, and require a more specific process and workflow more coupled to the data platform and its metadata