REST Catalog Interface
Audience: Integration Developers
Content Summary: Building a REST Catalog shim to integrate a catalog with Immuta requires implementing a simple REST interface. This page documents these endpoints and provides example requests and responses.
Authentication
Immuta requires HTTP Basic Authentication for REST Catalogs. When accessing
the /dataSource
and /tags
endpoints, Immuta will use the configured
username and password. If you choose to also protect the human-readable pages
with authentication, users will have to authenticate when they visit the
page.
Endpoints
/tags
Description
The tags endpoint returns all the tags the catalog can provide. It is used when refreshing Immuta's tag list imported from this catalog.
Schema
This endpoint responds to a GET request with a map of tags to tag objects. In
Immuta, a tag is a string and can be nested. This nested structure is encoded in the /tags
response by joining multiple tags with a .
. For more information on tags and
how they are created, managed, and displayed within Immuta, please see our tag
documentation.
For the purpose of REST Catalog integrations, a tag has the following format.
<string>[.<string>[.<string>...]]
Tags returned via a REST Catalog must meet the following constraints.
- They must be no longer than 500 characters. Longer tags will not throw an error but will be truncated silently at 500 characters.
- They must be composed of letters, digits, underscores, dashes, and white space characters. A period is semantic as described above.
A tag object has a single property: id
, which should be unique across the
catalog. If the catalog does not have an ID field available, hashing the tag
should provide a unique value that will work for this purpose.
{"id": <integer or string>}
The complete response looks like this.
{
<Tag>: <Tag Object>[, <Tag>: <Tag Object>[, <Tag>: <Tag Object>...]]
}
Example
curl http://my.custom.catalog/tags
{
"tag.one": {"id": 1},
"tag.two": {"id": 2},
"parent.child": {"id": 3},
"triple.nested.tag": {"id": 4}
}
/dataSource
Description
The data source endpoint does most of the work. It receives a POST
request
and returns the mapping of a data source and its columns to the applied tags.
Schema
Request
The request from Immuta is a POST. The body is a simple object containing only the ID of the data source within the catalog.
{
"catalogMetadata": {
"id": <integer>
}
}
Response
The schema for the data source response reuses the tag list from above and also uses the following set of metadata keys for both data sources and columns.
"catalogMetadata": {
# The ID of the data source on the catalog. It is expected to be the same
# ID that was POSTed in the request.
"id": <integer>
},
# This description will be displayed in the Immuta UI
"description": <string>,
"tags": {
<Tag>: <Tag Object>[, <Tag>: <Tag Object>[, <Tag>: <Tag Object>...]]
}
{
<Metadata>, # for the data source overall
# This maps the column names to metadata objects
"dictionary": {
<string>: {
<Metadata>
}
}
}
Examples
curl -X POST http://my.custom.catalog/dataSource \
--header "Content-Type: application/json" \
--data '{"catalogMetadata": {"id": 123}}'
{
"catalogMetadata": {
"name": "My Test Catalog",
"id": 123
},
"description": "This catalog was created as a documentation example",
"tags": {
"tag.one": {
"id": 1
},
"tag.two": {
"id": 2
},
"parent.child": {
"id": 3
}
},
"dictionary": {
"some_column_name": {
"catalogMetadata": {
"id": 10
},
"description": "This column has example data in it",
"tags": {
"tag.one": {
"id": 1
},
"triple.nested.tag": {
"id": 4
}
}
}
}
}
/dataSource/page/{ID}
Description
This endpoint returns the catalog's human-readable information page for the data source. Immuta links to this page as additional information about the data source.
If the catalog already provides such a page, this endpoint can redirect the user to that page.
Examples
curl http://my.custom.catalog/dataSource/page/123
<html>
<head>
<title>data source 123</title>
</head>
<body>
data source 123 is a data source that was created just for documentation.
</body>
</html>
/column/{ID}
Description
This endpoint returns the catalog's human-readable information page for the column. Immuta links to this page as additional information about the column.
If the catalog already provides such a page, this endpoint can redirect the user to that page.
Examples
curl http://my.custom.catalog/column/10
<html>
<head>
<title>data source 123 Column 10</title>
</head>
<body>
Column 10 is full of example data for documentation reasons.
</body>
</html>