Hadoop has the concept of a
Group Mapping Service,
which is a way to retrieve groups corresponding to a
provided user/Kerberos principal. By default, Hadoop services (HDFS, Hive, Impala, etc.) will retrieve a
user's groups from the local OS by way of the
With Immuta, this data can be enriched to include the user's current project. This can be helpful for systems where current project information could help provide access to data. For example, in Impala it is possible to GRANT access to a database or tables based on a user's membership in an Immuta project group. This way a user could read information from tables only when acting in the target project.
Immuta project group names are simply
project_id is just the Immuta
In Impala it is important that the
auth_to_localsetting is enabled in order to map Kerberos principals to short usernames in order to properly retrieve groups from Immuta for the corresponding principal. For example, Impala should map
bobin order to properly map
bobto the corresponding Immuta user account to determine the current project group (if any) for
If Immuta HDFS Native Workspaces are being created on the target cluster, then the Immuta Partition Service principal needs to be a Sentry Admin user in order to
CREATEdatabases and roles for use by Immuta.
If administrators want to allow users to
CREATEnon-data-source tables in the workspace database, the
immuta.workspace.allow.create.tableconfiguration option must be set to
truefor the Partition Service in
generator.xml. It is also recommended that Sentry Object Ownership is enabled and set to
ALLin this scenario, which allows users to
DROPtheir own tables. If Object Ownership is not enabled, users will not be able to
DROPtables and a Sentry Admin would need to clean up old tables.
For Hive, it is required that the ImmutaGroupsMapping service jar is added to the classpath for YARN applications. This can be done by updating the
yarn.application.classpathconfiguration value in
yarn-site.xml. In Cloudera manager the value
/opt/cloudera/parcels/IMMUTA/lib/immuta-group-mapping.jarshould be added under
YARN Application Classpathin Yarn's Cloudera Manager configuration page. Note that if you are using a non-standard parcel directory, you should replace
/opt/cloudera/parcels/with your custom directory.
Check the Existing Group Mapping Service
It's a good idea to start by checking the existing Group Mapping Service in the configuration item
hadoop.security.group.mapping. If this is not found in your configuration,
the default is
To use ImmutaGroupsMapping alongside another Group Mapping Service, there is an implementation called
org.apache.hadoop.security.CompositeGroupsMapping. This Group Mapping Service takes the results of multiple
Group Mapping Services and combines them. If
CompositeGroupsMapping is already being used before
adding ImmutaGroupsMapping, simply add ImmutaGroupsMapping as another provider in configuration.
This will be detailed below.
Configuration to Add
To enable the ImmutaGroupsMapping, the following configuration needs to be added to Hadoop XML configuration for any target systems where the groups mapping should be applied.
- If this should be applied across the cluster it should be added to the system-wide
- If it is only being applied to a single system (Impala for instance), then it should be added to an XML file specifically for Impala.
It is recommended that the group mapping service is applied only to target systems requiring Immuta project
groups to be determined for context-aware permissions. This can typically be limited to Hive and/or Impala.
In Cloudera Manager this configuration can be added to
Impala Daemon Advanced Configuration Snippet (Safety Valve) for core-site.xml and/or
Hive Service Advanced Configuration Snippet (Safety Valve) for core-site.xml.
The following configuration shows the ImmutaGroupsMapping provider being used alongside the
<property> <name>hadoop.security.group.mapping</name> <value>org.apache.hadoop.security.CompositeGroupsMapping</value> <final>true</final> </property> <property> <name>hadoop.security.group.mapping.providers</name> <value>jni,immuta</value> <final>true</final> </property> <property> <name>hadoop.security.group.mapping.providers.combined</name> <value>true</value> <description>Set to true if the results of all mapping services should be concatenated.</description> <final>true</final> </property> <property> <name>hadoop.security.group.mapping.provider.jni</name> <value>org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback</value> <final>true</final> </property> <property> <name>hadoop.security.group.mapping.provider.immuta</name> <value>com.immuta.security.ImmutaGroupsMapping</value> <final>true</final> </property>
If the group mapping service is being configured for a specific service (i.e., Hive or Impala), it is
immuta.system.api.key is also configured for that target service. The
ImmutaGroupsMapping provider requires the system API key in order to retrieve user group details.
Add something like the following to the properties defined above.
<property> <name>immuta.system.api.key</name> <value>mYIUy6REcWrnW1mtVjZpuZiyyRFVj3</value> <final>true</final> </property>
By default, Hadoop's group services cache the retrieved groups for 5 minutes. This may not be desirable for Immuta deployments using group mapping because switching project contexts would then take up to 5 minutes to have an effect on the cluster. In order to lower this cache time, add the following configuration to the same file as above:
<property> <name>hadoop.security.groups.cache.secs</name> <value>10</value> <description>Set to whatever value is most tolerable for the delay in group change for a user.</description> </property>