Py4j Security Error

Audience: Data Users and System Administrators

Content Summary: This page provides an explanation and solution for this common Databricks error.

Py4j Security Error Details

• Error Message: py4j.security.Py4JSecurityException: Constructor <> is not whitelisted

• Explanation: This error indicates you are being blocked by Py4j security rather than the Immuta Security Manager. Py4j security is strict and generally ends up blocking many ML libraries.

• Solution: Turn off Py4j security on the offending cluster by setting IMMUTA_SPARK_DATABRICKS_PY4J_STRICT_ENABLED=false in the environment variables section. Additionally, because there are limitations to the security mechanisms Immuta employs on-cluster when Py4j security is disabled, ensure that all users on the cluster have the same level of access to data, as users could theoretically see (policy-enforced) data that other users have queried.