Log Analysis Tool
Audience: System Administrators
Content Summary: This page details how to use the
immuta_hdfs_log_analyzer
tool to troubleshoot slowdowns in your CDH cluster.
Overview
Sub-optimal configuration of the Immuta HDFS NameNode plugin may cause cluster-wide slowdowns under certain conditions. The NameNode plugin contains a variety of cache settings to limit the number of network calls that occur within the NameNode's locked permission checking operation. If these settings are configured properly, there will be little to no impact on the performance of HDFS operations.
You can use the immuta_hdfs_log_analyzer
command-line utility to track the number of API calls coming from NameNode plugin to the Immuta Web Service.
Usage
You can download the log analysis tool here:
It can be invoked like so:
Options
START_TIME (
-s
,--start-time
): Timestamp for the beginning of the period to analyze.END_TIME (
-e
,--end-time
): Timestamp for the end of the period to analyze.GRANULARITY (
g
,--granularity
): Defines time buckets for analysis. Can beMINUTES
,HOURS
orDAYS
.TIME_FORMAT (
-t
,--time-format
): The format to use for timestamps. This should match the timestamp format in the Immuta Web Service logs.
Output
If you are able to correlate time buckets from this tool's output to periods of slow cluster performance, you may need to adjust configuration for the Immuta HDFS NameNode plugin.
Last updated