First Steps

• Obtain a copy of Apache Chukwa. You can find the latest release on the Apache Chukwa release page.
• Un-tar the release, via tar xzf.
• Make sure a copy of Apache Chukwa is available on each node being monitored.
• We refer to the directory containing Apache Chukwa as CHUKWA_HOME. It may be helpful to set CHUKWA_HOME explicitly in your environment, but Apache Chukwa does not require that you do so.

General Configuration

• Make sure that JAVA_HOME is set correctly and points to a Java 1.6 JRE. It's generally best to set this in CHUKWA_HOME/etc/chukwa/chukwa-env.sh.
• In CHUKWA_HOME/etc/chukwa/chukwa-env.sh, set CHUKWA_LOG_DIR and CHUKWA_PID_DIR to the directories where Apache Chukwa should store its console logs and pid files. The pid directory must not be shared between different Apache Chukwa instances: it should be local, not NFS-mounted.
• Optionally, set CHUKWA_IDENT_STRING. This string is used to name Apache Chukwa's own console log files.

Configuration

First, edit $CHUKWA_HOME/etc/chukwa/chukwa-env.sh In addition to the general directions given above, you should set HADOOP_CONF_DIR and HBASE_CONF_DIR. This should be Apache Hadoop deployment Apache Chukwa will use to store collected data. You will get a version mismatch error if this is configured incorrectly.

Edit the CHUKWA_HOME/etc/chukwa/initial_adaptors configuration file. This is where you tell Apache Chukwa what log files to monitor. See the adaptor configuration guide for a list of available adaptors.

There are a number of optional settings in $CHUKWA_HOME/etc/chukwa/chukwa-agent-conf.xml:

• The most important of these is the cluster/group name that identifies the monitored source nodes. This value is stored in each Chunk of collected data; you can therefore use it to distinguish data coming from different groups of machines.

    <property>
      <name>chukwaAgent.tags</name>
      <value>cluster="demo"</value>
      <description>The cluster's name for this agent</description>
    </property>

• Another important option is chukwaAgent.checkpoint.dir. This is the directory Apache Chukwa will use for its periodic checkpoints of running adaptors. It must not be a shared directory; use a local, not NFS-mount, directory.
• Setting the option chukwaAgent.control.remote will disallow remote connections to the agent control socket.

Starting, Stopping, And Monitoring

To run an agent process on a single node, use:

  sbin/chukwa-daemon.sh start agent

Typically, agents run as daemons. The script bin/start-agents.sh will ssh to each machine listed in etc/chukwa/agents and start an agent, running in the background. The script bin/stop-agents.sh does the reverse.

You can, of course, use any other daemon-management system you like. For instance, tools/init.d includes init scripts for running Apache Chukwa agents.

To check if an agent is working properly, you can telnet to the control port (9093 by default) and hit "enter". You will get a status message if the agent is running normally.

Configuration

Edit etc/chukwa/auth.conf and add authorized user to the list.

Starting, Stopping, And Monitoring

Hadoop Infrastructure Care Center (HICC) is Apache Chukwa web user interface. HICC is started by invoking

  sbin/chukwa-daemon.sh start hicc

Once the webcontainer with HICC has been started, point your favorite browser to:

  http://<server>:4080/hicc

UNIX Processes For Apache Chukwa Data Processes

Apache Chukwa Data Processors are identified by:

  org.apache.hadoop.chukwa.datacollection.agent.ChukwaAgent
  org.apache.hadoop.chukwa.hicc.HiccWebServer

The processes are scheduled execution, therefore they are not always visible from the process list.

Emergency Shutdown Procedure

If the system is not functioning properly and you cannot find an answer in the Administration Guide, execute the kill command. The current state of the java process will be written to the log files. You can analyze these files to determine the cause of the problem.

kill -3 <pid>