org.apache.hadoop.chukwa.datacollection.writer.localfs

Class LocalWriter

java.lang.Object

org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalWriter

All Implemented Interfaces:

ChukwaWriter

public class LocalWriter
extends Object
implements ChukwaWriter

This class is thread-safe -- rotate() and save() both synchronize on lock object.

Write data to a local fileSystem then move it to the remote HDFS
Warning:
There's no lock/waiting time for the remote client. The connection is released as soon as the last append is done, so therefore there is no guarantee that this class will not loose any data.
This class has been designed this way for performance reason.

In order to use this class, you need to define some parameters, in chukwa-collector-conf.xml

<property>
<name>chukwaCollector.localOutputDir</name>
<value>/grid/0/gs/chukwa/chukwa-0.1.2/dataSink/</value>
<description>Chukwa data sink directory</description>
</property>

<property>
<name>chukwaCollector.writerClass</name>
<value>org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalWriter</value>
<description>Local chukwa writer</description>
</property>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

ChukwaWriter.COMMIT_PENDING, ChukwaWriter.CommitStatus

Field Summary

Fields inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

COMMIT_FAIL, COMMIT_OK

Constructor Summary

Constructors

Constructor and Description
LocalWriter(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
ChukwaWriter.CommitStatus	add(List<Chunk> chunks) Best effort, there's no guarantee that chunks have really been written to disk
void	close() Called once, indicating that the writer should close files and prepare to exit.
protected void	computeTimePeriod()
protected String	getNewFileName()
void	init(org.apache.hadoop.conf.Configuration conf) Called once to initialize this writer.
protected void	rotate()
void	setup(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalWriter

public LocalWriter(org.apache.hadoop.conf.Configuration conf)
            throws WriterException

Throws:

WriterException

Method Detail

init

public void init(org.apache.hadoop.conf.Configuration conf)
          throws WriterException

Description copied from interface: ChukwaWriter

Called once to initialize this writer.

Specified by:

init in interface ChukwaWriter

Throws:

WriterException

setup

public void setup(org.apache.hadoop.conf.Configuration conf)
           throws WriterException

Throws:

WriterException

computeTimePeriod

protected void computeTimePeriod()

add

public ChukwaWriter.CommitStatus add(List<Chunk> chunks)
                              throws WriterException

Best effort, there's no guarantee that chunks have really been written to disk

Specified by:

add in interface ChukwaWriter

Returns:

CommitStatus

Throws:

WriterException

getNewFileName

protected String getNewFileName()

rotate

protected void rotate()
               throws WriterException

Throws:

WriterException

close

public void close()

Description copied from interface: ChukwaWriter

Called once, indicating that the writer should close files and prepare to exit.

Specified by:

close in interface ChukwaWriter