org.apache.hadoop.chukwa.datacollection.writer

Class SeqFileWriter

java.lang.Object

org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter

All Implemented Interfaces:

ChukwaWriter

Direct Known Subclasses:

FilePerPostWriter

public class SeqFileWriter
extends PipelineableWriter
implements ChukwaWriter

This class is thread-safe -- rotate() and save() both synchronize on this object.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	SeqFileWriter.StatReportingTask

Nested classes/interfaces inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

ChukwaWriter.COMMIT_PENDING, ChukwaWriter.CommitStatus

Field Summary

Fields

Modifier and Type	Field and Description
protected long	bytesThisRotate
protected org.apache.hadoop.conf.Configuration	conf
protected String	currentFileName
protected org.apache.hadoop.fs.FSDataOutputStream	currentOutputStr
protected org.apache.hadoop.fs.Path	currentPath
protected long	dataSize
static boolean	ENABLE_ROTATION_ON_CLOSE
static String	FIXED_INTERVAL_OFFSET_OPT
protected org.apache.hadoop.fs.FileSystem	fs
static String	IF_FIXED_INTERVAL_OPT
protected boolean	isRunning
protected static String	localHostAddr
protected Semaphore	lock
protected long	nextTimePeriodComputation
static String	OUTPUT_DIR_OPT
protected String	outputDir
static String	ROTATE_INTERVAL_OPT
protected Timer	rotateTimer
protected org.apache.hadoop.io.SequenceFile.Writer	seqFileWriter
protected int	STAT_INTERVAL_SECONDS
static String	STAT_PERIOD_OPT
protected Timer	statTimer
protected long	timePeriod

Fields inherited from class org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

next

Fields inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

COMMIT_FAIL, COMMIT_OK

Constructor Summary

Constructors

Constructor and Description
SeqFileWriter()

Method Summary

Methods

Modifier and Type	Method and Description
ChukwaWriter.CommitStatus	add(List<Chunk> chunks) Called repeatedly with data that should be serialized.
void	close() Called once, indicating that the writer should close files and prepare to exit.
protected void	computeTimePeriod()
long	getBytesWritten()
void	init(org.apache.hadoop.conf.Configuration conf) Called once to initialize this writer.

Methods inherited from class org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

setNextStage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENABLE_ROTATION_ON_CLOSE

public static boolean ENABLE_ROTATION_ON_CLOSE

STAT_INTERVAL_SECONDS

protected int STAT_INTERVAL_SECONDS

STAT_PERIOD_OPT

public static final String STAT_PERIOD_OPT

See Also:

Constant Field Values

ROTATE_INTERVAL_OPT

public static final String ROTATE_INTERVAL_OPT

See Also:

Constant Field Values

IF_FIXED_INTERVAL_OPT

public static final String IF_FIXED_INTERVAL_OPT

See Also:

Constant Field Values

FIXED_INTERVAL_OFFSET_OPT

public static final String FIXED_INTERVAL_OFFSET_OPT

See Also:

Constant Field Values

OUTPUT_DIR_OPT

public static final String OUTPUT_DIR_OPT

See Also:

Constant Field Values

localHostAddr

protected static String localHostAddr

lock

protected final Semaphore lock

fs

protected org.apache.hadoop.fs.FileSystem fs

conf

protected org.apache.hadoop.conf.Configuration conf

outputDir

protected String outputDir

currentPath

protected org.apache.hadoop.fs.Path currentPath

currentFileName

protected String currentFileName

currentOutputStr

protected org.apache.hadoop.fs.FSDataOutputStream currentOutputStr

seqFileWriter

protected org.apache.hadoop.io.SequenceFile.Writer seqFileWriter

timePeriod

protected long timePeriod

nextTimePeriodComputation

protected long nextTimePeriodComputation

rotateTimer

protected Timer rotateTimer

statTimer

protected Timer statTimer

dataSize

protected volatile long dataSize

bytesThisRotate

protected volatile long bytesThisRotate

isRunning

protected volatile boolean isRunning

Constructor Detail

SeqFileWriter

public SeqFileWriter()

Method Detail

getBytesWritten

public long getBytesWritten()

init

public void init(org.apache.hadoop.conf.Configuration conf)
          throws WriterException

Description copied from interface: ChukwaWriter

Called once to initialize this writer.

Specified by:

init in interface ChukwaWriter

Throws:

WriterException

computeTimePeriod

protected void computeTimePeriod()

add

public ChukwaWriter.CommitStatus add(List<Chunk> chunks)
                              throws WriterException

Description copied from interface: ChukwaWriter

Called repeatedly with data that should be serialized. Subclasses may assume that init() will be called before any calls to add(), and that add() won't be called after close().

Specified by:

add in interface ChukwaWriter

Overrides:

add in class PipelineableWriter

Returns:

Throws:

WriterException

close

public void close()

Description copied from interface: ChukwaWriter

Called once, indicating that the writer should close files and prepare to exit.

Specified by:

close in interface ChukwaWriter