org.apache.hadoop.chukwa.inputtools

Class ChukwaInputFormat

java.lang.Object

org.apache.hadoop.mapred.FileInputFormat<K,V>

org.apache.hadoop.mapred.SequenceFileInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

org.apache.hadoop.chukwa.inputtools.ChukwaInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

public class ChukwaInputFormat
extends org.apache.hadoop.mapred.SequenceFileInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

An InputFormat for processing logfiles in Chukwa. Designed to be a nearly drop-in replacement for the Hadoop default TextInputFormat so that code can be ported to use Chukwa with minimal modification. Has an optional configuration option, chukwa.inputfilter.datatype which can be used to filter the input by datatype. If need exists, this mechanism could be extended to also filter by other fields.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ChukwaInputFormat.ChukwaRecordReader

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop.mapred.FileInputFormat.Counter

Field Summary

Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES

Constructor Summary

Constructors

Constructor and Description
ChukwaInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class org.apache.hadoop.mapred.SequenceFileInputFormat

listStatus

Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, getSplits, isSplitable, makeSplit, makeSplit, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ChukwaInputFormat

public ChukwaInputFormat()

Method Detail

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                                          org.apache.hadoop.mapred.JobConf job,
                                                                                                                          org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                   throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Overrides:

getRecordReader in class org.apache.hadoop.mapred.SequenceFileInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException