org.apache.hadoop.chukwa.extraction.demux.processor.mapper

Class TsProcessor

java.lang.Object

org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

org.apache.hadoop.chukwa.extraction.demux.processor.mapper.TsProcessor

All Implemented Interfaces:

MapProcessor

public class TsProcessor
extends AbstractProcessor

TsProcessor is a generic processor that can be configured to find the timestamp in the text of a record. By default, this class expects that a record starts with a date in this format: yyyy-MM-dd HH:mm:ss,SSS

This format can be changed with the following configurations.

• TsProcessor.default.time.format - Changes the default time format used by all data types.
• TsProcessor.time.format.[some_data_type] - Overrides the default format for a specific data type.

If the time string is not at the beginning of the record you can configure a regular expression to locate the timestamp text with either of the following configurations. The text found in group 1 of the regular expression match will be used with the configured date format.

• TsProcessor.default.time.regex - Changes the default time location regex of the time text for all data types.
• TsProcessor.time.regex.[some_data_type] - Overrides the default time location regex for a specific data type.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_DATE_FORMAT
static String	DEFAULT_TIME_REGEX
static String	TIME_REGEX

Fields inherited from class org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

archiveKey, chunk, currentPos, key, startOffset

Constructor Summary

Constructors

Constructor and Description
TsProcessor()

Method Summary

Modifier and Type	Method and Description
protected void	parse(String recordEntry, org.apache.hadoop.mapred.OutputCollector<ChukwaRecordKey,ChukwaRecord> output, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

buildGenericRecord, getCurrentPos, getStartOffset, hasNext, nextLine, process, reset, saveChunkInError, setCurrentPos, setStartOffset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DATE_FORMAT

public static final String DEFAULT_DATE_FORMAT

See Also:

Constant Field Values

DEFAULT_TIME_REGEX

public static final String DEFAULT_TIME_REGEX

See Also:

Constant Field Values

TIME_REGEX

public static final String TIME_REGEX

See Also:

Constant Field Values

Constructor Detail

TsProcessor

public TsProcessor()

Method Detail

parse

protected void parse(String recordEntry,
                     org.apache.hadoop.mapred.OutputCollector<ChukwaRecordKey,ChukwaRecord> output,
                     org.apache.hadoop.mapred.Reporter reporter)
              throws Throwable

Specified by:

parse in class AbstractProcessor

Throws:

Throwable