org.apache.hadoop.chukwa.extraction.demux.processor.mapper

Class JobLog

java.lang.Object

org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

org.apache.hadoop.chukwa.extraction.demux.processor.mapper.JobLog

All Implemented Interfaces:

MapProcessor

public class JobLog
extends AbstractProcessor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JobLog.CounterHash Parse counter string to object Example string: {(org\.apache\.hadoop\.mapred\.JobInProgress$Counter)(Job Counters ) [(TOTAL_LAUNCHED_REDUCES)(Launched reduce tasks)(1)] [(TOTAL_LAUNCHED_MAPS)(Launched map tasks)(14)] [(DATA_LOCAL_MAPS)(Data-local map tasks)(14)] } {(FileSystemCounters)(FileSystemCounters) [(FILE_BYTES_READ)(FILE_BYTES_READ)(132)] [(HDFS_BYTES_READ)(HDFS_BYTES_READ)(20471)] [(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(790)] [(HDFS_BYTES_WRITTEN)(HDFS_BYTES_WRITTEN)(248)] }
static class	JobLog.JobLogLine

Field Summary

Fields inherited from class org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

archiveKey, chunk, key

Constructor Summary

Constructors

Constructor and Description
JobLog()

Method Summary

Methods

Modifier and Type	Method and Description
JobLog.JobLogLine	getJobLogLine(String recordEntry) Job logs could be split into multiple lines.
protected void	parse(String recordEntry, org.apache.hadoop.mapred.OutputCollector<ChukwaRecordKey,ChukwaRecord> output, org.apache.hadoop.mapred.Reporter reporter)
static ArrayList<String>	split(String s, String regex)

Methods inherited from class org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor

buildGenericRecord, hasNext, nextLine, process, reset, saveChunkInError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobLog

public JobLog()

Method Detail

getJobLogLine

public JobLog.JobLogLine getJobLogLine(String recordEntry)

Job logs could be split into multiple lines. If input recordEntry ends with '"' or '" .', process the line. Otherwise, save the log and wait for the next log.

Returns:

An object of JobLogLine if a full job log is found. Null otherwise.

parse

protected void parse(String recordEntry,
         org.apache.hadoop.mapred.OutputCollector<ChukwaRecordKey,ChukwaRecord> output,
         org.apache.hadoop.mapred.Reporter reporter)
              throws Throwable

Specified by:

parse in class AbstractProcessor

Throws:

Throwable

split

public static ArrayList<String> split(String s,
                      String regex)