org.apache.hadoop.chukwa.datacollection.adaptor.filetailer

Class LWFTAdaptor

java.lang.Object

org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor

org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.LWFTAdaptor

All Implemented Interfaces:

Adaptor

Direct Known Subclasses:

FileTailingAdaptor, RCheckFTAdaptor

public class LWFTAdaptor
extends AbstractAdaptor

A base class for file tailing adaptors. Intended to mandate as little policy as possible, and to use as few system resources as possible. If the file does not exist, this class will continue to retry quietly forever and will start tailing if it's eventually created.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.conf.Configuration	conf
static int	DEFAULT_MAX_READ_SIZE This is the maximum amount we'll read from any one file before moving on to the next.
protected long	fileReadOffset next PHYSICAL offset to read
protected long	lastSlurpTime The timestamp of last slurping.
static int	MAX_READ_SIZE
static String	MAX_READ_SIZE_OPT
protected long	offsetOfFirstByte The logical offset of the first byte of the file
protected static org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailer	tailer

Fields inherited from class org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor

adaptorID, control, dest, type

Constructor Summary

Constructors

Constructor and Description
LWFTAdaptor()

Method Summary

Methods

Modifier and Type	Method and Description
protected int	extractRecords(ChunkReceiver eq, long buffOffsetInFile, byte[] buf) Extract records from a byte sequence
String	getCurrentStatus() Return the adaptor's state Should not include class name or byte offset, which are written by caller.
String	getStreamName()
String	parseArgs(String params)
long	shutdown(AdaptorShutdownPolicy shutdownPolicy) Signals this adaptor to come to an orderly stop.
protected boolean	slurp(long len, RandomAccessFile reader)
void	start(long offset)
boolean	tailFile()
String	toString()

Methods inherited from class org.apache.hadoop.chukwa.datacollection.adaptor.AbstractAdaptor

deregisterAndStop, getType, hardStop, parseArgs, shutdown, start

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_MAX_READ_SIZE

public static final int DEFAULT_MAX_READ_SIZE

This is the maximum amount we'll read from any one file before moving on to the next. This way, we get quick response time for other files if one file is growing rapidly.

See Also:

Constant Field Values

MAX_READ_SIZE_OPT

public static final String MAX_READ_SIZE_OPT

See Also:

Constant Field Values

MAX_READ_SIZE

public static int MAX_READ_SIZE

tailer

protected static org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailer tailer

fileReadOffset

protected long fileReadOffset

next PHYSICAL offset to read

offsetOfFirstByte

protected long offsetOfFirstByte

The logical offset of the first byte of the file

conf

protected org.apache.hadoop.conf.Configuration conf

lastSlurpTime

protected long lastSlurpTime

The timestamp of last slurping.

Constructor Detail

LWFTAdaptor

public LWFTAdaptor()

Method Detail

start

public void start(long offset)

Specified by:

start in class AbstractAdaptor

getCurrentStatus

public String getCurrentStatus()

Description copied from interface: Adaptor

Return the adaptor's state Should not include class name or byte offset, which are written by caller. The datatype should, however, be written by this method.

Returns:

the adaptor state as a string

See Also:

Adaptor.getCurrentStatus()

toString

public String toString()

Overrides:

toString in class Object

getStreamName

public String getStreamName()

parseArgs

public String parseArgs(String params)

Specified by:

parseArgs in class AbstractAdaptor

shutdown

public long shutdown(AdaptorShutdownPolicy shutdownPolicy)
              throws AdaptorException

Description copied from interface: Adaptor

Signals this adaptor to come to an orderly stop. The adaptor ought to push out all the data it can before exiting depending of the shutdown policy

Returns:

the logical offset at which the adaptor was when the method return

Throws:

AdaptorException

extractRecords

protected int extractRecords(ChunkReceiver eq,
                 long buffOffsetInFile,
                 byte[] buf)
                      throws InterruptedException

Extract records from a byte sequence

Parameters:

eq - the queue to stick the new chunk[s] in

buffOffsetInFile - the byte offset in the stream at which buf[] begins

buf - the byte buffer to extract records from

Returns:

the number of bytes processed

Throws:

InterruptedException

slurp

protected boolean slurp(long len,
            RandomAccessFile reader)
                 throws IOException,
                        InterruptedException

Throws:

IOException

InterruptedException

tailFile

public boolean tailFile()
                 throws InterruptedException

Throws:

InterruptedException