org.apache.hadoop.chukwa.datacollection.adaptor.filetailer

Class FileTailingAdaptor

java.lang.Object

org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.FileTailingAdaptor

All Implemented Interfaces:

Adaptor

Direct Known Subclasses:

CharFileTailingAdaptorUTF8, CharFileTailingAdaptorUTF8NewLineEscaped

public class FileTailingAdaptor
extends Object
implements Adaptor

An adaptor that repeatedly tails a specified file, sending the new bytes. This class does not split out records, but just sends everything up to end of file. Subclasses can alter this behavior by overriding extractRecords().

Field Summary

Fields

Modifier and Type	Field and Description
protected long	adaptorID
protected static org.apache.hadoop.conf.Configuration	conf
static int	DEFAULT_MAX_READ_SIZE This is the maximum amount we'll read from any one file before moving on to the next.
protected long	fileReadOffset next PHYSICAL offset to read
static int	GRACEFUL_PERIOD
static int	MAX_READ_SIZE
static int	MAX_RETRIES
protected RandomAccessFile	reader
protected String	type

Constructor Summary

Constructors

Constructor and Description
FileTailingAdaptor()

Method Summary

Methods

Modifier and Type	Method and Description
protected int	extractRecords(ChunkReceiver eq, long buffOffsetInFile, byte[] buf) Extract records from a byte sequence
String	getCurrentStatus() Return the adaptor's state Should not include class name, datatype or byte offset, which are written by caller.
String	getStreamName() Return the stream name
String	getType()
void	hardStop() Stop tailing the file, effective immediately.
long	shutdown() Do one last tail, and then stop
void	start(long adaptorID, String type, String params, long bytes, ChunkReceiver dest) Start this adaptor
boolean	tailFile(ChunkReceiver eq) Looks at the tail of the associated file, adds some of it to event queue This method is not thread safe.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_MAX_READ_SIZE

public static final int DEFAULT_MAX_READ_SIZE

This is the maximum amount we'll read from any one file before moving on to the next. This way, we get quick response time for other files if one file is growing rapidly.

See Also:

Constant Field Values

MAX_READ_SIZE

public static int MAX_READ_SIZE

MAX_RETRIES

public static int MAX_RETRIES

GRACEFUL_PERIOD

public static int GRACEFUL_PERIOD

conf

protected static org.apache.hadoop.conf.Configuration conf

fileReadOffset

protected long fileReadOffset

next PHYSICAL offset to read

type

protected String type

reader

protected RandomAccessFile reader

adaptorID

protected long adaptorID

Constructor Detail

FileTailingAdaptor

public FileTailingAdaptor()

Method Detail

start

public void start(long adaptorID,
         String type,
         String params,
         long bytes,
         ChunkReceiver dest)

Description copied from interface: Adaptor

Start this adaptor

Specified by:

start in interface Adaptor

type - the application type, who is starting this adaptor

params - the status string to use for configuration.

bytes - the stream offset of the first byte sent by this adaptor

shutdown

public long shutdown()
              throws AdaptorException

Do one last tail, and then stop

Specified by:

shutdown in interface Adaptor

Returns:

the logical offset at which the adaptor stops

Throws:

AdaptorException

See Also:

Adaptor.shutdown()

hardStop

public void hardStop()
              throws AdaptorException

Stop tailing the file, effective immediately.

Specified by:

hardStop in interface Adaptor

Throws:

AdaptorException

getCurrentStatus

public String getCurrentStatus()

Description copied from interface: Adaptor

Return the adaptor's state Should not include class name, datatype or byte offset, which are written by caller.

Specified by:

getCurrentStatus in interface Adaptor

Returns:

the adaptor state as a string

See Also:

Adaptor.getCurrentStatus()

toString

public String toString()

Overrides:

toString in class Object

getStreamName

public String getStreamName()

Description copied from interface: Adaptor

Return the stream name

Specified by:

getStreamName in interface Adaptor

Returns:

Stream name as a string

tailFile

public boolean tailFile(ChunkReceiver eq)
                 throws InterruptedException

Looks at the tail of the associated file, adds some of it to event queue This method is not thread safe. Returns true if there's more data in the file

Parameters:

eq - the queue to write Chunks into

Throws:

InterruptedException

extractRecords

protected int extractRecords(ChunkReceiver eq,
                 long buffOffsetInFile,
                 byte[] buf)
                      throws InterruptedException

Extract records from a byte sequence

Parameters:

eq - the queue to stick the new chunk[s] in

buffOffsetInFile - the byte offset in the stream at which buf[] begins

buf - the byte buffer to extract records from

Returns:

the number of bytes processed

Throws:

InterruptedException

getType

public String getType()

Specified by:

getType in interface Adaptor