org.apache.hadoop.chukwa.datacollection.sender

Class ChukwaHttpSender

java.lang.Object

org.apache.hadoop.chukwa.datacollection.sender.ChukwaHttpSender

All Implemented Interfaces:

ChukwaSender

Direct Known Subclasses:

AsyncAckSender

public class ChukwaHttpSender
extends Object
implements ChukwaSender

Encapsulates all of the http setup and connection details needed for chunks to be delivered to a collector. This class should encapsulate the details of the low level data formatting. The Connector is responsible for picking what to send and to whom; retry policy is encoded in the collectors iterator. This class is not thread safe. Synchronization is the caller's responsibility.

On error, tries the list of available collectors, pauses for a minute, and then repeats.

Will wait forever for collectors to come up.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ChukwaHttpSender.CommitListEntry

Field Summary

Fields

Modifier and Type	Field and Description
static String	COLLECTOR_TIMEOUT_OPT

Constructor Summary

Constructors

Constructor and Description
ChukwaHttpSender(org.apache.hadoop.conf.Configuration c)

Method Summary

Modifier and Type	Method and Description
List<ChukwaHttpSender.CommitListEntry>	postAndParseResponse(org.apache.commons.httpclient.methods.PostMethod method, List<ChukwaHttpSender.CommitListEntry> expectedCommitResults)
List<ChukwaHttpSender.CommitListEntry>	send(List<Chunk> toSend) grab all of the chunks currently in the chunkQueue, stores a copy of them locally, calculates their size, sets them up
void	setCollectors(Iterator<String> collectors) Set up a list of connectors for this client to send Chunks to
void	stop()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COLLECTOR_TIMEOUT_OPT

public static final String COLLECTOR_TIMEOUT_OPT

See Also:

Constant Field Values

Constructor Detail

ChukwaHttpSender

public ChukwaHttpSender(org.apache.hadoop.conf.Configuration c)

Method Detail

setCollectors

public void setCollectors(Iterator<String> collectors)

Set up a list of connectors for this client to send Chunks to

Specified by:

setCollectors in interface ChukwaSender

Parameters:

collectors - is a list of collectors

send

public List<ChukwaHttpSender.CommitListEntry> send(List<Chunk> toSend)
                                            throws InterruptedException,
                                                   IOException

grab all of the chunks currently in the chunkQueue, stores a copy of them locally, calculates their size, sets them up

Specified by:

send in interface ChukwaSender

Parameters:

toSend - a list of chunks to commit

Returns:

array of chunk id's which were ACKed by collector

Throws:

InterruptedException - if interrupted while trying to send

IOException - when writing fails

postAndParseResponse

public List<ChukwaHttpSender.CommitListEntry> postAndParseResponse(org.apache.commons.httpclient.methods.PostMethod method,
                                                                   List<ChukwaHttpSender.CommitListEntry> expectedCommitResults)
                                                            throws IOException,
                                                                   InterruptedException

Parameters:

method - the data to push

expectedCommitResults - the list

Returns:

the list of committed chunks

Throws:

IOException - if error writing

InterruptedException - if shutdown has been initiated

stop

public void stop()

Specified by:

stop in interface ChukwaSender