[Buildbot-commits] buildbot/buildbot interfaces.py,1.25,1.26
Brian Warner
warner at users.sourceforge.net
Sun May 15 23:43:59 UTC 2005
Update of /cvsroot/buildbot/buildbot/buildbot
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv688/buildbot
Modified Files:
interfaces.py
Log Message:
Revision: arch at buildbot.sf.net--2004/buildbot--dev--0--patch-171
Creator: Brian Warner <warner at monolith.lothar.com>
handle large logfiles without consuming lots of memory
Merged from warner at monolith.lothar.com--2005 (patch 19-25)
Patches applied:
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-19
Merged from arch at buildbot.sf.net--2004 (patch 159-160)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-20
Merged from arch at buildbot.sf.net--2004 (patch 161-166)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-21
Merged from arch at buildbot.sf.net--2004 (patch 167)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-22
Merged from arch at buildbot.sf.net--2004 (patch 168)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-23
Merged from arch at buildbot.sf.net--2004 (patch 169)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-24
Merged from arch at buildbot.sf.net--2004 (patch 170)
* warner at monolith.lothar.com--2005/buildbot--dev--0--patch-25
handle large log files without using lots of memory
Index: interfaces.py
===================================================================
RCS file: /cvsroot/buildbot/buildbot/buildbot/interfaces.py,v
retrieving revision 1.25
retrieving revision 1.26
diff -u -d -r1.25 -r1.26
--- interfaces.py 12 May 2005 21:55:57 -0000 1.25
+++ interfaces.py 15 May 2005 23:43:56 -0000 1.26
@@ -411,41 +411,25 @@
"""Returns a single string with the color that should be used to
display this event. 'red' and 'yellow' are the most likely ones."""
-class IStatusLogStub(Interface):
- """I represent a single finished Log. The body of the log may live in a
- file on disk, but the metadata should all be available. To obtain the
- log contents, use IStatusLog(log).getText() and friends.
- """
-
- def getName():
- """Returns a short string with the name of this log, probably 'log'.
- """
-
- def getStep():
- """Returns the IBuildStepStatus which owns this log."""
- # TODO: can there be non-Step logs?
-
- def isFinished():
- """Return a boolean. True means the log has finished and is closed,
- False means it is still open and new chunks may be added to it."""
-
class IStatusLog(Interface):
"""I represent a single Log, which is a growing list of text items that
- contains some kind of output for a single BuildStep.
+ contains some kind of output for a single BuildStep. I might be finished,
+ in which case this list has stopped growing.
Each Log has a name, usually something boring like 'log' or 'output'.
These names are not guaranteed to be unique, however they are usually
chosen to be useful within the scope of a single step (i.e. the Compile
step might produce both 'log' and 'warnings'). The name may also have
- spaces. If you want something more globally meaningful, try::
+ spaces. If you want something more globally meaningful, at least within a
+ given Build, try::
'%s.%s' % (log.getStep.getName(), log.getName())
- The Log can be represented as plain text, or it can be accessed as a
- list of items, each of which has a channel indicator (header, stdout,
- stderr) and a text chunk. An HTML display might represent the
- interleaved channels with different styles, while a straight
- download-the-text interface would just want to retrieve a big string.
+ The Log can be presented as plain text, or it can be accessed as a list
+ of items, each of which has a channel indicator (header, stdout, stderr)
+ and a text chunk. An HTML display might represent the interleaved
+ channels with different styles, while a straight download-the-text
+ interface would just want to retrieve a big string.
The 'header' channel is used by ShellCommands to prepend a note about
which command is about to be run ('running command FOO in directory
@@ -461,6 +445,18 @@
Such synthetic Logs are usually finished as soon as they are created."""
+ def getName():
+ """Returns a short string with the name of this log, probably 'log'.
+ """
+
+ def getStep():
+ """Returns the IBuildStepStatus which owns this log."""
+ # TODO: can there be non-Step logs?
+
+ def isFinished():
+ """Return a boolean. True means the log has finished and is closed,
+ False means it is still open and new chunks may be added to it."""
+
def waitUntilFinished():
"""Return a Deferred that will fire when the log is closed. If the
log has already finished, this deferred will fire right away. The
@@ -475,14 +471,45 @@
If 'catchup' is True, the receiver will immediately be sent a series
of logChunk messages to bring it up to date with the partially-filled
log. This allows a status client to join a Log already in progress
- without missing any data."""
+ without missing any data. If the Log has already finished, it is too
+ late to catch up: just do getText() instead.
+
+ If the Log is very large, the receiver will be called many times with
+ a lot of data. There is no way to throttle this data. If the receiver
+ is planning on sending the data on to somewhere else, over a narrow
+ connection, you can get a throttleable subscription by using
+ C{subscribeConsumer} instead."""
def unsubscribe(receiver):
- """Remove a receiver previously registered with subscribe()."""
+ """Remove a receiver previously registered with subscribe(). Attempts
+ to remove a receiver which was not previously registered is a no-op.
+ """
+
+ def subscribeConsumer(consumer):
+ """Register an L{IStatusLogConsumer} to receive all chunks of the
+ logfile, including all the old entries and any that will arrive in
+ the future. The consumer will first have their C{registerProducer}
+ method invoked with a reference to an object that can be told
+ C{pauseProducing}, C{resumeProducing}, and C{stopProducing}. Then the
+ consumer's C{writeChunk} method will be called repeatedly with each
+ (channel, text) tuple in the log, starting with the very first. The
+ consumer will be notified with C{finish} when the log has been
+ exhausted (which can only happen when the log is finished). Note that
+ a small amount of data could be written via C{writeChunk} even after
+ C{pauseProducing} has been called.
+
+ To unsubscribe the consumer, use C{producer.stopProducing}."""
# once the log has finished, the following methods make sense. They can
- # be called earlier, but the results they return might not be finished
- # yet.
+ # be called earlier, but they will only return the contents of the log up
+ # to the point at which they were called. You will lose items that are
+ # added later. Use C{subscribe} or C{subscribeConsumer} to avoid missing
+ # anything.
+
+ def hasContents():
+ """Returns True if the LogFile still has contents available. Returns
+ False for logs that have been pruned. Clients should test this before
+ offering to show the contents of any log."""
def getText():
"""Return one big string with the contents of the Log. This merges
@@ -497,6 +524,35 @@
0 for stdout, 1 for stderr, 2 for header. (note that stderr is merged
into stdout if PTYs are in use)."""
+class IStatusLogConsumer(Interface):
+ """I am an object which can be passed to IStatusLog.subscribeConsumer().
+ I represent a target for writing the contents of an IStatusLog. This
+ differs from a regular IStatusReceiver in that it can pause the producer.
+ This makes it more suitable for use in streaming data over network
+ sockets, such as an HTTP request. Note that the consumer can only pause
+ the producer until it has caught up with all the old data. After that
+ point, C{pauseProducing} is ignored and all new output from the log is
+ sent directoy to the consumer."""
+
+ def registerProducer(producer, streaming):
+ """A producer is being hooked up to this consumer. The consumer only
+ has to handle a single producer. It should send .pauseProducing and
+ .resumeProducing messages to the producer when it wants to stop or
+ resume the flow of data. 'streaming' will be set to True because the
+ producer is always a PushProducer.
+ """
+
+ def unregisterProducer():
+ """The previously-registered producer has been removed. No further
+ pauseProducing or resumeProducing calls should be made. The consumer
+ should delete its reference to the Producer so it can be released."""
+
+ def writeChunk(chunk):
+ """A chunk (i.e. a tuple of (channel, text)) is being written to the
+ consumer."""
+
+ def finish():
+ """The log has finished sending chunks to the consumer."""
class IStatusReceiver(Interface):
"""I am an object which can receive build status updates. I may be
More information about the Commits
mailing list