[Buildbot-devel] status changes == source changes?

[A.T.Hofkamp]

Brian Warner wrote:
> "A.T.Hofkamp" <a.t.hofkamp at tue.nl> writes:
> 
> 
>>Wouldn't it be an alternative to view this output as yet another source of 
>>changes, and then send messages between both parties, just like messages get 
>>sent (and queue-ed) between source change, schedulers, and builders?
>>
>>We would then be able to collapse the status output infra structure and the 
>>builder infra structure together into a single mechanism.
>>
>>Would this be a good idea?
> 
> 
> I can't say that I understand what this would mean. The current architecture
> can be viewed as a big set of connected queues[1], each delivering different
> kinds of messages from one component to another. Status information messages

I guess I am suggesting a move in the direction of a seperate message delivery 
infra structure, that is, some (simple) form of publish/subscribe (pub/sub).
('simple' may for example mean that everybody is always subscribed to all 
message(type)s)

I believe such a move may be beneficial in the following ways:

- The generation of messages and the delivery of messages become seperate 
problems. Currently, a component that generates a message also must deliver 
it. With a pub/sub mechanism, the component generates a message and gives it 
to the pub/sub which then ensures it gets delivered to all interested parties.

As a result, the coupling between components can be much more loose.
(they don't need to know where the other party is, when they communicate using 
messages only).

- Abstraction from sender. An event is captured in a message, and who the 
sender is, is not important. A component can thus take messages from different 
sources without the need to have a direct link between the objects. For 
example, real-time notifiers such as IRC may use messages from builders (or 
builder steps) directly.


- Abstraction from receiver. As component you don't have to bother with 
keeping track of who wants to have your messages.

The abstraction of senders and receivers together also make each component 
much more flexible in the sense that you can eg split functionality (for 
example, split Status into a real-time forwarding service, and a history 
storage area) without any other component ever knowing.
(not sure whether this holds exactly for the Status component, but it is only 
an illsutration).

> flow from the Builders and the Builds and the Steps to their associated
> status objects (BuilderStatus, BuildStatus, BuildStepStatus respectively).
> These status messages are pretty different than source-code Changes.. they
> describe the results of a build, rather than the source code that went into a
> build.

My observation was mainly that status messages are also messages. Why are 
these not schedule-able/queue-able like the changes?
(ie have a 'build finished' message get queued for a downstream scheduler).

[ This all started when I saw the long list of hard-coded methodnames for
   messages in the base status class. This didn't seem like a good idea to me.
   By representing these events as a real message object, you can collapse such
   lists, and have a much more flexible/extendible set of message classes.

   Once you move to such a set of message classes, the step to some form of
   pub/sub seems beneficial too.
]

> So you need a different queue for Changes and status messages. They both use
> roughly the same architecture, but have different subscribers. I'm afraid I
> don't see how you might "unify" them any more than they currently are.

What happens mainly is that all events become globally available, and nobody 
knows (at a global level) who is sending or receiving what.
This means that adding new functionality (new messages) will not break 
anything which makes experiments and/or user-additions much easier.
The other way around, you can also eliminate functionality if you don't want 
it (ie drop the Status object, or only the history-part if it is seperate).

> [1]: http://buildbot.sourceforge.net/manual-0.7.4.html#System-Architecture

I know, I was more looking at the Status delivery architecture with all those 
rigid connections, ie

http://buildbot.sourceforge.net/manual-0.7.4.html#Status-Delivery-Architecture


(great pictures BTW, Brian).


Albert