[Buildbot-devel] scheduler questions

[Joe MacDonald]

Hi Brian (and everyone)

First, I've finally pushed my code through the review process here and it's
ready to share with anyone who wants it.  I'm attaching it to this mail.
The multiple-writer case hasn't been tested all that extensively, but the
single-writer, multiple reader path gets tested sixty-some times a day at my
work and it hasn't fallen over too many times for me, so it's not total
junk.  :-)

On 3/16/06, Brian Warner <warner-buildbot at lothar.com> wrote:
>
> > If anyone is interested, I'd implemented a multiple-reader,
> single-writer
> > subclass of the master locks for a very similar purpose about six months
> > ago.
>
> Interesting!
>
> So, one of the features of the existing Lock class is that a Step or a
> Build
> can wait for multiple locks, and it doesn't claim any of them until all of
> them are free (to avoid deadlocks). Do you happen to know if your
> multiple-reader class can handle this?


I haven't tested that path, and now that I think about it, I suspect it
might fall down if you had multiple locks of this type and you took one with
exclusive access and failed to take another with shared access.  I'd have to
draw the situation out on paper to be sure.

I'd been concerned about just that situation with the existing locks,
though, because I was concerned that the checks with isAvailable() and
claim() could get interleaved with other such checks and it would be
possible for someone to deadlock themselves by claim()ing a lock they had
determined was available but wasn't by the time they got to it.  I'd gone
ahead and rewritten a chunk of the builder code so that the isAvailable()
checks weren't necessary anymore, but then I was talking to you about the
semantics and decided what I was doing was probably over kill.  That the
situation I was imagining wouldn't happen with buildbot.

Could still be a problem with my class, though.


> Also, am I correct in believing that if nobody ever claims the "writer"
> lock,
> then your class can be used just as a multiple-holder semaphore?


Yep, that's the case.  I'm not sure what you'd use it for in that case,
since everyone asking for shared access will always succeed if nobody asks
for exclusive (ie. writer) access, but I guess it could be used to determine
if any builds are currently running if they wanted to claim the lock at the
start and only release it when they finished.  Which is sort of what I'm
using it for these days anyway.  The only difference is that I have one
build which must run as frequently as possible and no other builds can run
while it does.  But once it's finished, all the other builds can run
concurrently.  Kind of an odd situation, but our infrastructure demands this
is how we have to do them.

Anyway, I hope this turns out to be useful, or at least the basis of
something useful, to everyone.  I really should update our buildbot
installation to something newer than 0.7.0, I realize, but we're closing in
on beta right now and the build automation stuff isn't actually my real
job.  :-)


thanks,
> -Brian
>


--
-Joe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20060317/34e1a0a3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rwlocks.py.gz
Type: application/x-gzip
Size: 3047 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20060317/34e1a0a3/attachment.bin>