[Buildbot-devel] file transfer

[A.T.Hofkamp]

Hello all,

In the past weekend, I have written some code to allow file transfer as 
buildbot step. In short, you can do


s(step.FileUpload, slavesrc='buildresult', masterdest='storagename')
s(step.FileDownload, mastersrc='data.tar.gz', slavedest='data.tar.gz')

optional settings are 'workdir' for setting the sub directory at the slave, 
'maxsize' for putting an upper limit on the size of the transfer, and 
'blocksize' for limiting the amount of data transfered in each data transfer call.
Code is attached to this email. It is a svn diff w.r.t. installed 0.7.3 .

At the technical side, I created 2 pb.Referencable's that are used as file 
objects by the slave much like the suggestions made by Brian at this mailing 
list 'retrieving build results' , 2005-11-24 , 
http://sourceforge.net/mailarchive/message.php?msg_id=14010277 in my browser.
I didn't go as far as injecting data into the web server. I guess it can be 
done quite easily, but I have no idea how, I never looked at that part of 
buildbot.

While the code works (at least locally at my machine at home), I don't think 
it is finished yet.
- First of all, I am quite new at Twisted, pd, and buildbot, so no doubt I 
missed some setting somewhere.
- Secondly, at the master side, files are put in the dir where it is started, 
ie where 'master.cfg' is. Some improvement seems possible here...
- I also created a StatusRemoteCommand in order to transfer ok/failure back to 
the master. This code is a bit between RemoteCommand and LoggedRemoteCommand.
- After writing everything, I thought about the similarities between the 
*RemoteCommand classes, the remoteUpdate thingie that they use, and my FileIO 
stuff, and then I got the insight that they all do the same thing, they all 
transfer data, with the only difference being the way the data is handled at 
both ends.
My guess is that a more general mechanism may be useful here.


Suggestions on how to proceed are welcome,

Albert
-------------- next part --------------
A non-text attachment was scrubbed...
Name: buildbot_updown.patch.gz
Type: application/x-tar
Size: 3048 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20060612/d150c393/attachment.tar>