[Buildbot-devel] upcoming release plan: freeze 9/5, release 9/12

[Didier Trosset]

A.T.Hofkamp wrote:
> Didier Trosset wrote:
>> Hello,
>> 
>> I've uploaded a patch for a source step that pulls out the source files
>>  from a tar archive on an FTP server (instead of a version control 
>> system).
>> 
>> Ticket 333. Any thoughts about it?
> 
> Nice idea.
> 
> Some thoughts (not a real bb developer, so these may not count for 
> getting the patch accepted):
> 
> - Use FTP implementation from Twisted, you don't want to block the whole 
> bb slave application while downloading an ISO DVD image from a slow 
> server.

I didn't know about it. Will probably use it.

> - Ability to provide alternative sites?

Well, not my first goal. I won't get into it.

> - Create a way to avoid use of hard-coded tar command (ie maybe I want to
>  use a non-standard tar, different options (gzip or bzip2 
> decompression?), my tar doesn't know --strip, etc

Yes. I've already thought using the python tarfile library. I guess I will
have to, in the end. Does Windows have the tar command, BTW? But this brings
a lot of manual handling because the tarfile library does not have the
--strip option. So the extraction must be done by hand, the correct way, I
mean avoiding the pitfals of directory access rights.

Does anyone know if the python tarfile library could be enhanced?

> - Allow for some paranoia checking, upper limit of downloaded file size 
> (and since this step would usually be used to download 3rd-party code, I
>  can imagine even an exact byte count)

Shouldn't be that difficult to implement.

> Albert
> 

Jean-Paul Calderone wrote:
> 
> It would make more sense to have separate FTP and untar/gzip steps.

I wondered a long time on this one, yes. But in the end, I decided that I
wanted this step to be a source step. Using this source step would result in
having a directory ready-to-be-compiled.

> I agree with the other reply in this thread as well - if it takes too 
> long to download the file using the blocking FTP interface, the slave 
> will time out and the build will fail.
> 
> Jean-Paul
> 

Didier

-- 
Didier Trosset-Moreau
SND / Data Converters
Agilent Technologies
Geneva, Switzerland