[Buildbot-devel] Discussion on source steps

[Pradeepkumar Gayam]

On Tue, Jun 7, 2011 at 12:34 AM, Amber Yust <ayust at yelp.com> wrote:

> I actually really like the sound of a "generic source helper" step that
> would simply call user-defined scripts for the various actions in Buildbot's
> source abstraction, in addition to the more specific VCSes supported.


I like the idea too, but one problem may occur. Assume there are two/three
steps, say A, B, C. A being source clone/pull, B - switching
branch/revision, C - Clean up step. It is likely that user may fail to write
A,B,C compatible to each other.  Keeping custom clean-up command in mind my
plan is do the fallowing when ever I need to do anything related to clean
up.

"
if custom-command specified
   run it.
else
   run default VCS-clean command
"

 I didn't plan anything ahead of it. If senior devs think it's possible and
doesn't lead to any confusion/inconsistency please reply.

>
> ~Amber
>
> On Mon, Jun 6, 2011 at 11:38 AM, Dmitry Nezhevenko <dion at dion.org.ua>wrote:
>
>> On Mon, Jun 06, 2011 at 10:33:06PM +0530, Pradeepkumar Gayam wrote:
>> > I believe almost all modern VCS has this facility. If they don't, we
>> will
>> > implement a workaround even if we have to 6-7 all the commands. We are
>> also
>> > planning to give an option to run custom command to clean the build
>> > directory.
>>
>> This one looks very interesting for me. But maybe even better. Something
>> like
>> script-based VCS command that calls custom helper scripts for every
>> particular action:
>> - Cleanup
>> - Update to specific branch and revision.
>>
>> Update one should also return somehow got_revision. Or maybe just third
>> command "get current revision".
>>
>> These two (or three) commands will probably cover all checkout-related
>> stuff at
>> once. At least for most of proprietary VCS-es.
>>
>> If this is acceptable, I will probably try to contribute once interfaces
>> will be stable enough.
>>
>> Another VCS-related issue I'm experiencing is some kind of timeout
>> handling. Usually VCS operations depends on network or any other factors
>> that may fail. In case of network-related issues (like temporary 30
>> seconds downtime) it sounds reasonable to not fail source step and just
>> wait a bit and retry. At the same time not every VCS error should cause
>> retry. Like "unable to authenticate" error or "repository not found" has
>> no changes to recover.
>>
>> Currently buildbot doesn't provide any way to handle this some "standard"
>> way and it's up to slave command to handle everything. I'm doing
>> everything on slave by reading CLI client exit code and sometimes parsing
>> of it's stderr.
>>
>> So since you are trying to refactor VCS stuff, maybe it's good idea to
>> think about this.
>>
>> --
>> WBR, Dmitry
>>
>>
>> ------------------------------------------------------------------------------
>> Simplify data backup and recovery for your virtual environment with
>> vRanger.
>> Installation's a snap, and flexible recovery options mean your data is
>> safe,
>> secure and there when you need it. Discover what all the cheering's about.
>> Get your free trial download today.
>> http://p.sf.net/sfu/quest-dev2dev2
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>>
>
>
> ------------------------------------------------------------------------------
> Simplify data backup and recovery for your virtual environment with
> vRanger.
> Installation's a snap, and flexible recovery options mean your data is
> safe,
> secure and there when you need it. Discover what all the cheering's about.
> Get your free trial download today.
> http://p.sf.net/sfu/quest-dev2dev2
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110607/23aa4c06/attachment.html>