[Buildbot-devel] Discussion on source steps

[Amber Yust]

The common case where a user might want to keep ignored files is when those
files are things like .o files from a large c++ build process. for extremely
long compile times, it is often preferable to take advantage of incremental
recompiling to speed things up.

~Amber (sent from Droid, please forgive any typos!)
On May 16, 2011 2:31 AM, "A.T.Hofkamp" <a.t.hofkamp at tue.nl> wrote:
> On 05/16/2011 10:03 AM, Pradeepkumar Gayam wrote:
>> Hi,
>>
>> I am Pradeep working on GSoC project to improve source steps. I am
listing all
>> reasonable source step modes here. I am not familiar with users
requirement,
>
> Hello Pradeep,
>
>> As we are cleaning up the modes I think we should make it possible to
take
>> advantage of features of the version control systems along with the
common
>> modes. There were discussions on IRC and bug tracker[1] on required
modes.
>>
>> Proposed modes:
>> 1. Incremental
>> 2. Clean
>> 3. Fresh
>>
>> Incremental mode: This is same as current update. Update the source
keeping
>> old build artifacts around and pull the changes then build.
>>
>> Clean: As written in bug#669[1] this is equivalent to clobber with
>> ignore_ignores=False, i.e., remove all files generated in build process
that
>> are not under version control. Don't remove ignored files. Equivalent to
>> `git clean`
>>
>> Fresh: This is equivalent to clobber with ignore_ignores=True. That is
>> remove all the build artifact and ignored files. Equivalent to `git clean
>> -x`. This will create a pristine working directory to make fresh builds.
>
> I don't understand 'Clean' and 'Fresh'. The current 'clobber' simply
removes the directory, thus
> *everything* is gone. I don't see how you can switch between keeping or
removing generated files
> after removing the whole working directory.
> (Maybe 'git clean -x' explains it, but I am not a git user, so the command
means nothing to me, sorry.)
>
>> I guess this is pretty much all any user want to do. These three modes
can
>> be implemented across any VCS. Right now ignore_ignores is implemented
only
>
> Maybe it is my lack of understanding your other modes, but in my view
there are 2 choices to make.
> The main one is:
>
> 1. Throw away everything each time, or
> 2. Keep existing directory.
>
> In 2, you then have a second choice, namely
>
> 2a. Keep all files
> 2c. Throw away *all* files not in the current revision of the VCS, and
revert any changed source.
> (yep, 2b is missing, but it is on purpose, see below.)
>
> Comments:
> In 1, you need to do a full checkout every time. Very safe, but may be
costly in bandwidth.
> Option 2 tries to reduce bandwidth needed for updating to another revision
or branch by using the
> intelligence of the VCS.
> Option 2a is the simplest as it assumes the VCS can handle unmanaged
files, and the build system
> will not get confused by files which are not part of the current version.
(See also [1].)
> Option 2c reconstructs the situation after a checkout, before using the
VCS to update the revision
> or branch.
>
>
> My ideas are slightly different from your proposal, you don't have 2a, and
you have a option
> in-between them (let's call it option 2b), where you keep the ignored
files only.
>
> I don't understand why you have this mode.
> Do you have a use case in mind when this mode is useful?
>
> In my view, ignored files are the same as files that are not part of the
VCS checkout.
> The only difference is that they exist so often that output of eg 'svn
status' gets unreadable, so
> the VCS authors added a feature to suppress that output.
>
>
>
> I am very happy you will create 2c (if that is what you means by 'fresh'),
I wanted that for a long
> time!
>
> Greetings,
> Albert
>
>
> [1]: To understand the possible problems:
> Think of a project that has current revision X. A file gets added, and you
get revision Y.
> - If you build Y for the first time, the file gets added in the working
directory.
> - If I then build revision X again, the file stays. The VCS will silently
drop the new file from
> its administration, but not remove it (I am thinking worst case scenario
here). The project must
> be smart enough to consider the new file as part of the project while
building X.
> - Then I build revision Y again. The VCS needs to create the new file
again, but ho, a file with
> the given name already exists!
>
>
>
>
------------------------------------------------------------------------------
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110516/327204c5/attachment.html>