[users at bb.net] Trying for an odd git configuration in some builder code

[Randy J. Ray]

(I apologize in advance for anything I ask about that should be obvious. 
I've been tasked with extending an existing buildbot configuration with 
a new builder, and I am very new to buildbot. Also, I am very new to 
Python.)

My group has a buildbot configuration that we more or less "inherited" 
from a partner. We use it for regular builds of some packages (both 
internal and external), including OpenCV (http://opencv.org). I have 
been given the task of writing a new builder class to build the 3.1.0 
version of OpenCV, where our current config is build the 2.4.13 version.

This *should* have been pretty straightforward, since we're already 
building OpenCV. But it isn't, and here's why: the OpenCV source base 
(in git) has a specific branch for the 2.4 line, created when they 
started working on 3.0 (I presume). However, they never created branches 
for 3.0 or 3.1. As such, I cannot checkout 3.1 directly. I have to 
checkout master, then do "git checkout -b v3.1.0 3.1.0" to create a 
local branch from the 3.1.0 tag. To make it even more interesting, I 
also have to apply 1-2 upstream changes via "git cherry-pick", for 
things that are broken in 3.1.0 but not planned for general release 
until they release 3.2.0 (there is no plan for a 3.1.1, apparently).

I have to do this for two repositories-- opencv and opencv_extra (though 
opencv_extra does not need the cherry-pick steps).

I cannot seem to back-engineer how the version-control/source-control 
interaction takes place, partially because I am so unfamiliar with 
Python. In our existing code, we instantiate an instance of the Git 
class, imported from buildbot.steps.source.git. However, I don't see 
where our code makes any actual method calls on the object returned from 
the instantiation, so whatever is being done may all exist in the 
constructor/initialization.

I know from looking at the logs of the existing builders, that each git 
repo (we use 4 in total) seems to go through the following steps:

  * git --version
  * git clean -f -f -d
  * git fetch -t <url> 2.4
  * git reset --hard FETCH_HEAD --
  * git branch -M 2.4
  * git rev-parse HEAD

I have no idea how to proceed on this... I suspect that the above 
sequence is partly predicated on the fact that the repos currently 
exist, that is, if they didn't I'd see a "git clone" in the logs. But 
the "git fetch" command is fetching based on a remote branch, and I 
won't have a remote branch in this case (only a tag). In fact, I don't 
even *need* to fetch after the initial clone, because this tag won't 
move. Once I've initially done the set-up steps (clone, checkout, 
cherry-pick) subsequent builds won't need to do any git operations at 
all (the only thing changing is that we link OpenCV against NVIDIA's 
CUDA libraries, and the regularly-scheduled builds will be done to link 
against progressive development versions of CUDA). Heck, we may not even 
schedule this builder for regular builds, but rather just have it 
available for manual build requests for when we have a new CUDA to link 
against.

So, is this sort of configuration even possible? Most (if not all) of 
the remaining steps I need to do with this builder, I have been able to 
puzzle through. But the version-control portion has me stumped.

Randy

-- 
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Randy J. Ray                                Nvidia
Senior Build & Release Engineer             2701 San Tomas Expressway
rjray at nvidia.com                            Santa Clara, CA  95050


-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20161121/b93277a2/attachment.html>