[Buildbot-devel] IntegrityError on postgresql, "claimed_at" violates not-null constraint

[Benoît Allard]

I nailed this one !

It turns out ``buildbot.db.BuildRequestsConnectorComponent.claimBuildRequests`` was being called with '[]' as parameter, actually inserting empty rows (with default values) into the database.

This happened because the merge of the build requests returned '[]' because it was assumed that the user provided 'merge' function would always returns True when asked about twice the same argument.

The *good* news is that 0.8.8 (soon to be released) has a different way to schedule build requests. And doesn't seems to be vulnerable to this issue.

Now for 0.8.7, should I provide a patch, or is it assumed that everyone will go to 0.8.8 ?

Best Regards,
Benoît.

> -----Original Message-----
> From: Benoît Allard
> Sent: Friday, September 21, 2012 11:53 AM
> To: BuildBot Devel (buildbot-devel at lists.sourceforge.net)
> Subject: [Buildbot-devel] IntegrityError on postgresql, "claimed_at" violates
> not-null constraint
> 
> Hi there,
> 
> I just realized, the following message is filling up my postgresql logs for
> the past month, last two logs together are almost a Gig, only with this
> message ... And nothing in the twisted.log.
> 
> 2012-08-24 11:37:10 CEST ERROR:  null value in column "claimed_at" violates
> not-null constraint
> 2012-08-24 11:37:10 CEST STATEMENT:  INSERT INTO buildrequest_claims DEFAULT
> VALUES
> 2012-08-24 11:37:10 CEST ERROR:  null value in column "claimed_at" violates
> not-null constraint
> 2012-08-24 11:37:10 CEST STATEMENT:  INSERT INTO buildrequest_claims DEFAULT
> VALUES
> 2012-08-24 11:37:10 CEST ERROR:  null value in column "claimed_at" violates
> not-null constraint
> 2012-08-24 11:37:10 CEST STATEMENT:  INSERT INTO buildrequest_claims DEFAULT
> VALUES
> 2012-08-24 11:37:10 CEST ERROR:  null value in column "claimed_at" violates
> not-null constraint
> 2012-08-24 11:37:10 CEST STATEMENT:  INSERT INTO buildrequest_claims DEFAULT
> VALUES
> 2012-08-24 11:37:10 CEST ERROR:  null value in column "claimed_at" violates
> not-null constraint
> 2012-08-24 11:37:10 CEST STATEMENT:  INSERT INTO buildrequest_claims DEFAULT
> VALUES
> 
> Looks like, ``buildbot.process.Builder.maybeStartBuild`` has an infinite loop.
> 
> This function calls
> ``buildbot.db.BuildRequestsConnectorComponent.claimBuildRequests`` which (in
> our case) (wrongly) raises an ``AlreadyClaimedError``. And then try again.
> 
> Inside that ``claimBuildRequests``, we detect the ``AlreadyClaimedError``
> based on presence of ``sa.exc.IntegrityError``, or
> ``sa.exc.ProgrammingError``. The case I'm having is demonstrating that this is
> way too much, and that IntegrityErrors also happen in other cases.
> 
> I have no idea why the claimed_at is empty in this case.
> 
> Looks like restarting the master silent it (at least for the moment).
> 
> Does someone already has a fix in the pipeline for this ?
> 
> Regards,
> Benoît.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6467 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20130619/6ccdde11/attachment.bin>