<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hmm… Looking at isAvailable it does actually appear to be fair so it won’t acquire a counting lock if there’s an exclusive lock ahead of it.<div class="">I think there are 2 limitations:</div><div class=""><br class=""></div><div class="">1. Builds are started & block on locks that they can’t acquire. Any they can are acquired. The lock acquisition should be acquire all locks or none at all to avoid deadlocks since the lock acquisition order is not enforced at all. Dining philosopher’s problem.</div><div class="">2. There should be a way to distinguish greedy from non-greedy access.</div><div class=""><br class=""></div><div class="">Fixing #1 would ensure it’s almost impossible to set up a lock configuration that deadlocks with no diagnostics possible. It would also ensure that time estimates are more accurate since waiting for locks wouldn’t count as “build active” time as it does now.</div><div class="">Fixing #2 would ensure that it’s possible to have lock importance so that important tasks grabbing a counting lock could postpone less important exclusive lock tasks (e.g. processing some data in response to a user request vs running a background task that modifies the environment).</div><div class=""><br class=""></div><div class="">I’m not sure where to start fixing #1 - would appreciate any pointers.</div><div class="">I can tackle #2 if that sounds interesting. I’m thinking something like “preemptable_exclusive” as an additional access that returns false from isAvailable if there’s any counting jobs enqueued.</div><div class=""><br class=""></div><div class="">-Vitali<br class=""><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 4, 2015, at 2:56 PM, Dustin J. Mitchell <<a href="mailto:dustin@v.igoro.us" class="">dustin@v.igoro.us</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">I think your analysis is correct.<br class=""><div class=""><br class=""></div><div class="">Locks are entirely implemented within a master, which has a good side and a bad side. On the good side, it means all the data you need is there, somewhere in memory. On the bad side, it means they don't work too well on a multi-master setup.<br class=""><br class=""></div><div class="">I think it would make sense to keep list of lock requests, in order. In other words, a new counting lock can't acquire a lock that's already acquired in counting mode iff there is a pending exclusive lock.<br class=""><br class=""></div><div class="">Dustin<br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Nov 3, 2015 at 11:38 PM, Vitali Lovich <span dir="ltr" class=""><<a href="mailto:vlovich@gmail.com" target="_blank" class="">vlovich@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br class="">
<br class="">
My impression is that there is no fairness for locks currently & that exclusive locks can starve. Is that true?<br class="">
<br class="">
In other words, if I have job A that acquires a counting lock & job B acquires an exclusive lock, job B will correctly wait until A is done. However, if job C comes in & it in turn also acquires a counting lock,<br class="">
it will continue since it can acquire the lock thus preventing B from running.<br class="">
<br class="">
Assuming my analysis is correct, is anyone aware of an easy way to fix this (e.g. via some hook point within buildbot) or where I would even start to look for it?<br class="">
For example, is there a way to test if anything is waiting to acquire a given exclusive lock? I had a test that just tried to see if it could get the exclusive lock but I just realized that’s broken since it basically just transforms that lock into an exclusive lock.<br class="">
<br class="">
Thanks,<br class="">
Vitali<br class="">
_______________________________________________<br class="">
devel mailing list<br class="">
<a href="mailto:devel@buildbot.net" class="">devel@buildbot.net</a><br class="">
<a href="https://lists.buildbot.net/mailman/listinfo/devel" rel="noreferrer" target="_blank" class="">https://lists.buildbot.net/mailman/listinfo/devel</a></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></div></div></body></html>