<div dir="ltr"><div><br></div>SIGTERM at exactly 2AM. This sounds like a cronjob.<div><br></div><div>The connection lost message is related to the sigterm.</div><div><br></div><div>Hope this helps,<br></div><div>Pierre</div></div><br><div class="gmail_quote"><div dir="ltr">Le jeu. 23 mars 2017 à 21:47, Chris Spencer <<a href="mailto:chrisspen@gmail.com">chrisspen@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg">I'm noticing my Buildbot worker seems to be randomly stopping, causing all builds to stop.<br class="gmail_msg"><br class="gmail_msg"></div>The end of my worker's log shows:<br class="gmail_msg"><br class="gmail_msg"> [buildbot-server] out: 2017-03-22 19:10:55-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 19:20:55-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 19:30:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 19:40:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 19:50:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:00:55-0400 [HangCheckProtocol,client] message from master: keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:00:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:10:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:20:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:30:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:40:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 20:50:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:00:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:10:56-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:20:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:30:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:40:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 21:50:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:00:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:10:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:20:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:30:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:40:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 22:50:57-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:00:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:10:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:20:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:30:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:40:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-22 23:50:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:00:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:10:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:20:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:30:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:40:58-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 00:50:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:00:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:10:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:20:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:30:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:40:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 01:50:59-0400 [-] sending app-level keepalive<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 02:00:00-0400 [-] Received SIGTERM, shutting down.<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 02:00:00-0400 [HangCheckProtocol,client] Lost connection to localhost:9989<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 02:00:00-0400 [-] Stopping factory <buildbot_worker.pb.BotFactory instance at 0x7fdca6b3ccf8><br class="gmail_msg"> [buildbot-server] out: 2017-03-23 02:00:00-0400 [-] Main loop terminated.<br class="gmail_msg"> [buildbot-server] out: 2017-03-23 02:00:00-0400 [-] Server Shut Down.<br class="gmail_msg"><br class="gmail_msg"></div>What's causing this? What would have sent the worker a SIGTERM? The worker and master are on the same server, so I'm not sure what would be causing a "Lost connection" error.<br class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Master still appears to be running, as I can still access the web UI just fine.<br class="gmail_msg"><br class="gmail_msg"></div></div>
_______________________________________________<br class="gmail_msg">
users mailing list<br class="gmail_msg">
<a href="mailto:users@buildbot.net" class="gmail_msg" target="_blank">users@buildbot.net</a><br class="gmail_msg">
<a href="https://lists.buildbot.net/mailman/listinfo/users" rel="noreferrer" class="gmail_msg" target="_blank">https://lists.buildbot.net/mailman/listinfo/users</a></blockquote></div>