[Buildbot-devel] 504 timeouts & slow responses with nginx proxy

[Pierre Tardy]

Not sure if you are talking about nine but there is a sample config for
nginx in the doc.

http://buildbot.readthedocs.org/en/latest/manual/cfg-www.html#reverse-proxy-configuration

Time out can happen with web sockets add they are long lasting

Le ven. 10 avr. 2015 01:30, Edward Armes <edward.armes at gmail.com> a écrit :

>  The only differences I could think of the SSH connection makes is that
> the connection is already established so the timeout rules for nginx proxy
> don't apply (according to proxy timeout documentation). I would guess as
> well that also reduces the load on the BuildBot server as well as it
> doesn't have to establish anything below the application layer as it's
> using an already existing reliable transport, how this applies for data
> sending not a clue as it would depend on a huge number of factors
>
>
> Edward
>
>
> On 10/04/2015 00:21, Vitali Lovich wrote:
>
> Yup.  I know in theory that's the problem.  The challenge is that if I
> tunnel the port over SSH, I'm able to access the server without any of the
> issues I'm seeing (and it's just faster).  Of course it could be just
> observation bias as I haven't yet come up with a way to benchmark.
>
>  I have turned on twisted profiling so I'm going to take a look with
> QCacheGrind at the data to see if there's any buildbot bottlenecks.
>
>  I'll try increasing the proxy read/write timeout on the off chance that
> it is the issue, but I usually only get the 504 around 30s after the
> browser has tried the page which seems to line-up with the overall client
> request timeout I have set in Nginx.
>
> On Apr 9, 2015, at 4:14 PM, Edward Armes <edward.armes at gmail.com> wrote:
>
>   Hi Vitali,
>
> If you are not aware a 504 means Gateway Timeout Error. This means that
> you nginx server is not getting a response in time from the BuildBot server
> itself.
>
> Why this happens however could be for multiple reasons including:-
>
>    - The buildbot server is taken too long to respond
>    - The response/request got lost in the network
>    - The nginx server didn't finish processing the proxy result before
>    forwarding it on and dropped the request (unlikely).
>
> I would suspect it's the first one as I've noticed if the buildbot master
> is busy handling requests it can sometimes take a while to serve a page
> when it's loading a page with a lot of dynamic data (i.e. waterfall view).
>
> A simple fix might be for while the web status is building the response it
> might be an idea to send a 202 code back if the request takes a long time
> (to delay the 504 timeout), or (if this is possible) to increase the wait
> time for proxy directive from within nginx.
>
> Can you expand on this?  How would I do it?
>
>   Hope that helps
>
> Edward
>  On 09/04/2015 23:16, Vitali Lovich wrote:
>
> Hi,
>
> I’m wondering if anyone has had any experience with nginx sitting as a proxy server in front of buildbot.
> I’m getting random 504 timeouts which I *think* are something to do with nginx as opposed to buildbot itself,
> but I’m not sure how to confirm/triage this issue.
>
> Any tips would be great.
>
> My nginx configuration just sets proxy_pass to the buildbot instance.
> proxy_buffering is off & gzip proxying is on.
>
> Thanks,
> Vitali
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live exerciseshttp://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Buildbot-devel mailing listBuildbot-devel at lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
>
>
>  ------------------------------------------------------------
> ------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_
> campaign=VA_SF_______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150410/6d2712a4/attachment.html>