Varnish crashing when system starts to swap
Eden Li
eden at hulu.com
Tue Apr 29 09:47:36 CEST 2008
On Mon, Apr 21, 2008 at 3:08 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <BCA2D4755E06EF44A0278A1D9B217E3F08171B3361 at exchange01.ex.local>, Ca
>
> lle Korjus writes:
>
> >Varnish looks fine until it's had abour 1,5 million requests, then
> >we can see the kswapd0 and kswapd1 start working and load average
> >rises to about 200 and the machine gets totally unresponsive. Top
> >shows a lot of cpu beeing spent on i/o waits and varnish child
> >process restarts sometimes. In best
> >case the process restarts and the server starts behaving within 5
> >minutes but sometimes varnish dies completely. One thing we have
> >noticed is that the reserved memory for varnish keeps rising and
> >when it crashes it is usually around 14G.
>
> You didn't say which varnish version you are using, but we have fixed
> a number of memory leaks in -trunk recently.
I've just run into this exact issue varnish 1.1.2, except that varnish
was running for over a month until kswap started acting up. Also, our
backends are not on the same machine.
I tried running different revisions of varnish from trunk on our
machines (r2634 and r2436 on Linux CentOS 5.0), but it keeps asserting
when daemonizing the process (varnishd.c:557). Using an export of the
unstable branch kept at
http://varnish.projects.linpro.no/svn/branches/1.2 seems to be
working.
Do you know if this branch has some of the memory leak fixes you mention here?
More information about the varnish-misc
mailing list