abnormally high load?

[Jeremy Hinegardner]

Hi all,

I'm trying to figure out if this is a normal situation or not.  We have a
varnish instance in front of 12 tokyo tyrant instances with some inline C 
in the VCL to determine which backend to talk to.

Sporadically during the day the load on the dedicated varnish machine will spike
up into the 150-200 range and the response times from varnish drop by an order
of magnitude.  This is an 8CPU Centos 5.3 machine with 8GB of RAM
and the varnish commandline is:

    varnishd -a :6081 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -w 8,512 \
             -u varnish -g varnish \
             -h classic,1000003 \
             -s file,/var/lib/varnish/varnish_storage.bin,4G"

I could use any help on attempting to figure out why we get these spikes and
ways to mitigate them.   Anecdotally, these spikes appear to happen when the
cache hit ratio is low.

generally we have between 30 and 250 clients hitting the varnish instance.  The
high load definitely happens when there are large numbers of clients, but it is
not consistent.  At other times when there are large numbers of clients, the
load will be quite low.

I've read through many of the past threads on tuning varnish for high load and
attempted many of the configurations, yet none have really.

The client activity we are dealing with is, on the high end, around 400 client
programs hitting varnish with an average rate of say 30 requests / sec each.
We have large numbers of backend objects, and the need for caching is a few
hours tops.

If there are other questions that I should be asking, or other information that
anyone would like, just let me know.  An I'm okay with being told that I'm using
varnish in a way it wasn't meant to be used.

enjoy,

-jeremy

-- 
========================================================================
Jeremy Hinegardner                              jeremy at hinegardner.org