Varnish 503 Service unavailable error

[Rob S]

Paras Fadte wrote:
> Hi,
>
> Sometimes I receive 503 service unavailable error even though there
> are 4 backends . This would mean that all the backends are unavailable
> at a given time  which I don't think is the case . 
Can you replay your varnishlog file, and look for Backend_health items, 
and confirm that they did all go sick at the same time?  If they did, 
then you'll need to look at your backends themselves.  Are you 
separately monitoring them with Nagios, Zabbix, Pingdom or something 
like that?

If replaying varnishlog shows they weren't sick, then I suggest you get 
the varnish transaction ID from one of these 503 errors, and then 
extract the relevant portion of the varnishlog.  This might help explain 
the path taken by your request through the VCL, and help you diagnose a 
logic problem.

Finally, we define all our backends as being monitored by probe, but 
also redefine them without a probe. 

director failsafepool random {
        { .backend = serverAfailsafe; .weight = 1; }
        { .backend = serverBfailsafe; .weight = 1; }
        { .backend = serverCfailsafe; .weight = 1; }
        { .backend = serverDfailsafe; .weight = 1; }
}

We then use logic like:

    set req.backend = monitoredpool;

        if (!req.backend.healthy) {
                set req.backend = failsafepool;
        }

You can then look in your varnishncsa log to see if it used the normal 
or failsafe backends were used.


Rob