[Varnish] #1723: High CPU load and exponential peak of objects in a281a10

[Varnish]

#1723: High CPU load and exponential peak of objects in a281a10
---------------------------------+----------------------
 Reporter:  zaterio@…            |       Type:  defect
   Status:  new                  |   Priority:  normal
Milestone:  Varnish 4.0 release  |  Component:  varnishd
  Version:  trunk                |   Severity:  major
 Keywords:  load objects         |
---------------------------------+----------------------
 I am using varnishd (varnish-trunk review a281a10), with 2 storage
 backends:
 ram1: default storage (malloc, 8GB)
 vod1: only for mp4 files (file, 50 GB)

 in vcl_backend_response, whe have the following rule:


  set beresp.storage_hint = "ram1";
  set beresp.http.x-storage = "ram1";

  if (bereq.url ~ "\.mp4$")
                                 {
                                         set beresp.storage_hint = "vod1";
                                         set beresp.http.x-storage =
 "vod1";
                                         set beresp.do_stream = true;
                                         set beresp.ttl = 10h;
                                         set beresp.http.Cache-Control =
 "max-age=604800, public";
                                         set beresp.do_esi = true;
                                         return (deliver);
                                 }

 On this machine we have traffic peaks of 1 Gbits, but yesterady we
 accomodate new service and trafic increased to 2 Gbps.

 In this situation, every 30 min, the load increases, up to 100 (5
 minutes), and MAIN.n_objectcore and
 MAIN.n_objecthead rises to 16970000000000 (normal is 5.5k), and then
 uptime returns to 0, and load decreases.

 We have another server with identical hardware and varnish configuration
 (but more traffic, 5Gbps), This server does not have the above described
 behavior, this server is running varnish-trunk revision 7746e30.
 Whe downgrade the problematic machine from a281a10 to 7746e30 and after 10
 hours looks normal.

 DAEMON_OPTS="-a XXX.XXX.XXX.XXX:80, \
              -T XXX:XXX:XXX:XXX:6082 \
              -f /etc/varnish/default.vcl \
              -h classic,16383 \
              -s ram1=malloc,8G \
              -s vod1=file,/varnishcache/varnish.bin,50G \
              -p thread_pools=2 \
              -p thread_pool_min=500 \
              -p thread_pool_max=3000 \
              -p thread_pool_add_delay=2 \
              -p auto_restart=on \
              -p ping_interval=3 \
              -p send_timeout=5000 \
              -p workspace_session=1M \
              -p cli_timeout=25 \
 -p http_gzip_support=off \
 -p tcp_keepalive_time=600 \
 -p listen_depth=8192 \
 -p cli_buffer=32k \
 -p cli_limit=96k \
 -p ban_dups=on"

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1723>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator