Child panics on OpenSolaris

[Paul Wright]

On 13 February 2010 20:26, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:

> I'm still not 100% convinced that EBADF is not me screwing something
> up, so I have gone over the poll-waiter and made it even more
> paranoid and conservative than before.
>
> Can I get you to give -trunk r4561 a swing, and see if that changes
> anything ?

I just ran r4561 for about 10 minutes and observed the same error with
a client halfway through a KeepAlive session:

Child (3504) died signal=6
Child (3504) Panic message: Assert error in TCP_nonblocking(), tcp.c line 172:
  Condition((ioctl(sock, ((int)((uint32_t)(0x80000000|(((sizeof
(int))&0xff)<<16)| ('f'<<8)|126))), &i)) == 0) not true.
errno = 9 (Bad file number)
thread = (cache-worker)
ident = -smalloc,-hcritbit,poll
Backtrace:
  4457db: /opt/sbin/varnishd'pan_backtrace+0x1b [0x4457db]
  445ae5: /opt/sbin/varnishd'pan_ic+0x1c5 [0x445ae5]
  fffffd7ff2efdfec: /opt/lib/libvarnish.so.1.0.0'TCP_nonblocking+0x7c
[0xfffffd7ff2efdfec]
  419091: /opt/sbin/varnishd'vca_return_session+0x1b1 [0x419091]
  426aad: /opt/sbin/varnishd'cnt_wait+0x2bd [0x426aad]
  42bc3a: /opt/sbin/varnishd'CNT_Session+0x4ba [0x42bc3a]
  44835b: /opt/sbin/varnishd'wrk_do_cnt_sess+0x19b [0x44835b]
  447954: /opt/sbin/varnishd'wrk_thread_real+0x854 [0x447954]
  447eb3: /opt/sbin/varnishd'wrk_thread+0x123 [0x447eb3]
  fffffd7ff653acf5: /lib/amd64/libc.so.1'_thrp_setup+0x8d [0xfffffd7ff653acf5]
sp = 2b37678 {
  fd = 61, id = 61, xid = 0,
  client = 85.189.148.82:15824,
  step = STP_WAIT,
  handling = deliver,
  restarts = 0, esis = 0
  ws = 2b376e8 {
    id = "sess",
    {s,f,r,e} = {2b383f0,+20,+32788,+65536},
  },
  http[req] = {
    ws = 2b376e8[sess]
      "",
      "/i/video_graphics/starb_0.gif",
      "HTTP/1.1",
      "Accept: */*",
      "Referer:
http://www.firebox.com/product/993/Firebox-of-Retro-Sweets?aff=1108",
      "Accept-Language: en-gb",
      "UA-CPU: x86",
      "Accept-Encoding: gzip, deflate",
      "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;
.NET CLR 1.0.3705)",
      "Connection: Keep-Alive",
      "host: media.firebox.com",
      "X-Forwarded-For: 85.189.148.82",
  },
},

   61 ReqEnd       c 958037480 1266232359.325721025
1266232359.325860262 0.306401968 0.000052452 0.000086784
   61 Debug        c "herding"
   61 Debug        c "Poll: 0 / 1"
   61 ReqStart     c 85.189.148.82 15824 958037491
   61 RxRequest    c GET
   61 RxURL        c /i/video_graphics/starb_0.gif
   61 RxProtocol   c HTTP/1.1
   61 RxHeader     c Accept: */*
   61 RxHeader     c Referer:
http://www.firebox.com/product/993/Firebox-of-Retro-Sweets?aff=1108
   61 RxHeader     c Accept-Language: en-gb
   61 RxHeader     c UA-CPU: x86
   61 RxHeader     c Accept-Encoding: gzip, deflate
   61 RxHeader     c User-Agent: Mozilla/4.0 (compatible; MSIE 7.0;
Windows NT 5.1; .NET CLR 1.0.3705)
   61 RxHeader     c Host: media4.firebox.com
   61 RxHeader     c Connection: Keep-Alive
   61 RxHeader     c Cookie: user_session=XXXX;
locale=company_id%3A0%2Ccurrency_id%3A0%2Clanguage_id%3A0%2Ccountry%3AUnited+Kingdom;
xp_list=52%3D2; aff=1108; intl_aff=000%3D1108%252C1268824352;
__utma=64137912.663836520.1266233079.1266233079.1266
   61 VCL_call     c recv
   61 VCL_return   c lookup
   61 VCL_call     c hash
   61 VCL_return   c hash
   61 Hit          c 958027384
   61 VCL_call     c hit
   61 VCL_return   c deliver
   61 Length       c 125
   61 VCL_call     c deliver
   61 VCL_return   c deliver
   61 TxProtocol   c HTTP/1.1
   61 TxStatus     c 200
   61 TxResponse   c OK
   61 TxHeader     c Server: Apache
   61 TxHeader     c Last-Modified: Mon, 26 Feb 2007 16:53:44 GMT
   61 TxHeader     c ETag: "1708031-7d-42a63fbf35600"
   61 TxHeader     c Cache-Control: max-age=2592000
   61 TxHeader     c Expires: Wed, 17 Mar 2010 11:09:26 GMT
   61 TxHeader     c Content-Type: image/gif
   61 TxHeader     c Content-Length: 125
   61 TxHeader     c Date: Mon, 15 Feb 2010 11:12:39 GMT
   61 TxHeader     c X-Varnish: 958037491 958027384
   61 TxHeader     c Age: 193
   61 TxHeader     c Via: 1.1 varnish
   61 TxHeader     c Connection: keep-alive
   61 ReqEnd       c 958037491 1266232359.455624580
1266232359.455763578 0.129764318 0.000072718 0.000066280

Cheers,

Paul.