From nils.goroll at uplex.de  Mon Nov  2 10:02:14 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  2 Nov 2020 10:02:14 +0000 (UTC)
Subject: [master] 6094e3ffb test changing the path of a uds backend
Message-ID: <20201102100214.3B94210930A@lists.varnish-cache.org>


commit 6094e3ffb855513200a0afd89e1e7e71a1beead3
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 2 10:52:39 2020 +0100

    test changing the path of a uds backend
    
    complement the existing c88 test with changing the path of a backend.

diff --git a/bin/varnishtest/tests/c00088.vtc b/bin/varnishtest/tests/c00088.vtc
index d95477eae..1fe0b768e 100644
--- a/bin/varnishtest/tests/c00088.vtc
+++ b/bin/varnishtest/tests/c00088.vtc
@@ -1,4 +1,4 @@
-varnishtest "Dropping polling of a backend that listens at UDS"
+varnishtest "Change UDS backend: change path, drop poll"
 
 server s1 -listen "${tmpdir}/s1.sock" {
 	non_fatal
@@ -10,6 +10,16 @@ server s1 -listen "${tmpdir}/s1.sock" {
 	}
 } -start
 
+server s2 -listen "${tmpdir}/s2.sock" {
+	non_fatal
+	timeout 3
+	loop 40 {
+		rxreq
+		txresp
+		accept
+	}
+} -start
+
 varnish v1 -vcl {
 	probe default {
 		.window = 8;
@@ -18,13 +28,31 @@ varnish v1 -vcl {
 		.interval = 0.1s;
 	}
 	backend s1 {
-		.path = "${s1_sock}";
+		.path = "${s2_sock}";
 	}
 } -start
 
 delay 1
 
-varnish v1 -vcl+backend { } -cliok "vcl.use vcl2" -cliok "vcl.discard vcl1"
+varnish v1 -vcl {
+	probe default {
+		.window = 8;
+		.initial = 7;
+		.threshold = 8;
+		.interval = 0.1s;
+	}
+	backend s1 {
+		.path = "${s1_sock}";
+	}
+} -cliok "vcl.use vcl2" -cliok "vcl.discard vcl1"
+
+delay 1
+
+varnish v1 -vcl {
+	backend s1 {
+		.path = "${s1_sock}";
+	}
+} -cliok "vcl.use vcl3" -cliok "vcl.discard vcl2"
 
 delay 1
 

From phk at FreeBSD.org  Mon Nov  2 10:47:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  2 Nov 2020 10:47:09 +0000 (UTC)
Subject: [master] b11aebfbe Nitpickery
Message-ID: <20201102104709.5893C10A59F@lists.varnish-cache.org>


commit b11aebfbebd1ef20f4facbefbba623a5e67ac9f2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 08:48:11 2020 +0000

    Nitpickery

diff --git a/bin/varnishd/cache/cache_deliver_proc.c b/bin/varnishd/cache/cache_deliver_proc.c
index 6370ffe28..eb03ba239 100644
--- a/bin/varnishd/cache/cache_deliver_proc.c
+++ b/bin/varnishd/cache/cache_deliver_proc.c
@@ -216,7 +216,8 @@ vdp_objiterator(void *priv, unsigned flush, const void *ptr, ssize_t len)
 }
 
 
-int VDP_DeliverObj(struct vdp_ctx *vdc, struct objcore *oc)
+int
+VDP_DeliverObj(struct vdp_ctx *vdc, struct objcore *oc)
 {
 	int r, final;
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 2bf6c485f..a80d9ea30 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -860,7 +860,6 @@ cnt_recv_prep(struct req *req, const char *ci)
 		req->storage = NULL;
 	}
 
-	req->vdc->retval = 0;
 	req->is_hit = 0;
 	req->is_hitmiss = 0;
 	req->is_hitpass = 0;

From phk at FreeBSD.org  Mon Nov  2 10:47:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  2 Nov 2020 10:47:09 +0000 (UTC)
Subject: [master] 0b9d6927b More nitpickery
Message-ID: <20201102104709.7279410A5A2@lists.varnish-cache.org>


commit 0b9d6927b3e05ee6cf7b47939e772a29ef2ee328
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 09:04:49 2020 +0000

    More nitpickery

diff --git a/bin/varnishd/cache/cache_deliver_proc.c b/bin/varnishd/cache/cache_deliver_proc.c
index eb03ba239..a23ccec92 100644
--- a/bin/varnishd/cache/cache_deliver_proc.c
+++ b/bin/varnishd/cache/cache_deliver_proc.c
@@ -102,15 +102,16 @@ VDP_Init(struct vdp_ctx *vdc, struct worker *wrk, struct vsl_log *vsl,
  */
 
 int
-VDP_bytes(struct vdp_ctx *vdx, enum vdp_action act, const void *ptr, ssize_t len)
+VDP_bytes(struct vdp_ctx *vdc, enum vdp_action act,
+    const void *ptr, ssize_t len)
 {
 	int retval;
 	struct vdp_entry *vdpe;
 
-	CHECK_OBJ_NOTNULL(vdx, VDP_CTX_MAGIC);
-	if (vdx->retval)
-		return (vdx->retval);
-	vdpe = vdx->nxt;
+	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	if (vdc->retval)
+		return (vdc->retval);
+	vdpe = vdc->nxt;
 	CHECK_OBJ_NOTNULL(vdpe, VDP_ENTRY_MAGIC);
 
 	/* at most one VDP_END call */
@@ -124,15 +125,15 @@ VDP_bytes(struct vdp_ctx *vdx, enum vdp_action act, const void *ptr, ssize_t len
 		assert(act == VDP_FLUSH);
 
 	/* Call the present layer, while pointing to the next layer down */
-	vdx->nxt = VTAILQ_NEXT(vdpe, list);
+	vdc->nxt = VTAILQ_NEXT(vdpe, list);
 	vdpe->calls++;
-	vdx->bytes_done = len;
-	retval = vdpe->vdp->bytes(vdx, act, &vdpe->priv, ptr, len);
-	vdpe->bytes_in += vdx->bytes_done;
-	if (retval && (vdx->retval == 0 || retval < vdx->retval))
-		vdx->retval = retval; /* Latch error value */
-	vdx->nxt = vdpe;
-	return (vdx->retval);
+	vdc->bytes_done = len;
+	retval = vdpe->vdp->bytes(vdc, act, &vdpe->priv, ptr, len);
+	vdpe->bytes_in += vdc->bytes_done;
+	if (retval && (vdc->retval == 0 || retval < vdc->retval))
+		vdc->retval = retval; /* Latch error value */
+	vdc->nxt = vdpe;
+	return (vdc->retval);
 }
 
 int

From phk at FreeBSD.org  Mon Nov  2 10:47:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  2 Nov 2020 10:47:09 +0000 (UTC)
Subject: [master] 84b29acef ved_gzgz_init() has no business frobbing
 req->objcore directly,
 it is too far down the stack. Instead pass the objcore through the priv
 structure from ved_deliver() which has legitimate access.
Message-ID: <20201102104709.903FB10A5A5@lists.varnish-cache.org>


commit 84b29acefbe6e17eef5ab1d002fd1c7304230acf
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 10:06:55 2020 +0000

    ved_gzgz_init() has no business frobbing req->objcore directly,
    it is too far down the stack.  Instead pass the objcore through
    the priv structure from ved_deliver() which has legitimate access.

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index c34f77302..cd4755e19 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -578,6 +578,7 @@ struct ved_foo {
 	unsigned		magic;
 #define VED_FOO_MAGIC		0x6a5a262d
 	struct ecx		*ecx;
+	struct objcore		*objcore;
 	uint64_t		start, last, stop, lpad;
 	ssize_t			ll;
 	uint64_t		olen;
@@ -597,18 +598,19 @@ ved_gzgz_init(struct vdp_ctx *vdc, void **priv)
 	req = vdc->req;
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CAST_OBJ_NOTNULL(foo, *priv, VED_FOO_MAGIC);
+	CHECK_OBJ_NOTNULL(foo->objcore, OBJCORE_MAGIC);
 
 	memset(foo->tailbuf, 0xdd, sizeof foo->tailbuf);
 
-	AN(ObjCheckFlag(vdc->wrk, req->objcore, OF_GZIPED));
+	AN(ObjCheckFlag(vdc->wrk, foo->objcore, OF_GZIPED));
 
-	p = ObjGetAttr(vdc->wrk, req->objcore, OA_GZIPBITS, &l);
+	p = ObjGetAttr(vdc->wrk, foo->objcore, OA_GZIPBITS, &l);
 	AN(p);
 	assert(l == 32);
 	foo->start = vbe64dec(p);
 	foo->last = vbe64dec(p + 8);
 	foo->stop = vbe64dec(p + 16);
-	foo->olen = ObjGetLen(vdc->wrk, req->objcore);
+	foo->olen = ObjGetLen(vdc->wrk, foo->objcore);
 	assert(foo->start > 0 && foo->start < foo->olen * 8);
 	assert(foo->last > 0 && foo->last < foo->olen * 8);
 	assert(foo->stop > 0 && foo->stop < foo->olen * 8);
@@ -859,6 +861,7 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody)
 
 		INIT_OBJ(foo, VED_FOO_MAGIC);
 		foo->ecx = ecx;
+		foo->objcore = req->objcore;
 		i = VDP_Push(req->vdc, req->ws, &ved_gzgz, foo);
 
 	} else if (ecx->isgzip && !i) {

From phk at FreeBSD.org  Mon Nov  2 10:47:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  2 Nov 2020 10:47:09 +0000 (UTC)
Subject: [master] 1a818d8bc Pass the objcore to (only!) the topmost VDP
Message-ID: <20201102104709.B6AB810A5AB@lists.varnish-cache.org>


commit 1a818d8bc460c372aa144499d8075693fd70740d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 10:45:47 2020 +0000

    Pass the objcore to (only!) the topmost VDP

diff --git a/bin/varnishd/cache/cache_deliver_proc.c b/bin/varnishd/cache/cache_deliver_proc.c
index a23ccec92..85d294b2d 100644
--- a/bin/varnishd/cache/cache_deliver_proc.c
+++ b/bin/varnishd/cache/cache_deliver_proc.c
@@ -166,8 +166,14 @@ VDP_Push(struct vdp_ctx *vdc, struct ws *ws, const struct vdp *vdp, void *priv)
 	vdc->nxt = VTAILQ_FIRST(&vdc->vdp);
 
 	AZ(vdc->retval);
-	if (vdpe->vdp->init != NULL)
-		vdc->retval = vdpe->vdp->init(vdc, &vdpe->priv);
+	if (vdpe->vdp->init == NULL)
+		return (vdc->retval);
+
+	vdc->retval = vdpe->vdp->init(
+	    vdc,
+	    &vdpe->priv,
+	    vdpe == vdc->nxt ? vdc->req->objcore : NULL
+	);
 	if (vdc->retval > 0) {
 		VTAILQ_REMOVE(&vdc->vdp, vdpe, list);
 		vdc->nxt = VTAILQ_FIRST(&vdc->vdp);
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index cd4755e19..caed6f30c 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -254,20 +254,21 @@ ved_decode_len(struct vsl_log *vsl, const uint8_t **pp)
  */
 
 static int v_matchproto_(vdp_init_f)
-ved_vdp_esi_init(struct vdp_ctx *vdc, void **priv)
+ved_vdp_esi_init(struct vdp_ctx *vdc, void **priv, struct objcore *oc)
 {
 	struct ecx *ecx;
 	struct req *req;
 
 	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	CHECK_OBJ_ORNULL(oc, OBJCORE_MAGIC);
+	if (oc == NULL || !ObjHasAttr(vdc->wrk, oc, OA_ESIDATA))
+		return (1);
+
 	req = vdc->req;
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	AN(priv);
 	AZ(*priv);
 
-	if (!ObjHasAttr(vdc->wrk, req->objcore, OA_ESIDATA))
-		return (1);
-
 	ALLOC_OBJ(ecx, ECX_MAGIC);
 	AN(ecx);
 	assert(sizeof gzip_hdr == 10);
@@ -587,7 +588,7 @@ struct ved_foo {
 };
 
 static int v_matchproto_(vdp_fini_f)
-ved_gzgz_init(struct vdp_ctx *vdc, void **priv)
+ved_gzgz_init(struct vdp_ctx *vdc, void **priv, struct objcore *oc)
 {
 	ssize_t l;
 	const char *p;
@@ -595,6 +596,7 @@ ved_gzgz_init(struct vdp_ctx *vdc, void **priv)
 	struct req *req;
 
 	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	(void)oc;
 	req = vdc->req;
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CAST_OBJ_NOTNULL(foo, *priv, VED_FOO_MAGIC);
diff --git a/bin/varnishd/cache/cache_filter.h b/bin/varnishd/cache/cache_filter.h
index ff07ac563..451c5715b 100644
--- a/bin/varnishd/cache/cache_filter.h
+++ b/bin/varnishd/cache/cache_filter.h
@@ -102,7 +102,7 @@ enum vdp_action {
 	VDP_END,		/* Last buffer or after, implies VDP_FLUSH */
 };
 
-typedef int vdp_init_f(struct vdp_ctx *, void **priv);
+typedef int vdp_init_f(struct vdp_ctx *, void **priv, struct objcore *);
 /*
  * Return value:
  *	negative:	Error - abandon delivery
diff --git a/bin/varnishd/cache/cache_gzip.c b/bin/varnishd/cache/cache_gzip.c
index 147fe67f0..7e98ff4d9 100644
--- a/bin/varnishd/cache/cache_gzip.c
+++ b/bin/varnishd/cache/cache_gzip.c
@@ -288,7 +288,7 @@ VGZ_Gzip(struct vgz *vg, const void **pptr, ssize_t *plen, enum vgz_flag flags)
  */
 
 static int v_matchproto_(vdp_init_f)
-vdp_gunzip_init(struct vdp_ctx *vdp, void **priv)
+vdp_gunzip_init(struct vdp_ctx *vdp, void **priv, struct objcore *oc)
 {
 	struct vgz *vg;
 	struct boc *boc;
@@ -299,9 +299,9 @@ vdp_gunzip_init(struct vdp_ctx *vdp, void **priv)
 	uint64_t u;
 
 	CHECK_OBJ_NOTNULL(vdp, VDP_CTX_MAGIC);
+	CHECK_OBJ_ORNULL(oc, OBJCORE_MAGIC);
 	req = vdp->req;
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 
 	vg = VGZ_NewGunzip(req->vsl, "U D -");
 	AN(vg);
@@ -317,27 +317,22 @@ vdp_gunzip_init(struct vdp_ctx *vdp, void **priv)
 
 	req->resp_len = -1;
 
-	boc = HSH_RefBoc(req->objcore);
+	if (oc == NULL)
+		return (0);
+
+	boc = HSH_RefBoc(oc);
 	if (boc != NULL) {
 		CHECK_OBJ(boc, BOC_MAGIC);
 		bos = boc->state;
-		HSH_DerefBoc(req->wrk, req->objcore);
-	} else
-		bos = BOS_FINISHED;
-
-	/* OA_GZIPBITS is not stable yet */
-	if (bos < BOS_FINISHED)
-		return (0);
+		HSH_DerefBoc(req->wrk, oc);
+		if (bos < BOS_FINISHED)
+			return (0); /* OA_GZIPBITS is not stable yet */
+	}
 
-	p = ObjGetAttr(req->wrk, req->objcore, OA_GZIPBITS, &dl);
+	p = ObjGetAttr(req->wrk, oc, OA_GZIPBITS, &dl);
 	if (p != NULL && dl == 32) {
 		u = vbe64dec(p + 24);
-		/*
-		 * If the size is non-zero AND we are the top VDP
-		 * (ie: no ESI), we know what size the output will be.
-		 */
-		if (u != 0 &&
-		    VTAILQ_FIRST(&req->vdc->vdp)->vdp == &VDP_gunzip)
+		if (u != 0)
 			req->resp_len = u;
 	}
 	return (0);
diff --git a/bin/varnishd/cache/cache_range.c b/bin/varnishd/cache/cache_range.c
index b705cd526..007cf2c74 100644
--- a/bin/varnishd/cache/cache_range.c
+++ b/bin/varnishd/cache/cache_range.c
@@ -244,13 +244,14 @@ vrg_ifrange(struct req *req)
 }
 
 static int v_matchproto_(vdp_init_f)
-vrg_range_init(struct vdp_ctx *vdc, void **priv)
+vrg_range_init(struct vdp_ctx *vdc, void **priv, struct objcore *oc)
 {
 	const char *r;
 	const char *err;
 	struct req *req;
 
 	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	(void)oc;
 	req = vdc->req;
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	assert(http_GetHdr(req->http, H_Range, &r));
diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c
index 4ee707816..55a66290f 100644
--- a/bin/varnishd/http2/cache_http2_deliver.c
+++ b/bin/varnishd/http2/cache_http2_deliver.c
@@ -75,10 +75,11 @@ V2D_Init(void)
 /**********************************************************************/
 
 static int v_matchproto_(vdp_init_f)
-h2_init(struct vdp_ctx *vdc, void **priv)
+h2_init(struct vdp_ctx *vdc, void **priv, struct objcore *oc)
 {
 
 	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	(void)oc;
 	CHECK_OBJ_NOTNULL(vdc->req, REQ_MAGIC);
 	*priv = vdc->req->transport_priv;
 	return (0);

From phk at FreeBSD.org  Mon Nov  2 10:47:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  2 Nov 2020 10:47:09 +0000 (UTC)
Subject: [master] 0a2fe2f18 Update to new VDP->init() prototype
Message-ID: <20201102104709.D437610A5AF@lists.varnish-cache.org>


commit 0a2fe2f185e27203c6cfb58a037cd7feaeba31d5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 10:46:22 2020 +0000

    Update to new VDP->init() prototype

diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 44afe6098..429730c3b 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -101,9 +101,10 @@ static const struct vfp xyzzy_rot13 = {
 #define ROT13_BUFSZ 8
 
 static int v_matchproto_(vdp_init_f)
-xyzzy_rot13_init(struct vdp_ctx *vdc, void **priv)
+xyzzy_rot13_init(struct vdp_ctx *vdc, void **priv, struct objcore *oc)
 {
 	(void)vdc;
+	(void)oc;
 	AN(priv);
 	*priv = malloc(ROT13_BUFSZ);
 	if (*priv == NULL)

From dridi.boukelmoune at gmail.com  Mon Nov  2 14:58:07 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  2 Nov 2020 14:58:07 +0000 (UTC)
Subject: [master] 406c84760 vsl: New Timestamp:Connected for backend tasks
Message-ID: <20201102145807.7EF40114156@lists.varnish-cache.org>


commit 406c847600b787cabea47411afbb90629372afad
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Oct 22 13:45:32 2020 +0200

    vsl: New Timestamp:Connected for backend tasks
    
    Depending on the settings, a timeout can be ambiguous. It's also
    generally useful to be able to tell how long it takes, monitor it
    and possibly notice a degradation before errors start manifesting
    themselves.

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 763622798..056a5eb00 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -168,6 +168,7 @@ vbe_dir_getfd(VRT_CTX, struct worker *wrk, struct backend *bp,
 		return (NULL);
 	}
 
+	VSLb_ts_busyobj(bo, "Connected", W_TIM_real(wrk));
 	fdp = PFD_Fd(pfd);
 	AN(fdp);
 	assert(*fdp >= 0);
diff --git a/bin/varnishtest/tests/c00036.vtc b/bin/varnishtest/tests/c00036.vtc
index df0574138..75322844b 100644
--- a/bin/varnishtest/tests/c00036.vtc
+++ b/bin/varnishtest/tests/c00036.vtc
@@ -20,6 +20,7 @@ varnish v1 -vcl+backend {
 
 logexpect l1 -v v1 -q "vxid == 1004" {
 	expect * 1004 VCL_return      {^fetch}
+	expect 0 1004 Timestamp       {^Connected:}
 	expect 0 1004 BackendOpen     {^\d+ s1}
 	expect 0 1004 Timestamp       {^Bereq:}
 
@@ -27,6 +28,7 @@ logexpect l1 -v v1 -q "vxid == 1004" {
 	# backend goes away on a keepalive TCP connection:
 	expect 0 1004 FetchError      {^HTC eof .-1.}
 	expect 0 1004 BackendClose    {^\d+ s1}
+	expect 0 1004 Timestamp       {^Connected:}
 	expect 0 1004 BackendOpen     {^\d+ s1}
 
 	expect 0 1004 Timestamp       {^Bereq:}
diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 9b245344e..2d4b65b3c 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -86,6 +86,9 @@ Backend fetch timestamps
 Start
 	Start of the backend fetch processing.
 
+Connected
+	Successfully established or reused a backend connection.
+
 Bereq
 	Backend request sent.
 

From dridi.boukelmoune at gmail.com  Mon Nov  2 14:58:07 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  2 Nov 2020 14:58:07 +0000 (UTC)
Subject: [master] 5f5fb8b34 vsl: New Timestamp:Fetch for backend tasks
Message-ID: <20201102145807.9F0A4114159@lists.varnish-cache.org>


commit 5f5fb8b34c46ba007c17e37d28cc114004002776
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Oct 22 14:23:05 2020 +0200

    vsl: New Timestamp:Fetch for backend tasks
    
    If significant time is passed in vcl_backend_fetch it becomes difficult
    to interpret Timestamp:Connected, which this new timestamp remedies.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 13d240a83..02dd19f66 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -420,6 +420,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	if (wrk->handling == VCL_RET_ERROR)
 		return (F_STP_ERROR);
 
+	VSLb_ts_busyobj(bo, "Fetch", W_TIM_real(wrk));
 	i = VDI_GetHdr(bo);
 
 	now = W_TIM_real(wrk);
diff --git a/bin/varnishtest/tests/c00036.vtc b/bin/varnishtest/tests/c00036.vtc
index 75322844b..97b51d5cc 100644
--- a/bin/varnishtest/tests/c00036.vtc
+++ b/bin/varnishtest/tests/c00036.vtc
@@ -20,6 +20,7 @@ varnish v1 -vcl+backend {
 
 logexpect l1 -v v1 -q "vxid == 1004" {
 	expect * 1004 VCL_return      {^fetch}
+	expect 0 1004 Timestamp       {^Fetch:}
 	expect 0 1004 Timestamp       {^Connected:}
 	expect 0 1004 BackendOpen     {^\d+ s1}
 	expect 0 1004 Timestamp       {^Bereq:}
diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 2d4b65b3c..00cec7fc6 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -86,6 +86,9 @@ Backend fetch timestamps
 Start
 	Start of the backend fetch processing.
 
+Fetch
+	Came off vcl_backend_fetch ready to send the backend request.
+
 Connected
 	Successfully established or reused a backend connection.
 

From dridi.boukelmoune at gmail.com  Mon Nov  2 14:58:07 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  2 Nov 2020 14:58:07 +0000 (UTC)
Subject: [master] 4b15ca8f0 vsl: New Timestamp:Process for backend tasks
Message-ID: <20201102145807.C240D114161@lists.varnish-cache.org>


commit 4b15ca8f0ae89677e52206a12a2fa9e1fa7c93e4
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Oct 22 14:30:11 2020 +0200

    vsl: New Timestamp:Process for backend tasks
    
    Similarly to Timestamp:Fetch, it is otherwise not possible to distinguish
    between time spent in VCL and time spent strictly fetching the response
    body.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 02dd19f66..4d7b1547c 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -518,6 +518,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		return (F_STP_ERROR);
 	}
 
+	VSLb_ts_busyobj(bo, "Process", W_TIM_real(wrk));
 	assert(oc->boc->state <= BOS_REQ_DONE);
 	if (oc->boc->state != BOS_REQ_DONE) {
 		bo->req = NULL;
diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 00cec7fc6..e1d608edb 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -98,6 +98,9 @@ Bereq
 Beresp
 	Backend response headers received.
 
+Process
+	Processing finished, ready to fetch the response body.
+
 BerespBody
 	Backend response body received.
 

From reza at naghibi.com  Mon Nov  2 19:02:08 2020
From: reza at naghibi.com (Reza Naghibi)
Date: Mon,  2 Nov 2020 19:02:08 +0000 (UTC)
Subject: [6.0] 700788cb2 Move bereq.http to req->ws for vcl_pipe
Message-ID: <20201102190208.1E40711BBAA@lists.varnish-cache.org>


commit 700788cb28cd7cbd938f39875e6c2f05aabbb98e
Author: Reza Naghibi <reza at naghibi.com>
Date:   Mon Nov 2 14:01:11 2020 -0500

    Move bereq.http to req->ws for vcl_pipe
    
    Also make sure we didn't overflow before entering vcl_pipe. This would
    mean we have lost important connection headers.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 04470ab59..30b6fdc0b 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -770,7 +770,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	SES_Ref(bo->sp);
 	VCL_TaskEnter(bo->vcl, bo->privs);
 
-	HTTP_Setup(bo->bereq, bo->ws, bo->vsl, SLT_BereqMethod);
+	HTTP_Setup(bo->bereq, req->ws, bo->vsl, SLT_BereqMethod);
 	http_FilterReq(bo->bereq, req->http, 0);	// XXX: 0 ?
 	http_PrintfHeader(bo->bereq, "X-Varnish: %u", VXID(req->vsl->wid));
 	http_ForceHeader(bo->bereq, H_Connection, "close");
@@ -780,7 +780,10 @@ cnt_pipe(struct worker *wrk, struct req *req)
 		req->want100cont = 0;
 	}
 
-	VCL_pipe_method(req->vcl, wrk, req, bo, NULL);
+	if (WS_Overflowed(req->ws))
+		wrk->handling = VCL_RET_FAIL;
+	else
+		VCL_pipe_method(req->vcl, wrk, req, bo, NULL);
 
 	switch (wrk->handling) {
 	case VCL_RET_FAIL:
diff --git a/bin/varnishtest/tests/r03329.vtc b/bin/varnishtest/tests/r03329.vtc
index f3453bb61..8889f4061 100644
--- a/bin/varnishtest/tests/r03329.vtc
+++ b/bin/varnishtest/tests/r03329.vtc
@@ -3,11 +3,20 @@ varnishtest "Lost reason from pipe to synth"
 server s1 {} -start
 
 varnish v1 -vcl+backend {
+	import vtc;
 	sub vcl_recv {
+		if (req.http.workspace) {
+			vtc.workspace_alloc(client, -10);
+		}
 		return (pipe);
 	}
 	sub vcl_pipe {
-		return (synth(505, req.http.foo + "/bar"));
+		if (req.http.foo) {
+			return (synth(505, req.http.foo + "/bar"));
+		} else {
+			set bereq.http.baz = "baz";
+			return (synth(505, bereq.http.baz));
+		}
 	}
 } -start
 
@@ -15,4 +24,12 @@ client c1 {
 	txreq -hdr "foo: foo"
 	rxresp
 	expect resp.reason == "foo/bar"
+
+	txreq
+	rxresp
+	expect resp.reason == "baz"
+
+	txreq -hdr "workspace: true"
+	rxresp
+	expect resp.status == 503
 } -run

From hermunn at varnish-software.com  Wed Nov  4 12:31:13 2020
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Wed,  4 Nov 2020 12:31:13 +0000 (UTC)
Subject: [6.0] 4f99d164e Prepare for 6.0.7
Message-ID: <20201104123113.9EDF210F39E@lists.varnish-cache.org>


commit 4f99d164e7a43f2ae246daa4e73eefcb1fc78e5a
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Tue Nov 3 23:12:04 2020 +0100

    Prepare for 6.0.7

diff --git a/configure.ac b/configure.ac
index 1768ee068..65b88bda9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,8 +1,8 @@
 AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
-Copyright (c) 2006-2019 Varnish Software])
+Copyright (c) 2006-2020 Varnish Software])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [6.0.6], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [6.0.7], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 228c4210e..47862e467 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -27,7 +27,7 @@ individual releases. These documents are updated as part of the
 release process.
 
 ================================
-Varnish Cache 6.0.7 (YYYY-MM-DD)
+Varnish Cache 6.0.7 (2020-10-05)
 ================================
 
 * Add support for more HTTP response code reasons (3428_).
diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index e86d038fe..e5c33b1e4 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -37,7 +37,7 @@ Longer listings like example command output and VCL look like this::
     $ /opt/varnish/sbin/varnishd -V
     varnishd (varnish-trunk revision 199de9b)
     Copyright (c) 2006 Verdens Gang AS
-    Copyright (c) 2006-2019 Varnish Software AS
+    Copyright (c) 2006-2020 Varnish Software AS
 
 
 .. For maintainers:
diff --git a/lib/libvarnish/version.c b/lib/libvarnish/version.c
index b2258ccde..087289e51 100644
--- a/lib/libvarnish/version.c
+++ b/lib/libvarnish/version.c
@@ -44,5 +44,5 @@ VCS_Message(const char *progname)
 {
 	fprintf(stderr, "%s (%s)\n", progname, VCS_version);
 	fprintf(stderr, "Copyright (c) 2006 Verdens Gang AS\n");
-	fprintf(stderr, "Copyright (c) 2006-2019 Varnish Software AS\n");
+	fprintf(stderr, "Copyright (c) 2006-2020 Varnish Software AS\n");
 }

From hermunn at varnish-software.com  Wed Nov  4 12:34:07 2020
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Wed,  4 Nov 2020 12:34:07 +0000 (UTC)
Subject: [6.0] 1586cfc6c Revert "Prepare for 6.0.7"
Message-ID: <20201104123407.2BE2310F5E6@lists.varnish-cache.org>


commit 1586cfc6c21b4edb52c780b3a64398ba2d3f60d1
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Wed Nov 4 13:32:43 2020 +0100

    Revert "Prepare for 6.0.7"
    
    This reverts commit 4f99d164e7a43f2ae246daa4e73eefcb1fc78e5a.
    
    This was merged before review, by mistake. I am reverting this so that
    we can do the quality assurence before the actual merge. Sorry.

diff --git a/configure.ac b/configure.ac
index 65b88bda9..1768ee068 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,8 +1,8 @@
 AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
-Copyright (c) 2006-2020 Varnish Software])
+Copyright (c) 2006-2019 Varnish Software])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [6.0.7], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [6.0.6], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 47862e467..228c4210e 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -27,7 +27,7 @@ individual releases. These documents are updated as part of the
 release process.
 
 ================================
-Varnish Cache 6.0.7 (2020-10-05)
+Varnish Cache 6.0.7 (YYYY-MM-DD)
 ================================
 
 * Add support for more HTTP response code reasons (3428_).
diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index e5c33b1e4..e86d038fe 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -37,7 +37,7 @@ Longer listings like example command output and VCL look like this::
     $ /opt/varnish/sbin/varnishd -V
     varnishd (varnish-trunk revision 199de9b)
     Copyright (c) 2006 Verdens Gang AS
-    Copyright (c) 2006-2020 Varnish Software AS
+    Copyright (c) 2006-2019 Varnish Software AS
 
 
 .. For maintainers:
diff --git a/lib/libvarnish/version.c b/lib/libvarnish/version.c
index 087289e51..b2258ccde 100644
--- a/lib/libvarnish/version.c
+++ b/lib/libvarnish/version.c
@@ -44,5 +44,5 @@ VCS_Message(const char *progname)
 {
 	fprintf(stderr, "%s (%s)\n", progname, VCS_version);
 	fprintf(stderr, "Copyright (c) 2006 Verdens Gang AS\n");
-	fprintf(stderr, "Copyright (c) 2006-2020 Varnish Software AS\n");
+	fprintf(stderr, "Copyright (c) 2006-2019 Varnish Software AS\n");
 }

From guillaume at varnish-software.com  Wed Nov  4 18:55:08 2020
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Wed,  4 Nov 2020 18:55:08 +0000 (UTC)
Subject: [master] 1a6bef962 [cci] update the machine executor version
Message-ID: <20201104185508.52E901186A9@lists.varnish-cache.org>


commit 1a6bef9622c78d4adc0c7a59e9794b389c46a8cc
Author: Guillaume Quintard <guillaume at varnish-software.com>
Date:   Tue Nov 3 16:11:12 2020 -0800

    [cci] update the machine executor version
    
    Long story short: newer coreutils use `statx` which isn't supported by
    old kernel, so the `debian:bullseye` images blew up.

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 7b600094a..ea34dc7ab 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -97,7 +97,7 @@ jobs:
         description: the base Docker image for Dockerfile
         type: string
     machine:
-      image: ubuntu-1604:201903-01
+      image: ubuntu-2004:202010-01
     steps:
       - attach_workspace:
           at: ~/project

From phk at FreeBSD.org  Thu Nov  5 13:55:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu,  5 Nov 2020 13:55:09 +0000 (UTC)
Subject: [master] 1e09f422f Move Emit_UDS_Path() to vcc_backend.c where it
 belongs
Message-ID: <20201105135509.1EAC8110366@lists.varnish-cache.org>


commit 1e09f422f7890965990730b4a5c7826e29d4421c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 5 11:13:49 2020 +0000

    Move Emit_UDS_Path() to vcc_backend.c where it belongs

diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index f6a268800..6747a7544 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -34,6 +34,7 @@
 
 #include <stdlib.h>
 #include <string.h>
+#include <sys/stat.h>
 
 #include "vcc_compile.h"
 
@@ -86,6 +87,46 @@ Emit_Sockaddr(struct vcc *tl, const struct token *t_host,
 	Fb(tl, 0, "\t.path = (void *) 0,\n");
 }
 
+/*
+ * For UDS, we do not create a VSA. Check if it's an absolute path, can
+ * be accessed, and is a socket. If so, just emit the path field and set
+ * the IP suckaddrs to NULL.
+ */
+static void
+Emit_UDS_Path(struct vcc *tl, const struct token *t_path, const char *errid)
+{
+	struct stat st;
+
+	AN(t_path);
+	AN(t_path->dec);
+
+	if (*t_path->dec != '/') {
+		VSB_printf(tl->sb,
+			   "%s: Must be an absolute path:\n", errid);
+		vcc_ErrWhere(tl, t_path);
+		return;
+	}
+	errno = 0;
+	if (stat(t_path->dec, &st) != 0) {
+		int err = errno;
+		VSB_printf(tl->sb, "%s: Cannot stat: %s\n", errid,
+			   strerror(errno));
+		vcc_ErrWhere(tl, t_path);
+		if (err == ENOENT || err == EACCES)
+			vcc_Warn(tl);
+		else
+			return;
+	} else if (!S_ISSOCK(st.st_mode)) {
+		VSB_printf(tl->sb, "%s: Not a socket:\n", errid);
+		vcc_ErrWhere(tl, t_path);
+		return;
+	}
+	Fb(tl, 0, "\t.path = \"%s\",\n", t_path->dec);
+	Fb(tl, 0, "\t.ipv4_suckaddr = (void *) 0,\n");
+	Fb(tl, 0, "\t.ipv6_suckaddr = (void *) 0,\n");
+}
+
+
 /*--------------------------------------------------------------------
  * Disallow mutually exclusive field definitions
  */
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 173617e84..815b25918 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -360,8 +360,6 @@ void Resolve_Sockaddr(struct vcc *tl, const char *host, const char *defport,
     const char **ipv4, const char **ipv4_ascii, const char **ipv6,
     const char **ipv6_ascii, const char **p_ascii, int maxips,
     const struct token *t_err, const char *errid);
-void Emit_UDS_Path(struct vcc *tl, const struct token *t_path,
-    const char *errid);
 double vcc_DurationUnit(struct vcc *);
 void vcc_ByteVal(struct vcc *, double *);
 void vcc_Duration(struct vcc *tl, double *);
diff --git a/lib/libvcc/vcc_utils.c b/lib/libvcc/vcc_utils.c
index efe8f4861..8536d9984 100644
--- a/lib/libvcc/vcc_utils.c
+++ b/lib/libvcc/vcc_utils.c
@@ -257,45 +257,6 @@ Resolve_Sockaddr(struct vcc *tl,
 	ZERO_OBJ(rss, sizeof rss);
 }
 
-/*
- * For UDS, we do not create a VSA. Check if it's an absolute path, can
- * be accessed, and is a socket. If so, just emit the path field and set
- * the IP suckaddrs to NULL.
- */
-void
-Emit_UDS_Path(struct vcc *tl, const struct token *t_path, const char *errid)
-{
-	struct stat st;
-
-	AN(t_path);
-	AN(t_path->dec);
-
-	if (*t_path->dec != '/') {
-		VSB_printf(tl->sb,
-			   "%s: Must be an absolute path:\n", errid);
-		vcc_ErrWhere(tl, t_path);
-		return;
-	}
-	errno = 0;
-	if (stat(t_path->dec, &st) != 0) {
-		int err = errno;
-		VSB_printf(tl->sb, "%s: Cannot stat: %s\n", errid,
-			   strerror(errno));
-		vcc_ErrWhere(tl, t_path);
-		if (err == ENOENT || err == EACCES)
-			vcc_Warn(tl);
-		else
-			return;
-	} else if (!S_ISSOCK(st.st_mode)) {
-		VSB_printf(tl->sb, "%s: Not a socket:\n", errid);
-		vcc_ErrWhere(tl, t_path);
-		return;
-	}
-	Fb(tl, 0, "\t.path = \"%s\",\n", t_path->dec);
-	Fb(tl, 0, "\t.ipv4_suckaddr = (void *) 0,\n");
-	Fb(tl, 0, "\t.ipv6_suckaddr = (void *) 0,\n");
-}
-
 /*--------------------------------------------------------------------
  * Recognize and convert units of duration, return seconds.
  */

From dridi.boukelmoune at gmail.com  Thu Nov  5 14:47:09 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu,  5 Nov 2020 14:47:09 +0000 (UTC)
Subject: [master] 27be45a6d vtc: Whitespace OCD
Message-ID: <20201105144709.3607A11184E@lists.varnish-cache.org>


commit 27be45a6d6ad506a1df503e5020289ed3527bf57
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 5 09:29:41 2020 +0100

    vtc: Whitespace OCD

diff --git a/bin/varnishtest/tests/s00002.vtc b/bin/varnishtest/tests/s00002.vtc
index 3d8442eb8..1833cbed5 100644
--- a/bin/varnishtest/tests/s00002.vtc
+++ b/bin/varnishtest/tests/s00002.vtc
@@ -48,8 +48,8 @@ varnish v1 -vcl {
 			.window = 2;
 			.threshold = 1;
 			.initial = 0;
-			}
 		}
+	}
 	sub vcl_backend_response {
 		set beresp.ttl = 1s;
 		set beresp.grace = 1m;

From dridi.boukelmoune at gmail.com  Thu Nov  5 14:47:09 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu,  5 Nov 2020 14:47:09 +0000 (UTC)
Subject: [master] 593465d84 h2: Polish duplicate pseudo-header error
Message-ID: <20201105144709.58AD5111851@lists.varnish-cache.org>


commit 593465d84e269e60a66feebdd1f2cfa46694f44d
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 5 08:11:24 2020 +0100

    h2: Polish duplicate pseudo-header error
    
    I find ":path: /foo" more descriptive than "/foo", even though I could
    infer which one it was...

diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c
index 281fc7fac..6bc062ee1 100644
--- a/bin/varnishd/http2/cache_http2_hpack.c
+++ b/bin/varnishd/http2/cache_http2_hpack.c
@@ -95,6 +95,7 @@ static h2_error
 h2h_addhdr(struct http *hp, char *b, size_t namelen, size_t len)
 {
 	/* XXX: This might belong in cache/cache_http.c */
+	const char *b0;
 	unsigned n;
 
 	CHECK_OBJ_NOTNULL(hp, HTTP_MAGIC);
@@ -107,6 +108,7 @@ h2h_addhdr(struct http *hp, char *b, size_t namelen, size_t len)
 		return (H2SE_ENHANCE_YOUR_CALM);
 	}
 
+	b0 = b;
 	if (b[0] == ':') {
 		/* Match H/2 pseudo headers */
 		/* XXX: Should probably have some include tbl for
@@ -145,8 +147,8 @@ h2h_addhdr(struct http *hp, char *b, size_t namelen, size_t len)
 		/* Check for duplicate pseudo-header */
 		if (hp->hd[n].b != NULL) {
 			VSLb(hp->vsl, SLT_BogoHeader,
-			    "Duplicate pseudo-header: %.*s",
-			    (int)(len > 20 ? 20 : len), b);
+			    "Duplicate pseudo-header %.*s%.*s",
+			    (int)namelen, b0, (int)(len > 20 ? 20 : len), b);
 			return (H2SE_PROTOCOL_ERROR);	// rfc7540,l,3158,3162
 		}
 	} else {

From nils.goroll at uplex.de  Thu Nov  5 14:50:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  5 Nov 2020 14:50:08 +0000 (UTC)
Subject: [priv_copy] 0ffe142b1 SQUASHME: reset privs_copy for rollback
Message-ID: <20201105145008.3E37A111BE7@lists.varnish-cache.org>


commit 0ffe142b1a2b707d87f7ca7c41f80c1d1f56a4e2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 5 15:49:51 2020 +0100

    SQUASHME: reset privs_copy for rollback

diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index 2de283512..ca56a30a3 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -196,6 +196,7 @@ Req_Rollback(struct req *req)
 		VCL_TaskLeave(req->top->privs);
 	VCL_TaskLeave(req->privs);
 	VCL_TaskEnter(req->privs);
+	req->privs_copy = NULL;
 	if (IS_TOPREQ(req))
 		VCL_TaskEnter(req->top->privs);
 	HTTP_Clone(req->http, req->http0);

From dridi at varni.sh  Thu Nov  5 14:49:50 2020
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Thu, 5 Nov 2020 14:49:50 +0000
Subject: [master] 1a6bef962 [cci] update the machine executor version
In-Reply-To: <20201104185508.52E901186A9@lists.varnish-cache.org>
References: <20201104185508.52E901186A9@lists.varnish-cache.org>
Message-ID: <CABoVN9BW7xG6MwcCDJOQry3ha28_Vf1p4az9qKhfhp9jxBBLbA@mail.gmail.com>

On Wed, Nov 4, 2020 at 6:55 PM Guillaume Quintard
<guillaume at varnish-software.com> wrote:
>
>
> commit 1a6bef9622c78d4adc0c7a59e9794b389c46a8cc
> Author: Guillaume Quintard <guillaume at varnish-software.com>
> Date:   Tue Nov 3 16:11:12 2020 -0800
>
>     [cci] update the machine executor version
>
>     Long story short: newer coreutils use `statx` which isn't supported by
>     old kernel, so the `debian:bullseye` images blew up.

Kernel independence they said...

> diff --git a/.circleci/config.yml b/.circleci/config.yml
> index 7b600094a..ea34dc7ab 100644
> --- a/.circleci/config.yml
> +++ b/.circleci/config.yml
> @@ -97,7 +97,7 @@ jobs:
>          description: the base Docker image for Dockerfile
>          type: string
>      machine:
> -      image: ubuntu-1604:201903-01
> +      image: ubuntu-2004:202010-01
>      steps:
>        - attach_workspace:
>            at: ~/project
> _______________________________________________
> varnish-commit mailing list
> varnish-commit at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit

From nils.goroll at uplex.de  Thu Nov  5 15:21:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  5 Nov 2020 15:21:07 +0000 (UTC)
Subject: [master] f2fe1dd5b gc include
Message-ID: <20201105152107.ED99A112A39@lists.varnish-cache.org>


commit f2fe1dd5ba529211b3ebce7eb4168f91f1028141
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 5 16:05:25 2020 +0100

    gc include
    
    Ref: 1e09f422f7890965990730b4a5c7826e29d4421c

diff --git a/lib/libvcc/vcc_utils.c b/lib/libvcc/vcc_utils.c
index 8536d9984..ce149d8c5 100644
--- a/lib/libvcc/vcc_utils.c
+++ b/lib/libvcc/vcc_utils.c
@@ -36,7 +36,6 @@
 #include <stdlib.h>
 #include <sys/socket.h>
 #include <sys/types.h>
-#include <sys/stat.h>
 #include <unistd.h>
 
 #include "vcc_compile.h"

From nils.goroll at uplex.de  Thu Nov  5 15:21:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  5 Nov 2020 15:21:08 +0000 (UTC)
Subject: [master] 54419f891 sync function source file comments with reality
Message-ID: <20201105152108.11BC5112A3C@lists.varnish-cache.org>


commit 54419f89130248d395aad84b96eb617c051fd427
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 5 16:19:20 2020 +0100

    sync function source file comments with reality
    
    for VCL_* functions from cache_varnishd.h

diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index 835e74d7e..1fc5e41f7 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -430,23 +430,13 @@ enum vry_finish_flag { KEEP, DISCARD };
 void VRY_Finish(struct req *req, enum vry_finish_flag);
 
 /* cache_vcl.c */
-VCL_BACKEND VCL_DefaultDirector(const struct vcl *);
-const struct vrt_backend_probe *VCL_DefaultProbe(const struct vcl *);
-void VCL_Init(void);
-void VCL_Panic(struct vsb *, const char *nm, const struct vcl *);
-void VCL_Poll(void);
-void VCL_Ref(struct vcl *);
-void VCL_Refresh(struct vcl **);
-void VCL_Recache(struct worker *, struct vcl **);
-void VCL_Rel(struct vcl **);
-void VCL_TaskEnter(struct vrt_privs *);
-void VCL_TaskLeave(struct vrt_privs *);
-const char *VCL_Return_Name(unsigned);
-const char *VCL_Method_Name(unsigned);
 void VCL_Bo2Ctx(struct vrt_ctx *, struct busyobj *);
 void VCL_Req2Ctx(struct vrt_ctx *, struct req *);
 struct vrt_ctx *VCL_Get_CliCtx(int);
 struct vsb *VCL_Rel_CliCtx(struct vrt_ctx **);
+void VCL_Panic(struct vsb *, const char *nm, const struct vcl *);
+void VCL_Poll(void);
+void VCL_Init(void);
 
 #define VCL_MET_MAC(l,u,t,b) \
     void VCL_##l##_method(struct vcl *, struct worker *, struct req *, \
@@ -458,9 +448,6 @@ typedef int vcl_be_func(struct cli *, struct director *, void *);
 
 int VCL_IterDirector(struct cli *, const char *, vcl_be_func *, void *);
 
-/* cache_vcl_vrt.c */
-void VCL_VRT_Init(void);
-
 /* cache_vrt.c */
 void pan_privs(struct vsb *, const struct vrt_privs *);
 
@@ -468,9 +455,22 @@ void pan_privs(struct vsb *, const struct vrt_privs *);
 int VCL_StackVFP(struct vfp_ctx *, const struct vcl *, const char *);
 int VCL_StackVDP(struct req *, const struct vcl *, const char *);
 const char *resp_Get_Filter_List(struct req *req);
+void VCL_VRT_Init(void);
+
+/* cache_vrt_vcl.c */
+const char *VCL_Return_Name(unsigned);
+const char *VCL_Method_Name(unsigned);
+void VCL_Refresh(struct vcl **);
+void VCL_Recache(struct worker *, struct vcl **);
+void VCL_Ref(struct vcl *);
+void VCL_Rel(struct vcl **);
+VCL_BACKEND VCL_DefaultDirector(const struct vcl *);
+const struct vrt_backend_probe *VCL_DefaultProbe(const struct vcl *);
 
 /* cache_vrt_priv.c */
 extern struct vrt_privs cli_task_privs[1];
+void VCL_TaskEnter(struct vrt_privs *);
+void VCL_TaskLeave(struct vrt_privs *);
 
 /* cache_vrt_vmod.c */
 void VMOD_Init(void);

From nils.goroll at uplex.de  Thu Nov  5 15:38:35 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 5 Nov 2020 16:38:35 +0100
Subject: [priv_copy] 0ffe142b1 SQUASHME: reset privs_copy for rollback
In-Reply-To: <20201105145008.3E37A111BE7@lists.varnish-cache.org>
References: <20201105145008.3E37A111BE7@lists.varnish-cache.org>
Message-ID: <e459e31a-817f-e79f-1c97-2e328954195e@uplex.de>

Apologies for the branch pushed to the wrong repository.

I have also taken the opportunity to clean up some other branches which
accidentally got pushed to our main repo:

phk3316
tmp-test-6.5

@hermunn, should revert-3452-6.0-release-6.0.7 go away as well?



-- 

** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstra?e 32
22301 Hamburg

tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753

xmpp://slink at jabber.int.uplex.de/

http://uplex.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-commit/attachments/20201105/cf9cfa9a/attachment-0001.bin>

From martin at varnish-software.com  Fri Nov  6 10:32:14 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Fri,  6 Nov 2020 10:32:14 +0000 (UTC)
Subject: [6.0] aa16600c9 Make CircleCI script download dist tarball
Message-ID: <20201106103214.E7409108922@lists.varnish-cache.org>


commit aa16600c90a7f1be0a115df27d8db58ec06310d0
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Sep 25 15:37:27 2020 +0200

    Make CircleCI script download dist tarball
    
    If given a build parameter called 'dist-url', the build script downloads
    a tarball from the given URL instead of doing a 'make dist' step.

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 726d789db..571b7703f 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -13,6 +13,10 @@ parameters:
   pkg-commit:
     type: string
     default: ""
+  dist-url:
+    type: string
+    default: ""
+
 jobs:
   dist:
     description: Builds varnish-x.y.z.tar.gz that is used later for the packaging jobs
@@ -33,19 +37,36 @@ jobs:
                 make \
                 pcre-devel \
                 python \
-                python-sphinx
+                python-sphinx \
+                curl
       - checkout
-      - run:
-          name: Create the dist tarball
-          command: |
-            git checkout << pipeline.parameters.vc-commit >>
-            # if version is "trunk", it's a weekly tarball, override the version
-            if grep 'AC_INIT.*trunk.*' ./configure.ac; then
-                sed -i -e "s/^AC_INIT.*trunk.*/AC_INIT([Varnish], [$(date +%Y%m%d)], [varnish-dev at varnish-cache.org])/" ./configure.ac
-                touch .is_weekly
-            fi
-            ./autogen.des --quiet
-            make dist -j 16
+      - when:
+          condition: << pipeline.parameters.dist-url >>
+          steps:
+            - run:
+                name: Download the dist tarball
+                command: |
+                  curl -s << pipeline.parameters.dist-url >> -o varnish-dist.tar.gz
+                  mkdir parse-version-tmp
+                  cd parse-version-tmp
+                  tar xzf ../varnish-dist.tar.gz
+                  VERSION=$(varnish-*/configure --version | awk 'NR == 1 {print $NF}')
+                  cd ..
+                  mv varnish-dist.tar.gz varnish-${VERSION}.tar.gz
+      - unless:
+          condition: << pipeline.parameters.dist-url >>
+          steps:
+            - run:
+                name: Create the dist tarball
+                command: |
+                  git checkout << pipeline.parameters.vc-commit >>
+                  # if version is "trunk", it's a weekly tarball, override the version
+                  if grep 'AC_INIT.*trunk.*' ./configure.ac; then
+                      sed -i -e "s/^AC_INIT.*trunk.*/AC_INIT([Varnish], [$(date +%Y%m%d)], [varnish-dev at varnish-cache.org])/" ./configure.ac
+                      touch .is_weekly
+                  fi
+                  ./autogen.des --quiet
+                  make dist -j 16
       - persist_to_workspace:
           root: .
           paths:

From dridi at varni.sh  Fri Nov  6 10:41:50 2020
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Fri, 6 Nov 2020 10:41:50 +0000
Subject: [6.0] aa16600c9 Make CircleCI script download dist tarball
In-Reply-To: <20201106103214.E7409108922@lists.varnish-cache.org>
References: <20201106103214.E7409108922@lists.varnish-cache.org>
Message-ID: <CABoVN9DbwFNV4LTWaAystMw4MUnONFrHKR23=91aFs6zVGu80g@mail.gmail.com>

On Fri, Nov 6, 2020 at 10:32 AM Martin Blix Grydeland
<martin at varnish-software.com> wrote:
>
>
> commit aa16600c90a7f1be0a115df27d8db58ec06310d0
> Author: Martin Blix Grydeland <martin at varnish-software.com>
> Date:   Fri Sep 25 15:37:27 2020 +0200
>
>     Make CircleCI script download dist tarball
>
>     If given a build parameter called 'dist-url', the build script downloads
>     a tarball from the given URL instead of doing a 'make dist' step.
>
> diff --git a/.circleci/config.yml b/.circleci/config.yml
> index 726d789db..571b7703f 100644
> --- a/.circleci/config.yml
> +++ b/.circleci/config.yml
> @@ -13,6 +13,10 @@ parameters:
>    pkg-commit:
>      type: string
>      default: ""
> +  dist-url:
> +    type: string
> +    default: ""
> +
>  jobs:
>    dist:
>      description: Builds varnish-x.y.z.tar.gz that is used later for the packaging jobs
> @@ -33,19 +37,36 @@ jobs:
>                  make \
>                  pcre-devel \
>                  python \
> -                python-sphinx
> +                python-sphinx \
> +                curl
>        - checkout
> -      - run:
> -          name: Create the dist tarball
> -          command: |
> -            git checkout << pipeline.parameters.vc-commit >>
> -            # if version is "trunk", it's a weekly tarball, override the version
> -            if grep 'AC_INIT.*trunk.*' ./configure.ac; then
> -                sed -i -e "s/^AC_INIT.*trunk.*/AC_INIT([Varnish], [$(date +%Y%m%d)], [varnish-dev at varnish-cache.org])/" ./configure.ac
> -                touch .is_weekly
> -            fi
> -            ./autogen.des --quiet
> -            make dist -j 16
> +      - when:
> +          condition: << pipeline.parameters.dist-url >>
> +          steps:
> +            - run:
> +                name: Download the dist tarball
> +                command: |
> +                  curl -s << pipeline.parameters.dist-url >> -o varnish-dist.tar.gz
> +                  mkdir parse-version-tmp
> +                  cd parse-version-tmp
> +                  tar xzf ../varnish-dist.tar.gz
> +                  VERSION=$(varnish-*/configure --version | awk 'NR == 1 {print $NF}')
> +                  cd ..
> +                  mv varnish-dist.tar.gz varnish-${VERSION}.tar.gz
> +      - unless:
> +          condition: << pipeline.parameters.dist-url >>
> +          steps:
> +            - run:
> +                name: Create the dist tarball
> +                command: |
> +                  git checkout << pipeline.parameters.vc-commit >>
> +                  # if version is "trunk", it's a weekly tarball, override the version
> +                  if grep 'AC_INIT.*trunk.*' ./configure.ac; then
> +                      sed -i -e "s/^AC_INIT.*trunk.*/AC_INIT([Varnish], [$(date +%Y%m%d)], [varnish-dev at varnish-cache.org])/" ./configure.ac
> +                      touch .is_weekly
> +                  fi

First of all, why not grab the version like several lines above, by
running  ./configure --version instead of a hazardous grep?

Second, on a stable branch, we can get rid of the whole weekly check.

> +                  ./autogen.des --quiet
> +                  make dist -j 16
>        - persist_to_workspace:
>            root: .
>            paths:
> _______________________________________________
> varnish-commit mailing list
> varnish-commit at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit

From martin at varnish-software.com  Fri Nov  6 10:50:12 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Fri,  6 Nov 2020 10:50:12 +0000 (UTC)
Subject: [6.0] cd5036cef Make the 6.0 branch use the 6.0-lts pkg-varnish-cache
 branch
Message-ID: <20201106105012.2F7B9109276@lists.varnish-cache.org>


commit cd5036ceff3fc6d37f57ecb33e37ac4181b56311
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Nov 6 11:49:14 2020 +0100

    Make the 6.0 branch use the 6.0-lts pkg-varnish-cache branch

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 571b7703f..0b8da355f 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -12,7 +12,7 @@ parameters:
     default: "HEAD"
   pkg-commit:
     type: string
-    default: ""
+    default: "6.0-lts"
   dist-url:
     type: string
     default: ""

From martin at varnish-software.com  Fri Nov  6 14:30:12 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Fri,  6 Nov 2020 14:30:12 +0000 (UTC)
Subject: [6.0] 525d371e3 Prepare for 6.0.7
Message-ID: <20201106143012.94712110A07@lists.varnish-cache.org>


commit 525d371e3ea0e0c38edd7baf0f80dc226560f26e
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Nov 6 11:50:28 2020 +0100

    Prepare for 6.0.7

diff --git a/configure.ac b/configure.ac
index 1768ee068..65b88bda9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,8 +1,8 @@
 AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
-Copyright (c) 2006-2019 Varnish Software])
+Copyright (c) 2006-2020 Varnish Software])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [6.0.6], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [6.0.7], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 228c4210e..a386b2a5b 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -27,7 +27,7 @@ individual releases. These documents are updated as part of the
 release process.
 
 ================================
-Varnish Cache 6.0.7 (YYYY-MM-DD)
+Varnish Cache 6.0.7 (2020-11-06)
 ================================
 
 * Add support for more HTTP response code reasons (3428_).
diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index e86d038fe..e5c33b1e4 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -37,7 +37,7 @@ Longer listings like example command output and VCL look like this::
     $ /opt/varnish/sbin/varnishd -V
     varnishd (varnish-trunk revision 199de9b)
     Copyright (c) 2006 Verdens Gang AS
-    Copyright (c) 2006-2019 Varnish Software AS
+    Copyright (c) 2006-2020 Varnish Software AS
 
 
 .. For maintainers:
diff --git a/lib/libvarnish/version.c b/lib/libvarnish/version.c
index b2258ccde..087289e51 100644
--- a/lib/libvarnish/version.c
+++ b/lib/libvarnish/version.c
@@ -44,5 +44,5 @@ VCS_Message(const char *progname)
 {
 	fprintf(stderr, "%s (%s)\n", progname, VCS_version);
 	fprintf(stderr, "Copyright (c) 2006 Verdens Gang AS\n");
-	fprintf(stderr, "Copyright (c) 2006-2019 Varnish Software AS\n");
+	fprintf(stderr, "Copyright (c) 2006-2020 Varnish Software AS\n");
 }

From phk at FreeBSD.org  Fri Nov  6 17:03:09 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri,  6 Nov 2020 17:03:09 +0000 (UTC)
Subject: [master] 3ba46b255 Encapsulat the backend connection endpoint data in
 a separate structure.
Message-ID: <20201106170309.6F874114506@lists.varnish-cache.org>


commit 3ba46b255538330de29bb111bff0b54dda9bce9f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Nov 6 17:01:54 2020 +0000

    Encapsulat the backend connection endpoint data in a separate structure.

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 056a5eb00..f4a9aadc4 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -560,21 +560,22 @@ VRT_backend_vsm_need(VRT_CTX)
 }
 
 static uint64_t
-vrt_hash_be(const struct vrt_backend *vrt)
+vrt_hash_be(const struct vrt_endpoint *vep)
 {
 	struct VSHA256Context cx[1];
 	unsigned char ident[VSHA256_DIGEST_LENGTH];
 
+	CHECK_OBJ_NOTNULL(vep, VRT_ENDPOINT_MAGIC);
 	VSHA256_Init(cx);
 	VSHA256_Update(cx, vbe_proto_ident, strlen(vbe_proto_ident));
-	if (vrt->ipv4_suckaddr != NULL)
-		VSHA256_Update(cx, vrt->ipv4_suckaddr, vsa_suckaddr_len);
-	if (vrt->ipv6_suckaddr != NULL)
-		VSHA256_Update(cx, vrt->ipv6_suckaddr, vsa_suckaddr_len);
-	if (vrt->path != NULL)
-		VSHA256_Update(cx, vrt->path, strlen(vrt->path));
-	if (vrt->prefix_ptr != NULL)
-		VSHA256_Update(cx, vrt->prefix_ptr, vrt->prefix_len);
+	if (vep->ipv4 != NULL)
+		VSHA256_Update(cx, vep->ipv4, vsa_suckaddr_len);
+	if (vep->ipv6 != NULL)
+		VSHA256_Update(cx, vep->ipv6, vsa_suckaddr_len);
+	if (vep->uds_path != NULL)
+		VSHA256_Update(cx, vep->uds_path, strlen(vep->uds_path));
+	if (vep->ident != NULL)
+		VSHA256_Update(cx, vep->ident, vep->ident_len);
 	VSHA256_Final(ident, cx);
 	return (vbe64dec(ident));
 }
@@ -586,20 +587,22 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 	struct backend *be;
 	struct vcl *vcl;
 	const struct vrt_backend_probe *vbp;
+	const struct vrt_endpoint *vep;
 	const struct vdi_methods *m;
 	const struct suckaddr *sa;
 	char abuf[VTCP_ADDRBUFSIZE];
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(vrt, VRT_BACKEND_MAGIC);
-	if (vrt->path == NULL) {
-		if (vrt->ipv4_suckaddr == NULL && vrt->ipv6_suckaddr == NULL) {
+	vep = vrt->endpoint;
+	CHECK_OBJ_NOTNULL(vep, VRT_ENDPOINT_MAGIC);
+	if (vep->uds_path == NULL) {
+		if (vep->ipv4 == NULL && vep->ipv6 == NULL) {
 			VRT_fail(ctx, "%s: Illegal IP", __func__);
 			return (NULL);
 		}
 	} else
-		assert(vrt->ipv4_suckaddr == NULL &&
-		    vrt->ipv6_suckaddr == NULL);
+		assert(vep->ipv4== NULL && vep->ipv6== NULL);
 
 	vcl = ctx->vcl;
 	AN(vcl);
@@ -617,14 +620,14 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 #undef DN
 
 	if (be->hosthdr == NULL) {
-		if (vrt->path != NULL)
+		if (vrt->endpoint->uds_path != NULL)
 			sa = bogo_ip;
-		else if (cache_param->prefer_ipv6 && vrt->ipv6_suckaddr != NULL)
-			sa = vrt->ipv6_suckaddr;
-		else if (vrt->ipv4_suckaddr != NULL)
-			sa = vrt->ipv4_suckaddr;
+		else if (cache_param->prefer_ipv6 && vep->ipv6 != NULL)
+			sa = vep->ipv6;
+		else if (vep->ipv4!= NULL)
+			sa = vep->ipv4;
 		else
-			sa = vrt->ipv6_suckaddr;
+			sa = vep->ipv6;
 		VTCP_name(sa, abuf, sizeof abuf, NULL, 0);
 		REPLACE(be->hosthdr, abuf);
 	}
@@ -633,8 +636,8 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 	    "%s.%s", VCL_Name(ctx->vcl), vrt->vcl_name);
 	AN(be->vsc);
 
-	be->tcp_pool = VTP_Ref(vrt->ipv4_suckaddr, vrt->ipv6_suckaddr,
-	    vrt->path, vrt_hash_be(vrt));
+	be->tcp_pool = VTP_Ref(vep->ipv4, vep->ipv6,
+	    vep->uds_path, vrt_hash_be(vrt->endpoint));
 	AN(be->tcp_pool);
 
 	vbp = vrt->probe;
@@ -670,6 +673,7 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 VCL_BACKEND
 VRT_new_backend(VRT_CTX, const struct vrt_backend *vrt)
 {
+	CHECK_OBJ_NOTNULL(vrt->endpoint, VRT_ENDPOINT_MAGIC);
 	return (VRT_new_backend_clustered(ctx, NULL, vrt));
 }
 
diff --git a/include/vrt.h b/include/vrt.h
index f9894ed73..4dea8c50a 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -380,20 +380,27 @@ extern const void * const vrt_magic_string_unset;
  * One of those things...
  */
 
+struct vrt_endpoint {
+	unsigned			magic;
+#define VRT_ENDPOINT_MAGIC		0xcc419347
+	VCL_IP				ipv4;
+	VCL_IP				ipv6;
+	const char			*uds_path;
+	const void			*ident;
+	int				ident_len;
+};
+
 #define VRT_BACKEND_FIELDS(rigid)				\
 	rigid char			*vcl_name;		\
 	rigid char			*ipv4_addr;		\
 	rigid char			*ipv6_addr;		\
 	rigid char			*port;			\
-	rigid char			*path;			\
 	rigid char			*hosthdr;		\
 	vtim_dur			connect_timeout;	\
 	vtim_dur			first_byte_timeout;	\
 	vtim_dur			between_bytes_timeout;	\
 	unsigned			max_connections;	\
-	unsigned			proxy_header;		\
-	void				*prefix_ptr;		\
-	unsigned			prefix_len;
+	unsigned			proxy_header;
 
 #define VRT_BACKEND_HANDLE()			\
 	do {					\
@@ -401,7 +408,6 @@ extern const void * const vrt_magic_string_unset;
 		DA(ipv4_addr);			\
 		DA(ipv6_addr);			\
 		DA(port);			\
-		DA(path);			\
 		DA(hosthdr);			\
 		DN(connect_timeout);		\
 		DN(first_byte_timeout);		\
@@ -413,9 +419,8 @@ extern const void * const vrt_magic_string_unset;
 struct vrt_backend {
 	unsigned			magic;
 #define VRT_BACKEND_MAGIC		0x4799ce6c
+	const struct vrt_endpoint	*endpoint;
 	VRT_BACKEND_FIELDS(const)
-	VCL_IP				ipv4_suckaddr;
-	VCL_IP				ipv6_suckaddr;
 	VCL_PROBE			probe;
 };
 
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index 6747a7544..41674efd2 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -58,11 +58,11 @@ vcc_default_probe(struct vcc *tl)
  */
 
 static void
-Emit_Sockaddr(struct vcc *tl, const struct token *t_host,
+Emit_Sockaddr(struct vcc *tl, struct vsb *vsb1, const struct token *t_host,
     const struct token *t_port)
 {
 	const char *ipv4, *ipv4a, *ipv6, *ipv6a, *pa;
-	char buf[256];
+	char buf[BUFSIZ];
 
 	AN(t_host->dec);
 
@@ -74,17 +74,19 @@ Emit_Sockaddr(struct vcc *tl, const struct token *t_host,
 	    &ipv4, &ipv4a, &ipv6, &ipv6a, &pa, 2, t_host, "Backend host");
 	ERRCHK(tl);
 	if (ipv4 != NULL) {
-		Fb(tl, 0, "\t.ipv4_suckaddr = (const struct suckaddr *)%s,\n",
+		VSB_printf(vsb1,
+		    "\t.ipv4 = (const struct suckaddr *)%s,\n",
 		    ipv4);
 		Fb(tl, 0, "\t.ipv4_addr = \"%s\",\n", ipv4a);
 	}
 	if (ipv6 != NULL) {
-		Fb(tl, 0, "\t.ipv6_suckaddr = (const struct suckaddr *)%s,\n",
+		VSB_printf(vsb1,
+		    "\t.ipv6 = (const struct suckaddr *)%s,\n",
 		    ipv6);
 		Fb(tl, 0, "\t.ipv6_addr = \"%s\",\n", ipv6a);
 	}
 	Fb(tl, 0, "\t.port = \"%s\",\n", pa);
-	Fb(tl, 0, "\t.path = (void *) 0,\n");
+	VSB_printf(vsb1, "\t.uds_path = (void *) 0,\n");
 }
 
 /*
@@ -93,7 +95,8 @@ Emit_Sockaddr(struct vcc *tl, const struct token *t_host,
  * the IP suckaddrs to NULL.
  */
 static void
-Emit_UDS_Path(struct vcc *tl, const struct token *t_path, const char *errid)
+Emit_UDS_Path(struct vcc *tl, struct vsb *vsb1,
+    const struct token *t_path, const char *errid)
 {
 	struct stat st;
 
@@ -121,12 +124,11 @@ Emit_UDS_Path(struct vcc *tl, const struct token *t_path, const char *errid)
 		vcc_ErrWhere(tl, t_path);
 		return;
 	}
-	Fb(tl, 0, "\t.path = \"%s\",\n", t_path->dec);
-	Fb(tl, 0, "\t.ipv4_suckaddr = (void *) 0,\n");
-	Fb(tl, 0, "\t.ipv6_suckaddr = (void *) 0,\n");
+	VSB_printf(vsb1, "\t.uds_path = \"%s\",\n", t_path->dec);
+	VSB_printf(vsb1, "\t.ipv4 = (void *) 0,\n");
+	VSB_printf(vsb1, "\t.ipv6 = (void *) 0,\n");
 }
 
-
 /*--------------------------------------------------------------------
  * Disallow mutually exclusive field definitions
  */
@@ -346,7 +348,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	struct token *t_did = NULL;
 	struct fld_spec *fs;
 	struct inifin *ifp;
-	struct vsb *vsb;
+	struct vsb *vsb1, *vsb2;
 	char *p;
 	unsigned u;
 	double t;
@@ -364,9 +366,11 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	    "?proxy_header",
 	    NULL);
 
-	vsb = VSB_new_auto();
-	AN(vsb);
-	tl->fb = vsb;
+	vsb1 = VSB_new_auto();
+	AN(vsb1);
+	vsb2 = VSB_new_auto();
+	AN(vsb2);
+	tl->fb = vsb2;
 
 	if (tl->t->tok == ID &&
 	    (vcc_IdIs(tl->t, "none") || vcc_IdIs(tl->t, "None"))) {
@@ -381,10 +385,16 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 
 	SkipToken(tl, '{');
 
+	VSB_printf(vsb1,
+	    "\nstatic const struct vrt_endpoint vgc_dir_ep_%s = {\n",
+	    vgcname);
+	VSB_printf(vsb1, "\t.magic = VRT_ENDPOINT_MAGIC,\n");
+
 	Fb(tl, 0, "\nstatic const struct vrt_backend vgc_dir_priv_%s = {\n",
 	    vgcname);
 
 	Fb(tl, 0, "\t.magic = VRT_BACKEND_MAGIC,\n");
+	Fb(tl, 0, "\t.endpoint = &vgc_dir_ep_%s,\n", vgcname);
 	Fb(tl, 0, "\t.vcl_name = \"%.*s", PF(t_be));
 	Fb(tl, 0, "\",\n");
 
@@ -517,10 +527,10 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	assert(t_host != NULL || t_path != NULL);
 	if (t_host != NULL)
 		/* Check that the hostname makes sense */
-		Emit_Sockaddr(tl, t_host, t_port);
+		Emit_Sockaddr(tl, vsb1, t_host, t_port);
 	else
 		/* Check that the path can be a legal UDS */
-		Emit_UDS_Path(tl, t_path, "Backend path");
+		Emit_UDS_Path(tl, vsb1, t_path, "Backend path");
 	ERRCHK(tl);
 
 	ExpectErr(tl, '}');
@@ -543,10 +553,15 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 
 	vcc_NextToken(tl);
 
+	VSB_printf(vsb1, "};\n");
+	AZ(VSB_finish(vsb1));
+	Fh(tl, 0, "%s", VSB_data(vsb1));
+	VSB_destroy(&vsb1);
+
 	tl->fb = NULL;
-	AZ(VSB_finish(vsb));
-	Fh(tl, 0, "%s", VSB_data(vsb));
-	VSB_destroy(&vsb);
+	AZ(VSB_finish(vsb2));
+	Fh(tl, 0, "%s", VSB_data(vsb2));
+	VSB_destroy(&vsb2);
 
 	ifp = New_IniFin(tl);
 	VSB_printf(ifp->ini,
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 429730c3b..a3cfa0dae 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -522,10 +522,13 @@ cooldown_thread(void *priv)
 static VCL_BACKEND
 create_cold_backend(VRT_CTX)
 {
+	struct vrt_endpoint vep[1];
 	struct vrt_backend be[1];
 
+	INIT_OBJ(vep, VRT_ENDPOINT_MAGIC);
+	vep->uds_path = "/";
 	INIT_OBJ(be, VRT_BACKEND_MAGIC);
-	be->path = "/";
+	be->endpoint = vep;
 	be->vcl_name = "doomed";
 	return (VRT_new_backend(ctx, be));
 }
diff --git a/lib/libvmod_debug/vmod_debug_dyn.c b/lib/libvmod_debug/vmod_debug_dyn.c
index 3ef74544c..68739f641 100644
--- a/lib/libvmod_debug/vmod_debug_dyn.c
+++ b/lib/libvmod_debug/vmod_debug_dyn.c
@@ -65,13 +65,16 @@ dyn_dir_init(VRT_CTX, struct xyzzy_debug_dyn *dyn,
 {
 	struct suckaddr *sa;
 	VCL_BACKEND dir, dir2;
+	struct vrt_endpoint vep;
 	struct vrt_backend vrt;
 
 	CHECK_OBJ_NOTNULL(dyn, VMOD_DEBUG_DYN_MAGIC);
 	XXXAN(addr);
 	XXXAN(port);
 
+	INIT_OBJ(&vep, VRT_ENDPOINT_MAGIC);
 	INIT_OBJ(&vrt, VRT_BACKEND_MAGIC);
+	vrt.endpoint = &vep;
 	vrt.port = port;
 	vrt.vcl_name = dyn->vcl_name;
 	vrt.hosthdr = addr;
@@ -81,10 +84,10 @@ dyn_dir_init(VRT_CTX, struct xyzzy_debug_dyn *dyn,
 	AN(sa);
 	if (VSA_Get_Proto(sa) == AF_INET) {
 		vrt.ipv4_addr = addr;
-		vrt.ipv4_suckaddr = sa;
+		vep.ipv4 = sa;
 	} else if (VSA_Get_Proto(sa) == AF_INET6) {
 		vrt.ipv6_addr = addr;
-		vrt.ipv6_suckaddr = sa;
+		vep.ipv6 = sa;
 	} else
 		WRONG("Wrong proto family");
 
@@ -171,6 +174,7 @@ static int
 dyn_uds_init(VRT_CTX, struct xyzzy_debug_dyn_uds *uds, VCL_STRING path)
 {
 	VCL_BACKEND dir, dir2;
+	struct vrt_endpoint vep;
 	struct vrt_backend vrt;
 	struct stat st;
 
@@ -197,12 +201,14 @@ dyn_uds_init(VRT_CTX, struct xyzzy_debug_dyn_uds *uds, VCL_STRING path)
 		return (-1);
 	}
 
+	INIT_OBJ(&vep, VRT_ENDPOINT_MAGIC);
 	INIT_OBJ(&vrt, VRT_BACKEND_MAGIC);
-	vrt.path = path;
+	vrt.endpoint = &vep;
+	vep.uds_path = path;
 	vrt.vcl_name = uds->vcl_name;
 	vrt.hosthdr = "localhost";
-	vrt.ipv4_suckaddr = NULL;
-	vrt.ipv6_suckaddr = NULL;
+	vep.ipv4 = NULL;
+	vep.ipv6 = NULL;
 
 	if ((dir = VRT_new_backend(ctx, &vrt)) == NULL)
 		return (-1);

From phk at FreeBSD.org  Mon Nov  9 12:39:12 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  9 Nov 2020 12:39:12 +0000 (UTC)
Subject: [master] c93f9bc6d Fix inconsequential memory leak in
 vcc_ParseHostDef()
Message-ID: <20201109123912.CCAFF10711E@lists.varnish-cache.org>


commit c93f9bc6de6f246ba19a4c14ef7e0225b13cd86d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 9 12:37:38 2020 +0000

    Fix inconsequential memory leak in vcc_ParseHostDef()

diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index 41674efd2..4a88eeb8b 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -348,7 +348,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	struct token *t_did = NULL;
 	struct fld_spec *fs;
 	struct inifin *ifp;
-	struct vsb *vsb1, *vsb2;
+	struct vsb *vsb1;
 	char *p;
 	unsigned u;
 	double t;
@@ -366,15 +366,9 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	    "?proxy_header",
 	    NULL);
 
-	vsb1 = VSB_new_auto();
-	AN(vsb1);
-	vsb2 = VSB_new_auto();
-	AN(vsb2);
-	tl->fb = vsb2;
 
 	if (tl->t->tok == ID &&
 	    (vcc_IdIs(tl->t, "none") || vcc_IdIs(tl->t, "None"))) {
-		Fb(tl, 0, "\n\t%s = (NULL);\n", vgcname);
 		vcc_NextToken(tl);
 		SkipToken(tl, ';');
 		ifp = New_IniFin(tl);
@@ -385,19 +379,6 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 
 	SkipToken(tl, '{');
 
-	VSB_printf(vsb1,
-	    "\nstatic const struct vrt_endpoint vgc_dir_ep_%s = {\n",
-	    vgcname);
-	VSB_printf(vsb1, "\t.magic = VRT_ENDPOINT_MAGIC,\n");
-
-	Fb(tl, 0, "\nstatic const struct vrt_backend vgc_dir_priv_%s = {\n",
-	    vgcname);
-
-	Fb(tl, 0, "\t.magic = VRT_BACKEND_MAGIC,\n");
-	Fb(tl, 0, "\t.endpoint = &vgc_dir_ep_%s,\n", vgcname);
-	Fb(tl, 0, "\t.vcl_name = \"%.*s", PF(t_be));
-	Fb(tl, 0, "\",\n");
-
 	/* Check for old syntax */
 	if (tl->t->tok == ID && vcc_IdIs(tl->t, "set")) {
 		VSB_cat(tl->sb,
@@ -410,6 +391,17 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 		return;
 	}
 
+	tl->fb = VSB_new_auto();
+	AN(tl->fb);
+
+	Fb(tl, 0, "\nstatic const struct vrt_backend vgc_dir_priv_%s = {\n",
+	    vgcname);
+
+	Fb(tl, 0, "\t.magic = VRT_BACKEND_MAGIC,\n");
+	Fb(tl, 0, "\t.endpoint = &vgc_dir_ep_%s,\n", vgcname);
+	Fb(tl, 0, "\t.vcl_name = \"%.*s", PF(t_be));
+	Fb(tl, 0, "\",\n");
+
 	while (tl->t->tok != '}') {
 
 		vcc_IsField(tl, &t_field, fs);
@@ -436,6 +428,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 				vcc_ErrToken(tl, tl->t);
 				VSB_cat(tl->sb, " at ");
 				vcc_ErrWhere(tl, tl->t);
+				VSB_destroy(&tl->fb);
 				return;
 			}
 			vcc_Redef(tl, "Address", &t_did, t_field);
@@ -482,6 +475,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 				VSB_cat(tl->sb,
 				    ".proxy_header must be 1 or 2\n");
 				vcc_ErrWhere(tl, t_val);
+				VSB_destroy(&tl->fb);
 				return;
 			}
 			SkipToken(tl, ';');
@@ -507,9 +501,11 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 			vcc_ErrToken(tl, tl->t);
 			VSB_cat(tl->sb, " at\n");
 			vcc_ErrWhere(tl, tl->t);
+			VSB_destroy(&tl->fb);
 			return;
 		} else {
 			ErrInternal(tl);
+			VSB_destroy(&tl->fb);
 			return;
 		}
 
@@ -518,12 +514,22 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	vcc_FieldsOk(tl, fs);
 	ERRCHK(tl);
 
+	ExpectErr(tl, '}');
+
 	if (t_host == NULL && t_path == NULL) {
 		VSB_cat(tl->sb, "Expected .host or .path.\n");
 		vcc_ErrWhere(tl, t_be);
+		VSB_destroy(&tl->fb);
 		return;
 	}
 
+	vsb1 = VSB_new_auto();
+	AN(vsb1);
+	VSB_printf(vsb1,
+	    "\nstatic const struct vrt_endpoint vgc_dir_ep_%s = {\n",
+	    vgcname);
+	VSB_printf(vsb1, "\t.magic = VRT_ENDPOINT_MAGIC,\n");
+
 	assert(t_host != NULL || t_path != NULL);
 	if (t_host != NULL)
 		/* Check that the hostname makes sense */
@@ -533,9 +539,10 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 		Emit_UDS_Path(tl, vsb1, t_path, "Backend path");
 	ERRCHK(tl);
 
-	ExpectErr(tl, '}');
-
-	/* We have parsed it all, emit the ident string */
+	VSB_printf(vsb1, "};\n");
+	AZ(VSB_finish(vsb1));
+	Fh(tl, 0, "%s", VSB_data(vsb1));
+	VSB_destroy(&vsb1);
 
 	/* Emit the hosthdr field, fall back to .host if not specified */
 	/* If .path is specified, set "0.0.0.0". */
@@ -553,15 +560,9 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 
 	vcc_NextToken(tl);
 
-	VSB_printf(vsb1, "};\n");
-	AZ(VSB_finish(vsb1));
-	Fh(tl, 0, "%s", VSB_data(vsb1));
-	VSB_destroy(&vsb1);
-
-	tl->fb = NULL;
-	AZ(VSB_finish(vsb2));
-	Fh(tl, 0, "%s", VSB_data(vsb2));
-	VSB_destroy(&vsb2);
+	AZ(VSB_finish(tl->fb));
+	Fh(tl, 0, "%s", VSB_data(tl->fb));
+	VSB_destroy(&tl->fb);
 
 	ifp = New_IniFin(tl);
 	VSB_printf(ifp->ini,

From martin at varnish-software.com  Mon Nov  9 15:13:08 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:08 +0000 (UTC)
Subject: [master] 0954500dc Use VTCP_Assert() also in VTCP_close()
Message-ID: <20201109151308.5F93010D0E2@lists.varnish-cache.org>


commit 0954500dcf1e302e60c479d0f2a0e573092f5228
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 14:35:23 2020 +0200

    Use VTCP_Assert() also in VTCP_close()
    
    Consistently use VTCP_Assert when asserting on the result of VTCP_Check().

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index bdd5d4e53..45f1c98b7 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -344,7 +344,7 @@ VTCP_close(int *s)
 
 	i = close(*s);
 
-	assert(VTCP_Check(i));
+	VTCP_Assert(i);
 	*s = -1;
 }
 

From martin at varnish-software.com  Mon Nov  9 15:13:08 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:08 +0000 (UTC)
Subject: [master] f8a8b362e Make VTCP_Check() accept EAGAIN/EWOULDBLOCK as
 acceptable errno
Message-ID: <20201109151308.7947D10D0E5@lists.varnish-cache.org>


commit f8a8b362eef918e6211427ec2f6b814410041f61
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 14:56:38 2020 +0200

    Make VTCP_Check() accept EAGAIN/EWOULDBLOCK as acceptable errno
    
    When a socket timeout is set on a socket and the timeout expires, read()
    and write() calls on that socket will return (-1) with errno set to
    EAGAIN/EWOULDBLOCK.

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 45f1c98b7..53335aedf 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -564,6 +564,14 @@ VTCP_Check(int a)
 		return (1);
 	if (errno == ECONNRESET || errno == ENOTCONN || errno == EPIPE)
 		return (1);
+	/* Accept EAGAIN (and EWOULDBLOCK in case they are not the same)
+	 * as errno values. Even though our sockets are all non-blocking,
+	 * when a SO_{SND|RCV}TIMEO expires, read() or write() on the
+	 * socket will return (-1) and errno set to EAGAIN. (This is not
+	 * documented in the read(2) and write(2) manpages, but is
+	 * described in the socket(7) manpage.) */
+	if (errno == EAGAIN || errno == EWOULDBLOCK)
+		return (1);
 #if (defined (__SVR4) && defined (__sun))
 	if (errno == EPROTO)
 		return (1);

From martin at varnish-software.com  Mon Nov  9 15:13:08 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:08 +0000 (UTC)
Subject: [master] 30ec003dd Change VTCP_Check() to take a ssize_t
Message-ID: <20201109151308.A421010D0E8@lists.varnish-cache.org>


commit 30ec003dd8de69bf5881b848d7dd07414b1bc4e6
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 15:07:46 2020 +0200

    Change VTCP_Check() to take a ssize_t
    
    Since the input value is sometimes the result of a read()/write() call,
    avoid truncating the ssize_t value on calling it.

diff --git a/include/vtcp.h b/include/vtcp.h
index 05244f74b..c561c2567 100644
--- a/include/vtcp.h
+++ b/include/vtcp.h
@@ -37,7 +37,7 @@ struct suckaddr;
 #define VTCP_ADDRBUFSIZE		64
 #define VTCP_PORTBUFSIZE		16
 
-int VTCP_Check(int a);
+int VTCP_Check(ssize_t a);
 #define VTCP_Assert(a) assert(VTCP_Check(a))
 
 struct suckaddr *VTCP_my_suckaddr(int sock);
diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 53335aedf..7baee4b06 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -558,7 +558,7 @@ VTCP_check_hup(int sock)
  */
 
 int
-VTCP_Check(int a)
+VTCP_Check(ssize_t a)
 {
 	if (a == 0)
 		return (1);

From martin at varnish-software.com  Mon Nov  9 15:13:08 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:08 +0000 (UTC)
Subject: [master] 6afd6dc6f Enable the use of VTCP_Assert in v1f_read()
Message-ID: <20201109151308.CEA1C10D0ED@lists.varnish-cache.org>


commit 6afd6dc6f8c9492335cafdea41ce47e2fb6294eb
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 15:35:13 2020 +0200

    Enable the use of VTCP_Assert in v1f_read()
    
    Now that VTCP_Assert() accepts EAGAIN as a legal errno value for read()
    errors, uncomment this check.

diff --git a/bin/varnishd/http1/cache_http1_vfp.c b/bin/varnishd/http1/cache_http1_vfp.c
index 5e3b5cfa0..13e042e6f 100644
--- a/bin/varnishd/http1/cache_http1_vfp.c
+++ b/bin/varnishd/http1/cache_http1_vfp.c
@@ -44,6 +44,7 @@
 #include "cache_http1.h"
 
 #include "vct.h"
+#include "vtcp.h"
 
 /*--------------------------------------------------------------------
  * Read up to len bytes, returning pipelined data first.
@@ -76,7 +77,7 @@ v1f_read(const struct vfp_ctx *vc, struct http_conn *htc, void *d, ssize_t len)
 	if (len > 0) {
 		i = read(*htc->rfd, p, len);
 		if (i < 0) {
-			// XXX: VTCP_Assert(i); // but also: EAGAIN
+			VTCP_Assert(i);
 			VSLb(vc->wrk->vsl, SLT_FetchError,
 			    "%s", vstrerror(errno));
 			return (i);

From martin at varnish-software.com  Mon Nov  9 15:13:08 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:08 +0000 (UTC)
Subject: [master] 58a21da77 Allow a > 0 in VTCP_Check()
Message-ID: <20201109151308.F2D4B10D0F4@lists.varnish-cache.org>


commit 58a21da7736295e674ebb7e8ae9eee9529c083b0
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 15:58:14 2020 +0200

    Allow a > 0 in VTCP_Check()
    
    When used to check the result of read() and write() calls, it is useful
    that a positive return value is accepted in VTCP_Check().

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 7baee4b06..851e3137e 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -562,6 +562,8 @@ VTCP_Check(ssize_t a)
 {
 	if (a == 0)
 		return (1);
+	if (a > 0)
+		return (1);
 	if (errno == ECONNRESET || errno == ENOTCONN || errno == EPIPE)
 		return (1);
 	/* Accept EAGAIN (and EWOULDBLOCK in case they are not the same)

From martin at varnish-software.com  Mon Nov  9 15:13:09 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:09 +0000 (UTC)
Subject: [master] 81cc3b3be Use VTCP_Assert() in VTCP_read()
Message-ID: <20201109151309.240D110D104@lists.varnish-cache.org>


commit 81cc3b3be05a735e8d572a0e9cb08f521979f2b7
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 16:04:21 2020 +0200

    Use VTCP_Assert() in VTCP_read()

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 851e3137e..1cf21b0a1 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -619,5 +619,6 @@ VTCP_read(int fd, void *ptr, size_t len, vtim_dur tmo)
 			return (-2);
 	}
 	i = read(fd, ptr, len);
+	VTCP_Assert(i);
 	return (i < 0 ? -1 : i);
 }

From martin at varnish-software.com  Mon Nov  9 15:13:09 2020
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon,  9 Nov 2020 15:13:09 +0000 (UTC)
Subject: [master] 13db39049 Add VTCP_Assert() in various places that call
 read() and write()
Message-ID: <20201109151309.7F11210D10C@lists.varnish-cache.org>


commit 13db390498511366de5c5f4f4bae2a02a519e333
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Oct 7 16:17:44 2020 +0200

    Add VTCP_Assert() in various places that call read() and write()
    
    This adds VTCP_Assert() on the result of read and write calls that deals
    with TCP sockets.

diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c
index bf41b952a..8432c8582 100644
--- a/bin/varnishd/cache/cache_backend_probe.c
+++ b/bin/varnishd/cache/cache_backend_probe.c
@@ -229,6 +229,7 @@ vbp_write(struct vbp_target *vt, int *sock, const void *buf, size_t len)
 	int i;
 
 	i = write(*sock, buf, len);
+	VTCP_Assert(i);
 	if (i != len) {
 		if (i < 0) {
 			vt->err_xmit |= 1;
@@ -371,6 +372,7 @@ vbp_poke(struct vbp_target *vt)
 			    sizeof vt->resp_buf - rlen);
 		else
 			i = read(s, buf, sizeof buf);
+		VTCP_Assert(i);
 		if (i <= 0) {
 			if (i < 0)
 				bprintf(vt->resp_buf, "Read error %d (%s)",
diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c
index 77ef66f3f..b051499ee 100644
--- a/bin/varnishd/http1/cache_http1_deliver.c
+++ b/bin/varnishd/http1/cache_http1_deliver.c
@@ -35,6 +35,8 @@
 #include "cache/cache_filter.h"
 #include "cache_http1.h"
 
+#include "vtcp.h"
+
 /*--------------------------------------------------------------------*/
 
 static int v_matchproto_(vdp_bytes_f)
@@ -76,7 +78,7 @@ v1d_error(struct req *req, const char *msg)
 	VSLb(req->vsl, SLT_RespReason, "Internal Server Error");
 
 	req->wrk->stats->client_resp_500++;
-	(void)write(req->sp->fd, r_500, sizeof r_500 - 1);
+	VTCP_Assert(write(req->sp->fd, r_500, sizeof r_500 - 1));
 	req->doclose = SC_TX_EOF;
 }
 
diff --git a/bin/varnishd/http1/cache_http1_pipe.c b/bin/varnishd/http1/cache_http1_pipe.c
index 2ddaf7f44..d273ae4f4 100644
--- a/bin/varnishd/http1/cache_http1_pipe.c
+++ b/bin/varnishd/http1/cache_http1_pipe.c
@@ -39,6 +39,7 @@
 #include <stdio.h>
 
 #include "cache_http1.h"
+#include "vtcp.h"
 
 #include "VSC_vbe.h"
 
@@ -51,10 +52,12 @@ rdf(int fd0, int fd1, uint64_t *pcnt)
 	char buf[BUFSIZ], *p;
 
 	i = read(fd0, buf, sizeof buf);
+	VTCP_Assert(i);
 	if (i <= 0)
 		return (1);
 	for (p = buf; i > 0; i -= j, p += j) {
 		j = write(fd1, p, i);
+		VTCP_Assert(j);
 		if (j <= 0)
 			return (1);
 		*pcnt += j;
@@ -123,6 +126,7 @@ V1P_Process(const struct req *req, int fd, struct v1p_acct *v1a)
 	if (req->htc->pipeline_b != NULL) {
 		j = write(fd,  req->htc->pipeline_b,
 		    req->htc->pipeline_e - req->htc->pipeline_b);
+		VTCP_Assert(j);
 		if (j < 0)
 			return;
 		req->htc->pipeline_b = NULL;
diff --git a/bin/varnishd/http2/cache_http2_session.c b/bin/varnishd/http2/cache_http2_session.c
index 0637984b8..cc1816f54 100644
--- a/bin/varnishd/http2/cache_http2_session.c
+++ b/bin/varnishd/http2/cache_http2_session.c
@@ -40,6 +40,7 @@
 
 #include "vend.h"
 #include "vtim.h"
+#include "vtcp.h"
 
 static const char h2_resp_101[] =
 	"HTTP/1.1 101 Switching Protocols\r\n"
@@ -255,6 +256,7 @@ h2_ou_session(struct worker *wrk, struct h2_sess *h2,
 	}
 
 	sz = write(h2->sess->fd, h2_resp_101, strlen(h2_resp_101));
+	VTCP_Assert(sz);
 	if (sz != strlen(h2_resp_101)) {
 		VSLb(h2->vsl, SLT_Debug, "H2: Upgrade: Error writing 101"
 		    " response: %s\n", vstrerror(errno));
diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 36ffabff2..e9a2ce164 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -744,6 +744,7 @@ VPX_Send_Proxy(int fd, int version, const struct sess *sp)
 		WRONG("Wrong proxy version");
 
 	r = write(fd, VSB_data(vsb), VSB_len(vsb));
+	VTCP_Assert(r);
 
 	if (!DO_DEBUG(DBG_PROTOCOL))
 		return (r);

From phk at FreeBSD.org  Tue Nov 10 16:31:11 2020
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 Nov 2020 16:31:11 +0000 (UTC)
Subject: [master] ad7ab013c I'm not even going comment on how SunOS returns
 this to a read(2)...
Message-ID: <20201110163111.314E310E61A@lists.varnish-cache.org>


commit ad7ab013ce120fe7478f38530f52a61fb1300341
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 10 16:30:07 2020 +0000

    I'm not even going comment on how SunOS returns this to a read(2)...

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 1cf21b0a1..90857c329 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -575,6 +575,8 @@ VTCP_Check(ssize_t a)
 	if (errno == EAGAIN || errno == EWOULDBLOCK)
 		return (1);
 #if (defined (__SVR4) && defined (__sun))
+	if (errno == ECONNREFUSED)	// in r02702.vtc
+		return (1);
 	if (errno == EPROTO)
 		return (1);
 #endif

From nils.goroll at uplex.de  Thu Nov 12 16:31:11 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:31:11 +0000 (UTC)
Subject: [master] cab53eb33 shard: fix associated parameter set with
 resolve=NOW
Message-ID: <20201112163111.9F5D99E504@lists.varnish-cache.org>


commit cab53eb33b267a96161a00abd0d26416a2a3a9bc
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 16:45:40 2020 +0100

    shard: fix associated parameter set with resolve=NOW
    
    When a parameter set is associated at the director level, it should be
    in effect with resolve=NOW as with resolve=LAZY.

diff --git a/bin/varnishtest/tests/d00021.vtc b/bin/varnishtest/tests/d00021.vtc
index e691def28..cf0a1be27 100644
--- a/bin/varnishtest/tests/d00021.vtc
+++ b/bin/varnishtest/tests/d00021.vtc
@@ -97,8 +97,9 @@ varnish v1 -vcl+backend {
 	}
 
 	sub vcl_backend_response {
-	    if (bereq.http.layered) {
+	    if (bereq.http.layered || bereq.http.resolve) {
 		set beresp.http.healthy = std.healthy(ll.backend());
+		set beresp.http.backend-now = l.backend(resolve=NOW);
 	    } else {
 		set beresp.http.healthy = std.healthy(
 		  vd.backend(resolve=LAZY, by=KEY, key=1));
@@ -137,6 +138,7 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "ll"
 	expect resp.http.backend == "s1"
+	expect resp.http.backend == resp.http.backend-now
 
 	txreq -url /2 -hdr "layered: true"
 	rxresp
@@ -144,6 +146,7 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "ll"
 	expect resp.http.backend == "s2"
+	expect resp.http.backend == resp.http.backend-now
 
 	txreq -url /3 -hdr "layered: true"
 	rxresp
@@ -151,6 +154,7 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "ll"
 	expect resp.http.backend == "s3"
+	expect resp.http.backend == resp.http.backend-now
 
 	txreq -url /1 -hdr "resolve: true"
 	rxresp
@@ -158,6 +162,7 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "s1"
 	expect resp.http.backend == "s1"
+	expect resp.http.backend == resp.http.backend-now
 
 	txreq -url /2 -hdr "resolve: true"
 	rxresp
@@ -165,6 +170,7 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "s2"
 	expect resp.http.backend == "s2"
+	expect resp.http.backend == resp.http.backend-now
 
 	txreq -url /3 -hdr "resolve: true"
 	rxresp
@@ -172,4 +178,5 @@ client c1 {
 	expect resp.http.healthy == "true"
 	expect resp.http.director == "s3"
 	expect resp.http.backend == "s3"
+	expect resp.http.backend == resp.http.backend-now
 } -run
diff --git a/lib/libvmod_directors/vmod_shard.c b/lib/libvmod_directors/vmod_shard.c
index 5e0137bbc..a88095c26 100644
--- a/lib/libvmod_directors/vmod_shard.c
+++ b/lib/libvmod_directors/vmod_shard.c
@@ -608,6 +608,14 @@ vmod_shard_backend(VRT_CTX, struct vmod_directors_shard *vshard,
 	else
 		resolve = VENUM(NOW);
 
+	if (ctx->method & SHARD_VCL_TASK_BEREQ) {
+		pp = shard_param_task(ctx, vshard->shardd,
+		    vshard->shardd->param);
+		if (pp == NULL)
+			return (NULL);
+		pp->vcl_name = vshard->shardd->name;
+	}
+
 	if (resolve == VENUM(LAZY)) {
 		if ((args & ~arg_resolve) == 0) {
 			AN(vshard->dir);
@@ -621,14 +629,6 @@ vmod_shard_backend(VRT_CTX, struct vmod_directors_shard *vshard,
 			    "context");
 			return (NULL);
 		}
-
-		assert(ctx->method & SHARD_VCL_TASK_BEREQ);
-
-		pp = shard_param_task(ctx, vshard->shardd,
-				      vshard->shardd->param);
-		if (pp == NULL)
-			return (NULL);
-		pp->vcl_name = vshard->shardd->name;
 	} else if (resolve == VENUM(NOW)) {
 		if (ctx->method & VCL_MET_TASK_H) {
 			shard_fail(ctx, vshard->shardd->name, "%s",
@@ -636,8 +636,9 @@ vmod_shard_backend(VRT_CTX, struct vmod_directors_shard *vshard,
 			    "used in vcl_init{}/vcl_fini{}");
 			return (NULL);
 		}
-		pp = shard_param_stack(&pstk, vshard->shardd->param,
-				       vshard->shardd->name);
+		pp = shard_param_stack(&pstk,
+		    pp != NULL ? pp : vshard->shardd->param,
+		    vshard->shardd->name);
 	} else {
 		WRONG("resolve enum");
 	}

From nils.goroll at uplex.de  Thu Nov 12 16:31:11 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:31:11 +0000 (UTC)
Subject: [master] 8372154d5 shard: test alt parameter limiting
Message-ID: <20201112163111.C3CDB9E51A@lists.varnish-cache.org>


commit 8372154d50406da3645deda94fcff381d5f4ccd2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 17:01:53 2020 +0100

    shard: test alt parameter limiting

diff --git a/bin/varnishtest/tests/d00022.vtc b/bin/varnishtest/tests/d00022.vtc
index e8cf95a5c..c11cdaa0d 100644
--- a/bin/varnishtest/tests/d00022.vtc
+++ b/bin/varnishtest/tests/d00022.vtc
@@ -71,6 +71,7 @@ varnish v1 -vcl+backend {
 		set resp.http.all-alt-0 = vd.backend(by=KEY, key=1756955383, alt=0, healthy=ALL);
 		set resp.http.all-alt-1 = vd.backend(by=KEY, key=1756955383, alt=1, healthy=ALL);
 		set resp.http.all-alt-2 = vd.backend(by=KEY, key=1756955383, alt=2, healthy=ALL);
+		set resp.http.all-alt-3 = vd.backend(by=KEY, key=1756955383, alt=3, healthy=ALL);
 		set resp.http.chosen-alt-0 = vd.backend(by=KEY, key=1756955383, alt=0, healthy=CHOSEN);
 		set resp.http.chosen-alt-1 = vd.backend(by=KEY, key=1756955383, alt=1, healthy=CHOSEN);
 		set resp.http.chosen-alt-2 = vd.backend(by=KEY, key=1756955383, alt=2, healthy=CHOSEN);
@@ -86,6 +87,7 @@ client c1 {
 	expect resp.http.all-alt-0 == "s1"
 	expect resp.http.all-alt-1 == "s2"
 	expect resp.http.all-alt-2 == "s3"
+	expect resp.http.all-alt-3 == resp.http.all-alt-2
 	expect resp.http.chosen-alt-0 == "s1"
 	expect resp.http.chosen-alt-1 == "s2"
 	expect resp.http.chosen-alt-2 == "s3"
@@ -96,6 +98,7 @@ client c1 {
 	expect resp.http.all-alt-0 == "s1"
 	expect resp.http.all-alt-1 == "s2"
 	expect resp.http.all-alt-2 == "s3"
+	expect resp.http.all-alt-3 == resp.http.all-alt-2
 	expect resp.http.chosen-alt-0 == "s1"
 	expect resp.http.chosen-alt-1 == "s2"
 	expect resp.http.chosen-alt-2 == "s3"
@@ -106,6 +109,7 @@ client c1 {
 	expect resp.http.all-alt-0 == "s1"
 	expect resp.http.all-alt-1 == "s2"
 	expect resp.http.all-alt-2 == "s3"
+	expect resp.http.all-alt-3 == resp.http.all-alt-2
 	expect resp.http.chosen-alt-0 == "s1"
 	expect resp.http.chosen-alt-1 == "s2"
 	expect resp.http.chosen-alt-2 == "s3"

From nils.goroll at uplex.de  Thu Nov 12 16:31:11 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:31:11 +0000 (UTC)
Subject: [master] 8afbf7f63 tabularize bereq flags (only bereq.is_bgfetch for
 now)
Message-ID: <20201112163111.EDA249E520@lists.varnish-cache.org>


commit 8afbf7f6310662b73ca88505ea41cd2ecad74bd2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 17:17:53 2020 +0100

    tabularize bereq flags (only bereq.is_bgfetch for now)

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index b8dbd1303..e4c77ee07 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -399,7 +399,7 @@ struct busyobj {
 
 	struct pool_task	fetch_task[1];
 
-#define BO_FLAG(l, r, w, f, d) unsigned	l:1;
+#define BO_FLAG(l, r, rr, rw, f, d) unsigned	l:1;
 #include "tbl/bo_flags.h"
 
 	/* Timeouts */
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index c0443ebed..409f536d5 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -390,7 +390,7 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	VSB_cat(vsb, "flags = {");
 	p = "";
 /*lint -save -esym(438,p) -e539 */
-#define BO_FLAG(l, r, w, f, d)					\
+#define BO_FLAG(l, r, rr, rw, f, d)				\
 	if (bo->l) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
 #include "tbl/bo_flags.h"
 /*lint -restore */
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index b9165d96b..4a1e50860 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -230,9 +230,20 @@ VRT_r_beresp_##field(VRT_CTX)						\
 	return (ctx->bo->field);					\
 }
 
-#define BO_FLAG(l, r, w, f, d)			\
-	VBERESPR##r(l, #l, f)			\
-	VBERESPW##w(l, #l, f)
+#define VBEREQR0(field, str, fltchk)
+#define VBEREQR1(field, str, fltchk)					\
+VCL_BOOL								\
+VRT_r_bereq_##field(VRT_CTX)						\
+{									\
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);				\
+	CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);			\
+	return (ctx->bo->field);					\
+}
+
+#define BO_FLAG(l, r, rr, rw, f, d)		\
+	VBEREQR##r(l, #l, f)			\
+	VBERESPR##rr(l, #l, f)			\
+	VBERESPW##rw(l, #l, f)
 #include "tbl/bo_flags.h"
 
 #undef VBERESPWF0
@@ -246,14 +257,6 @@ VRT_r_beresp_##field(VRT_CTX)						\
 #undef VBERESPR1
 /*--------------------------------------------------------------------*/
 
-VCL_BOOL
-VRT_r_bereq_is_bgfetch(VRT_CTX)
-{
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
-	return (ctx->bo->is_bgfetch);
-}
-
 VCL_BOOL
 VRT_r_bereq_uncacheable(VRT_CTX)
 {
diff --git a/include/tbl/bo_flags.h b/include/tbl/bo_flags.h
index 9d26d6a97..e96e21a86 100644
--- a/include/tbl/bo_flags.h
+++ b/include/tbl/bo_flags.h
@@ -34,15 +34,15 @@
 /*
  * filters: whether this flag determines beresp.filters default
  *
- * lower, vcl_r, vcl_w, filters, doc */
-BO_FLAG(do_esi,		1, 1, 1, "")
-BO_FLAG(do_gzip,	1, 1, 1, "")
-BO_FLAG(do_gunzip,	1, 1, 1, "")
-BO_FLAG(do_stream,	1, 1, 0, "")
-BO_FLAG(do_pass,	0, 0, 0, "")
-BO_FLAG(uncacheable,	0, 0, 0, "")
-BO_FLAG(was_304,	1, 0, 0, "")
-BO_FLAG(is_bgfetch,	0, 0, 0, "")
+ * lower, vcl_r, vcl_beresp_r, vcl_beresp_w, filters, doc */
+BO_FLAG(do_esi,		0, 1, 1, 1, "")
+BO_FLAG(do_gzip,	0, 1, 1, 1, "")
+BO_FLAG(do_gunzip,	0, 1, 1, 1, "")
+BO_FLAG(do_stream,	0, 1, 1, 0, "")
+BO_FLAG(do_pass,	0, 0, 0, 0, "")
+BO_FLAG(uncacheable,	0, 0, 0, 0, "")
+BO_FLAG(was_304,	0, 1, 0, 0, "")
+BO_FLAG(is_bgfetch,	1, 0, 0, 0, "")
 #undef BO_FLAG
 
 /*lint -restore */

From nils.goroll at uplex.de  Thu Nov 12 16:31:12 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:31:12 +0000 (UTC)
Subject: [master] 49d32683d add bereq.is_hitpass and bereq.is_hitmiss
Message-ID: <20201112163112.26EA59E52D@lists.varnish-cache.org>


commit 49d32683d96a6558a53e523713c584abf12424f0
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 17:28:14 2020 +0100

    add bereq.is_hitpass and bereq.is_hitmiss
    
    as copies of the respective client side flags

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 4d7b1547c..0ba276571 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -1114,6 +1114,9 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 		WRONG("Wrong fetch mode");
 	}
 
+	bo->is_hitpass = req->is_hitpass;
+	bo->is_hitmiss = req->is_hitmiss;
+
 	VSLb(bo->vsl, SLT_Begin, "bereq %u %s", VXID(req->vsl->wid), how);
 	VSLb(bo->vsl, SLT_VCL_use, "%s", VCL_Name(bo->vcl));
 	VSLb(req->vsl, SLT_Link, "bereq %u %s", VXID(bo->vsl->wid), how);
diff --git a/bin/varnishtest/tests/c00075.vtc b/bin/varnishtest/tests/c00075.vtc
index 6caacbfaa..4609862ba 100644
--- a/bin/varnishtest/tests/c00075.vtc
+++ b/bin/varnishtest/tests/c00075.vtc
@@ -53,6 +53,8 @@ server s1 {
 varnish v1 \
 	-arg "-s default,1m" -vcl+backend {
 	sub vcl_backend_response {
+		set beresp.http.be-hitmiss = bereq.is_hitmiss;
+		set beresp.http.be-hitpass = bereq.is_hitpass;
 		if (bereq.url == "/hfp") {
 			return (pass(10m));
 		}
@@ -75,6 +77,8 @@ client c1 {
 	expect resp.bodylen == 65536
 	expect resp.http.is-hitmiss == false
 	expect resp.http.is-hitpass == false
+	expect resp.http.be-hitmiss == resp.http.is-hitmiss
+	expect resp.http.be-hitpass == resp.http.is-hitpass
 	expect_pattern
 } -start
 
@@ -98,6 +102,8 @@ client c1 {
 	expect resp.bodylen == 65536
 	expect resp.http.is-hitmiss == true
 	expect resp.http.is-hitpass == false
+	expect resp.http.be-hitmiss == resp.http.is-hitmiss
+	expect resp.http.be-hitpass == resp.http.is-hitpass
 	expect_pattern
 } -start
 
@@ -122,6 +128,8 @@ client c1 {
 	expect resp.bodylen == 65536
 	expect resp.http.is-hitmiss == false
 	expect resp.http.is-hitpass == false
+	expect resp.http.be-hitmiss == resp.http.is-hitmiss
+	expect resp.http.be-hitpass == resp.http.is-hitpass
 	expect_pattern
 } -start
 
@@ -145,6 +153,8 @@ client c1 {
 	expect resp.bodylen == 65536
 	expect resp.http.is-hitmiss == false
 	expect resp.http.is-hitpass == true
+	expect resp.http.be-hitmiss == resp.http.is-hitmiss
+	expect resp.http.be-hitpass == resp.http.is-hitpass
 	expect_pattern
 } -start
 
diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst
index 72098379a..27707dd7f 100644
--- a/doc/sphinx/reference/vcl_var.rst
+++ b/doc/sphinx/reference/vcl_var.rst
@@ -650,6 +650,22 @@ bereq.is_bgfetch
 	grace, and this fetch was kicked of in the background to get
 	a fresh copy.
 
+bereq.is_hitmiss
+
+	Type: BOOL
+
+	Readable from: backend
+
+	If this backend request was caused by a hitmiss.
+
+bereq.is_hitpass
+
+	Type: BOOL
+
+	Readable from: backend
+
+	If this backend request was caused by a hitpass.
+
 beresp
 ~~~~~~
 
diff --git a/include/tbl/bo_flags.h b/include/tbl/bo_flags.h
index e96e21a86..ac0fdf241 100644
--- a/include/tbl/bo_flags.h
+++ b/include/tbl/bo_flags.h
@@ -43,6 +43,8 @@ BO_FLAG(do_pass,	0, 0, 0, 0, "")
 BO_FLAG(uncacheable,	0, 0, 0, 0, "")
 BO_FLAG(was_304,	0, 1, 0, 0, "")
 BO_FLAG(is_bgfetch,	1, 0, 0, 0, "")
+BO_FLAG(is_hitmiss,	1, 0, 0, 0, "")
+BO_FLAG(is_hitpass,	1, 0, 0, 0, "")
 #undef BO_FLAG
 
 /*lint -restore */

From nils.goroll at uplex.de  Thu Nov 12 16:48:06 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:48:06 +0000 (UTC)
Subject: [master] 2508b65a8 flexelint c7ac5266377ed6171548fcd61cbf5f095f98fa41
Message-ID: <20201112164806.9F454A1773@lists.varnish-cache.org>


commit 2508b65a81a5e96bbaf96f68da7caa807b9c2320
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 17:44:04 2020 +0100

    flexelint c7ac5266377ed6171548fcd61cbf5f095f98fa41
    
    constify and avoid (char *) = strchr((const char *), x)
    
    error:
    
    vtc_varnish.c  559  Error 158: Assignment to variable 'la' (line 548)
    increases capability

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 52447b11b..db8fc91ee 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -545,10 +545,10 @@ varnish_launch(struct varnish *v)
  */
 
 static void
-varnish_listen(struct varnish *v, char *la)
+varnish_listen(const struct varnish *v, char *la)
 {
-	const char *a, *p, *n, *n2;
-	char m[64], s[256];
+	const char *a, *p, *n2;
+	char *n, m[64], s[256];
 	unsigned first;
 
 	n2 = "";

From nils.goroll at uplex.de  Thu Nov 12 16:56:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 12 Nov 2020 16:56:07 +0000 (UTC)
Subject: [master] 4148837d2 flexelint c7ac5266377ed6171548fcd61cbf5f095f98fa41
Message-ID: <20201112165607.43747A1C56@lists.varnish-cache.org>


commit 4148837d2304ab86fdda7cb9bbedd6f980c25421
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 12 17:54:38 2020 +0100

    flexelint c7ac5266377ed6171548fcd61cbf5f095f98fa41
    
    polish differently and more sensibly

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index db8fc91ee..9912da43b 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -547,8 +547,8 @@ varnish_launch(struct varnish *v)
 static void
 varnish_listen(const struct varnish *v, char *la)
 {
-	const char *a, *p, *n2;
-	char *n, m[64], s[256];
+	const char *a, *p, *n, *n2;
+	char m[64], s[256];
 	unsigned first;
 
 	n2 = "";
@@ -556,15 +556,15 @@ varnish_listen(const struct varnish *v, char *la)
 
 	while (*la != '\0') {
 		n = la;
-		la = strchr(n, ' ');
+		la = strchr(la, ' ');
 		AN(la);
 		*la = '\0';
-		a = la + 1;
-		la = strchr(a, ' ');
+		a = ++la;
+		la = strchr(la, ' ');
 		AN(la);
 		*la = '\0';
-		p = la + 1;
-		la = strchr(p, '\n');
+		p = ++la;
+		la = strchr(la, '\n');
 		AN(la);
 		*la = '\0';
 		la++;

From dridi.boukelmoune at gmail.com  Fri Nov 13 11:11:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 13 Nov 2020 11:11:15 +0000 (UTC)
Subject: [master] e12880497 Revert "HSH_DerefObjCore signedness fix"
Message-ID: <20201113111115.3782193ED8@lists.varnish-cache.org>


commit e128804977f21abc6becff03efd853cd71fd3c69
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Nov 2 11:31:25 2020 +0100

    Revert "HSH_DerefObjCore signedness fix"
    
    This reverts commit 5367be3f62f45380ef7b5348eefb88a9c1507318.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index c9eafa92b..9e80efcb1 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -975,7 +975,7 @@ HSH_DerefBoc(struct worker *wrk, struct objcore *oc)
  * Returns zero if target was destroyed.
  */
 
-unsigned
+int
 HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp, int rushmax)
 {
 	struct objcore *oc;
diff --git a/bin/varnishd/cache/cache_objhead.h b/bin/varnishd/cache/cache_objhead.h
index d9a015417..bc1782379 100644
--- a/bin/varnishd/cache/cache_objhead.h
+++ b/bin/varnishd/cache/cache_objhead.h
@@ -66,7 +66,7 @@ struct boc *HSH_RefBoc(const struct objcore *);
 void HSH_DerefBoc(struct worker *wrk, struct objcore *);
 void HSH_DeleteObjHead(const struct worker *, struct objhead *);
 
-unsigned HSH_DerefObjCore(struct worker *, struct objcore **, int rushmax);
+int HSH_DerefObjCore(struct worker *, struct objcore **, int rushmax);
 #define HSH_RUSH_POLICY -1
 
 enum lookup_e HSH_Lookup(struct req *, struct objcore **, struct objcore **);
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index 9a52edf5b..19d05f768 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -260,7 +260,7 @@ VRB_Ignore(struct req *req)
 void
 VRB_Free(struct req *req)
 {
-	unsigned r;
+	int r;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 
@@ -269,6 +269,7 @@ VRB_Free(struct req *req)
 
 	r = HSH_DerefObjCore(req->wrk, &req->body_oc, 0);
 
+
 	// each busyobj may have gained a reference
 	assert (r <= req->restarts + 1);
 }

From dridi.boukelmoune at gmail.com  Fri Nov 13 11:11:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 13 Nov 2020 11:11:15 +0000 (UTC)
Subject: [master] 9939e2b9b param: Give a typedef to bits parameters
Message-ID: <20201113111115.466F693EDB@lists.varnish-cache.org>


commit 9939e2b9b8f40ecf4773eb1a8e122cdff7b55c36
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Oct 22 07:56:04 2020 +0200

    param: Give a typedef to bits parameters
    
    A nice and tidy single token to integrate with the parameters table.
    
    Refs #3250

diff --git a/bin/varnishd/common/common_param.h b/bin/varnishd/common/common_param.h
index fd4cfc0fd..e11967a2a 100644
--- a/bin/varnishd/common/common_param.h
+++ b/bin/varnishd/common/common_param.h
@@ -71,6 +71,11 @@ struct poolparam {
 	double			max_age;
 };
 
+
+typedef uint8_t vsl_mask_t[256>>3];
+typedef uint8_t debug_t[(DBG_Reserved+7)>>3];
+typedef uint8_t feature_t[(FEATURE_Reserved+7)>>3];
+
 struct params {
 
 #define ptyp_bool		unsigned
@@ -105,7 +110,7 @@ struct params {
 
 	struct vre_limits	vre_limits;
 
-	uint8_t			vsl_mask[256>>3];
-	uint8_t			debug_bits[(DBG_Reserved+7)>>3];
-	uint8_t			feature_bits[(FEATURE_Reserved+7)>>3];
+	vsl_mask_t		vsl_mask;
+	debug_t			debug_bits;
+	feature_t		feature_bits;
 };

From dridi.boukelmoune at gmail.com  Fri Nov 13 11:11:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 13 Nov 2020 11:11:15 +0000 (UTC)
Subject: [master] 13ac4047e panic: Whitespace OCD
Message-ID: <20201113111115.69A9B93EDE@lists.varnish-cache.org>


commit 13ac4047e88641ce2f69e0cb1321051a2138976f
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 13 12:06:28 2020 +0100

    panic: Whitespace OCD

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 409f536d5..84b4a77c8 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -702,8 +702,8 @@ pan_argv(struct vsb *vsb)
 		VSB_quote(vsb, heritage.argv[i], -1, VSB_QUOTE_CSTR);
 		VSB_cat(vsb, ",\n");
 	}
-	VSB_cat(vsb, "}\n");
 	VSB_indent(vsb, -2);
+	VSB_cat(vsb, "}\n");
 
 }
 /*--------------------------------------------------------------------*/

From dridi.boukelmoune at gmail.com  Fri Nov 13 11:11:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 13 Nov 2020 11:11:15 +0000 (UTC)
Subject: [master] c582cefd5 vtc: Make t02011.vtc failures more visible
Message-ID: <20201113111115.8F17C93EE1@lists.varnish-cache.org>


commit c582cefd5d085d63b2caefe6b56cc1385348190f
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 13 12:07:04 2020 +0100

    vtc: Make t02011.vtc failures more visible
    
    Seeing a test failure on vtest I think there might be a race here that I
    hope to make more explicit next time it occurs.

diff --git a/bin/varnishtest/tests/t02011.vtc b/bin/varnishtest/tests/t02011.vtc
index 10ed15286..6991e1025 100644
--- a/bin/varnishtest/tests/t02011.vtc
+++ b/bin/varnishtest/tests/t02011.vtc
@@ -32,7 +32,7 @@ varnish v1 -vcl+backend {
 
 	sub vcl_recv {
 		if (req.http.should == "reset") {
-			return (fail);
+			vtc.panic("Expected stream reset REFUSED_STREAM");
 		}
 	}
 

From dridi.boukelmoune at gmail.com  Fri Nov 13 11:31:09 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 13 Nov 2020 11:31:09 +0000 (UTC)
Subject: [master] 948661aac hash: Fix refcount type in HSH_DerefObjCore()
Message-ID: <20201113113109.165B294DB9@lists.varnish-cache.org>


commit 948661aacf62983c2af4ec6bac25b9041029ae90
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 13 12:28:17 2020 +0100

    hash: Fix refcount type in HSH_DerefObjCore()
    
    We want to avoid unsigned integers for reference counting since they
    would hide underflows.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 9e80efcb1..daf8a9cf7 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -981,7 +981,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp, int rushmax)
 	struct objcore *oc;
 	struct objhead *oh;
 	struct rush rush;
-	unsigned r;
+	int r;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	TAKE_OBJ_NOTNULL(oc, ocp, OBJCORE_MAGIC);
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index 19d05f768..bedd342c5 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -269,7 +269,6 @@ VRB_Free(struct req *req)
 
 	r = HSH_DerefObjCore(req->wrk, &req->body_oc, 0);
 
-
 	// each busyobj may have gained a reference
 	assert (r <= req->restarts + 1);
 }

From nils.goroll at uplex.de  Mon Nov 16 14:00:12 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 16 Nov 2020 14:00:12 +0000 (UTC)
Subject: [master] d0a23b2b2 std.set_ip_tos() to support IPv6
Message-ID: <20201116140012.6B6D8965C1@lists.varnish-cache.org>


commit d0a23b2b25f6717bc3260f20c99847d0a3fc465c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 16 14:38:37 2020 +0100

    std.set_ip_tos() to support IPv6

diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c
index 948a0972b..57b12c6a6 100644
--- a/lib/libvmod_std/vmod_std.c
+++ b/lib/libvmod_std/vmod_std.c
@@ -55,15 +55,26 @@ VCL_VOID v_matchproto_(td_std_set_ip_tos)
 vmod_set_ip_tos(VRT_CTX, VCL_INT tos)
 {
 	struct suckaddr *sa;
-	int itos = tos;
+	int fam, itos = tos;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AZ(SES_Get_local_addr(ctx->req->sp, &sa));
 	/* Silently ignore for non-IP addresses. */
 	if (VSA_Compare(sa, bogo_ip) == 0)
 		return;
-	VTCP_Assert(setsockopt(ctx->req->sp->fd,
-	    IPPROTO_IP, IP_TOS, &itos, sizeof(itos)));
+	fam = VSA_Get_Proto(sa);
+	switch (fam) {
+	case PF_INET:
+		VTCP_Assert(setsockopt(ctx->req->sp->fd,
+		    IPPROTO_IP, IP_TOS, &itos, sizeof(itos)));
+		break;
+	case PF_INET6:
+		VTCP_Assert(setsockopt(ctx->req->sp->fd,
+		    IPPROTO_IPV6, IPV6_TCLASS, &itos, sizeof(itos)));
+		break;
+	default:
+		INCOMPL();
+	}
 }
 
 static const char *
diff --git a/lib/libvmod_std/vmod_std.vcc b/lib/libvmod_std/vmod_std.vcc
index 961e90ae1..d0aea5707 100644
--- a/lib/libvmod_std/vmod_std.vcc
+++ b/lib/libvmod_std/vmod_std.vcc
@@ -538,12 +538,14 @@ Example::
 
 $Function VOID set_ip_tos(INT tos)
 
-Sets the IP type-of-service (TOS) field for the current session to
-*tos*. Silently ignored if the listen address is a Unix domain socket.
-
-Please note that the TOS field is not removed by the end of the
-request so probably want to set it on every request should you utilize
-it.
+Sets the Differentiated Services Codepoint (DSCP) / IPv4 Type of
+Service (TOS) / IPv6 Traffic Class (TCLASS) byte for the current
+session to *tos*. Silently ignored if the listen address is a Unix
+domain socket.
+
+Please note that setting the traffic class affects all requests on the
+same http1.1 / http2 TCP connection and, in particular, is not removed
+at the end of the request.
 
 Example::
 

From dridi.boukelmoune at gmail.com  Mon Nov 16 16:45:08 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 16 Nov 2020 16:45:08 +0000 (UTC)
Subject: [master] 31164908c clean out vmod cache at startup
Message-ID: <20201116164508.D2337A03F9@lists.varnish-cache.org>


commit 31164908ce8e81d116e5a30b7fc404e00f4cc4b3
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 16 16:06:50 2020 +0100

    clean out vmod cache at startup
    
    Fixes #3243

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 65ccfcb3d..db886f1e6 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -787,6 +787,10 @@ main(int argc, char * const *argv)
 		ARGV_ERR("Cannot create working directory (%s): %s\n",
 		    workdir, vstrerror(errno));
 
+	VJ_master(JAIL_MASTER_FILE);
+	AZ(system("rm -rf vmod_cache"));
+	VJ_master(JAIL_MASTER_LOW);
+
 	if (VJ_make_subdir("vmod_cache", "VMOD cache", NULL)) {
 		ARGV_ERR(
 		    "Cannot create vmod directory (%s/vmod_cache): %s\n",

From nils.goroll at uplex.de  Wed Nov 18 14:35:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 14:35:08 +0000 (UTC)
Subject: [master] a2330f6af flexelint b78cb58ab820e30696f5591279e9081a5b3e8cf1
Message-ID: <20201118143508.7B276968A6@lists.varnish-cache.org>


commit a2330f6afbfe7fc8a526dad1f53465cd6953c123
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 15:02:52 2020 +0100

    flexelint b78cb58ab820e30696f5591279e9081a5b3e8cf1
    
    Yes, we do not actually use the mode name, but we still want to have it
    for debugging and clarity.
    
    Info 754: local struct member 'vsc_sf_mode::name' (line 57, file
        ../../lib/libvarnishapi/vsc.c) not referenced

diff --git a/lib/libvarnishapi/vsc.c b/lib/libvarnishapi/vsc.c
index 445c55b79..05e3c11e2 100644
--- a/lib/libvarnishapi/vsc.c
+++ b/lib/libvarnishapi/vsc.c
@@ -152,6 +152,7 @@ vsc_sf_arg(struct vsc *vsc, const char *glob, const struct vsc_sf_mode *mode)
 	AN(sf);
 	REPLACE(sf->pattern, glob);
 	sf->mode = mode;
+	AN(mode->name);
 	if (mode->append)
 		VTAILQ_INSERT_TAIL(&vsc->sf_list, sf, list);
 	else

From nils.goroll at uplex.de  Wed Nov 18 14:35:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 14:35:08 +0000 (UTC)
Subject: [master] c72248915 flexelint vcc
Message-ID: <20201118143508.96BAB968AB@lists.varnish-cache.org>


commit c722489158eb28c11d0a54283595b1b5261bcbc7
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 15:07:38 2020 +0100

    flexelint vcc
    
    Another case of struct instance names which we want to have, even though
    we do not currently use them.

diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index b97b20bca..e3c07f630 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -264,6 +264,7 @@ VCC_SymbolGet(struct vcc *tl, vcc_ns_t ns, vcc_kind_t kind,
 
 	AN(tl);
 	CHECK_OBJ_NOTNULL(ns, VCC_NAMESPACE_MAGIC);
+	AN(ns->name);
 	CHECK_OBJ_NOTNULL(kind, KIND_MAGIC);
 	AN(e);
 	AN(x);
diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c
index 0bd9f2aed..b6f08bd71 100644
--- a/lib/libvcc/vcc_xref.c
+++ b/lib/libvcc/vcc_xref.c
@@ -119,6 +119,7 @@ vcc_AddUses(struct vcc *tl, const struct token *t1, const struct token *t2,
 	AN(pu);
 	AN(sym);
 	AN(use);
+	AN(use->name);
 	pu->t1 = t1;
 	pu->t2 = t2;
 	if (pu->t2 == NULL)

From nils.goroll at uplex.de  Wed Nov 18 14:35:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 14:35:08 +0000 (UTC)
Subject: [master] 96430d192 flexelint params.h
Message-ID: <20201118143508.CAB91968AF@lists.varnish-cache.org>


commit 96430d192d56e104de21c0f6d6db0240500779a1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 15:11:41 2020 +0100

    flexelint params.h
    
    we do want to use FLAGX | FLAGY even if zero for clarity
    
    error:
    
    Info 835: A zero has been given as right argument to operator '|'

diff --git a/include/tbl/params.h b/include/tbl/params.h
index 51ad42456..af751fdf2 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -34,7 +34,7 @@
  * ...: struct parspec fields, except the non-const string fields
  */
 
-/*lint -save -e525 -e539 */
+/*lint -save -e525 -e539 -e835 */
 
 /*--------------------------------------------------------------------
  *  * Simple parameters

From nils.goroll at uplex.de  Wed Nov 18 14:35:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 14:35:08 +0000 (UTC)
Subject: [master] 0bf8e9c78 flexelint e128804977f21abc6becff03efd853cd71fd3c69
Message-ID: <20201118143509.0BFB2968BF@lists.varnish-cache.org>


commit 0bf8e9c78062710c4cfee9819cbaa1ddc28b60e1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 15:15:30 2020 +0100

    flexelint e128804977f21abc6becff03efd853cd71fd3c69

diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index bedd342c5..f4eedc1f5 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -270,7 +270,8 @@ VRB_Free(struct req *req)
 	r = HSH_DerefObjCore(req->wrk, &req->body_oc, 0);
 
 	// each busyobj may have gained a reference
-	assert (r <= req->restarts + 1);
+	assert (r >= 0);
+	assert ((unsigned)r <= req->restarts + 1);
 }
 
 /*----------------------------------------------------------------------

From nils.goroll at uplex.de  Wed Nov 18 14:48:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 14:48:07 +0000 (UTC)
Subject: [master] 4b6ade1de use the right jail level for a system() call
Message-ID: <20201118144807.7A17D9751E@lists.varnish-cache.org>


commit 4b6ade1de5750e1d1c32cc0021adfad57ea9fa35
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 15:45:49 2020 +0100

    use the right jail level for a system() call
    
    Ref 31164908ce8e81d116e5a30b7fc404e00f4cc4b3
    
    Fixes #3243

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index db886f1e6..b848df76c 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -787,7 +787,7 @@ main(int argc, char * const *argv)
 		ARGV_ERR("Cannot create working directory (%s): %s\n",
 		    workdir, vstrerror(errno));
 
-	VJ_master(JAIL_MASTER_FILE);
+	VJ_master(JAIL_MASTER_SYSTEM);
 	AZ(system("rm -rf vmod_cache"));
 	VJ_master(JAIL_MASTER_LOW);
 

From nils.goroll at uplex.de  Wed Nov 18 15:18:06 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 15:18:06 +0000 (UTC)
Subject: [master] 2342a05d6 flexelint vmod_unix
Message-ID: <20201118151806.AC2159B473@lists.varnish-cache.org>


commit 2342a05d6554f2d72ec1939b64eeb8c4ad7fdd6a
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 16:15:11 2020 +0100

    flexelint vmod_unix
    
    Solve comparison between unsigned and -1 by moving a cast to where we need it.
    
    the existing code already implied that a VCL_INT be large enough to hold
    uid_t / gid_t.

diff --git a/lib/libvmod_unix/vmod_unix.c b/lib/libvmod_unix/vmod_unix.c
index d7b882693..0dfbd5ae2 100644
--- a/lib/libvmod_unix/vmod_unix.c
+++ b/lib/libvmod_unix/vmod_unix.c
@@ -123,16 +123,16 @@ VCL_STRING							\
 vmod_##func(VRT_CTX)						\
 {								\
 	struct type *s;						\
-	id##_t i;						\
+	VCL_INT ret;						\
 	VCL_STRING name;					\
 								\
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);			\
-	i = (id##_t) vmod_##id(ctx);				\
-	if (i == -1)						\
+	ret = vmod_##id(ctx);					\
+	if (ret == -1)						\
 		return (NULL);					\
 								\
 	errno = 0;						\
-	s = get(i);						\
+	s = get((id##_t) ret);					\
 	if (s == NULL) {					\
 		ERRNOCREDS(ctx);				\
 		return (NULL);					\

From nils.goroll at uplex.de  Wed Nov 18 15:47:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 15:47:07 +0000 (UTC)
Subject: [master] d40966416 silence flexelint for -1 argument to chown()
Message-ID: <20201118154707.2251E9C367@lists.varnish-cache.org>


commit d409664160da3ad50fc52d4ffe266faf047bfd87
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 16:26:59 2020 +0100

    silence flexelint for -1 argument to chown()

diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 3caf03b01..f1c65166b 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -125,7 +125,7 @@
 -e850	// for loop index variable '___' whose type category is '___'
 	// is modified in body of the for loop that began at '___'
 
-
+-esym(570, chown)	   // loss of sign for unchanged argument
 -esym(765, vcc_ProcAction) // could be made static
 -esym(759, vcc_ProcAction) // could be moved to module
 -esym(714, vcc_ProcAction) // not ref.

From nils.goroll at uplex.de  Wed Nov 18 15:47:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 15:47:07 +0000 (UTC)
Subject: [master] d8549f89c flexelint silencing
Message-ID: <20201118154707.3B5719C36A@lists.varnish-cache.org>


commit d8549f89c531d5b68573d2d5cb96299d7580858e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 16:39:46 2020 +0100

    flexelint silencing
    
    leaving these in the "to be reviewed" section because I am not entirely
    sure.
    
    I think these cases are:
    
    - ses_set_attr / beresp_filter_fixed: flexelint bug? Looks like it does not
      register use via macros
    
    - ObjGetU32/Lck_DestroyClass API
    
    - HTTP_IterHdrPack / SYMTAB_NOERR cases we do not want to make static
      by design

diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index f1c65166b..4cb63a3a7 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -162,3 +162,18 @@
 -e778   // Constant expression evaluates to 0 in operation '___'
 -e736	// Loss of precision (___) (___ bits to ___ bits)
 -e655	// bitwise compatible enums
+
+// cache_session.c
+-esym(528, ses_set_attr)	// Not referenced
+// cache_vrt_var.c
+-esym(528, beresp_filter_fixed) // Not referenced
+
+-esym(714, Lck_DestroyClass)	// Not referenced
+-esym(759, HTTP_IterHdrPack)	// Could be moved to module
+-esym(759, ObjGetU32)		// Could be moved to module
+-esym(759, Lck_DestroyClass)	// Could be moved to module
+-esym(759, SYMTAB_NOERR)	// Could be moved to module
+-esym(765, HTTP_IterHdrPack)	// Could be made static
+-esym(765, ObjGetU32)		// Could be made static
+-esym(765, Lck_DestroyClass)	// Could be made static
+-esym(765, SYMTAB_NOERR)	// Could be made static

From nils.goroll at uplex.de  Wed Nov 18 16:06:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 16:06:07 +0000 (UTC)
Subject: [master] cde002016 silence flexelint "vsa.h not used"
Message-ID: <20201118160607.A5FB59CEC5@lists.varnish-cache.org>


commit cde002016c68a88cdb0a82dc3635a4ae4be1e493
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 16:54:28 2020 +0100

    silence flexelint "vsa.h not used"
    
    it does not register use via tbl/sess_attr.h

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index f927538fd..5c5f94b20 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -35,6 +35,7 @@
  * sess fields, for instance ->fd being negative ->reason.
  *
  */
+//lint -e{766}
 
 #include "config.h"
 

From nils.goroll at uplex.de  Wed Nov 18 16:06:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 16:06:07 +0000 (UTC)
Subject: [master] 7847a8e14 silence flexelint on obj_attr::OA__MAX
Message-ID: <20201118160607.B7FEC9CEC7@lists.varnish-cache.org>


commit 7847a8e140a46da656ecc0f475fdcd39853d950e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 16:59:36 2020 +0100

    silence flexelint on obj_attr::OA__MAX

diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 4cb63a3a7..488c8dbe4 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -177,3 +177,4 @@
 -esym(765, ObjGetU32)		// Could be made static
 -esym(765, Lck_DestroyClass)	// Could be made static
 -esym(765, SYMTAB_NOERR)	// Could be made static
+-esum(769, obj_attr::OA__MAX)	// Not referenced

From nils.goroll at uplex.de  Wed Nov 18 16:06:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 16:06:07 +0000 (UTC)
Subject: [master] fb9aea68f move flexelint silencing from global to module
Message-ID: <20201118160607.D25AA9CECA@lists.varnish-cache.org>


commit fb9aea68fa02bbaa38a40a17870d0553383d5e4f
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 17:04:46 2020 +0100

    move flexelint silencing from global to module
    
    we see this issue in two places (for libvcc and varnishd), hope this
    will silence both

diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 488c8dbe4..e2af26b69 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -172,9 +172,7 @@
 -esym(759, HTTP_IterHdrPack)	// Could be moved to module
 -esym(759, ObjGetU32)		// Could be moved to module
 -esym(759, Lck_DestroyClass)	// Could be moved to module
--esym(759, SYMTAB_NOERR)	// Could be moved to module
 -esym(765, HTTP_IterHdrPack)	// Could be made static
 -esym(765, ObjGetU32)		// Could be made static
 -esym(765, Lck_DestroyClass)	// Could be made static
--esym(765, SYMTAB_NOERR)	// Could be made static
 -esum(769, obj_attr::OA__MAX)	// Not referenced
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index e3c07f630..807015c09 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -27,6 +27,7 @@
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  */
+/*lint -save -esym(759, SYMTAB_NOERR) -esym(765, SYMTAB_NOERR)*/
 
 #include "config.h"
 
@@ -534,3 +535,4 @@ VCC_HandleSymbol(struct vcc *tl, vcc_type_t fmt, const char *pfx)
 		sym->def_b = t;
 	return (sym);
 }
+/*lint -restore */

From nils.goroll at uplex.de  Wed Nov 18 16:11:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 16:11:07 +0000 (UTC)
Subject: [master] 2698789de flexelint: try to address chown(-1) diffrently
Message-ID: <20201118161107.EF017A05E4@lists.varnish-cache.org>


commit 2698789de2698ecda0e1b26b58cd075f00544a93
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 17:09:19 2020 +0100

    flexelint: try to address chown(-1) diffrently
    
    I do not understand the global -esym does not work, but the warning is
    definitely bogus, the (unisgned)-1 argument is intentional and a
    documented feature.

diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index e2af26b69..82e1891e8 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -125,7 +125,6 @@
 -e850	// for loop index variable '___' whose type category is '___'
 	// is modified in body of the for loop that began at '___'
 
--esym(570, chown)	   // loss of sign for unchanged argument
 -esym(765, vcc_ProcAction) // could be made static
 -esym(759, vcc_ProcAction) // could be moved to module
 -esym(714, vcc_ProcAction) // not ref.
diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c
index 15aed013a..9d2f2755a 100644
--- a/bin/varnishd/mgt/mgt_jail_unix.c
+++ b/bin/varnishd/mgt/mgt_jail_unix.c
@@ -283,7 +283,9 @@ vju_make_workdir(const char *dname, const char *what, struct vsb *vsb)
 		    dname, vstrerror(errno));
 		return (1);
 	}
+/*lint -save -e570 */
 	AZ(chown(dname, -1, vju_gid));
+/*lint -restore */
 	AZ(seteuid(vju_uid));
 	return (0);
 }

From nils.goroll at uplex.de  Wed Nov 18 16:27:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 18 Nov 2020 16:27:07 +0000 (UTC)
Subject: [master] b6325269b polish flexelint options and fix typo
Message-ID: <20201118162707.5F0D9A0F0E@lists.varnish-cache.org>


commit b6325269bc9bbb16923548675948fc25c96b91b2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 17:25:47 2020 +0100

    polish flexelint options and fix typo

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 5c5f94b20..e9ca3f215 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -35,7 +35,7 @@
  * sess fields, for instance ->fd being negative ->reason.
  *
  */
-//lint -e{766}
+//lint --e{766}
 
 #include "config.h"
 
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 82e1891e8..ffb48351c 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -174,4 +174,4 @@
 -esym(765, HTTP_IterHdrPack)	// Could be made static
 -esym(765, ObjGetU32)		// Could be made static
 -esym(765, Lck_DestroyClass)	// Could be made static
--esum(769, obj_attr::OA__MAX)	// Not referenced
+-esym(769, obj_attr::OA__MAX)	// Not referenced
diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c
index 9d2f2755a..c0d5c50d9 100644
--- a/bin/varnishd/mgt/mgt_jail_unix.c
+++ b/bin/varnishd/mgt/mgt_jail_unix.c
@@ -283,9 +283,8 @@ vju_make_workdir(const char *dname, const char *what, struct vsb *vsb)
 		    dname, vstrerror(errno));
 		return (1);
 	}
-/*lint -save -e570 */
+	//lint -e{570}
 	AZ(chown(dname, -1, vju_gid));
-/*lint -restore */
 	AZ(seteuid(vju_uid));
 	return (0);
 }

From nils.goroll at uplex.de  Fri Nov 20 15:23:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 20 Nov 2020 15:23:07 +0000 (UTC)
Subject: [master] f61dde46f vcc: Pass literal VCL_STRANDS arguments
Message-ID: <20201120152308.11B5A9B225@lists.varnish-cache.org>


commit f61dde46f4e3863543a21ebcbe64767d2ef5807b
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Oct 27 23:42:08 2020 +0100

    vcc: Pass literal VCL_STRANDS arguments
    
    avoiding local variables and VPI_BundleStrands()
    
    Fixes #3392

diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index ceb6f3441..f5fa84ea3 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -222,27 +222,6 @@ VRT_AllocStrandsWS(struct ws *ws, int n)
 	return (s);
 }
 
-/*--------------------------------------------------------------------
- * Build STRANDS from what is essentially a STRING_LIST
- */
-
-VCL_STRANDS
-VPI_BundleStrands(int n, struct strands *s, char const **d, const char *f, ...)
-{
-	va_list ap;
-
-	assert(n > 0);
-	s->n = n;
-	s->p = d;
-	*d++ = f;
-	va_start(ap, f);
-	while(--n)
-		*d++ = va_arg(ap, const char *);
-	assert(va_arg(ap, const char *) == vrt_magic_string_end);
-	va_end(ap);
-	return (s);
-}
-
 /*--------------------------------------------------------------------
  * Compare two STRANDS
  */
diff --git a/include/vcc_interface.h b/include/vcc_interface.h
index 9d9f46d6d..c754af479 100644
--- a/include/vcc_interface.h
+++ b/include/vcc_interface.h
@@ -71,6 +71,3 @@ struct vpi_ii {
 	const void *			p;
 	const char * const		name;
 };
-
-VCL_STRANDS VPI_BundleStrands(int, struct strands *, char const **,
-    const char *f, ...);
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 1447ec2b3..376e8f614 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -153,6 +153,8 @@ vcc_expr_edit(struct vcc *tl, vcc_type_t fmt, const char *p, struct expr *e1,
 	struct expr *e, *e3;
 	int nl = 1;
 
+	(void) tl;
+
 	AN(e1);
 	e = vcc_new_expr(fmt);
 	while (*p != '\0') {
@@ -184,16 +186,10 @@ vcc_expr_edit(struct vcc *tl, vcc_type_t fmt, const char *p, struct expr *e1,
 		case 't':
 			e3 = (*p == 'T' ? e1 : e2);
 			AN(e3);
-			VSB_printf(tl->curproc->prologue,
-			    "  struct strands strs_%u_a;\n"
-			    "  const char * strs_%u_s[%d];\n",
-			    tl->unique, tl->unique, e3->nstr);
 			VSB_printf(e->vsb,
-			    "VPI_BundleStrands(%d, &strs_%u_a, strs_%u_s,"
-			    "\v+\n%s,\nvrt_magic_string_end\v-\n)",
-			    e3->nstr, tl->unique, tl->unique,
-			VSB_data(e3->vsb));
-			tl->unique++;
+			    "&(struct strands){.n = %d, .p = "
+			    "(const char *[%d]){\n%s\n}}",
+			    e3->nstr, e3->nstr, VSB_data(e3->vsb));
 			break;
 		case '1':
 			VSB_cat(e->vsb, VSB_data(e1->vsb));

From nils.goroll at uplex.de  Fri Nov 20 15:45:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 20 Nov 2020 15:45:08 +0000 (UTC)
Subject: [master] 4d858a111 Documentation note on header names
Message-ID: <20201120154508.5BF379BE02@lists.varnish-cache.org>


commit 4d858a111c105d9fb6be9a2b8fa18a69d04ac7de
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 20 16:43:44 2020 +0100

    Documentation note on header names
    
    closes #3379

diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst
index 27707dd7f..828c1e60e 100644
--- a/doc/sphinx/reference/vcl_var.rst
+++ b/doc/sphinx/reference/vcl_var.rst
@@ -254,6 +254,10 @@ req.http.*
 	The RFCs allow multiple headers with the same name, and both
 	``set`` and ``unset`` will remove *all* headers with the name given.
 
+	The header name ``*`` is a VCL symbol and as such cannot, for
+	example, start with a numeral. Custom VMODs exist for handling
+	of such header names.
+
 
 req.restarts
 
@@ -438,6 +442,8 @@ req_top.http.*
 	HTTP headers of the top-level request in a tree of ESI requests.
 	Identical to req.http. in non-ESI requests.
 
+	See ``req.http.*`` for general notes.
+
 
 req_top.proto
 
@@ -580,6 +586,7 @@ bereq.http.*
 
 	The headers to be sent to the backend.
 
+	See ``req.http.*`` for general notes.
 
 bereq.uncacheable
 
@@ -742,6 +749,8 @@ beresp.http.*
 
 	The HTTP headers returned from the server.
 
+	See ``req.http.*`` for general notes.
+
 beresp.do_esi
 
 	Type: BOOL
@@ -1071,6 +1080,7 @@ obj.http.*
 
 	The HTTP headers stored in the object.
 
+	See ``req.http.*`` for general notes.
 
 obj.ttl
 
@@ -1234,10 +1244,9 @@ resp.http.*
 
 	Unsetable from: vcl_deliver, vcl_synth
 
-
 	The HTTP headers that will be returned.
 
-.. XXX does vcl_synth make any sense?
+	See ``req.http.*`` for general notes.
 
 resp.do_esi	``VCL >= 4.1``
 

From dridi.boukelmoune at gmail.com  Fri Nov 20 16:08:07 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 20 Nov 2020 16:08:07 +0000 (UTC)
Subject: [master] e9c2a488d Add MAIN.esi_req counter for ESI subrequests
Message-ID: <20201120160807.8AC899CADC@lists.varnish-cache.org>


commit e9c2a488d291b3129838c411cac7312f8801ff1e
Author: Reza Naghibi <reza at naghibi.com>
Date:   Tue Nov 10 16:05:55 2020 -0500

    Add MAIN.esi_req counter for ESI subrequests

diff --git a/bin/varnishd/VSC_main.vsc b/bin/varnishd/VSC_main.vsc
index 41c2b1228..4bede0a9a 100644
--- a/bin/varnishd/VSC_main.vsc
+++ b/bin/varnishd/VSC_main.vsc
@@ -97,6 +97,12 @@
 
 	The count of parseable client requests seen.
 
+.. varnish_vsc:: esi_req
+	:group: wrk
+	:oneliner:	ESI subrequests
+
+	Number of ESI subrequests made.
+
 .. varnish_vsc:: cache_hit
 	:group: wrk
 	:oneliner:	Cache hits
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index caed6f30c..a1a5afe04 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -139,6 +139,7 @@ ved_include(struct req *preq, const char *src, const char *host,
 
 	VSLb_ts_req(req, "Start", W_TIM_real(wrk));
 
+	wrk->stats->esi_req++;
 	req->esi_level = preq->esi_level + 1;
 
 	memset(req->top, 0, sizeof *req->top);
diff --git a/bin/varnishtest/tests/e00003.vtc b/bin/varnishtest/tests/e00003.vtc
index 23041400c..c93c6aec1 100644
--- a/bin/varnishtest/tests/e00003.vtc
+++ b/bin/varnishtest/tests/e00003.vtc
@@ -83,6 +83,7 @@ client c1 {
 }
 
 client c1 -run
+varnish v1 -expect esi_req == 2
 varnish v1 -expect esi_errors == 0
 varnish v1 -expect MAIN.s_resp_bodybytes == 150
 

From dridi.boukelmoune at gmail.com  Mon Nov 23 05:15:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 23 Nov 2020 05:15:15 +0000 (UTC)
Subject: [master] 1c7294c92 cli: Split vcl.discard in two
Message-ID: <20201123051515.583A6B15D9@lists.varnish-cache.org>


commit 1c7294c921fe3549692f745eebe7fc655e53b1c2
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Oct 12 11:49:32 2020 +0200

    cli: Split vcl.discard in two
    
    The first operation consists in checking the ability to discard the
    requested VCL, the second part effectively performs the VCL deletion.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 359877aa4..4b5f907d0 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -653,43 +653,16 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 	free(p);
 }
 
-static void v_matchproto_(cli_func_t)
-mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
+static void
+mgt_vcl_discard(struct cli *cli, struct vclprog *vp)
 {
-	unsigned status;
 	char *p = NULL;
-	struct vclprog *vp;
-	struct vcldep *vd;
-	int n;
+	unsigned status;
+
+	AN(vp);
+	assert(vp != active_vcl);
+	assert(VTAILQ_EMPTY(&vp->dto));
 
-	(void)priv;
-	vp = mcf_find_vcl(cli, av[2]);
-	if (vp == NULL)
-		return;
-	if (vp == active_vcl) {
-		VCLI_SetResult(cli, CLIS_CANT);
-		VCLI_Out(cli, "Cannot discard active VCL program\n");
-		return;
-	}
-	if (!VTAILQ_EMPTY(&vp->dto)) {
-		VCLI_SetResult(cli, CLIS_CANT);
-		AN(vp->warm);
-		if (!mcf_is_label(vp))
-			VCLI_Out(cli, "Cannot discard labeled VCL program.\n");
-		else
-			VCLI_Out(cli,
-			    "Cannot discard this VCL label, "
-			    "other VCLs depend on it.\n");
-		n = 0;
-		VTAILQ_FOREACH(vd, &vp->dto, lto) {
-			if (n++ == 5) {
-				VCLI_Out(cli, "\t[...]");
-				break;
-			}
-			VCLI_Out(cli, "\t%s\n", vd->from->name);
-		}
-		return;
-	}
 	if (mcf_is_label(vp)) {
 		AN(vp->warm);
 		vp->warm = 0;
@@ -698,13 +671,66 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 	}
 	if (MCH_Running()) {
 		AZ(vp->warm);
-		if (mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]))
+		if (mgt_cli_askchild(&status, &p, "vcl.discard %s\n", vp->name))
 			assert(status == CLIS_OK || status == CLIS_COMMS);
 		free(p);
 	}
 	mgt_vcl_del(vp);
 }
 
+static struct vclprog *
+mgt_vcl_can_discard(struct cli *cli, const char * const *av)
+{
+	struct vclprog *vp;
+	struct vcldep *vd;
+	int n;
+
+	AN(cli);
+	AN(av);
+	AN(*av);
+
+	vp = mcf_find_vcl(cli, *av);
+	if (vp == NULL)
+		return (NULL);
+	if (vp == active_vcl) {
+		VCLI_SetResult(cli, CLIS_CANT);
+		VCLI_Out(cli, "Cannot discard active VCL program\n");
+		return (NULL);
+	}
+	if (VTAILQ_EMPTY(&vp->dto))
+		return (vp);
+
+	VCLI_SetResult(cli, CLIS_CANT);
+	AN(vp->warm);
+	if (!mcf_is_label(vp))
+		VCLI_Out(cli, "Cannot discard labeled VCL program.\n");
+	else
+		VCLI_Out(cli,
+		    "Cannot discard this VCL label, "
+		    "other VCLs depend on it.\n");
+	n = 0;
+	VTAILQ_FOREACH(vd, &vp->dto, lto) {
+		if (n++ == 5) {
+			VCLI_Out(cli, "\t[...]");
+			break;
+		}
+		VCLI_Out(cli, "\t%s\n", vd->from->name);
+	}
+	return (NULL);
+}
+
+static void v_matchproto_(cli_func_t)
+mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
+{
+	struct vclprog *vp;
+
+	(void)priv;
+	vp = mgt_vcl_can_discard(cli, av + 2);
+	if (vp == NULL)
+		return;
+	mgt_vcl_discard(cli, vp);
+}
+
 static void v_matchproto_(cli_func_t)
 mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 {

From dridi.boukelmoune at gmail.com  Mon Nov 23 05:15:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 23 Nov 2020 05:15:15 +0000 (UTC)
Subject: [master] 66db3c5e2 cli: Always mention VCL names in vcl.discard errors
Message-ID: <20201123051515.64C20B15DB@lists.varnish-cache.org>


commit 66db3c5e29341a3ba03bb30a96469bce4f701226
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Oct 12 13:26:05 2020 +0200

    cli: Always mention VCL names in vcl.discard errors
    
    This is going to be useful once vcl.discard can take multiple VCL names
    at once.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 4b5f907d0..37ca013e9 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -128,7 +128,7 @@ mcf_find_vcl(struct cli *cli, const char *name)
 	vp = mcf_vcl_byname(name);
 	if (vp == NULL) {
 		VCLI_SetResult(cli, CLIS_PARAM);
-		VCLI_Out(cli, "No VCL named %s known.", name);
+		VCLI_Out(cli, "No VCL named %s known\n", name);
 	}
 	return (vp);
 }
@@ -694,7 +694,8 @@ mgt_vcl_can_discard(struct cli *cli, const char * const *av)
 		return (NULL);
 	if (vp == active_vcl) {
 		VCLI_SetResult(cli, CLIS_CANT);
-		VCLI_Out(cli, "Cannot discard active VCL program\n");
+		VCLI_Out(cli, "Cannot discard active VCL program %s\n",
+		    vp->name);
 		return (NULL);
 	}
 	if (VTAILQ_EMPTY(&vp->dto))
@@ -703,11 +704,12 @@ mgt_vcl_can_discard(struct cli *cli, const char * const *av)
 	VCLI_SetResult(cli, CLIS_CANT);
 	AN(vp->warm);
 	if (!mcf_is_label(vp))
-		VCLI_Out(cli, "Cannot discard labeled VCL program.\n");
+		VCLI_Out(cli, "Cannot discard labeled VCL program %s:\n",
+		    vp->name);
 	else
 		VCLI_Out(cli,
-		    "Cannot discard this VCL label, "
-		    "other VCLs depend on it.\n");
+		    "Cannot discard VCL label %s, other VCLs depend on it:\n",
+		    vp->name);
 	n = 0;
 	VTAILQ_FOREACH(vd, &vp->dto, lto) {
 		if (n++ == 5) {

From dridi.boukelmoune at gmail.com  Mon Nov 23 05:15:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 23 Nov 2020 05:15:15 +0000 (UTC)
Subject: [master] 9947307d3 cli: Teach vcl.discard to operate on multiple VCLs
Message-ID: <20201123051515.9C26FB15DE@lists.varnish-cache.org>


commit 9947307d357247823bd8b48f034f5b3d253e6664
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Oct 13 10:45:23 2020 +0200

    cli: Teach vcl.discard to operate on multiple VCLs
    
    To put it simply, let's take a simple CLI script:
    
        vcl.discard vcl1
        vcl.discard vcl2
        [...]
        vcl.discard vclX
    
    We can now achieve the same with a single command:
    
        vcl.discard vcl1 vcl2 ... vclX
    
    But there is slighty more to it, because vcl.discard operates on both
    VCLs and VCL labels, and dependencies can exist between both. So in
    addition to operate on multiple VCLs it also does so in the correct
    order.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 37ca013e9..af4cffc11 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -678,28 +678,15 @@ mgt_vcl_discard(struct cli *cli, struct vclprog *vp)
 	mgt_vcl_del(vp);
 }
 
-static struct vclprog *
-mgt_vcl_can_discard(struct cli *cli, const char * const *av)
+static void
+mgt_vcl_discard_depfail(struct cli *cli, struct vclprog *vp)
 {
-	struct vclprog *vp;
 	struct vcldep *vd;
 	int n;
 
 	AN(cli);
-	AN(av);
-	AN(*av);
-
-	vp = mcf_find_vcl(cli, *av);
-	if (vp == NULL)
-		return (NULL);
-	if (vp == active_vcl) {
-		VCLI_SetResult(cli, CLIS_CANT);
-		VCLI_Out(cli, "Cannot discard active VCL program %s\n",
-		    vp->name);
-		return (NULL);
-	}
-	if (VTAILQ_EMPTY(&vp->dto))
-		return (vp);
+	AN(vp);
+	assert(!VTAILQ_EMPTY(&vp->dto));
 
 	VCLI_SetResult(cli, CLIS_CANT);
 	AN(vp->warm);
@@ -718,19 +705,114 @@ mgt_vcl_can_discard(struct cli *cli, const char * const *av)
 		}
 		VCLI_Out(cli, "\t%s\n", vd->from->name);
 	}
-	return (NULL);
+}
+
+static struct vclprog **
+mgt_vcl_discard_depcheck(struct cli *cli, const char * const *names,
+    unsigned *lp)
+{
+	struct vclprog **res, *vp;
+	struct vcldep *vd;
+	const char * const *s;
+	unsigned i, j, l;
+
+	l = 0;
+	s = names;
+	while (*s != NULL) {
+		l++;
+		s++;
+	}
+	AN(l);
+
+	res = calloc(l, sizeof *res);
+	AN(res);
+
+	/* NB: Build a list of VCL programs and labels to discard. A null
+	 * pointer in the array is a VCL program that no longer needs to be
+	 * discarded.
+	 */
+	for (i = 0; i < l; i++) {
+		vp = mcf_find_vcl(cli, names[i]);
+		if (vp == NULL)
+			break;
+		if (vp == active_vcl) {
+			VCLI_SetResult(cli, CLIS_CANT);
+			VCLI_Out(cli, "Cannot discard active VCL program %s\n",
+			    vp->name);
+			break;
+		}
+		for (j = 0; j < i; j++)
+			if (vp == res[j])
+				break;
+		if (j < i)
+			continue; /* skip duplicates */
+		res[i] = vp;
+	}
+
+	if (i < l) {
+		free(res);
+		return (NULL);
+	}
+
+	/* NB: Check that all direct dependent VCL programs and labels belong
+	 * to the list of VCLs to be discarded. This mechanically ensures that
+	 * indirect dependent VCLs also belong.
+	 */
+	for (i = 0; i < l; i++) {
+		if (res[i] == NULL || VTAILQ_EMPTY(&res[i]->dto))
+			continue;
+		VTAILQ_FOREACH(vd, &res[i]->dto, lto) {
+			for (j = 0; j < l; j++)
+				if (res[j] == vd->from)
+					break;
+			if (j == l)
+				break;
+		}
+		if (vd != NULL)
+			break;
+	}
+
+	if (i < l) {
+		mgt_vcl_discard_depfail(cli, res[i]);
+		free(res);
+		return (NULL);
+	}
+
+	*lp = l;
+	return (res);
 }
 
 static void v_matchproto_(cli_func_t)
 mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 {
-	struct vclprog *vp;
+	struct vclprog **vp;
+	unsigned i, l, done;
 
 	(void)priv;
-	vp = mgt_vcl_can_discard(cli, av + 2);
+	vp = mgt_vcl_discard_depcheck(cli, av + 2, &l);
 	if (vp == NULL)
 		return;
-	mgt_vcl_discard(cli, vp);
+
+	/* NB: discard VCLs in topological order. At this point it
+	 * can only succeed, but the loop ensures that it eventually
+	 * completes even if the dependency check is ever broken.
+	 */
+	AN(l);
+	do {
+		done = 0;
+		for (i = 0; i < l; i++) {
+			if (vp[i] == NULL || vp[i]->nto > 0)
+				continue;
+			mgt_vcl_discard(cli, vp[i]);
+			vp[i] = NULL;
+			done++;
+		}
+	} while (done > 0);
+
+	for (i = 0; i < l; i++)
+		if (vp[i] != NULL)
+			WRONG(vp[i]->name);
+	free(vp);
 }
 
 static void v_matchproto_(cli_func_t)
diff --git a/bin/varnishtest/tests/c00077.vtc b/bin/varnishtest/tests/c00077.vtc
index c25d9e4c5..f176bc948 100644
--- a/bin/varnishtest/tests/c00077.vtc
+++ b/bin/varnishtest/tests/c00077.vtc
@@ -64,15 +64,5 @@ varnish v1 -clierr 106 "vcl.label vclA vcl3"
 
 varnish v1 -cliok "vcl.symtab"
 
-varnish v1 -cliok "vcl.discard vcl3"
-varnish v1 -cliok "vcl.discard vcl4"
-varnish v1 -cliok "vcl.discard vcl5"
-varnish v1 -cliok "vcl.discard vcl6"
-varnish v1 -cliok "vcl.discard vcl7"
-
-varnish v1 -cliok "vcl.discard vclB"
-varnish v1 -cliok "vcl.discard vcl2"
-varnish v1 -cliok "vcl.discard vclA"
-varnish v1 -cliok "vcl.discard vcl1"
-
-
+varnish v1 -clierr 300 "vcl.discard vcl1 vcl2 vcl3 vcl4 vcl5 vcl6 vcl7"
+varnish v1 -cliok "vcl.discard vcl1 vcl2 vcl3 vcl4 vcl5 vcl6 vcl7 vclA vclB"
diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
index 3e737c641..6fef4be1f 100644
--- a/include/tbl/cli_cmds.h
+++ b/include/tbl/cli_cmds.h
@@ -99,10 +99,12 @@ CLI_CMD(VCL_STATE,
 
 CLI_CMD(VCL_DISCARD,
 	"vcl.discard",
-	"vcl.discard <configname|label>",
-	"Unload the named configuration (when possible).",
-	"",
-	1, 1
+	"vcl.discard <configname|label>...", /* XXX: allow globs */
+	"Unload the named configurations (if possible).",
+	" If more than one named configuration is specified the command"
+	" is equivalent to individual commands in the right order with"
+	" respect to configurations dependencies.",
+	1, -1
 )
 
 CLI_CMD(VCL_LIST,

From dridi.boukelmoune at gmail.com  Mon Nov 23 05:15:15 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 23 Nov 2020 05:15:15 +0000 (UTC)
Subject: [master] f7aeba94f cli: New vcl.deps [-j] command
Message-ID: <20201123051515.C3117B15E3@lists.varnish-cache.org>


commit f7aeba94f089d22256d56ff6935cbc2b9e64e06e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Aug 5 09:34:45 2020 +0200

    cli: New vcl.deps [-j] command
    
    The plain text output is a suitable input for tsort(1), initially
    meant to find the correct discard order, I realized that it would
    be more reliable to let vcl.discard figure that one out.
    
    It wouldn't really work in my initial scenario:
    
        varnishadm vcl.deps |
        <filter some VCLs out> |
        tsort |
        xargs -n 1 varnishadm vcl.discard
    
    The filtering part only worked for direct dependencies but we can
    have two levels with return(vcl).
    
    I'm keeping this command only for setup analysis, the output can
    easily be turned into a graphviz dot file for example.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index af4cffc11..4eb713320 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -1006,6 +1006,66 @@ mcf_vcl_label(struct cli *cli, const char * const *av, void *priv)
 	free(p);
 }
 
+static void v_matchproto_(cli_func_t)
+mcf_vcl_deps(struct cli *cli, const char * const *av, void *priv)
+{
+	struct vclprog *vp;
+	struct vcldep *vd;
+	struct vsb *vsb;
+
+	(void)av;
+	(void)priv;
+
+	vsb = VSB_new_auto();
+	AN(vsb);
+
+	VTAILQ_FOREACH(vp, &vclhead, list) {
+		if (VTAILQ_EMPTY(&vp->dfrom)) {
+			VSB_printf(vsb, "%s\n", vp->name);
+			continue;
+		}
+		VTAILQ_FOREACH(vd, &vp->dfrom, lfrom)
+			VSB_printf(vsb, "%s\t%s\n", vp->name, vd->to->name);
+	}
+	VCLI_VTE(cli, &vsb, 80);
+}
+
+static void v_matchproto_(cli_func_t)
+mcf_vcl_deps_json(struct cli *cli, const char * const *av, void *priv)
+{
+	struct vclprog *vp;
+	struct vcldep *vd;
+	const char *sepd, *sepa;
+
+	(void)priv;
+
+	VCLI_JSON_begin(cli, 1, av);
+
+	VTAILQ_FOREACH(vp, &vclhead, list) {
+		VCLI_Out(cli, ",\n");
+		VCLI_Out(cli, "{\n");
+		VSB_indent(cli->sb, 2);
+		VCLI_Out(cli, "\"name\": \"%s\",\n", vp->name);
+		VCLI_Out(cli, "\"deps\": [");
+		VSB_indent(cli->sb, 2);
+		sepd = "";
+		sepa = "";
+		VTAILQ_FOREACH(vd, &vp->dfrom, lfrom) {
+			VCLI_Out(cli, "%s\n", sepd);
+			VCLI_Out(cli, "\"%s\"", vd->to->name);
+			sepd = ",";
+			sepa = "\n";
+		}
+		VSB_indent(cli->sb, -2);
+		VCLI_Out(cli, "%s", sepa);
+		VCLI_Out(cli, "]\n");
+		VSB_indent(cli->sb, -2);
+		VCLI_Out(cli, "}");
+	}
+
+	VCLI_JSON_end(cli);
+}
+
 /*--------------------------------------------------------------------*/
 
 static int v_matchproto_(vev_cb_f)
@@ -1037,6 +1097,7 @@ static struct cli_proto cli_vcl[] = {
 	{ CLICMD_VCL_STATE,		"", mcf_vcl_state },
 	{ CLICMD_VCL_DISCARD,		"", mcf_vcl_discard },
 	{ CLICMD_VCL_LIST,		"", mcf_vcl_list, mcf_vcl_list_json },
+	{ CLICMD_VCL_DEPS,		"", mcf_vcl_deps, mcf_vcl_deps_json },
 	{ CLICMD_VCL_LABEL,		"", mcf_vcl_label },
 	{ CLICMD_DEBUG_VCL_SYMTAB,	"d", mcf_vcl_symtab },
 	{ NULL }
diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
index 6fef4be1f..02d853b68 100644
--- a/include/tbl/cli_cmds.h
+++ b/include/tbl/cli_cmds.h
@@ -125,6 +125,20 @@ CLI_CMD(VCL_LIST,
 	0, 0
 )
 
+CLI_CMD(VCL_DEPS,
+	"vcl.deps",
+	"vcl.deps [-j]",
+	"List all loaded configuration and their dependencies.",
+	"  Unless ``-j`` is specified for JSON output, the"
+	" output format is up to two columns of dynamic width"
+	" separated by white space with the fields:\n\n"
+	"  * VCL: a VCL program\n\n"
+	"  * Dependency: another VCL program it depends on\n\n"
+	"Only direct dependencies are listed, and VCLs with"
+	" multiple dependencies are listed multiple times.",
+	0, 0
+)
+
 CLI_CMD(VCL_SHOW,
 	"vcl.show",
 	"vcl.show [-v] <configname>",

From nils.goroll at uplex.de  Mon Nov 23 11:14:15 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 23 Nov 2020 11:14:15 +0000 (UTC)
Subject: [master] b6d062455 silence flexelint
Message-ID: <20201123111415.DCB2B624FB@lists.varnish-cache.org>


commit b6d062455bcb4b4c8bf972b90d9a252e6dcf23fb
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 23 11:57:17 2020 +0100

    silence flexelint
    
    Dridi, phk and myself all stared at this piece of code and see no issue:
    res has been calloc'ed l elements. For any res[i] and res[j] access, i
    and j are less than l, respectively. l is not increased after the
    initial count of elements in s.
    
    Ref 9947307d357247823bd8b48f034f5b3d253e6664
    
    Flexelint error:
    
                               _
                    res[i] = vp;
    mgt/mgt_vcl.c  749  Warning 661: Possible access of out-of-bounds
        pointer (1 beyond end of data) by operator '[' [Reference:
        file mgt/mgt_vcl.c: lines 719, 722, 727, 744, 747, 749]
    mgt/mgt_vcl.c  719  Info 831: Reference cited in prior message
    mgt/mgt_vcl.c  722  Info 831: Reference cited in prior message
    mgt/mgt_vcl.c  727  Info 831: Reference cited in prior message
    mgt/mgt_vcl.c  744  Info 831: Reference cited in prior message
    mgt/mgt_vcl.c  747  Info 831: Reference cited in prior message
    mgt/mgt_vcl.c  749  Info 831: Reference cited in prior message

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 4eb713320..ab635d7fc 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -746,6 +746,7 @@ mgt_vcl_discard_depcheck(struct cli *cli, const char * const *names,
 				break;
 		if (j < i)
 			continue; /* skip duplicates */
+		//lint -e{661}
 		res[i] = vp;
 	}
 

From nils.goroll at uplex.de  Mon Nov 23 14:10:04 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 23 Nov 2020 14:10:04 +0000 (UTC)
Subject: [master] 7a12b1c3f clarify first byte timeout FetchError message
Message-ID: <20201123141004.A4D63935A1@lists.varnish-cache.org>


commit 7a12b1c3ff670d83970b9299882b4808fe2ab5df
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 20 09:42:44 2020 +0100

    clarify first byte timeout FetchError message
    
    For a first byte timeout on a new connection, we emit
    
            FetchError: first byte timeout
    
    For a first byte timeout being reached on a recycled connection, we
    should keep the error message similar.
    
    Ref eecd409d13f145765de3aeee4984179e0ae008b5

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index f4a9aadc4..826ca94df 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -301,7 +301,7 @@ vbe_dir_gethdrs(VRT_CTX, VCL_BACKEND d)
 			    bo->htc->first_byte_timeout) != 0) {
 				bo->htc->doclose = SC_RX_TIMEOUT;
 				VSLb(bo->vsl, SLT_FetchError,
-				     "Timed out reusing backend connection");
+				     "first byte timeout (reused connection)");
 				extrachance = 0;
 			}
 		}

From dridi.boukelmoune at gmail.com  Tue Nov 24 14:26:10 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 24 Nov 2020 14:26:10 +0000 (UTC)
Subject: [master] 6f587a30b Add """ ... """ as a long string format in VCL
Message-ID: <20201124142610.B966D93839@lists.varnish-cache.org>


commit 6f587a30b5098417a54497bf42ede03314287e64
Author: Andrew Wiik <andrew at varnish-software.com>
Date:   Mon Oct 26 16:06:56 2020 -0400

    Add """ ... """ as a long string format in VCL

diff --git a/bin/varnishtest/tests/v00019.vtc b/bin/varnishtest/tests/v00019.vtc
index fc98616a6..d53a5aeb2 100644
--- a/bin/varnishtest/tests/v00019.vtc
+++ b/bin/varnishtest/tests/v00019.vtc
@@ -18,6 +18,11 @@ varnish v1 -errvcl {Unterminated long-string, starting at} {
 	{" }
 }
 
+varnish v1 -errvcl {Unterminated long-string, starting at} {
+	backend b { .host = "127.0.0.1"; }
+	""" ""
+}
+
 varnish v1 -errvcl {Unterminated string at} {
 	backend b { .host = "127.0.0.1"; }
 	"
diff --git a/bin/varnishtest/tests/v00066.vtc b/bin/varnishtest/tests/v00066.vtc
new file mode 100644
index 000000000..3fd9d5deb
--- /dev/null
+++ b/bin/varnishtest/tests/v00066.vtc
@@ -0,0 +1,22 @@
+varnishtest "long string coverage"
+
+varnish v1 -vcl {
+	backend default none;
+
+	sub vcl_recv {
+		return (synth(200));
+	}
+
+	sub vcl_synth {
+                set resp.body = """{"key":"value"}""";
+                return (deliver);
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+        expect resp.bodylen == 15
+	expect resp.body == {{"key":"value"}}
+} -run
diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index 00e21b414..d8d80491d 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -462,7 +462,7 @@ vcc_Lexer(struct vcc *tl, const struct source *sp, int eoi)
 			return;
 		}
 
-		/* Recognize long-strings */
+		/* Recognize long-strings {" "} */
 		if (*p == '{' && p[1] == '"') {
 			for (q = p + 2; q < sp->e; q++) {
 				if (*q == '"' && q[1] == '}') {
@@ -487,6 +487,31 @@ vcc_Lexer(struct vcc *tl, const struct source *sp, int eoi)
 			return;
 		}
 
+		/* Recognize long-strings """ """ */
+		if (*p == '"' && p[1] == '"' && p[2] == '"') {
+			for (q = p + 3; q < sp->e; q++) {
+				if (*q == '"' && q[1] == '"' && q[2] == '"') {
+					vcc_addtoken(tl, CSTR, sp, p, q + 3);
+					break;
+				}
+			}
+			if (q < sp->e) {
+				p = q + 3;
+				u = tl->t->e - tl->t->b;
+				u -= 6;		/* """ ... """ */
+				tl->t->dec = TlAlloc(tl, u + 1 );
+				AN(tl->t->dec);
+				memcpy(tl->t->dec, tl->t->b + 3, u);
+				tl->t->dec[u] = '\0';
+				continue;
+			}
+			vcc_addtoken(tl, EOI, sp, p, p + 3);
+			VSB_cat(tl->sb,
+			    "Unterminated long-string, starting at\n");
+			vcc_ErrWhere(tl, tl->t);
+			return;
+		}
+
 		/* Recognize BLOB (= SF-binary) */
 		if (*p == ':') {
 			vsb = VSB_new_auto();

From dridi.boukelmoune at gmail.com  Tue Nov 24 14:26:10 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 24 Nov 2020 14:26:10 +0000 (UTC)
Subject: [master] 2211bcc73 Add documentation for new long string format
Message-ID: <20201124142610.D666B9383C@lists.varnish-cache.org>


commit 2211bcc7355ee4b965cebcfc62575a1a6938da52
Author: Andrew Wiik <andrew at varnish-software.com>
Date:   Fri Nov 20 11:39:05 2020 -0500

    Add documentation for new long string format

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index a532c96fd..2e7767c17 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -84,7 +84,7 @@ Strings
 
 Basic strings are enclosed in double quotes ``"``\ *...*\ ``"``, and
 may not contain newlines. Long strings are enclosed in
-``{"``\ *...*\ ``"}``. They may contain any character including single
+``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""``. They may contain any character including single
 double quotes ``"``, newline and other control characters except for the
 *NUL* (0x00) character.
 
@@ -511,7 +511,7 @@ ban(STRING)
 
       Either a literal string or a regular expression. Note that
       *<arg>* does not use any of the string delimiters like ``"`` or
-      ``{"``\ *...*\ ``"}`` used elsewhere in varnish. To match
+      ``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""`` used elsewhere in varnish. To match
       against strings containing whitespace, regular expressions
       containing ``\s`` can be used.
 
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index ea97402cc..22b71ac57 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -23,7 +23,7 @@ to do the "count-the-backslashes" polka::
 
   regsub("barf", "(b)(a)(r)(f)", "\4\3\2p") -> "frap"
 
-Long strings are enclosed in {" ... "}. They may contain any character
+Long strings are enclosed in {" ... "} or """ ... """. They may contain any character
 including ", newline and other control characters except for the NUL
 (0x00) character. If you really want NUL characters in a string there
 is a VMOD that makes it possible to create such strings.

From dridi.boukelmoune at gmail.com  Tue Nov 24 14:26:10 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 24 Nov 2020 14:26:10 +0000 (UTC)
Subject: [master] cf80fe68a vcc: Kill unused parameter
Message-ID: <20201124142611.16BBD93841@lists.varnish-cache.org>


commit cf80fe68a546df1f639134a7306a095a0023379f
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 24 10:11:03 2020 +0100

    vcc: Kill unused parameter
    
    And with that we know for sure that we always have null-terminated
    VCL sources.

diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index f3e5b3b0b..6deaff9e4 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -506,18 +506,17 @@ EmitStruct(const struct vcc *tl)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_new_source(const char *b, const char *e, const char *name)
+vcc_new_source(const char *src, const char *name)
 {
 	struct source *sp;
 
-	if (e == NULL)
-		e = strchr(b, '\0');
+	AN(src);
+	AN(name);
 	sp = calloc(1, sizeof *sp);
-	assert(sp != NULL);
-	sp->name = strdup(name);
-	AN(sp->name);
-	sp->b = b;
-	sp->e = e;
+	AN(sp);
+	REPLACE(sp->name, name);
+	sp->b = src;
+	sp->e = strchr(src, '\0');
 	return (sp);
 }
 
@@ -557,7 +556,7 @@ vcc_file_source(const struct vcc *tl, const char *fn)
 		free(fnp);
 		return (NULL);
 	}
-	sp = vcc_new_source(f, NULL, fnp);
+	sp = vcc_new_source(f, fnp);
 	free(fnp);
 	sp->freeit = f;
 	return (sp);
@@ -694,7 +693,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp, const char *jfile)
 		return (NULL);
 
 	/* Register and lex the builtin VCL */
-	sp = vcc_new_source(tl->builtin_vcl, NULL, "Builtin");
+	sp = vcc_new_source(tl->builtin_vcl, "Builtin");
 	assert(sp != NULL);
 	VTAILQ_INSERT_TAIL(&tl->sources, sp, list);
 	sp->idx = tl->nsources++;
@@ -818,7 +817,7 @@ VCC_Compile(struct vcc *tl, struct vsb **sb,
 	AN(ofile);
 	AN(jfile);
 	if (vclsrc != NULL)
-		sp = vcc_new_source(vclsrc, NULL, vclsrcfile);
+		sp = vcc_new_source(vclsrc, vclsrcfile);
 	else
 		sp = vcc_file_source(tl, vclsrcfile);
 

From dridi.boukelmoune at gmail.com  Tue Nov 24 14:26:11 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 24 Nov 2020 14:26:11 +0000 (UTC)
Subject: [master] 5a99d056f vcc: Fold string parsing into a single function
Message-ID: <20201124142611.4AC6F93847@lists.varnish-cache.org>


commit 5a99d056f4bd02896972d90a5d491094bebbdfa7
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 24 10:32:32 2020 +0100

    vcc: Fold string parsing into a single function
    
    Having now the guarantee that VCL sources are always null-terminated we
    can use standard string.h functions to detect long strings. We can also
    generalize vcc_decstr() to work with all kinds of strings.
    
    Instead of duplicating the code in 3 locations with slight variations,
    encode the specificities of each type of string into a struct and have
    one location to parse them.
    
    Bonus deduplication, inline C parsing is identical.
    
    This is loosely inspired by vcl_fixed_token() and we could further
    reduce the size of the loop in vcc_Lexer() by extracting more logic.

diff --git a/bin/varnishtest/tests/v00019.vtc b/bin/varnishtest/tests/v00019.vtc
index d53a5aeb2..904586790 100644
--- a/bin/varnishtest/tests/v00019.vtc
+++ b/bin/varnishtest/tests/v00019.vtc
@@ -23,7 +23,7 @@ varnish v1 -errvcl {Unterminated long-string, starting at} {
 	""" ""
 }
 
-varnish v1 -errvcl {Unterminated string at} {
+varnish v1 -errvcl {Unterminated string, starting at} {
 	backend b { .host = "127.0.0.1"; }
 	"
 }
diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index d8d80491d..34ee49ad4 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -344,20 +344,19 @@ vcc_ExpectVid(struct vcc *tl, const char *what)
  * Decode a string
  */
 
-static int
-vcc_decstr(struct vcc *tl)
+static void
+vcc_decstr(struct vcc *tl, unsigned sep)
 {
 	char *q;
 	unsigned int l;
 
 	assert(tl->t->tok == CSTR);
-	l = (tl->t->e - tl->t->b) - 2;
+	l = (tl->t->e - tl->t->b) - (sep * 2);
 	tl->t->dec = TlAlloc(tl, l + 1);
-	assert(tl->t->dec != NULL);
+	AN(tl->t->dec);
 	q = tl->t->dec;
-	memcpy(q, tl->t->b + 1, l);
+	memcpy(q, tl->t->b + sep, l);
 	q[l] = '\0';
-	return (0);
 }
 
 /*--------------------------------------------------------------------
@@ -383,6 +382,64 @@ vcc_addtoken(struct vcc *tl, unsigned tok,
 	tl->t = t;
 }
 
+/*--------------------------------------------------------------------
+ * Find a delimited token
+ */
+
+static const struct delim_def {
+	const char	*name;
+	const char	*b;
+	const char	*e;
+	unsigned	len;	/* NB: must be the same for both delimiters */
+	unsigned	crlf;
+	unsigned	tok;
+} delim_defs[] = {
+#define DELIM_DEF(nm, l, r, c, t)		\
+	{ nm, l, r, sizeof (l) - 1, c, t }
+	DELIM_DEF("long-string", "\"\"\"", "\"\"\"", 1, CSTR),	/* """...""" */
+	DELIM_DEF("long-string", "{\"", "\"}", 1, CSTR),	/*  {"..."}  */
+	DELIM_DEF("string", "\"", "\"", 0, CSTR),		/*   "..."   */
+	DELIM_DEF("inline C source", "C{", "}C", 1, CSRC),	/*  C{...}C  */
+#undef DELIM_DEF
+	{ NULL }
+};
+
+static unsigned
+vcc_delim_token(struct vcc *tl, const struct source *sp, const char *p,
+    const char **qp)
+{
+	const struct delim_def *dd;
+	const char *q, *r;
+
+	for (dd = delim_defs; dd->name != NULL; dd++)
+		if (!strncmp(p, dd->b, dd->len))
+			break;
+
+	if (dd->name == NULL)
+		return (0);
+
+	q = strstr(p + dd->len, dd->e);
+	if (q != NULL && !dd->crlf) {
+		r = strpbrk(p + dd->len, "\r\n");
+		if (r != NULL && r < q)
+			q = NULL;
+	}
+
+	if (q == NULL) {
+		vcc_addtoken(tl, EOI, sp, p, p + dd->len);
+		VSB_printf(tl->sb, "Unterminated %s, starting at\n", dd->name);
+		vcc_ErrWhere(tl, tl->t);
+		return (0);
+	}
+
+	assert(q < sp->e);
+	vcc_addtoken(tl, dd->tok, sp, p, q + dd->len);
+	if (dd->tok == CSTR)
+		vcc_decstr(tl, dd->len);
+	*qp = q + dd->len;
+	return (1);
+}
+
 /*--------------------------------------------------------------------
  * Lexical analysis and token generation
  */
@@ -443,75 +500,6 @@ vcc_Lexer(struct vcc *tl, const struct source *sp, int eoi)
 			continue;
 		}
 
-		/* Recognize inline C-code */
-		if (*p == 'C' && p[1] == '{') {
-			for (q = p + 2; q < sp->e; q++) {
-				if (*q == '}' && q[1] == 'C') {
-					vcc_addtoken(tl, CSRC, sp, p, q + 2);
-					break;
-				}
-			}
-			if (q < sp->e) {
-				p = q + 2;
-				continue;
-			}
-			vcc_addtoken(tl, EOI, sp, p, p + 2);
-			VSB_cat(tl->sb,
-			    "Unterminated inline C source, starting at\n");
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
-
-		/* Recognize long-strings {" "} */
-		if (*p == '{' && p[1] == '"') {
-			for (q = p + 2; q < sp->e; q++) {
-				if (*q == '"' && q[1] == '}') {
-					vcc_addtoken(tl, CSTR, sp, p, q + 2);
-					break;
-				}
-			}
-			if (q < sp->e) {
-				p = q + 2;
-				u = tl->t->e - tl->t->b;
-				u -= 4;		/* {" ... "} */
-				tl->t->dec = TlAlloc(tl, u + 1 );
-				AN(tl->t->dec);
-				memcpy(tl->t->dec, tl->t->b + 2, u);
-				tl->t->dec[u] = '\0';
-				continue;
-			}
-			vcc_addtoken(tl, EOI, sp, p, p + 2);
-			VSB_cat(tl->sb,
-			    "Unterminated long-string, starting at\n");
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
-
-		/* Recognize long-strings """ """ */
-		if (*p == '"' && p[1] == '"' && p[2] == '"') {
-			for (q = p + 3; q < sp->e; q++) {
-				if (*q == '"' && q[1] == '"' && q[2] == '"') {
-					vcc_addtoken(tl, CSTR, sp, p, q + 3);
-					break;
-				}
-			}
-			if (q < sp->e) {
-				p = q + 3;
-				u = tl->t->e - tl->t->b;
-				u -= 6;		/* """ ... """ */
-				tl->t->dec = TlAlloc(tl, u + 1 );
-				AN(tl->t->dec);
-				memcpy(tl->t->dec, tl->t->b + 3, u);
-				tl->t->dec[u] = '\0';
-				continue;
-			}
-			vcc_addtoken(tl, EOI, sp, p, p + 3);
-			VSB_cat(tl->sb,
-			    "Unterminated long-string, starting at\n");
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
-
 		/* Recognize BLOB (= SF-binary) */
 		if (*p == ':') {
 			vsb = VSB_new_auto();
@@ -574,32 +562,17 @@ vcc_Lexer(struct vcc *tl, const struct source *sp, int eoi)
 			continue;
 		}
 
-		/* Match for the fixed tokens (see generate.py) */
-		u = vcl_fixed_token(p, &q);
-		if (u != 0) {
-			vcc_addtoken(tl, u, sp, p, q);
+		/* Match delimited tokens */
+		if (vcc_delim_token(tl, sp, p, &q) != 0) {
 			p = q;
 			continue;
 		}
+		ERRCHK(tl);
 
-		/* Match strings */
-		if (*p == '"') {
-			for (q = p + 1; q < sp->e; q++) {
-				if (*q == '"') {
-					q++;
-					break;
-				}
-				if (*q == '\r' || *q == '\n') {
-					vcc_addtoken(tl, EOI, sp, p, q);
-					VSB_cat(tl->sb,
-					    "Unterminated string at\n");
-					vcc_ErrWhere(tl, tl->t);
-					return;
-				}
-			}
-			vcc_addtoken(tl, CSTR, sp, p, q);
-			if (vcc_decstr(tl))
-				return;
+		/* Match for the fixed tokens (see generate.py) */
+		u = vcl_fixed_token(p, &q);
+		if (u != 0) {
+			vcc_addtoken(tl, u, sp, p, q);
 			p = q;
 			continue;
 		}

From dridi.boukelmoune at gmail.com  Tue Nov 24 14:26:11 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 24 Nov 2020 14:26:11 +0000 (UTC)
Subject: [master] 3ec74a3aa doc: Format OCD
Message-ID: <20201124142611.793119384F@lists.varnish-cache.org>


commit 3ec74a3aa2fd1880d8c998de813eb5d3c61c32ea
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 24 15:23:16 2020 +0100

    doc: Format OCD
    
    Better diff with the --word-diff --ignore-all-space options.

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 2e7767c17..b442b88d3 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -84,9 +84,9 @@ Strings
 
 Basic strings are enclosed in double quotes ``"``\ *...*\ ``"``, and
 may not contain newlines. Long strings are enclosed in
-``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""``. They may contain any character including single
-double quotes ``"``, newline and other control characters except for the
-*NUL* (0x00) character.
+``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""``. They may contain any
+character including single double quotes ``"``, newline and other control
+characters except for the *NUL* (0x00) character.
 
 Booleans
 ~~~~~~~~
@@ -511,9 +511,9 @@ ban(STRING)
 
       Either a literal string or a regular expression. Note that
       *<arg>* does not use any of the string delimiters like ``"`` or
-      ``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""`` used elsewhere in varnish. To match
-      against strings containing whitespace, regular expressions
-      containing ``\s`` can be used.
+      ``{"``\ *...*\ ``"}`` or ``"""``\ *...*\ ``"""`` used elsewhere
+      in varnish. To match against strings containing whitespace,
+      regular expressions containing ``\s`` can be used.
 
     * for duration fields:
 
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index 22b71ac57..73acf995f 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -23,10 +23,10 @@ to do the "count-the-backslashes" polka::
 
   regsub("barf", "(b)(a)(r)(f)", "\4\3\2p") -> "frap"
 
-Long strings are enclosed in {" ... "} or """ ... """. They may contain any character
-including ", newline and other control characters except for the NUL
-(0x00) character. If you really want NUL characters in a string there
-is a VMOD that makes it possible to create such strings.
+Long strings are enclosed in {" ... "} or """ ... """. They may contain
+any character including ", newline and other control characters except
+for the NUL (0x00) character. If you really want NUL characters in a
+string there is a VMOD that makes it possible to create such strings.
 
 .. _vcl_syntax_acl:
 

From nils.goroll at uplex.de  Wed Nov 25 11:24:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 25 Nov 2020 11:24:07 +0000 (UTC)
Subject: [master] 8b6649355 Added missing timestamp
Message-ID: <20201125112407.3AD6B61139@lists.varnish-cache.org>


commit 8b66493557b79c8ac8d84516240cedb61bbf9e01
Author: ThijsFeryn <thijs at feryn.eu>
Date:   Wed Nov 25 12:14:02 2020 +0100

    Added missing timestamp

diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 016cc5784..c98646d26 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -80,14 +80,15 @@ SLTM(SessOpen, 0, "Client connection opened",
 	"The first record for a client connection, with the socket-endpoints"
 	" of the connection.\n\n"
 	"The format is::\n\n"
-	"\t%s %d %s %s %s %d\n"
-	"\t|  |  |  |  |  |\n"
-	"\t|  |  |  |  |  +- File descriptor number\n"
-	"\t|  |  |  |  +---- Local TCP port / 0 for UDS\n"
-	"\t|  |  |  +------- Local IPv4/6 address / 0.0.0.0 for UDS\n"
-	"\t|  |  +---------- Socket name (from -a argument)\n"
-	"\t|  +------------- Remote TCP port / 0 for UDS\n"
-	"\t+---------------- Remote IPv4/6 address / 0.0.0.0 for UDS\n"
+	"\t%s %d %s %s %s %f %d\n"
+	"\t|  |  |  |  |  |  |\n"
+	"\t|  |  |  |  |  |  +- File descriptor number\n"
+	"\t|  |  |  |  |  +---- Session start time (unix epoch)\n"
+	"\t|  |  |  |  +------- Local TCP port / 0 for UDS\n"
+	"\t|  |  |  +---------- Local IPv4/6 address / 0.0.0.0 for UDS\n"
+	"\t|  |  +------------- Socket name (from -a argument)\n"
+	"\t|  +---------------- Remote TCP port / 0 for UDS\n"
+	"\t+------------------- Remote IPv4/6 address / 0.0.0.0 for UDS\n"
 	"\n"
 )
 

From fgsch at lodoss.net  Wed Nov 25 13:11:07 2020
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 25 Nov 2020 13:11:07 +0000 (UTC)
Subject: [master] 184736fad Workaround build issue in macos
Message-ID: <20201125131107.592C5642D8@lists.varnish-cache.org>


commit 184736fad473070da79b77a16eab38faf393516e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 25 11:06:51 2020 +0000

    Workaround build issue in macos
    
    stdbool.h gets pulled in indirectly, which defines bool to _Bool.
    Since we are not using the bool type here, just undefine it.

diff --git a/bin/varnishd/common/common_param.h b/bin/varnishd/common/common_param.h
index e11967a2a..b7f07ace3 100644
--- a/bin/varnishd/common/common_param.h
+++ b/bin/varnishd/common/common_param.h
@@ -40,6 +40,10 @@
 
 #define VSM_CLASS_PARAM		"Params"
 
+#ifdef __MACH__
+#undef bool
+#endif
+
 enum debug_bits {
 #define DEBUG_BIT(U, l, d) DBG_##U,
 #include "tbl/debug_bits.h"

From dridi.boukelmoune at gmail.com  Wed Nov 25 16:37:06 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 25 Nov 2020 16:37:06 +0000 (UTC)
Subject: [master] 001485e67 cache: Move the pdiff() function to vas.h
Message-ID: <20201125163706.F16DC96025@lists.varnish-cache.org>


commit 001485e6705976bd1029279cb639a394c4dee59c
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 25 17:22:42 2020 +0100

    cache: Move the pdiff() function to vas.h
    
    This way it can be used almost anywhere in the code base and out of tree,
    not just in the cache process. The function was slightly polished after
    prior discussions with phk and slink.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index e4c77ee07..10bd3079b 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -824,20 +824,6 @@ int RFC2616_Do_Cond(const struct req *sp);
 void RFC2616_Weaken_Etag(struct http *hp);
 void RFC2616_Vary_AE(struct http *hp);
 
-/*
- * A normal pointer difference is signed, but we never want a negative value
- * so this little tool will make sure we don't get that.
- */
-
-static inline unsigned
-pdiff(const void *b, const void *e)
-{
-
-	assert(b <= e);
-	return
-	    ((unsigned)((const unsigned char *)e - (const unsigned char *)b));
-}
-
 #define Tcheck(t) do {						\
 		AN((t).b);					\
 		AN((t).e);					\
diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index 338eac7f8..36edfd68e 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -45,7 +45,7 @@ WS_Assert(const struct ws *ws)
 {
 
 	CHECK_OBJ_NOTNULL(ws, WS_MAGIC);
-	DSL(DBG_WORKSPACE, 0, "WS(%p) = (%s, %p %u %u %u)",
+	DSL(DBG_WORKSPACE, 0, "WS(%p) = (%s, %p %zu %zu %zu)",
 	    ws, ws->id, ws->s, pdiff(ws->s, ws->f),
 	    ws->r == NULL ? 0 : pdiff(ws->f, ws->r),
 	    pdiff(ws->s, ws->e));
@@ -312,7 +312,7 @@ WS_ReserveSize(struct ws *ws, unsigned bytes)
 		return (0);
 	}
 	ws->r = ws->f + b2;
-	DSL(DBG_WORKSPACE, 0, "WS_ReserveSize(%p, %u/%u) = %u",
+	DSL(DBG_WORKSPACE, 0, "WS_ReserveSize(%p, %u/%u) = %zu",
 	    ws, b2, bytes, pdiff(ws->f, ws->r));
 	WS_Assert(ws);
 	return (pdiff(ws->f, ws->r));
diff --git a/include/vas.h b/include/vas.h
index 90fac3482..32a092241 100644
--- a/include/vas.h
+++ b/include/vas.h
@@ -117,4 +117,17 @@ do {									\
 		typedef char __vassert_## y[(x) ? 1 : -1] v_unused_
 #endif
 
+/*
+ * A normal pointer difference is signed, but when we don't want a negative
+ * value this little tool will make sure we don't get that.
+ */
+
+static inline size_t
+pdiff(const void *b, const void *e)
+{
+
+	assert(b <= e);
+	return ((size_t)((const char *)e - (const char *)b));
+}
+
 #endif

From dridi.boukelmoune at gmail.com  Wed Nov 25 16:37:07 2020
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 25 Nov 2020 16:37:07 +0000 (UTC)
Subject: [master] 3b12edaf9 vcc: Avoid suspicious truncation with pdiff()
Message-ID: <20201125163707.133FC96028@lists.varnish-cache.org>


commit 3b12edaf95c1de1477c2cee329301b590d247af0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 25 17:25:21 2020 +0100

    vcc: Avoid suspicious truncation with pdiff()
    
    Flexelint complains that this truncation is suspicious, which is fair.
    The pdiff() function allows for a more expressive, and hopefully less
    suspicious alternative with a bit of hardening in the process.

diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index 34ee49ad4..167b6ed28 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -351,7 +351,7 @@ vcc_decstr(struct vcc *tl, unsigned sep)
 	unsigned int l;
 
 	assert(tl->t->tok == CSTR);
-	l = (tl->t->e - tl->t->b) - (sep * 2);
+	l = pdiff(tl->t->b + sep, tl->t->e - sep);
 	tl->t->dec = TlAlloc(tl, l + 1);
 	AN(tl->t->dec);
 	q = tl->t->dec;

From nils.goroll at uplex.de  Fri Nov 27 14:30:09 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 27 Nov 2020 14:30:09 +0000 (UTC)
Subject: [master] 5cbb35866 assert magic of vrt_privs when finalizing
Message-ID: <20201127143009.3B78863B7D@lists.varnish-cache.org>


commit 5cbb3586649839b83438bd99f557cf30c9c78a66
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 27 15:27:31 2020 +0100

    assert magic of vrt_privs when finalizing

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index 44692af56..3d2ec57ee 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -225,7 +225,9 @@ VCL_TaskLeave(struct vrt_privs *privs)
 	 * a costly operation. Instead we safely walk the whole tree and clear
 	 * the head at the very end.
 	 */
-	VRBT_FOREACH_SAFE(vp, vrt_privs, privs, vp1)
+	VRBT_FOREACH_SAFE(vp, vrt_privs, privs, vp1) {
+		CHECK_OBJ(vp, VRT_PRIV_MAGIC);
 		VRT_priv_fini(vp->priv);
+	}
 	ZERO_OBJ(privs, sizeof *privs);
 }

From nils.goroll at uplex.de  Fri Nov 27 18:23:08 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 27 Nov 2020 18:23:08 +0000 (UTC)
Subject: [master] 683c7773f Revert "assert magic of vrt_privs when finalizing"
Message-ID: <20201127182308.0F5A796340@lists.varnish-cache.org>


commit 683c7773fec527a1544ded6113f308c3462b4fcf
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 27 19:21:25 2020 +0100

    Revert "assert magic of vrt_privs when finalizing"
    
    This change also exposed #3385
    
    See #3470 for a suggestion how to get out of this
    
    This reverts commit 5cbb3586649839b83438bd99f557cf30c9c78a66.

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index 3d2ec57ee..44692af56 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -225,9 +225,7 @@ VCL_TaskLeave(struct vrt_privs *privs)
 	 * a costly operation. Instead we safely walk the whole tree and clear
 	 * the head at the very end.
 	 */
-	VRBT_FOREACH_SAFE(vp, vrt_privs, privs, vp1) {
-		CHECK_OBJ(vp, VRT_PRIV_MAGIC);
+	VRBT_FOREACH_SAFE(vp, vrt_privs, privs, vp1)
 		VRT_priv_fini(vp->priv);
-	}
 	ZERO_OBJ(privs, sizeof *privs);
 }

From nils.goroll at uplex.de  Mon Nov 30 10:34:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 30 Nov 2020 10:34:07 +0000 (UTC)
Subject: [master] 681c11998 Methods for vmod_priv
Message-ID: <20201130103407.3BF91B2580@lists.varnish-cache.org>


commit 681c11998ec56dc4460a30585210f95a7850ea3b
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 18 18:55:03 2020 +0100

    Methods for vmod_priv
    
    This is the refactoring we agreed on to enable an alternative
    implementation of #3454. This PR does not yet introduce the copy
    callback needed to add the functionality suggested in #3454.
    
    We replace the .free pointer of struct vmod_priv with a pointer to a
    methods struct with callbacks. As of this commit, it only contains
    the .free callback renamed to .fini. The purpose of the refactoring is
    to allow addition of more callbacks later.
    
    The new struct vmod_priv_methods also contains a .type member pointing
    to a string to contain an arbitrary description of the type of data any
    priv holds which uses these methods.
    
    Implementation:
    
    relevant changes are in cache_vrt_priv.c and vrt.h, other changes are to
    the documentation and bundled vmods.
    
    The implementation is a simple refactoring for indirection of the call
    to the .fini callback via the methods structure.

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index 44692af56..bb4d41ebd 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -61,6 +61,8 @@ void
 pan_privs(struct vsb *vsb, const struct vrt_privs *privs)
 {
 	struct vrt_priv *vp;
+	const struct vmod_priv *p;
+	const struct vmod_priv_methods *m;
 
 	VSB_printf(vsb, "privs = %p {\n", privs);
 	if (PAN_already(vsb, privs))
@@ -69,11 +71,18 @@ pan_privs(struct vsb *vsb, const struct vrt_privs *privs)
 	if (privs != NULL) {
 		VRBT_FOREACH(vp, vrt_privs, privs) {
 			PAN_CheckMagic(vsb, vp, VRT_PRIV_MAGIC);
+			p = vp->priv;
+			if (p == NULL) {
+				// should never happen
+				VSB_printf(vsb, "priv NULL vmod %jx\n",
+				    (uintmax_t)vp->vmod_id);
+				continue;
+			}
+			m = p->methods;
 			VSB_printf(vsb,
-			    "priv {p %p l %ld f %p} vmod %jx\n",
-			    vp->priv->priv,
-			    vp->priv->len,
-			    vp->priv->free,
+			    "priv {p %p l %ld m %p t \"%s\"} vmod %jx\n",
+			    p->priv, p->len, m,
+			    m != NULL ? m->type : "",
 			    (uintmax_t)vp->vmod_id
 			);
 		}
@@ -201,9 +210,17 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 void
 VRT_priv_fini(const struct vmod_priv *p)
 {
+	const struct vmod_priv_methods *m;
 
-	if (p->priv != NULL && p->free != NULL)
-		p->free(p->priv);
+	m = p->methods;
+	if (m == NULL)
+		return;
+
+	CHECK_OBJ(m, VMOD_PRIV_METHODS_MAGIC);
+	if (p->priv == NULL || m->fini == NULL)
+		return;
+
+	m->fini(p->priv);
 }
 
 /*--------------------------------------------------------------------*/
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 4497fe095..afc97d820 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -502,28 +502,48 @@ specified.
 
 This structure contains three members::
 
-	typedef void vmod_priv_free_f(void *);
 	struct vmod_priv {
-		void                    *priv;
-		int			len;
-		vmod_priv_free_f        *free;
+		void				*priv;
+		long				len;
+		const struct vmod_priv_methods  *methods;
 	};
 
-The "priv" and "len" elements can be used for whatever the vmod
-code wants to use them for, and the "free" element provides a
-callback to clean them up.
+The ``.priv`` and ``.len`` elements can be used for whatever the vmod
+code wants to use them for.
 
-If both the "priv" and "free" pointers are non-NULL when the scope
-ends, the "free" function will be called with the "priv" pointer
-as its only argument.
+``.methods`` can be an optional pointer to a struct of callbacks::
+
+	typedef void vmod_priv_fini_f(void *);
+
+	struct vmod_priv_methods {
+		unsigned			magic;
+		const char			*type;
+		vmod_priv_fini_f		*fini;
+	};
+
+``.magic`` has to be initialized to
+``VMOD_PRIV_METHODS_MAGIC``. ``.type`` should be a descriptive name to
+help debugging.
+
+``.fini`` will be called for a non-NULL ``.priv`` of the ``struct
+vmod_priv`` when the scope ends with that ``.priv`` pointer as its
+only argument.
 
 In the common case where a private data structure is allocated with
 malloc(3) would look like this::
 
+	static const struct vmod_priv_methods mymethods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "mystate",
+		.fini = free	/* free(3) */
+	}};
+
+	// ....
+
 	if (priv->priv == NULL) {
 		priv->priv = calloc(1, sizeof(struct myfoo));
 		AN(priv->priv);
-		priv->free = free;	/* free(3) */
+		priv->methods = mymethods;
 		mystate = priv->priv;
 		mystate->foo = 21;
 		...
diff --git a/include/vrt.h b/include/vrt.h
index 4dea8c50a..7508a5b22 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -55,6 +55,8 @@
  * 13.0 (2021-03-15)
  *	Calling convention for VDP implementation changed
  *	Added VRT_ValidHdr()
+ *	struct vmod_priv_methods added
+ *	struct vmod_priv free member replaced with methods
  * 12.0 (2020-09-15)
  *	Added VRT_DirectorResolve()
  *	Added VCL_STRING VRT_BLOB_string(VRT_CTX, VCL_BLOB)
@@ -583,11 +585,20 @@ void VRT_Format_Proxy(struct vsb *, VCL_INT, VCL_IP, VCL_IP, VCL_STRING);
 
 typedef int vmod_event_f(VRT_CTX, struct vmod_priv *, enum vcl_event_e);
 
-typedef void vmod_priv_free_f(void *);
+/* vmod_priv related */
+typedef void vmod_priv_fini_f(void *);
+
+struct vmod_priv_methods {
+	unsigned			magic;
+#define VMOD_PRIV_METHODS_MAGIC	0xcea5ff99
+	const char			*type;
+	vmod_priv_fini_f		*fini;
+};
+
 struct vmod_priv {
-	void			*priv;
-	long			len;
-	vmod_priv_free_f	*free;
+	void				*priv;
+	long				len;
+	const struct vmod_priv_methods	*methods;
 };
 
 void VRT_priv_fini(const struct vmod_priv *p);
diff --git a/lib/libvmod_cookie/vmod_cookie.c b/lib/libvmod_cookie/vmod_cookie.c
index 6c8b6a3a4..fdaa2f594 100644
--- a/lib/libvmod_cookie/vmod_cookie.c
+++ b/lib/libvmod_cookie/vmod_cookie.c
@@ -84,6 +84,12 @@ cobj_free(void *p)
 	FREE_OBJ(vcp);
 }
 
+static const struct vmod_priv_methods cookie_cobj_priv_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "vmod_cookie_cobj",
+		.fini = cobj_free
+}};
+
 static struct vmod_cookie *
 cobj_get(struct vmod_priv *priv)
 {
@@ -94,7 +100,7 @@ cobj_get(struct vmod_priv *priv)
 		AN(vcp);
 		VTAILQ_INIT(&vcp->cookielist);
 		priv->priv = vcp;
-		priv->free = cobj_free;
+		priv->methods = cookie_cobj_priv_methods;
 	} else
 		CAST_OBJ_NOTNULL(vcp, priv->priv, VMOD_COOKIE_MAGIC);
 
@@ -258,6 +264,12 @@ free_re(void *priv)
 	AZ(vre);
 }
 
+static const struct vmod_priv_methods cookie_re_priv_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "vmod_cookie_re",
+		.fini = free_re
+}};
+
 VCL_STRING
 vmod_get_re(VRT_CTX, struct vmod_priv *priv, struct vmod_priv *priv_call,
     VCL_STRING expression)
@@ -281,7 +293,7 @@ vmod_get_re(VRT_CTX, struct vmod_priv *priv, struct vmod_priv *priv_call,
 		}
 
 		priv_call->priv = vre;
-		priv_call->free = free_re;
+		priv_call->methods = cookie_re_priv_methods;
 		AZ(pthread_mutex_unlock(&mtx));
 	}
 
@@ -431,7 +443,7 @@ re_filter(VRT_CTX, struct vmod_priv *priv, struct vmod_priv *priv_call,
 		}
 
 		priv_call->priv = vre;
-		priv_call->free = free_re;
+		priv_call->methods = cookie_re_priv_methods;
 		AZ(pthread_mutex_unlock(&mtx));
 	}
 
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index a3cfa0dae..7c50561f5 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -225,6 +225,12 @@ xyzzy_author(VRT_CTX, VCL_ENUM person, VCL_ENUM someone)
 	WRONG("Illegal VMOD enum");
 }
 
+static const struct vmod_priv_methods xyzzy_test_priv_call_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "debug_test_priv_call",
+		.fini = free
+}};
+
 VCL_VOID v_matchproto_(td_debug_test_priv_call)
 xyzzy_test_priv_call(VRT_CTX, struct vmod_priv *priv)
 {
@@ -232,7 +238,7 @@ xyzzy_test_priv_call(VRT_CTX, struct vmod_priv *priv)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (priv->priv == NULL) {
 		priv->priv = strdup("BAR");
-		priv->free = free;
+		priv->methods = xyzzy_test_priv_call_methods;
 	} else {
 		assert(!strcmp(priv->priv, "BAR"));
 	}
@@ -246,6 +252,12 @@ priv_task_free(void *ptr)
 	free(ptr);
 }
 
+static const struct vmod_priv_methods xyzzy_test_priv_task_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "debug_test_priv_task",
+		.fini = priv_task_free
+}};
+
 VCL_STRING v_matchproto_(td_debug_test_priv_task)
 xyzzy_test_priv_task(VRT_CTX, struct vmod_priv *priv, VCL_STRING s)
 {
@@ -256,7 +268,7 @@ xyzzy_test_priv_task(VRT_CTX, struct vmod_priv *priv, VCL_STRING s)
 		    priv, priv->priv);
 	} else if (priv->priv == NULL) {
 		priv->priv = strdup(s);
-		priv->free = priv_task_free;
+		priv->methods = xyzzy_test_priv_task_methods;
 		VSL(SLT_Debug, 0, "test_priv_task(%p) = %p (new)",
 		    priv, priv->priv);
 	} else {
@@ -271,10 +283,16 @@ xyzzy_test_priv_task(VRT_CTX, struct vmod_priv *priv, VCL_STRING s)
 		    priv, priv->priv);
 	}
 	if (priv->priv != NULL)
-		assert(priv->free == priv_task_free);
+		assert(priv->methods == xyzzy_test_priv_task_methods);
 	return (priv->priv);
 }
 
+static const struct vmod_priv_methods xyzzy_test_priv_top_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "debug_test_priv_top",
+		.fini = free
+}};
+
 VCL_STRING v_matchproto_(td_debug_test_priv_top)
 xyzzy_test_priv_top(VRT_CTX, struct vmod_priv *priv, VCL_STRING s)
 {
@@ -282,7 +300,7 @@ xyzzy_test_priv_top(VRT_CTX, struct vmod_priv *priv, VCL_STRING s)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (priv->priv == NULL) {
 		priv->priv = strdup(s);
-		priv->free = free;
+		priv->methods = xyzzy_test_priv_top_methods;
 	}
 	return (priv->priv);
 }
@@ -376,7 +394,7 @@ xyzzy_fail2(VRT_CTX)
 	return (1);
 }
 
-static void v_matchproto_(vmod_priv_free_f)
+static void v_matchproto_(vmod_priv_fini_f)
 priv_vcl_free(void *priv)
 {
 	struct priv_vcl *priv_vcl;
@@ -394,6 +412,12 @@ priv_vcl_free(void *priv)
 	AZ(priv_vcl);
 }
 
+static const struct vmod_priv_methods priv_vcl_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "debug_priv_vcl_free",
+		.fini = priv_vcl_free
+}};
+
 static int
 event_load(VRT_CTX, struct vmod_priv *priv)
 {
@@ -413,7 +437,7 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 	priv_vcl->foo = strdup("FOO");
 	AN(priv_vcl->foo);
 	priv->priv = priv_vcl;
-	priv->free = priv_vcl_free;
+	priv->methods = priv_vcl_methods;
 
 	VRT_AddVFP(ctx, &xyzzy_rot13);
 
@@ -977,7 +1001,7 @@ xyzzy_store_ip(VRT_CTX, VCL_IP ip)
 		return;
 	}
 
-	AZ(priv->free);
+	AZ(priv->methods);
 	assert(VSA_Sane(ip));
 	priv->priv = TRUST_ME(ip);
 }
@@ -992,7 +1016,7 @@ xyzzy_get_ip(VRT_CTX)
 
 	priv = VRT_priv_task(ctx, &store_ip_token);
 	AN(priv);
-	AZ(priv->free);
+	AZ(priv->methods);
 
 	ip = priv->priv;
 	assert(VSA_Sane(ip));
@@ -1146,6 +1170,12 @@ fail_f(void *priv)
 	VRT_fail(ctx, "thou shalt not rollet back");
 }
 
+static const struct vmod_priv_methods xyzzy_fail_rollback_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "debug_fail_rollback",
+		.fini = fail_f
+}};
+
 VCL_VOID v_matchproto_(td_xyzzy_debug_fail_rollback)
 xyzzy_fail_rollback(VRT_CTX)
 {
@@ -1161,12 +1191,12 @@ xyzzy_fail_rollback(VRT_CTX)
 
 	if (p->priv != NULL) {
 		assert(p->priv == ctx);
-		assert(p->free == fail_f);
+		assert(p->methods == xyzzy_fail_rollback_methods);
 		return;
 	}
 
 	p->priv = TRUST_ME(ctx);
-	p->free = fail_f;
+	p->methods = xyzzy_fail_rollback_methods;
 }
 
 VCL_VOID v_matchproto_(td_xyzzy_debug_ok_rollback)
@@ -1183,7 +1213,7 @@ xyzzy_ok_rollback(VRT_CTX)
 	}
 
 	p->priv = NULL;
-	p->free = NULL;
+	p->methods = NULL;
 }
 
 VCL_STRING v_matchproto_(td_xyzzy_debug_re_quote)
diff --git a/lib/libvmod_directors/shard_cfg.c b/lib/libvmod_directors/shard_cfg.c
index 312e9c722..1a0cb40e4 100644
--- a/lib/libvmod_directors/shard_cfg.c
+++ b/lib/libvmod_directors/shard_cfg.c
@@ -86,7 +86,7 @@ change_reconfigure(struct shard_change *change, VCL_INT replicas);
  * a PRIV_TASK state, which we work in reconfigure.
  */
 
-static void v_matchproto_(vmod_priv_free_f)
+static void v_matchproto_(vmod_priv_fini_f)
 shard_change_fini(void * priv)
 {
 	struct shard_change *change;
@@ -99,6 +99,12 @@ shard_change_fini(void * priv)
 	(void) change_reconfigure(change, 67);
 }
 
+static const struct vmod_priv_methods shard_change_priv_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "vmod_directors_shard_cfg",
+		.fini = shard_change_fini
+}};
+
 static struct shard_change *
 shard_change_get(VRT_CTX, struct sharddir * const shardd)
 {
@@ -132,7 +138,7 @@ shard_change_get(VRT_CTX, struct sharddir * const shardd)
 	change->shardd = shardd;
 	VSTAILQ_INIT(&change->tasks);
 	task->priv = change;
-	task->free = shard_change_fini;
+	task->methods = shard_change_priv_methods;
 
 	return (change);
 }
diff --git a/lib/libvmod_std/vmod_std_fileread.c b/lib/libvmod_std/vmod_std_fileread.c
index df7a160d1..1da8b670b 100644
--- a/lib/libvmod_std/vmod_std_fileread.c
+++ b/lib/libvmod_std/vmod_std_fileread.c
@@ -83,6 +83,12 @@ free_frfile(void *ptr)
 	}
 }
 
+static const struct vmod_priv_methods frfile_methods[1] = {{
+		.magic = VMOD_PRIV_METHODS_MAGIC,
+		.type = "vmod_std_fileread",
+		.fini = free_frfile
+}};
+
 static struct frfile *
 find_frfile(struct vmod_priv *priv, VCL_STRING file_name)
 {
@@ -112,7 +118,7 @@ find_frfile(struct vmod_priv *priv, VCL_STRING file_name)
 	}
 	AZ(pthread_mutex_unlock(&frmtx));
 	if (frf != NULL) {
-		priv->free = free_frfile;
+		priv->methods = frfile_methods;
 		priv->priv = frf;
 		return (frf);
 	}
@@ -127,7 +133,7 @@ find_frfile(struct vmod_priv *priv, VCL_STRING file_name)
 		frf->contents = s;
 		frf->blob->blob = s;
 		frf->blob->len = (size_t)sz;
-		priv->free = free_frfile;
+		priv->methods = frfile_methods;
 		priv->priv = frf;
 		AZ(pthread_mutex_lock(&frmtx));
 		VTAILQ_INSERT_HEAD(&frlist, frf, list);

From nils.goroll at uplex.de  Mon Nov 30 18:02:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 30 Nov 2020 18:02:07 +0000 (UTC)
Subject: [master] f825ef93a Utility function for more thorough VRT_CTX sanity
 checking
Message-ID: <20201130180207.0FE0064DF4@lists.varnish-cache.org>


commit f825ef93a9fd47701866c1b6e5a695828465ab01
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 30 18:50:29 2020 +0100

    Utility function for more thorough VRT_CTX sanity checking
    
    this needs to live within the exclusive circle of cache_vcl.h includers, so I
    went for cache_vrt_vcl.c just because of the name.

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index eda1fe3db..6c419136a 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -313,6 +313,30 @@ VCL_DefaultProbe(const struct vcl *vcl)
 
 /*--------------------------------------------------------------------*/
 
+void
+VRT_CTX_Assert(VRT_CTX)
+{
+	CHECK_OBJ_ORNULL(ctx, VRT_CTX_MAGIC);
+
+	if (ctx->msg != NULL)
+		CHECK_OBJ(ctx->msg, VSB_MAGIC);
+	else
+		AN(ctx->vsl);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	WS_Assert(ctx->ws);
+
+	CHECK_OBJ_ORNULL(ctx->sp, SESS_MAGIC);
+
+	CHECK_OBJ_ORNULL(ctx->req, REQ_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->http_req, HTTP_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->http_req_top, HTTP_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->http_resp, HTTP_MAGIC);
+
+	CHECK_OBJ_ORNULL(ctx->bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->http_bereq, HTTP_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->http_beresp, HTTP_MAGIC);
+}
+
 struct vclref *
 VRT_VCL_Prevent_Cold(VRT_CTX, const char *desc)
 {
diff --git a/include/vrt.h b/include/vrt.h
index 7508a5b22..6f8b8cb52 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -57,6 +57,7 @@
  *	Added VRT_ValidHdr()
  *	struct vmod_priv_methods added
  *	struct vmod_priv free member replaced with methods
+ *	VRT_CTX_Assert() added
  * 12.0 (2020-09-15)
  *	Added VRT_DirectorResolve()
  *	Added VCL_STRING VRT_BLOB_string(VRT_CTX, VCL_BLOB)
@@ -638,6 +639,8 @@ void VRT_VSC_Hide(const struct vsc_seg *);
 void VRT_VSC_Reveal(const struct vsc_seg *);
 size_t VRT_VSC_Overhead(size_t);
 
+void VRT_CTX_Assert(VRT_CTX);
+
 /*
  * API to restrict the VCL in various ways
  */

From nils.goroll at uplex.de  Mon Nov 30 18:28:07 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 30 Nov 2020 18:28:07 +0000 (UTC)
Subject: [master] 462c0f2e8 stupid glitch
Message-ID: <20201130182807.DA3B565A4A@lists.varnish-cache.org>


commit 462c0f2e8fdf63c5709a8c845595c555a8097fbf
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 30 19:27:20 2020 +0100

    stupid glitch

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 6c419136a..9b4d44c3c 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -316,7 +316,7 @@ VCL_DefaultProbe(const struct vcl *vcl)
 void
 VRT_CTX_Assert(VRT_CTX)
 {
-	CHECK_OBJ_ORNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 
 	if (ctx->msg != NULL)
 		CHECK_OBJ(ctx->msg, VSB_MAGIC);

From nils.goroll at uplex.de  Mon Nov 30 18:31:06 2020
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 30 Nov 2020 18:31:06 +0000 (UTC)
Subject: [master] e1b0a9303 silence flexelint on a condition which is known to
 be always false
Message-ID: <20201130183106.8466B65D17@lists.varnish-cache.org>


commit e1b0a93034ff0646c14adbf2ee159a0626570532
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 30 19:29:15 2020 +0100

    silence flexelint on a condition which is known to be always false
    
    ... unless we f*up, which is the sole purpose of its existence

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index bb4d41ebd..53183489f 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -72,6 +72,7 @@ pan_privs(struct vsb *vsb, const struct vrt_privs *privs)
 		VRBT_FOREACH(vp, vrt_privs, privs) {
 			PAN_CheckMagic(vsb, vp, VRT_PRIV_MAGIC);
 			p = vp->priv;
+			//lint -e{774}
 			if (p == NULL) {
 				// should never happen
 				VSB_printf(vsb, "priv NULL vmod %jx\n",