From phk at FreeBSD.org  Mon May  1 19:16:11 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  1 May 2023 19:16:11 +0000 (UTC)
Subject: [master] 8bcbc8655 Update vtest.sh to correctly report vmod tests
Message-ID: <20230501191611.3C94D11DE31@lists.varnish-cache.org>


commit 8bcbc8655801d0bfc8b49718a8d997a7bfd562f2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 1 19:15:12 2023 +0000

    Update vtest.sh to correctly report vmod tests

diff --git a/tools/vtest.sh b/tools/vtest.sh
index f695f1f56..00620502f 100755
--- a/tools/vtest.sh
+++ b/tools/vtest.sh
@@ -190,14 +190,15 @@ failedtests () (
 	test ! -d $LOGDIR &&
 	LOGDIR="bin/varnishtest/tests"
 
-	grep -l ':test-result: FAIL' "$LOGDIR"/*.trs |
+	find . -name '*.trs' -print | xargs grep -l ':test-result: FAIL' |
 	while read trs
 	do
 		name=`basename "${trs}" .trs`
-		vtc="${name}.vtc"
+		vtc=`echo $trs | sed -e 's/trs$/vtc/' -e 's/.*sub\///' `
+		logfile=`echo $trs | sed -e 's/trs$/log/'`
 		log="${name}.log"
-		rev=`git log -n 1 --pretty=format:%H "${VTCDIR}/${vtc}"`
-		cp "${LOGDIR}/${log}" "${REPORTDIR}/_${log}"
+		rev=`git log -n 1 --pretty=format:%H "${vtc}"`
+		cp "${logfile}" "${REPORTDIR}/_${log}"
 		echo "VTCGITREV ${name} ${rev}"
 		echo "MANIFEST _${log}"
 	done

From nils.goroll at uplex.de  Thu May  4 09:50:13 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  4 May 2023 09:50:13 +0000 (UTC)
Subject: [master] 836ecdbc1 Changelog TLC
Message-ID: <20230504095013.155F111DA4C@lists.varnish-cache.org>


commit 836ecdbc1296b1c3b88f4f63d397b10d84690f77
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu May 4 11:43:23 2023 +0200

    Changelog TLC
    
    not yet complete, I just had this change locally and want to avoid
    repeating work on it.

diff --git a/doc/changes.rst b/doc/changes.rst
index aef12cd4f..d280a9c7f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -35,6 +35,15 @@ release process.
 Varnish Cache NEXT (2023-09-15)
 ===============================
 
+.. PLEASE keep this roughly in commit order as shown by git-log / tig
+   (new to old)
+
+* Call sites of VMOD fuctions and methods can now be restricted to
+  built-in subroutines using the ``$Restrict`` stanza in the VCC file.
+
+* Varnish Delivery Processors (VDPs) are now also properly closed for
+  error conditions, avoiding potential minor memory leaks.
+
 .. _3908: https://github.com/varnishcache/varnish-cache/pull/3908
 
 * The newly introduced abstract socket support was incompatible with

From nils.goroll at uplex.de  Thu May  4 09:50:13 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  4 May 2023 09:50:13 +0000 (UTC)
Subject: [master] 71de7ae93 Silence flexelint complaints about libunwind.h
Message-ID: <20230504095013.3437F11DA4F@lists.varnish-cache.org>


commit 71de7ae93f6b0da4192cba666faeddb392bda8e3
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu May 4 11:49:31 2023 +0200

    Silence flexelint complaints about libunwind.h

diff --git a/bin/flint.lnt b/bin/flint.lnt
index e6cee798a..3bb395da5 100644
--- a/bin/flint.lnt
+++ b/bin/flint.lnt
@@ -39,6 +39,9 @@
 -elib(659)		// no tokens after struct def.
 -elib(123)		// macro def. with arg at, (just warn)
 
++libh(/usr/include/libunwind.h)
+-elib(849)
+
 -emacro(702, WEXITSTATUS)	// signed shift right
 
 -e825	// control flows into case/default without -fallthrough comment

From phk at FreeBSD.org  Thu May  4 22:49:17 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu,  4 May 2023 22:49:17 +0000 (UTC)
Subject: [master] 8d33dacfc Make directors_c00004.vtc more solid with a slight
 pause
Message-ID: <20230504224917.291AE110B92@lists.varnish-cache.org>


commit 8d33dacfc0cc80d19bbacd29df7512f9d9407bae
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 4 20:42:11 2023 +0000

    Make directors_c00004.vtc more solid with a slight pause

diff --git a/vmod/tests/directors_c00004.vtc b/vmod/tests/directors_c00004.vtc
index 668d6c5a8..72f00666c 100644
--- a/vmod/tests/directors_c00004.vtc
+++ b/vmod/tests/directors_c00004.vtc
@@ -96,7 +96,11 @@ client c1 {
 	rxresp
 	expect resp.body == "xiuFi3Pe"
 	expect resp.http.backend == "vd"
+} -run
+
+varnish v1 -vsl_catchup
 
+client c1 {
 	txreq -url /1 -hdr "pipe: true"
 	rxresp
 	expect resp.body == "ech3Ooj"

From nils.goroll at uplex.de  Mon May  8 14:50:08 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  8 May 2023 14:50:08 +0000 (UTC)
Subject: [master] f60995b8b Improve http1_iovs_flush documentation
Message-ID: <20230508145008.3449911DBF0@lists.varnish-cache.org>


commit f60995b8bedf9bdd0f99c2c3c3b7738cf4798fe2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon May 8 16:36:12 2023 +0200

    Improve http1_iovs_flush documentation

diff --git a/lib/libvsc/VSC_main.vsc b/lib/libvsc/VSC_main.vsc
index 2500a0260..46655ca72 100644
--- a/lib/libvsc/VSC_main.vsc
+++ b/lib/libvsc/VSC_main.vsc
@@ -946,4 +946,11 @@
 .. varnish_vsc:: http1_iovs_flush
 	:oneliner:	Premature iovec flushes
 
+	Number of additional writes performed on HTTP1 connections
+	because the number of IO vectors was too small to submit all
+	possible IO in one go. This number is configured through the
+	``http1_iovs`` parameter for client connections and implicitly
+	defined by the amount of free workspace for backend
+	connections.
+
 .. varnish_vsc_end::	main

From nils.goroll at uplex.de  Mon May  8 14:50:08 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  8 May 2023 14:50:08 +0000 (UTC)
Subject: [master] 10b1eff8c Changelog TLC
Message-ID: <20230508145008.5187211DBF3@lists.varnish-cache.org>


commit 10b1eff8c028a9f307249592a6062d2a6ab441ca
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon May 8 16:47:05 2023 +0200

    Changelog TLC

diff --git a/doc/changes.rst b/doc/changes.rst
index d280a9c7f..c9c8d1d00 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -38,17 +38,39 @@ Varnish Cache NEXT (2023-09-15)
 .. PLEASE keep this roughly in commit order as shown by git-log / tig
    (new to old)
 
-* Call sites of VMOD fuctions and methods can now be restricted to
-  built-in subroutines using the ``$Restrict`` stanza in the VCC file.
-
 * Varnish Delivery Processors (VDPs) are now also properly closed for
   error conditions, avoiding potential minor memory leaks.
 
-.. _3908: https://github.com/varnishcache/varnish-cache/pull/3908
+* A regression introduced with Varnish Cache 7.3.0 was fixed: On
+  HTTP/2 connections, URLs starting with ``//`` no longer trigger a
+  protocol error (3911_).
+
+* Call sites of VMOD functions and methods can now be restricted to
+  built-in subroutines using the ``$Restrict`` stanza in the VCC file.
+
+* The counter ``MAIN.http1_iovs_flush`` has been added to track the
+  number of premature ``writev()`` calls due to an insufficient number
+  of IO vectors. This number is configured through the ``http1_iovs``
+  parameter for client connections and implicitly defined by the
+  amount of free workspace for backend connections.
+
+* Object creation failures by the selected storage engine are now
+  logged under the ``Error`` tag as ``Failed to create object object
+  from %s %s``.
+
+* ``varnishtest`` gained the macro ``varnishd_args`` to globally
+  append additional arguments to the ``varnishd`` command line. Macros
+  in this macro's value will be expanded.
+
+* The limit on the size of ``varnishtest`` macros has been raised to
+  2KB.
 
 * The newly introduced abstract socket support was incompatible with
   other implementations, this has been fixed (3908_).
 
+.. _3908: https://github.com/varnishcache/varnish-cache/pull/3908
+.. _3911: https://github.com/varnishcache/varnish-cache/issues/3911
+
 ================================
 Varnish Cache 7.3.0 (2023-03-15)
 ================================

From nils.goroll at uplex.de  Mon May  8 14:53:03 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  8 May 2023 14:53:03 +0000 (UTC)
Subject: [master] f6ef5927b Add (be)req.trace and feature +trace to control
 generation of VCL_trace logs
Message-ID: <20230508145303.49BDC11E041@lists.varnish-cache.org>


commit f6ef5927bca7a5bb05cd67674c0e26f233852648
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 8 16:15:54 2022 +0100

    Add (be)req.trace and feature +trace to control generation of VCL_trace logs
    
    To potentially emit VCL_trace VSL records, we called VPI_count() from
    VGC for, in the extreme, every line of VCL.
    
    For a normal setup, VPI_count() would call into VSL only to find out
    that VCL_trace is masked and not have any effect.
    
    Issuing two additional function calls for each line of VCL obviously
    is something we should avoid unless there is a real benefit.
    
    To avoid the overhead, we move the check for the tracing flag to VGC
    and decorate it with UNLIKELY(), which has no effect for now but is
    expected to become a branch prediction hint soon.
    
    This should bring the cost down close to zero.
    
    Being at it, we also add VCL control over tracing through req.trace
    and bereq.trace. req.trace gets initialized from feature +trace, and
    bereq.trace inherits the value from req.trace. For now, tracing of the
    housekeeping subs vcl_init and vcl_fini can only be controlled through
    feature +trace.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 5313268b3..fabddba5b 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -891,6 +891,7 @@ cnt_recv_prep(struct req *req, const char *ci)
 		req->hash_ignore_vary = 0;
 		req->client_identity = NULL;
 		req->storage = NULL;
+		req->trace = FEATURE(FEATURE_TRACE);
 	}
 
 	req->is_hit = 0;
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 0f3245c8c..ad811c4eb 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -93,6 +93,7 @@ VCL_Bo2Ctx(struct vrt_ctx *ctx, struct busyobj *bo)
 	ctx->ws = bo->ws;
 	ctx->vpi = bo->wrk->vpi;
 	ctx->vpi->handling = 0;
+	ctx->vpi->trace = bo->trace;
 }
 
 void
@@ -116,6 +117,7 @@ VCL_Req2Ctx(struct vrt_ctx *ctx, struct req *req)
 	ctx->ws = req->ws;
 	ctx->vpi = req->wrk->vpi;
 	ctx->vpi->handling = 0;
+	ctx->vpi->trace = req->trace;
 }
 
 /*--------------------------------------------------------------------*/
@@ -128,6 +130,7 @@ VCL_Get_CliCtx(int msg)
 	INIT_OBJ(&ctx_cli, VRT_CTX_MAGIC);
 	INIT_OBJ(&wrk_vpi_cli, WRK_VPI_MAGIC);
 	ctx_cli.vpi = &wrk_vpi_cli;
+	wrk_vpi_cli.trace = FEATURE(FEATURE_TRACE);
 	ctx_cli.now = VTIM_real();
 	if (msg) {
 		ctx_cli.msg = VSB_new_auto();
diff --git a/bin/varnishd/cache/cache_vpi.c b/bin/varnishd/cache/cache_vpi.c
index 0d8e792bf..9b3b3db1c 100644
--- a/bin/varnishd/cache/cache_vpi.c
+++ b/bin/varnishd/cache/cache_vpi.c
@@ -62,14 +62,13 @@ VPI_wrk_init(struct worker *wrk, void *p, size_t spc)
 }
 
 void
-VPI_count(VRT_CTX, unsigned u)
+VPI_trace(VRT_CTX, unsigned u)
 {
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC);
 	assert(u < ctx->vcl->conf->nref);
-	ctx->vpi->ref = u;
 	if (ctx->vsl != NULL)
 		VSLb(ctx->vsl, SLT_VCL_trace, "%s %u %u.%u.%u",
 		    ctx->vcl->loaded_name, u, ctx->vcl->conf->ref[u].source,
diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index cc4f1bffb..d8630d347 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -658,6 +658,15 @@ VRT_handled(VRT_CTX)
 	return (ctx->vpi->handling);
 }
 
+/* the trace value is cached in the VPI for efficiency */
+VCL_VOID
+VRT_trace(VRT_CTX, VCL_BOOL a)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->vpi);
+	ctx->vpi->trace = a;
+}
+
 /*--------------------------------------------------------------------*/
 
 VCL_VOID
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index fe4f8a81e..2bd56b71a 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -301,6 +301,23 @@ VRT_r_beresp_uncacheable(VRT_CTX)
 	return (ctx->bo->uncacheable);
 }
 
+VCL_VOID
+VRT_l_req_trace(VRT_CTX, VCL_BOOL a)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
+	ctx->req->trace = a;
+	VRT_trace(ctx, a);
+}
+VCL_VOID
+VRT_l_bereq_trace(VRT_CTX, VCL_BOOL a)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
+	ctx->bo->trace = a;
+	VRT_trace(ctx, a);
+}
+
 /*--------------------------------------------------------------------*/
 
 VCL_BYTES
diff --git a/bin/varnishtest/tests/c00005.vtc b/bin/varnishtest/tests/c00005.vtc
index 255f42232..4520088c7 100644
--- a/bin/varnishtest/tests/c00005.vtc
+++ b/bin/varnishtest/tests/c00005.vtc
@@ -9,7 +9,7 @@ server s1 {
 	txresp -body "2222\n"
 } -start
 
-varnish v1 -arg "-p vsl_mask=+VCL_trace"
+varnish v1 -arg "-p feature=+trace"
 
 varnish v1 -errvcl {Name acl1 must have type 'acl'.} {
 	sub vcl_recv {
diff --git a/bin/varnishtest/tests/c00054.vtc b/bin/varnishtest/tests/c00054.vtc
index 266cc13a8..420386b89 100644
--- a/bin/varnishtest/tests/c00054.vtc
+++ b/bin/varnishtest/tests/c00054.vtc
@@ -9,7 +9,7 @@ server s1 {
 varnish v1 -vcl+backend {} -start
 
 varnish v1 -cliok "param.show vsl_mask"
-varnish v1 -cliok "param.set vsl_mask +VCL_trace"
+varnish v1 -cliok "param.set vsl_mask -VCL_trace"
 varnish v1 -cliok "param.show vsl_mask"
 varnish v1 -cliok "param.set vsl_mask -WorkThread,-TTL"
 varnish v1 -cliok "param.show vsl_mask"
diff --git a/bin/varnishtest/tests/r02413.vtc b/bin/varnishtest/tests/r02413.vtc
index 15ed32fe5..ba85102b1 100644
--- a/bin/varnishtest/tests/r02413.vtc
+++ b/bin/varnishtest/tests/r02413.vtc
@@ -1,11 +1,11 @@
-varnishtest "Test vcl_trace"
+varnishtest "Test feature trace"
 
 server s1 {
 	rxreq
 	txresp
 } -start
 
-varnish v1 -arg "-p vsl_mask=+VCL_trace" -vcl+backend {
+varnish v1 -arg "-p feature=+trace" -vcl+backend {
 	sub vcl_deliver {
 		set resp.http.vcl = "vclA";
 	}
@@ -18,6 +18,7 @@ varnish v1 -vcl+backend {
 		if (req.http.vcl == "vcl1") {
 			return (vcl(vclA));
 		}
+		set req.trace = false;
 	}
 	sub vcl_deliver {
 		set resp.http.vcl = "vcl2";
@@ -28,16 +29,24 @@ varnish v1 -cliok "vcl.label vclB vcl2"
 varnish v1 -cliok "vcl.list"
 
 logexpect l1 -v v1 -g raw {
+	expect * 1002	VCL_call	"BACKEND_FETCH"
+	expect 0 1002	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}
 	expect * 1001	VCL_call	"RECV"
 	expect 0 1001	VCL_trace	{^vcl2 \d+ \d+\.\d+\.\d+$}
 	expect * 1001	VCL_call	"RECV"
 	expect 0 1001	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}
+	expect * 1003	VCL_call	"DELIVER"
+	expect 0 1003	RespHeader	{^vcl: vcl2}
 } -start
 
 client c1 {
 	txreq -hdr "vcl: vcl1"
 	rxresp
 	expect resp.http.vcl == vclA
+
+	txreq
+	rxresp
+	expect resp.http.vcl == vcl2
 } -run
 
 logexpect l1 -wait
diff --git a/doc/changes.rst b/doc/changes.rst
index c9c8d1d00..e30c24924 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -38,6 +38,19 @@ Varnish Cache NEXT (2023-09-15)
 .. PLEASE keep this roughly in commit order as shown by git-log / tig
    (new to old)
 
+* VCL tracing now needs to be explicitly activated by setting the
+  ``req.trace`` or ``bereq.trace`` VCL variables, which are
+  initialized from the ``feature +trace`` flag. Only if the trace
+  variables are set will ``VCL_trace`` log records be generated.
+
+  Consequently, ``VCL_trace`` has been removed from the default
+  ``vsl_mask``, so any trace records will be emitted by
+  default. ``vsl_mask`` can still be used to filter ``VCL_trace``
+  records.
+
+  To trace ``vcl_init {}`` and ``vcl_fini {}``, set the ``feature
+  +trace`` flag while the vcl is loaded/discarded.
+
 * Varnish Delivery Processors (VDPs) are now also properly closed for
   error conditions, avoiding potential minor memory leaks.
 
diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst
index edb9761bb..b8a28c453 100644
--- a/doc/sphinx/reference/vcl_var.rst
+++ b/doc/sphinx/reference/vcl_var.rst
@@ -451,6 +451,23 @@ req.time
 	across restarts.
 
 
+.. _req.trace:
+
+req.trace
+
+	Type: BOOL
+
+	Readable from: client
+
+	Writable from: client
+
+	Controls if ``VCL_trace`` VSL records are emitted for the current
+	request, see :ref:`vsl(7)`.
+
+	Defaults to the setting of the ``feature trace`` parameter,
+	see :ref:`varnishd(1)`. Does not get reset by a rollback.
+
+
 .. _req.transport:
 
 req.transport
@@ -794,6 +811,23 @@ bereq.time
 	remains constant across retries.
 
 
+.. _bereq.trace:
+
+bereq.trace
+
+	Type: BOOL
+
+	Readable from: backend
+
+	Writable from: backend
+
+	Controls if ``VCL_trace`` VSL records are emitted for the current
+	request, see :ref:`vsl(7)`.
+
+	Inherits the value of ``req.trace`` when the backend request
+	is created. Does not get reset by a rollback.
+
+
 .. _bereq.uncacheable:
 
 bereq.uncacheable
diff --git a/include/tbl/feature_bits.h b/include/tbl/feature_bits.h
index 68ae39839..1ff2a361f 100644
--- a/include/tbl/feature_bits.h
+++ b/include/tbl/feature_bits.h
@@ -86,6 +86,11 @@ FEATURE_BIT(BUSY_STATS_RATE,	busy_stats_rate,
     "Make busy workers comply with thread_stats_rate."
 )
 
+FEATURE_BIT(TRACE,			trace,
+    "Enable VCL tracing by default (enable (be)req.trace). "
+    "Required for tracing vcl_init / vcl_fini"
+)
+
 #undef FEATURE_BIT
 
 /*lint -restore */
diff --git a/include/tbl/params.h b/include/tbl/params.h
index f5fb58695..fcf47e31c 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -1817,7 +1817,6 @@ PARAM_BITS(
 	"-ObjProtocol,"
 	"-ObjReason,"
 	"-ObjStatus,"
-	"-VCL_trace,"
 	"-VdpAcct,"
 	"-VfpAcct,"
 	"-WorkThread",
diff --git a/include/tbl/req_bereq_flags.h b/include/tbl/req_bereq_flags.h
index 36f868d2a..1cbde5d12 100644
--- a/include/tbl/req_bereq_flags.h
+++ b/include/tbl/req_bereq_flags.h
@@ -34,6 +34,7 @@
 /* lower, vcl_r, vcl_w, doc */
 REQ_BEREQ_FLAG(is_hitmiss,		1, 0, "")
 REQ_BEREQ_FLAG(is_hitpass,		1, 0, "")
+REQ_BEREQ_FLAG(trace,			1, 0, "")
 #undef REQ_BEREQ_FLAG
 
 /*lint -restore */
diff --git a/include/vcc_interface.h b/include/vcc_interface.h
index 83ef7f096..529ab9ed1 100644
--- a/include/vcc_interface.h
+++ b/include/vcc_interface.h
@@ -60,11 +60,16 @@ struct wrk_vpi {
 	unsigned	magic;
 #define WRK_VPI_MAGIC	0xaa3d3df3
 	unsigned	handling;
+	unsigned	trace;	// caches (be)req.trace
 	unsigned	ref;	// index into (struct vpi_ref)[]
 };
 
-
-void VPI_count(VRT_CTX, unsigned);
+#define VPI_count(ctx, max, u) \
+	do {							\
+		assert(u < max);				\
+		(ctx)->vpi->ref = u;				\
+	} while(0)
+void VPI_trace(VRT_CTX, unsigned);
 void VPI_vcl_fini(VRT_CTX);
 
 int VPI_Vmod_Init(VRT_CTX, struct vmod **hdl, unsigned nbr, void *ptr, int len,
diff --git a/include/vdef.h b/include/vdef.h
index faa4877c4..36e4fa854 100644
--- a/include/vdef.h
+++ b/include/vdef.h
@@ -257,3 +257,7 @@ typedef struct {
 
 #define Tcheck(t)	do { (void)pdiff((t).b, (t).e); } while (0)
 #define Tlen(t)		(pdiff((t).b, (t).e))
+
+/* #3020 dummy definitions until PR is merged*/
+#define LIKELY(x)	(x)
+#define UNLIKELY(x)	(x)
diff --git a/include/vrt.h b/include/vrt.h
index 68a2ff5a5..9dd7f044b 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -59,6 +59,7 @@
  *
  * NEXT (2023-09-15)
  *	[cache_filter.h] struct vdp gained priv1 member
+ *	VRT_trace() added
  * 17.0 (2023-03-15)
  *	VXID is 64 bit
  *	[cache.h] http_GetRange() changed
@@ -676,6 +677,7 @@ VCL_VOID VRT_UnsetHdr(VRT_CTX, VCL_HEADER);
 VCL_VOID VRT_SetHdr(VRT_CTX, VCL_HEADER, const char *pfx, VCL_STRANDS);
 VCL_VOID VRT_handling(VRT_CTX, unsigned hand);
 unsigned VRT_handled(VRT_CTX);
+VCL_VOID VRT_trace(VRT_CTX, VCL_BOOL);
 VCL_VOID VRT_fail(VRT_CTX, const char *fmt, ...) v_printflike_(2,3);
 VCL_VOID VRT_hashdata(VRT_CTX, VCL_STRANDS);
 
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index cd9832569..f579e2d0b 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -47,9 +47,11 @@ static void vcc_Compound(struct vcc *tl);
 	tl->indent -= INDENT;	\
 } while (0)
 
-#define C(tl, sep)	do {					\
-	Fb(tl, 1, "VPI_count(ctx, %u)%s\n", ++tl->cnt, sep);	\
-	tl->t->cnt = tl->cnt;					\
+#define C(tl, sep)	do {						\
+	Fb(tl, 1, "VPI_count(ctx, VGC_NREFS, %u)%s\n", ++tl->cnt, sep); \
+	Fb(tl, 1, "if (UNLIKELY(ctx->vpi->trace)) VPI_trace(ctx, %u)%s\n", \
+	   tl->cnt, sep);						\
+	tl->t->cnt = tl->cnt;						\
 } while (0)
 
 /*--------------------------------------------------------------------

From phk at FreeBSD.org  Tue May  9 05:49:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 05:49:08 +0000 (UTC)
Subject: [master] 9f48ee000 Handle the -gzipbody and -gziplen arguments
 centrally.
Message-ID: <20230509054908.9A17E114480@lists.varnish-cache.org>


commit 9f48ee000efcf816aa29f3c03bc7b87ded5002f5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 05:47:15 2023 +0000

    Handle the -gzipbody and -gziplen arguments centrally.

diff --git a/bin/varnishtest/vtc.h b/bin/varnishtest/vtc.h
index ab0b8031a..92d9acf12 100644
--- a/bin/varnishtest/vtc.h
+++ b/bin/varnishtest/vtc.h
@@ -150,8 +150,8 @@ void stop_h2(struct http *hp);
 void b64_settings(const struct http *hp, const char *s);
 
 /* vtc_gzip.c */
-void vtc_gzip(struct http *, const char *, char **, long *);
 void vtc_gunzip(struct http *, char *, long *);
+void vtc_gzip_cmd(struct http *hp, char * const *argv, char **body, long *bodylen);
 
 /* vtc_subr.c */
 struct vsb *vtc_hex_to_bin(struct vtclog *vl, const char *arg);
diff --git a/bin/varnishtest/vtc_gzip.c b/bin/varnishtest/vtc_gzip.c
index 02700bf69..a3c4b3c0f 100644
--- a/bin/varnishtest/vtc_gzip.c
+++ b/bin/varnishtest/vtc_gzip.c
@@ -103,7 +103,7 @@ vtc_gzip_vsb(struct vtclog *vl, int fatal, int gzip_level, const struct vsb *vin
 	return (vout);
 }
 
-void
+static void
 vtc_gzip(struct http *hp, const char *input, char **body, long *bodylen)
 {
 	struct vsb *vin, *vout;
@@ -193,3 +193,27 @@ vtc_gunzip(struct http *hp, char *body, long *bodylen)
 	vtc_dump(hp->vl, 4, "body", body, *bodylen);
 	bprintf(hp->bodylen, "%ld", *bodylen);
 }
+
+void
+vtc_gzip_cmd(struct http *hp, char * const *argv, char **body, long *bodylen)
+{
+	char *b;
+
+	AN(hp);
+	AN(argv);
+	AN(body);
+	AN(bodylen);
+
+	if (!strcmp(*argv, "-gzipbody")) {
+		AZ(*body);
+		vtc_gzip(hp, argv[1], body, bodylen);
+		AN(*body);
+	} else if (!strcmp(*argv, "-gziplen")) {
+		b = synth_body(argv[1], 1);
+		vtc_gzip(hp, b, body, bodylen);
+		AN(*body);
+		free(b);
+	} else {
+		WRONG("Wrong cmd til vtc_gzip_cmd");
+	}
+}
diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index 4c646d337..aab1626da 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -848,18 +848,16 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp,
 		} else if (!strcmp(*av, "-gziplen")) {
 			assert(body == nullbody);
 			free(body);
-			b = synth_body(av[1], 1);
-			vtc_gzip(hp, b, &body, &bodylen);
-			free(b);
+			body = NULL;
+			vtc_gzip_cmd(hp, av, &body, &bodylen);
 			VSB_printf(hp->vsb, "Content-Encoding: gzip%s", nl);
-			// vtc_hexdump(hp->vl, 4, "gzip", (void*)body, bodylen);
 			av++;
 		} else if (!strcmp(*av, "-gzipbody")) {
 			assert(body == nullbody);
 			free(body);
-			vtc_gzip(hp, av[1], &body, &bodylen);
+			body = NULL;
+			vtc_gzip_cmd(hp, av, &body, &bodylen);
 			VSB_printf(hp->vsb, "Content-Encoding: gzip%s", nl);
-			// vtc_hexdump(hp->vl, 4, "gzip", (void*)body, bodylen);
 			av++;
 		} else
 			break;
diff --git a/bin/varnishtest/vtc_http2.c b/bin/varnishtest/vtc_http2.c
index 42410e45f..68c36b4f9 100644
--- a/bin/varnishtest/vtc_http2.c
+++ b/bin/varnishtest/vtc_http2.c
@@ -1443,7 +1443,7 @@ cmd_tx11obj(CMD_ARGS)
 	/*XXX: do we need a better api? yes we do */
 	struct hpk_hdr hdr;
 	char *cmd_str = *av;
-	char *b, *p;
+	char *p;
 
 	CAST_OBJ_NOTNULL(s, priv, STREAM_MAGIC);
 	INIT_FRAME(f, CONTINUATION, 0, s->id, END_HEADERS);
@@ -1598,19 +1598,13 @@ cmd_tx11obj(CMD_ARGS)
 				av++;
 			}
 			else if (!strcmp(*av, "-gzipbody")) {
-				AZ(body);
-				vtc_gzip(s->hp, av[1], &body, &bodylen);
-				AN(body);
+				vtc_gzip_cmd(s->hp, av, &body, &bodylen);
 				ENC(hdr, ":content-encoding", "gzip");
 				f.flags &= ~END_STREAM;
 				av++;
 			}
 			else if (!strcmp(*av, "-gziplen")) {
-				AZ(body);
-				b = synth_body(av[1], 1);
-				vtc_gzip(s->hp, b, &body, &bodylen);
-				AN(body);
-				free(b);
+				vtc_gzip_cmd(s->hp, av, &body, &bodylen);
 				ENC(hdr, ":content-encoding", "gzip");
 				f.flags &= ~END_STREAM;
 				av++;

From phk at FreeBSD.org  Tue May  9 07:53:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 07:53:08 +0000 (UTC)
Subject: [master] 7fa722836 Collect all the body-gzip stuff one place.
Message-ID: <20230509075308.E2D54118714@lists.varnish-cache.org>


commit 7fa722836fd55d09a6fbe29bb30b475acdc01f78
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 06:51:47 2023 +0000

    Collect all the body-gzip stuff one place.

diff --git a/bin/varnishtest/vtc.h b/bin/varnishtest/vtc.h
index 92d9acf12..e7513092e 100644
--- a/bin/varnishtest/vtc.h
+++ b/bin/varnishtest/vtc.h
@@ -151,7 +151,7 @@ void b64_settings(const struct http *hp, const char *s);
 
 /* vtc_gzip.c */
 void vtc_gunzip(struct http *, char *, long *);
-void vtc_gzip_cmd(struct http *hp, char * const *argv, char **body, long *bodylen);
+int vtc_gzip_cmd(struct http *hp, char * const *argv, char **body, long *bodylen);
 
 /* vtc_subr.c */
 struct vsb *vtc_hex_to_bin(struct vtclog *vl, const char *arg);
diff --git a/bin/varnishtest/vtc_gzip.c b/bin/varnishtest/vtc_gzip.c
index a3c4b3c0f..f38699b33 100644
--- a/bin/varnishtest/vtc_gzip.c
+++ b/bin/varnishtest/vtc_gzip.c
@@ -194,26 +194,41 @@ vtc_gunzip(struct http *hp, char *body, long *bodylen)
 	bprintf(hp->bodylen, "%ld", *bodylen);
 }
 
-void
-vtc_gzip_cmd(struct http *hp, char * const *argv, char **body, long *bodylen)
+int
+vtc_gzip_cmd(struct http *hp, char * const *av, char **body, long *bodylen)
 {
 	char *b;
 
 	AN(hp);
-	AN(argv);
+	AN(av);
 	AN(body);
 	AN(bodylen);
 
-	if (!strcmp(*argv, "-gzipbody")) {
-		AZ(*body);
-		vtc_gzip(hp, argv[1], body, bodylen);
+	if (!strcmp(*av, "-gzipresidual")) {
+		hp->gzipresidual = strtoul(av[1], NULL, 0);
+		return (1);
+	}
+	if (!strcmp(*av, "-gziplevel")) {
+		hp->gziplevel = strtoul(av[1], NULL, 0);
+		return (1);
+	}
+	if (!strcmp(*av, "-gzipbody")) {
+		if (*body != NULL)
+			free(*body);
+		*body = NULL;
+		vtc_gzip(hp, av[1], body, bodylen);
 		AN(*body);
-	} else if (!strcmp(*argv, "-gziplen")) {
-		b = synth_body(argv[1], 1);
+		return (2);
+	}
+	if (!strcmp(*av, "-gziplen")) {
+		if (*body != NULL)
+			free(*body);
+		*body = NULL;
+		b = synth_body(av[1], 1);
 		vtc_gzip(hp, b, body, bodylen);
 		AN(*body);
 		free(b);
-	} else {
-		WRONG("Wrong cmd til vtc_gzip_cmd");
+		return (2);
 	}
+	return (0);
 }
diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index aab1626da..68b9d5936 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -839,26 +839,13 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp,
 			body = synth_body(av[1], 0);
 			bodylen = strlen(body);
 			av++;
-		} else if (!strcmp(*av, "-gzipresidual")) {
-			hp->gzipresidual = strtoul(av[1], NULL, 0);
-			av++;
-		} else if (!strcmp(*av, "-gziplevel")) {
-			hp->gziplevel = strtoul(av[1], NULL, 0);
-			av++;
-		} else if (!strcmp(*av, "-gziplen")) {
-			assert(body == nullbody);
-			free(body);
-			body = NULL;
-			vtc_gzip_cmd(hp, av, &body, &bodylen);
-			VSB_printf(hp->vsb, "Content-Encoding: gzip%s", nl);
-			av++;
-		} else if (!strcmp(*av, "-gzipbody")) {
-			assert(body == nullbody);
-			free(body);
-			body = NULL;
-			vtc_gzip_cmd(hp, av, &body, &bodylen);
-			VSB_printf(hp->vsb, "Content-Encoding: gzip%s", nl);
-			av++;
+		} else if (!strncmp(*av, "-gzip", 5)) {
+			l = vtc_gzip_cmd(hp, av, &body, &bodylen);
+			if (l == 0)
+				break;
+			av += l;
+			if (l > 1)
+				VSB_printf(hp->vsb, "Content-Encoding: gzip%s", nl);
 		} else
 			break;
 	}
diff --git a/bin/varnishtest/vtc_http2.c b/bin/varnishtest/vtc_http2.c
index 68c36b4f9..72a4399fc 100644
--- a/bin/varnishtest/vtc_http2.c
+++ b/bin/varnishtest/vtc_http2.c
@@ -1427,6 +1427,7 @@ static void
 cmd_tx11obj(CMD_ARGS)
 {
 	struct stream *s;
+	int i;
 	int status_done = 1;
 	int method_done = 1;
 	int path_done = 1;
@@ -1597,17 +1598,15 @@ cmd_tx11obj(CMD_ARGS)
 				f.flags &= ~END_STREAM;
 				av++;
 			}
-			else if (!strcmp(*av, "-gzipbody")) {
-				vtc_gzip_cmd(s->hp, av, &body, &bodylen);
-				ENC(hdr, ":content-encoding", "gzip");
-				f.flags &= ~END_STREAM;
-				av++;
-			}
-			else if (!strcmp(*av, "-gziplen")) {
-				vtc_gzip_cmd(s->hp, av, &body, &bodylen);
-				ENC(hdr, ":content-encoding", "gzip");
-				f.flags &= ~END_STREAM;
-				av++;
+			else if (!strncmp(*av, "-gzip", 5)) {
+				i = vtc_gzip_cmd(s->hp, av, &body, &bodylen);
+				if (i == 0)
+					break;
+				av += i;
+				if (i > 1) {
+					ENC(hdr, ":content-encoding", "gzip");
+					f.flags &= ~END_STREAM;
+				}
 			}
 			else if (AV_IS("-dep")) {
 				STRTOU32_CHECK(stid, av, p, vl, "-dep", 0);

From phk at FreeBSD.org  Tue May  9 07:53:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 07:53:08 +0000 (UTC)
Subject: [master] a91d3f2cd Polish the body-gzippery a bit.
Message-ID: <20230509075308.F0939118715@lists.varnish-cache.org>


commit a91d3f2cdb225bd1681cbb2545fd954b9cf68b35
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 07:52:10 2023 +0000

    Polish the body-gzippery a bit.

diff --git a/bin/varnishtest/vtc_gzip.c b/bin/varnishtest/vtc_gzip.c
index f38699b33..e2b210885 100644
--- a/bin/varnishtest/vtc_gzip.c
+++ b/bin/varnishtest/vtc_gzip.c
@@ -61,15 +61,18 @@ APOS(ssize_t sz)
 	return (sz);
 }
 
+/**********************************************************************
+ * GUNZIPery
+ */
+
 static struct vsb *
-vtc_gzip_vsb(struct vtclog *vl, int fatal, int gzip_level, const struct vsb *vin, int *residual)
+vtc_gunzip_vsb(struct vtclog *vl, int fatal, const struct vsb *vin)
 {
 	z_stream vz;
 	struct vsb *vout;
 	int i;
 	char buf[BUFSIZ];
 
-	AN(residual);
 	memset(&vz, 0, sizeof vz);
 	vout = VSB_new_auto();
 	AN(vout);
@@ -77,120 +80,123 @@ vtc_gzip_vsb(struct vtclog *vl, int fatal, int gzip_level, const struct vsb *vin
 	vz.next_in = (void*)VSB_data(vin);
 	vz.avail_in = APOS(VSB_len(vin));
 
-	assert(Z_OK == deflateInit2(&vz,
-	    gzip_level, Z_DEFLATED, 31, 9, Z_DEFAULT_STRATEGY));
+	assert(Z_OK == inflateInit2(&vz, 31));
 
 	do {
 		vz.next_out = (void*)buf;
 		vz.avail_out = sizeof buf;
-		i = deflate(&vz, Z_FINISH);
+		i = inflate(&vz, Z_FINISH);
 		if (vz.avail_out != sizeof buf)
 			VSB_bcat(vout, buf, sizeof buf - vz.avail_out);
 	} while (i == Z_OK || i == Z_BUF_ERROR);
 	if (i != Z_STREAM_END)
 		vtc_log(vl, fatal,
-		    "Gzip error = %d (%s) in:%jd out:%jd len:%zd",
+		    "Gunzip error = %d (%s) in:%jd out:%jd len:%zd",
 		    i, vz.msg, (intmax_t)vz.total_in, (intmax_t)vz.total_out,
 		    VSB_len(vin));
 	AZ(VSB_finish(vout));
 #ifdef VGZ_EXTENSIONS
-	*residual = vz.stop_bit & 7;
 	vtc_report_gz_bits(vl, &vz);
-#else
-	*residual = 0;
 #endif
-	assert(Z_OK == deflateEnd(&vz));
+	assert(Z_OK == inflateEnd(&vz));
 	return (vout);
 }
 
-static void
-vtc_gzip(struct http *hp, const char *input, char **body, long *bodylen)
+void
+vtc_gunzip(struct http *hp, char *body, long *bodylen)
 {
 	struct vsb *vin, *vout;
-	int res;
+
+	AN(body);
+	if (body[0] != (char)0x1f || body[1] != (char)0x8b)
+		vtc_log(hp->vl, hp->fatal,
+		    "Gunzip error: body lacks gzip magic");
 
 	vin = VSB_new_auto();
 	AN(vin);
-	VSB_bcat(vin, input, strlen(input));
+	VSB_bcat(vin, body, *bodylen);
 	AZ(VSB_finish(vin));
-	vout = vtc_gzip_vsb(hp->vl, hp->fatal, hp->gziplevel, vin, &res);
+	vout = vtc_gunzip_vsb(hp->vl, hp->fatal, vin);
 	VSB_destroy(&vin);
 
-#ifdef VGZ_EXTENSIONS
-	if (hp->gzipresidual >= 0 && hp->gzipresidual != res)
-		vtc_log(hp->vl, hp->fatal,
-		    "Wrong gzip residual got %d wanted %d",
-		    res, hp->gzipresidual);
-#endif
-	*body = malloc(APOS(VSB_len(vout) + 1));
-	AN(*body);
-	memcpy(*body, VSB_data(vout), APOS(VSB_len(vout) + 1));
+	memcpy(body, VSB_data(vout), APOS(VSB_len(vout) + 1));
 	*bodylen = APOS(VSB_len(vout));
 	VSB_destroy(&vout);
 	vtc_log(hp->vl, 3, "new bodylen %ld", *bodylen);
-	vtc_dump(hp->vl, 4, "body", *body, *bodylen);
+	vtc_dump(hp->vl, 4, "body", body, *bodylen);
 	bprintf(hp->bodylen, "%ld", *bodylen);
 }
 
-static struct vsb *
-vtc_gunzip_vsb(struct vtclog *vl, int fatal, const struct vsb *vin)
+/**********************************************************************
+ * GZIPery
+ */
+
+static int
+vtc_gzip_chunk(z_stream *vz, struct vsb *vout, const void *in, size_t inlen, int flush)
 {
-	z_stream vz;
-	struct vsb *vout;
 	int i;
 	char buf[BUFSIZ];
 
+	vz->next_in = TRUST_ME(in);
+	vz->avail_in = APOS(inlen);
+	do {
+		vz->next_out = (void*)buf;
+		vz->avail_out = sizeof buf;
+		i = deflate(vz, flush);
+		if (vz->avail_out != sizeof buf)
+			VSB_bcat(vout, buf, sizeof buf - vz->avail_out);
+	} while (i == Z_OK || i == Z_BUF_ERROR);
+	vz->next_out = NULL;
+	vz->avail_out = 0;
+	vz->next_in = NULL;
+	AZ(vz->avail_in);
+	vz->avail_in = 0;
+	return (i);
+}
+
+static void
+vtc_gzip(struct http *hp, const void *input, char **body, long *bodylen)
+{
+	struct vsb *vout;
+	int i, res;
+	z_stream vz;
+
 	memset(&vz, 0, sizeof vz);
 	vout = VSB_new_auto();
 	AN(vout);
 
-	vz.next_in = (void*)VSB_data(vin);
-	vz.avail_in = APOS(VSB_len(vin));
-
-	assert(Z_OK == inflateInit2(&vz, 31));
+	assert(Z_OK == deflateInit2(&vz,
+	    hp->gziplevel, Z_DEFLATED, 31, 9, Z_DEFAULT_STRATEGY));
 
-	do {
-		vz.next_out = (void*)buf;
-		vz.avail_out = sizeof buf;
-		i = inflate(&vz, Z_FINISH);
-		if (vz.avail_out != sizeof buf)
-			VSB_bcat(vout, buf, sizeof buf - vz.avail_out);
-	} while (i == Z_OK || i == Z_BUF_ERROR);
-	if (i != Z_STREAM_END)
-		vtc_log(vl, fatal,
-		    "Gunzip error = %d (%s) in:%jd out:%jd len:%zd",
+	i = vtc_gzip_chunk(&vz, vout, input, strlen(input), Z_FINISH);
+	if (i != Z_STREAM_END) {
+		vtc_log(hp->vl, hp->fatal,
+		    "Gzip error = %d (%s) in:%jd out:%jd len:%zd",
 		    i, vz.msg, (intmax_t)vz.total_in, (intmax_t)vz.total_out,
-		    VSB_len(vin));
+		    strlen(input));
+	}
 	AZ(VSB_finish(vout));
 #ifdef VGZ_EXTENSIONS
-	vtc_report_gz_bits(vl, &vz);
+	res = vz.stop_bit & 7;
+	vtc_report_gz_bits(hp->vl, &vz);
+#else
+	res = 0;
 #endif
-	assert(Z_OK == inflateEnd(&vz));
-	return (vout);
-}
-
-void
-vtc_gunzip(struct http *hp, char *body, long *bodylen)
-{
-	struct vsb *vin, *vout;
+	assert(Z_OK == deflateEnd(&vz));
 
-	AN(body);
-	if (body[0] != (char)0x1f || body[1] != (char)0x8b)
+#ifdef VGZ_EXTENSIONS
+	if (hp->gzipresidual >= 0 && hp->gzipresidual != res)
 		vtc_log(hp->vl, hp->fatal,
-		    "Gunzip error: body lacks gzip magic");
-
-	vin = VSB_new_auto();
-	AN(vin);
-	VSB_bcat(vin, body, *bodylen);
-	AZ(VSB_finish(vin));
-	vout = vtc_gunzip_vsb(hp->vl, hp->fatal, vin);
-	VSB_destroy(&vin);
-
-	memcpy(body, VSB_data(vout), APOS(VSB_len(vout) + 1));
+		    "Wrong gzip residual got %d wanted %d",
+		    res, hp->gzipresidual);
+#endif
+	*body = malloc(APOS(VSB_len(vout) + 1));
+	AN(*body);
+	memcpy(*body, VSB_data(vout), APOS(VSB_len(vout) + 1));
 	*bodylen = APOS(VSB_len(vout));
 	VSB_destroy(&vout);
 	vtc_log(hp->vl, 3, "new bodylen %ld", *bodylen);
-	vtc_dump(hp->vl, 4, "body", body, *bodylen);
+	vtc_dump(hp->vl, 4, "body", *body, *bodylen);
 	bprintf(hp->bodylen, "%ld", *bodylen);
 }
 

From phk at FreeBSD.org  Tue May  9 07:59:05 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 07:59:05 +0000 (UTC)
Subject: [master] ed1a4f23e Ensure old VSL's do not confuse expectations
Message-ID: <20230509075905.2DFC7118DFB@lists.varnish-cache.org>


commit ed1a4f23e2bc7f173476e6b324d0286ab7e0bb9a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 07:58:30 2023 +0000

    Ensure old VSL's do not confuse expectations

diff --git a/bin/varnishtest/tests/r02413.vtc b/bin/varnishtest/tests/r02413.vtc
index ba85102b1..64715f6e9 100644
--- a/bin/varnishtest/tests/r02413.vtc
+++ b/bin/varnishtest/tests/r02413.vtc
@@ -28,6 +28,9 @@ varnish v1 -vcl+backend {
 varnish v1 -cliok "vcl.label vclB vcl2"
 varnish v1 -cliok "vcl.list"
 
+# Ensure old VSLs do not confuse l1
+varnish v1 -vsl_catchup
+
 logexpect l1 -v v1 -g raw {
 	expect * 1002	VCL_call	"BACKEND_FETCH"
 	expect 0 1002	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}

From phk at FreeBSD.org  Tue May  9 08:55:06 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 08:55:06 +0000 (UTC)
Subject: [master] 7565cdba7 Make -gziplen flush underways to exercise more
 cornercases.
Message-ID: <20230509085506.7921011AE2A@lists.varnish-cache.org>


commit 7565cdba7d1027669dc805820e8dd8ae55e1be9c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 08:53:06 2023 +0000

    Make -gziplen flush underways to exercise more cornercases.

diff --git a/bin/varnishtest/vtc_gzip.c b/bin/varnishtest/vtc_gzip.c
index e2b210885..b07e7218d 100644
--- a/bin/varnishtest/vtc_gzip.c
+++ b/bin/varnishtest/vtc_gzip.c
@@ -132,7 +132,7 @@ vtc_gunzip(struct http *hp, char *body, long *bodylen)
  */
 
 static int
-vtc_gzip_chunk(z_stream *vz, struct vsb *vout, const void *in, size_t inlen, int flush)
+vtc_gzip_chunk(z_stream *vz, struct vsb *vout, const char *in, size_t inlen, int flush)
 {
 	int i;
 	char buf[BUFSIZ];
@@ -145,7 +145,7 @@ vtc_gzip_chunk(z_stream *vz, struct vsb *vout, const void *in, size_t inlen, int
 		i = deflate(vz, flush);
 		if (vz->avail_out != sizeof buf)
 			VSB_bcat(vout, buf, sizeof buf - vz->avail_out);
-	} while (i == Z_OK || i == Z_BUF_ERROR);
+	} while (i == Z_OK || vz->avail_in > 0);
 	vz->next_out = NULL;
 	vz->avail_out = 0;
 	vz->next_in = NULL;
@@ -155,10 +155,11 @@ vtc_gzip_chunk(z_stream *vz, struct vsb *vout, const void *in, size_t inlen, int
 }
 
 static void
-vtc_gzip(struct http *hp, const void *input, char **body, long *bodylen)
+vtc_gzip(struct http *hp, const char *input, char **body, long *bodylen, int fragment)
 {
 	struct vsb *vout;
 	int i, res;
+	size_t inlen = strlen(input);
 	z_stream vz;
 
 	memset(&vz, 0, sizeof vz);
@@ -168,7 +169,20 @@ vtc_gzip(struct http *hp, const void *input, char **body, long *bodylen)
 	assert(Z_OK == deflateInit2(&vz,
 	    hp->gziplevel, Z_DEFLATED, 31, 9, Z_DEFAULT_STRATEGY));
 
-	i = vtc_gzip_chunk(&vz, vout, input, strlen(input), Z_FINISH);
+	while (fragment && inlen > 3) {
+		res = inlen / 3;
+		i = vtc_gzip_chunk(&vz, vout, input, res, Z_BLOCK);
+		if (i != Z_OK && i != Z_BUF_ERROR) {
+			vtc_log(hp->vl, hp->fatal,
+			    "Gzip error = %d (%s) in:%jd out:%jd len:%zd",
+			    i, vz.msg, (intmax_t)vz.total_in,
+			    (intmax_t)vz.total_out, strlen(input));
+		}
+		input += res;
+		inlen -= res;
+	}
+
+	i = vtc_gzip_chunk(&vz, vout, input, inlen, Z_FINISH);
 	if (i != Z_STREAM_END) {
 		vtc_log(hp->vl, hp->fatal,
 		    "Gzip error = %d (%s) in:%jd out:%jd len:%zd",
@@ -222,7 +236,7 @@ vtc_gzip_cmd(struct http *hp, char * const *av, char **body, long *bodylen)
 		if (*body != NULL)
 			free(*body);
 		*body = NULL;
-		vtc_gzip(hp, av[1], body, bodylen);
+		vtc_gzip(hp, av[1], body, bodylen, 0);
 		AN(*body);
 		return (2);
 	}
@@ -231,7 +245,7 @@ vtc_gzip_cmd(struct http *hp, char * const *av, char **body, long *bodylen)
 			free(*body);
 		*body = NULL;
 		b = synth_body(av[1], 1);
-		vtc_gzip(hp, b, body, bodylen);
+		vtc_gzip(hp, b, body, bodylen, 1);
 		AN(*body);
 		free(b);
 		return (2);

From phk at FreeBSD.org  Tue May  9 11:47:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  9 May 2023 11:47:08 +0000 (UTC)
Subject: [master] f4f2830c1 Take another stab at stabilizing r02413
Message-ID: <20230509114708.B7016120676@lists.varnish-cache.org>


commit f4f2830c16242bd4f5390bdb7782e435b628b907
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 9 11:45:21 2023 +0000

    Take another stab at stabilizing r02413

diff --git a/bin/varnishtest/tests/r02413.vtc b/bin/varnishtest/tests/r02413.vtc
index 64715f6e9..578fa5409 100644
--- a/bin/varnishtest/tests/r02413.vtc
+++ b/bin/varnishtest/tests/r02413.vtc
@@ -11,6 +11,8 @@ varnish v1 -arg "-p feature=+trace" -vcl+backend {
 	}
 } -start
 
+varnish v1 -cliok "param.set debug +syncvsl"
+
 varnish v1 -cliok "vcl.label vclA vcl1"
 
 varnish v1 -vcl+backend {
@@ -32,12 +34,12 @@ varnish v1 -cliok "vcl.list"
 varnish v1 -vsl_catchup
 
 logexpect l1 -v v1 -g raw {
-	expect * 1002	VCL_call	"BACKEND_FETCH"
-	expect 0 1002	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}
 	expect * 1001	VCL_call	"RECV"
 	expect 0 1001	VCL_trace	{^vcl2 \d+ \d+\.\d+\.\d+$}
 	expect * 1001	VCL_call	"RECV"
 	expect 0 1001	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}
+	expect * 1002	VCL_call	"BACKEND_FETCH"
+	expect 0 1002	VCL_trace	{^vcl1 \d+ \d+\.\d+\.\d+$}
 	expect * 1003	VCL_call	"DELIVER"
 	expect 0 1003	RespHeader	{^vcl: vcl2}
 } -start

From phk at FreeBSD.org  Wed May 10 08:38:11 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 10 May 2023 08:38:11 +0000 (UTC)
Subject: [master] 66531cefc Rearrange r03159 to not leave varnishd processes
 around after failure
Message-ID: <20230510083811.61393122958@lists.varnish-cache.org>


commit 66531cefcfcb65f9e1413711c9beb3225fb91b11
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 10 08:36:25 2023 +0000

    Rearrange r03159 to not leave varnishd processes around after failure

diff --git a/bin/varnishtest/tests/r03159.vtc b/bin/varnishtest/tests/r03159.vtc
index 66085f189..aed842a6c 100644
--- a/bin/varnishtest/tests/r03159.vtc
+++ b/bin/varnishtest/tests/r03159.vtc
@@ -2,15 +2,13 @@ varnishtest "double sub unref warning / warnings output for -f"
 
 # Also tests #3160
 
-shell {
-	cat >unref.vcl <<-EOF
+filewrite ${tmpdir}/unref.vcl {
 	vcl 4.1;
 	backend be none;
 	sub foo { }
-	EOF
 }
 
-process p1 -winsz 100 80 -log {
+process p1 -log {
 	varnishd -F -n "${tmpdir}/t" -a "${tmpdir}/sock" \
 	    -p vcc_feature=-err_unref -f "${tmpdir}/unref.vcl" \
 	    -l 2m 2>&1
@@ -23,11 +21,12 @@ process p1 -screen_dump
 
 process p2 -log {varnishadm -n ${tmpdir}/t "vcl.load unref ${tmpdir}/unref.vcl"} -run
 
-process p2 -expect-text 0 1 "Unused sub foo, defined:"
-process p2 -expect-text 0 1 "(That was just a warning)"
+process p2 -wait
+process p1 -kill TERM
 
 process p2 -screen_dump
 
-process p1 -kill TERM
+process p2 -expect-text 0 1 "Unused sub foo, defined:"
+process p2 -expect-text 0 1 "(That was just a warning)"
 
 process p1 -expect-exit 0x40 -wait

From phk at FreeBSD.org  Wed May 10 11:18:06 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 10 May 2023 11:18:06 +0000 (UTC)
Subject: [master] 1d166a9ba Attempt to stabilize r03159 on arm32 vtester
Message-ID: <20230510111806.9D29F100C73@lists.varnish-cache.org>


commit 1d166a9baec1838f795c2084a69761478c5efc17
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 10 11:17:07 2023 +0000

    Attempt to stabilize r03159 on arm32 vtester

diff --git a/bin/varnishtest/tests/r03159.vtc b/bin/varnishtest/tests/r03159.vtc
index aed842a6c..b95abf320 100644
--- a/bin/varnishtest/tests/r03159.vtc
+++ b/bin/varnishtest/tests/r03159.vtc
@@ -12,14 +12,19 @@ process p1 -log {
 	varnishd -F -n "${tmpdir}/t" -a "${tmpdir}/sock" \
 	    -p vcc_feature=-err_unref -f "${tmpdir}/unref.vcl" \
 	    -l 2m 2>&1
-} -start
+} -start -expect-exit 0x40
 
 process p1 -expect-text 0 1 "Unused sub foo, defined:"
 process p1 -expect-text 0 1 "(That was just a warning)"
 
 process p1 -screen_dump
 
-process p2 -log {varnishadm -n ${tmpdir}/t "vcl.load unref ${tmpdir}/unref.vcl"} -run
+process p2 -log {
+	set -e
+	varnishadm -n ${tmpdir}/t "vcl.list"
+	varnishadm -n ${tmpdir}/t -t 20 "vcl.load unref ${tmpdir}/unref.vcl"
+	varnishadm -n ${tmpdir}/t "vcl.list"
+} -run
 
 process p2 -wait
 process p1 -kill TERM
@@ -29,4 +34,4 @@ process p2 -screen_dump
 process p2 -expect-text 0 1 "Unused sub foo, defined:"
 process p2 -expect-text 0 1 "(That was just a warning)"
 
-process p1 -expect-exit 0x40 -wait
+process p1 -wait

From phk at FreeBSD.org  Wed May 10 11:56:07 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 10 May 2023 11:56:07 +0000 (UTC)
Subject: [master] 565159689 Exit(2) if we get a CLI timeout.
Message-ID: <20230510115607.16AC8101F5B@lists.varnish-cache.org>


commit 5651596898557696b2506794744d342ff99d60dc
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 10 11:55:18 2023 +0000

    Exit(2) if we get a CLI timeout.

diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c
index e9d280f52..d08da8278 100644
--- a/bin/varnishadm/varnishadm.c
+++ b/bin/varnishadm/varnishadm.c
@@ -174,8 +174,10 @@ pass_answer(int fd, enum pass_mode_e mode)
 
 	u = VCLI_ReadResult(fd, &status, &answer, timeout);
 	if (u) {
-		if (status == CLIS_COMMS)
-			RL_EXIT(0);
+		if (status == CLIS_COMMS) {
+			fprintf(stderr, "%s\n", answer);
+			RL_EXIT(2);
+		}
 		if (answer)
 			fprintf(stderr, "%s\n", answer);
 		RL_EXIT(1);
@@ -304,6 +306,7 @@ interactive(int sock)
 {
 	struct pollfd fds[2];
 	int i;
+	unsigned status;
 	line_sock = sock;
 	rl_already_prompted = 1;
 	rl_callback_handler_install("varnish> ", send_line);
@@ -324,8 +327,10 @@ interactive(int sock)
 		if (fds[0].revents & POLLIN) {
 			/* Get rid of the prompt, kinda hackish */
 			printf("\r           \r");
-			(void)pass_answer(fds[0].fd, pass_interactive);
+			status = pass_answer(fds[0].fd, pass_interactive);
 			rl_forced_update_display();
+			if (status == CLIS_CLOSE)
+				RL_EXIT(0);
 		}
 		if (fds[1].revents & POLLIN) {
 			rl_callback_read_char();

From phk at FreeBSD.org  Fri May 12 22:01:13 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 12 May 2023 22:01:13 +0000 (UTC)
Subject: [master] c83cc81fb Fix a bug in our vgz-extension which is only
 triggered by vtest and only after 7565cdba7d10
Message-ID: <20230512220113.8E5E512375B@lists.varnish-cache.org>


commit c83cc81fb8d3086eb5be69d8a246457c2848c0b5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri May 12 21:59:06 2023 +0000

    Fix a bug in our vgz-extension which is only triggered by vtest and only after 7565cdba7d10

diff --git a/lib/libvgz/deflate.c b/lib/libvgz/deflate.c
index ed9ac5190..75277afd4 100644
--- a/lib/libvgz/deflate.c
+++ b/lib/libvgz/deflate.c
@@ -1810,6 +1810,9 @@ local block_state deflate_stored(s, flush)
             s->strm->total_out += len;
         }
     } while (last == 0);
+    if (last) 
+        s->strm->stop_bit =
+           (s->strm->total_out + s->pending) * 8 + s->bi_valid;
 
     /* Update the sliding window with the last s->w_size bytes of the copied
      * data, or append all of the copied data to the existing window if less

From phk at FreeBSD.org  Mon May 15 10:15:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 15 May 2023 10:15:08 +0000 (UTC)
Subject: [master] 5bd96abdb Dont take a pointless extra round through VGZ if
 we are already done.
Message-ID: <20230515101508.A2A9E11EE0A@lists.varnish-cache.org>


commit 5bd96abdb40e51b9e3266bc79bb06495a34725dd
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 15 10:13:24 2023 +0000

    Dont take a pointless extra round through VGZ if we are already done.

diff --git a/bin/varnishd/cache/cache_gzip.c b/bin/varnishd/cache/cache_gzip.c
index b73ca938e..d97985734 100644
--- a/bin/varnishd/cache/cache_gzip.c
+++ b/bin/varnishd/cache/cache_gzip.c
@@ -623,7 +623,8 @@ vfp_gzip_pull(struct vfp_ctx *vc, struct vfp_entry *vfe, void *p,
 				VGZ_UpdateObj(vc, vg, VUA_UPDATE);
 				*lp = dl;
 				assert(dp == p);
-				return (VFP_OK);
+				if (vr != VGZ_END || !VGZ_IbufEmpty(vg))
+					return (VFP_OK);
 			}
 		}
 		AN(VGZ_IbufEmpty(vg));

From nils.goroll at uplex.de  Mon May 15 13:26:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 15 May 2023 13:26:06 +0000 (UTC)
Subject: [master] ab0214c80 Move HSH_Cancel() to VDP_Close() to bring it under
 transport control
Message-ID: <20230515132606.9656966D9@lists.varnish-cache.org>


commit ab0214c8015a1694996e54e4df101816953963db
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 10 13:07:40 2023 +0200

    Move HSH_Cancel() to VDP_Close() to bring it under transport control
    
    Transports should be free to keep a reference on the object to be
    delivered until after their transport function returns, but
    HSH_Cancel() in cnt_transmit() prevented the object from being of any
    use for the case that it is final (pass/hfm/hfp).
    
    We solve this by moving the HSH_Cancel() close to VDP_Close, which
    also makes sense from the perspective of the VDP design: Until the VDP
    close, filters could still reference object data.
    
    HSH_Cancel() needs the objcore, which could be reachable also via
    vdc->req. But that member is unset in VDP_DeliverObj(), presumably to
    make it clear that VDP .bytes callbacks should not access request
    data. Thus, we pass it as a new argument to VDP_Close() as well as any
    boc being held by the caller.
    
    The objcore can also be NULL for the case where a transport generates
    the body without holding an objcore at all.

diff --git a/bin/varnishd/cache/cache_deliver_proc.c b/bin/varnishd/cache/cache_deliver_proc.c
index 44507eb5a..a6d9832ed 100644
--- a/bin/varnishd/cache/cache_deliver_proc.c
+++ b/bin/varnishd/cache/cache_deliver_proc.c
@@ -33,6 +33,7 @@
 
 #include "cache_varnishd.h"
 #include "cache_filter.h"
+#include "cache_objhead.h"
 
 void
 VDP_Panic(struct vsb *vsb, const struct vdp_ctx *vdc)
@@ -182,12 +183,16 @@ VDP_Push(VRT_CTX, struct vdp_ctx *vdc, struct ws *ws, const struct vdp *vdp,
 }
 
 uint64_t
-VDP_Close(struct vdp_ctx *vdc)
+VDP_Close(struct vdp_ctx *vdc, struct objcore *oc, struct boc *boc)
 {
 	struct vdp_entry *vdpe;
 	uint64_t rv = 0;
 
 	CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(vdc->wrk, WORKER_MAGIC);
+	CHECK_OBJ_ORNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
+
 	while (!VTAILQ_EMPTY(&vdc->vdp)) {
 		vdpe = VTAILQ_FIRST(&vdc->vdp);
 		rv = vdpe->bytes_in;
@@ -209,6 +214,8 @@ VDP_Close(struct vdp_ctx *vdc)
 			assert(vdpe->end == VDP_END);
 #endif
 	}
+	if (oc != NULL)
+		HSH_Cancel(vdc->wrk, oc, boc);
 	return (rv);
 }
 
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 56f4b82d2..762e42ae2 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -924,7 +924,7 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody)
 	if (i && req->doclose == SC_NULL)
 		req->doclose = SC_REM_CLOSE;
 
-	req->acct.resp_bodybytes += VDP_Close(req->vdc);
+	req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc);
 
 	if (i && !ecx->incl_cont) {
 		req->top->topreq->vdc->retval = -1;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index fabddba5b..789cea540 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -488,8 +488,6 @@ cnt_transmit(struct worker *wrk, struct req *req)
 
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
-	HSH_Cancel(wrk, req->objcore, boc);
-
 	if (req->doclose == SC_NULL && (req->objcore->flags & OC_F_FAILED)) {
 		/* The object we delivered failed due to a streaming error.
 		 * Fail the request. */
@@ -497,7 +495,7 @@ cnt_transmit(struct worker *wrk, struct req *req)
 	}
 
 	if (req->doclose != SC_NULL)
-		req->acct.resp_bodybytes += VDP_Close(req->vdc);
+		req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc);
 
 	if (boc != NULL)
 		HSH_DerefBoc(wrk, req->objcore);
diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index 08763874c..b534eb174 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -191,7 +191,7 @@ void VDI_Init(void);
 /* cache_deliver_proc.c */
 void VDP_Init(struct vdp_ctx *vdx, struct worker *wrk, struct vsl_log *vsl,
     struct req *req);
-uint64_t VDP_Close(struct vdp_ctx *);
+uint64_t VDP_Close(struct vdp_ctx *, struct objcore *, struct boc *);
 void VDP_Panic(struct vsb *vsb, const struct vdp_ctx *vdc);
 int VDP_Push(VRT_CTX, struct vdp_ctx *, struct ws *, const struct vdp *,
     void *priv);
diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c
index 00331091e..86a2dfd0e 100644
--- a/bin/varnishd/http1/cache_http1_deliver.c
+++ b/bin/varnishd/http1/cache_http1_deliver.c
@@ -163,7 +163,7 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody)
 	AZ(req->wrk->v1l);
 
 	req->acct.resp_hdrbytes += hdrbytes;
-	req->acct.resp_bodybytes += VDP_Close(req->vdc);
+	req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc);
 
 	if (sc == SC_NULL && err && req->sp->fd >= 0)
 		sc = SC_REM_CLOSE;
diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c
index a19e936d5..632d999ef 100644
--- a/bin/varnishd/http2/cache_http2_deliver.c
+++ b/bin/varnishd/http2/cache_http2_deliver.c
@@ -351,5 +351,5 @@ h2_deliver(struct req *req, struct boc *boc, int sendbody)
 	}
 
 	AZ(req->wrk->v1l);
-	req->acct.resp_bodybytes += VDP_Close(req->vdc);
+	req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc);
 }

From nils.goroll at uplex.de  Mon May 15 13:27:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 15 May 2023 13:27:06 +0000 (UTC)
Subject: [master] 8f48758fc man: Move beresp initialization to vcl-var(7)
Message-ID: <20230515132706.738116946@lists.varnish-cache.org>


commit 8f48758fc68c7f3183922be82d06624209734636
Author: Walid Boudebouda <walid.boudebouda at gmail.com>
Date:   Mon Mar 27 17:59:30 2023 +0200

    man: Move beresp initialization to vcl-var(7)
    
    The 304 handling and initial values for beresp.ttl, beresp.grace and
    beresp.keep belong in a reference manual.

diff --git a/doc/sphinx/reference/vcl-var.rst b/doc/sphinx/reference/vcl-var.rst
index b8278b57a..f2d832306 100644
--- a/doc/sphinx/reference/vcl-var.rst
+++ b/doc/sphinx/reference/vcl-var.rst
@@ -74,6 +74,79 @@ Assigning an HTTP standardized code to ``resp.status`` or ``beresp.status``
 will also set ``resp.reason`` or ``beresp.reason``  to the corresponding
 status message.
 
+
+304 handling
+~~~~~~~~~~~~
+
+For a 304 response, Varnish core code amends ``beresp`` before calling
+`vcl_backend_response`:
+
+* If the gzip status changed, ``Content-Encoding`` is unset and any
+  ``Etag`` is weakened
+
+* Any headers not present in the 304 response are copied from the
+  existing cache object. ``Content-Length`` is copied if present in
+  the existing cache object and discarded otherwise.
+
+* The status gets set to 200.
+
+`beresp.was_304` marks that this conditional response processing has
+happened.
+
+Note: Backend conditional requests are independent of client
+conditional requests, so clients may receive 304 responses no matter
+if a backend request was conditional.
+
+beresp.ttl / beresp.grace / beresp.keep
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before calling `vcl_backend_response`, core code sets ``beresp.ttl``
+based on the response status and the response headers ``Age``,
+``Cache-Control`` or ``Expires`` and ``Date`` as follows:
+
+* If present and valid, the value of the ``Age`` header is effectively
+  deduced from all ttl calculations.
+
+* For status codes 200, 203, 204, 300, 301, 304, 404, 410 and 414:
+
+  * If ``Cache-Control`` contains an ``s-maxage`` or ``max-age`` field
+    (in that order of preference), the ttl is set to the respective
+    non-negative value or 0 if negative.
+
+  * Otherwise, if no ``Expires`` header exists, the default ttl is
+    used.
+
+  * Otherwise, if ``Expires`` contains a time stamp before ``Date``,
+    the ttl is set to 0.
+
+  * Otherwise, if no ``Date`` header is present or the ``Date`` header
+    timestamp differs from the local clock by no more than the
+    `clock_skew` parameter, the ttl is set to
+
+    * 0 if ``Expires`` denotes a past timestamp or
+
+    * the difference between the local clock and the ``Expires``
+      header otherwise.
+
+  * Otherwise, the ttl is set to the difference between ``Expires``
+    and ``Date``
+
+* For status codes 302 and 307, the calculation is identical except
+  that the default ttl is not used and -1 is returned if neither
+  ``Cache-Control`` nor ``Expires`` exists.
+
+* For all other status codes, ttl -1 is returned.
+
+``beresp.grace`` defaults to the `default_grace` parameter.
+
+For a non-negative ttl, if ``Cache-Control`` contains a
+``stale-while-revalidate`` field value, ``beresp.grace`` is
+set to that value if non-negative or 0 otherwise.
+
+``beresp.keep`` defaults to the `default_keep` parameter.
+
+
+
 SEE ALSO
 ========
 
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index b614a8428..3d6b67f04 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -28,6 +28,7 @@ subroutines are documented herein.
 A default behavior is provided for all :ref:`reference-states` in the
 :ref:`vcl-built-in-code` code.
 
+
 client side
 -----------
 
@@ -382,76 +383,6 @@ The `vcl_backend_response` subroutine may terminate with calling
     ``beresp.reason`` being preset to the arguments of ``error()`` if
     arguments are provided.
 
-304 handling
-~~~~~~~~~~~~
-
-For a 304 response, Varnish core code amends ``beresp`` before calling
-`vcl_backend_response`:
-
-* If the gzip status changed, ``Content-Encoding`` is unset and any
-  ``Etag`` is weakened
-
-* Any headers not present in the 304 response are copied from the
-  existing cache object. ``Content-Length`` is copied if present in
-  the existing cache object and discarded otherwise.
-
-* The status gets set to 200.
-
-`beresp.was_304` marks that this conditional response processing has
-happened.
-
-Note: Backend conditional requests are independent of client
-conditional requests, so clients may receive 304 responses no matter
-if a backend request was conditional.
-
-beresp.ttl / beresp.grace / beresp.keep
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Before calling `vcl_backend_response`, core code sets ``beresp.ttl``
-based on the response status and the response headers ``Age``,
-``Cache-Control`` or ``Expires`` and ``Date`` as follows:
-
-* If present and valid, the value of the ``Age`` header is effectively
-  deduced from all ttl calculations.
-
-* For status codes 200, 203, 204, 300, 301, 304, 404, 410 and 414:
-
-  * If ``Cache-Control`` contains an ``s-maxage`` or ``max-age`` field
-    (in that order of preference), the ttl is set to the respective
-    non-negative value or 0 if negative.
-
-  * Otherwise, if no ``Expires`` header exists, the default ttl is
-    used.
-
-  * Otherwise, if ``Expires`` contains a time stamp before ``Date``,
-    the ttl is set to 0.
-
-  * Otherwise, if no ``Date`` header is present or the ``Date`` header
-    timestamp differs from the local clock by no more than the
-    `clock_skew` parameter, the ttl is set to
-
-    * 0 if ``Expires`` denotes a past timestamp or
-
-    * the difference between the local clock and the ``Expires``
-      header otherwise.
-
-  * Otherwise, the ttl is set to the difference between ``Expires``
-    and ``Date``
-
-* For status codes 302 and 307, the calculation is identical except
-  that the default ttl is not used and -1 is returned if neither
-  ``Cache-Control`` nor ``Expires`` exists.
-
-* For all other status codes, ttl -1 is returned.
-
-``beresp.grace`` defaults to the `default_grace` parameter.
-
-For a non-negative ttl, if ``Cache-Control`` contains a
-``stale-while-revalidate`` field value, ``beresp.grace`` is
-set to that value if non-negative or 0 otherwise.
-
-``beresp.keep`` defaults to the `default_keep` parameter.
-
 .. _vcl_backend_error:
 
 vcl_backend_error

From nils.goroll at uplex.de  Mon May 15 13:27:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 15 May 2023 13:27:06 +0000 (UTC)
Subject: [master] c65d257ab man: Document built-in subs in a reference manual
Message-ID: <20230515132706.B0381694F@lists.varnish-cache.org>


commit c65d257ab9e7cee5a77a35aa0128f9fce701db1c
Author: Walid Boudebouda <walid.boudebouda at gmail.com>
Date:   Tue Mar 28 16:10:56 2023 +0200

    man: Document built-in subs in a reference manual
    
    Instead of documenting built-in subroutines in the user guide, a new
    vcl-step(7) manual is added. It is structured with a vcl-step.rst file
    that includes a vcl_step.rst file containing the actual list of steps.
    
    This is similar to how the vcl-var(7) manual is constructed to later
    make it authoritative.

diff --git a/doc/sphinx/conf.py.in b/doc/sphinx/conf.py.in
index 30c527a50..a0bb5d40b 100644
--- a/doc/sphinx/conf.py.in
+++ b/doc/sphinx/conf.py.in
@@ -64,7 +64,7 @@ release = '@VERSION@'
 
 # List of directories, relative to source directory, that shouldn't be searched
 # for source files.
-exclude_patterns = ['build','include/*.rst','reference/vcl_var.rst']
+exclude_patterns = ['build','include/*.rst','reference/vcl_*.rst']
 
 # The reST default role (used for this markup: `text`) to use for all documents.
 #default_role = None
diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst
index cb50ea6ff..8eec67414 100644
--- a/doc/sphinx/reference/index.rst
+++ b/doc/sphinx/reference/index.rst
@@ -19,6 +19,7 @@ The VCL language
 
 	VCL - The Varnish Configuration Language <vcl>
 	VCL Variables <vcl-var>
+	VCL Steps <vcl-step>
 	VCL backend configuration <vcl-backend>
 	VCL backend health probe <vcl-probe>
 	states.rst
diff --git a/doc/sphinx/reference/states.rst b/doc/sphinx/reference/states.rst
index b816c25d7..2a4cbbacc 100644
--- a/doc/sphinx/reference/states.rst
+++ b/doc/sphinx/reference/states.rst
@@ -18,7 +18,7 @@ state machines. Whenever a state is entered, a C function is called,
 which in turn calls the appropriate Varnish core code functions to
 process the request or response at this stage. For most states, core
 code also calls into a state-specific function compiled from VCL, a
-VCL subroutine (see :ref:`vcl-built-in-subs` ).
+VCL subroutine (see :ref:`vcl_steps`).
 
 As a general guideline, core code aims to prepare objects accessible
 from VCL with good defaults for the most common cases before calling
diff --git a/doc/sphinx/reference/vcl-step.rst b/doc/sphinx/reference/vcl-step.rst
new file mode 100644
index 000000000..011e84e00
--- /dev/null
+++ b/doc/sphinx/reference/vcl-step.rst
@@ -0,0 +1,63 @@
+..
+	Copyright (c) 2013-2021 Varnish Software AS
+	SPDX-License-Identifier: BSD-2-Clause
+	See LICENSE file for full text of license
+
+
+.. role:: ref(emphasis)
+
+.. _vcl-steps(7):
+
+=========
+VCL-steps
+=========
+
+--------------------
+Built-in subroutines
+--------------------
+
+:Manual section: 7
+
+
+DESCRIPTION
+===========
+
+Various built-in subroutines are called during processing of client
+and backend requests as well as upon ``vcl.load`` and ``vcl.discard``.
+
+See :ref:`reference-states` for a detailed graphical overview of the
+states and how they relate to core code functions and VCL subroutines.
+
+Built-in subroutines always terminate with a ``return (<action>)``,
+where ``<action>`` determines how processing continues in the request
+processing state machine.
+
+The behaviour of actions is identical or at least similar across
+subroutines, so differences are only documented where relevant.
+
+Common actions are documented in
+:ref:`user-guide-vcl_actions`. Actions specific to only one or some
+subroutines are documented herein.
+
+A default behavior is provided for all :ref:`reference-states` in the
+:ref:`vcl-built-in-code` code.
+
+
+.. include:: vcl_step.rst
+
+
+
+SEE ALSO
+========
+
+* :ref:`varnishd(1)`
+* :ref:`vcl(7)`
+
+COPYRIGHT
+=========
+
+This document is licensed under the same license as Varnish
+itself. See LICENSE for details.
+
+* Copyright (c) 2006 Verdens Gang AS
+* Copyright (c) 2006-2021 Varnish Software AS
diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 3770dc626..5143f0ff0 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -445,6 +445,7 @@ SEE ALSO
 * :ref:`varnishd(1)`
 * :ref:`vcl-backend(7)`
 * :ref:`vcl-probe(7)`
+* :ref:`vcl-steps(7)`
 * :ref:`vcl-var(7)`
 * :ref:`vmod_directors(3)`
 * :ref:`vmod_std(3)`
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/reference/vcl_step.rst
similarity index 52%
rename from doc/sphinx/users-guide/vcl-built-in-subs.rst
rename to doc/sphinx/reference/vcl_step.rst
index 3d6b67f04..30d946a9a 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/reference/vcl_step.rst
@@ -1,36 +1,11 @@
-..
-	Copyright (c) 2013-2021 Varnish Software AS
-	SPDX-License-Identifier: BSD-2-Clause
-	See LICENSE file for full text of license
 
-.. _vcl-built-in-subs:
+.. _vcl_steps:
 
-Built-in subroutines
-====================
+VCL Steps
+=========
 
-Various built-in subroutines are called during processing of client-
-and backend requests as well as upon ``vcl.load`` and ``vcl.discard``.
-
-See :ref:`reference-states` for a detailed graphical overview of the
-states and how they relate to core code functions and VCL subroutines.
-
-Built-in subroutines always terminate with a ``return (<action>)``,
-where ``<action>`` determines how processing continues in the request
-processing state machine.
-
-The behaviour of actions is identical or at least similar across
-subroutines, so differences are only documented where relevant.
-
-Common actions are documented in
-:ref:`user-guide-vcl_actions`. Actions specific to only one or some
-subroutines are documented herein.
-
-A default behavior is provided for all :ref:`reference-states` in the
-:ref:`vcl-built-in-code` code.
-
-
-client side
------------
+Client side
+###########
 
 .. _vcl_recv:
 
@@ -48,40 +23,41 @@ be set as a default for the backend processing side.
 The `vcl_recv` subroutine may terminate with calling ``return()`` on one
 of the following keywords:
 
-  ``fail``
-    see :ref:`fail`
-
-  ``synth(status code, reason)``
-    see :ref:`synth`
-
-  ``restart``
-    see :ref:`restart`
-
-  ``pass``
-    see :ref:`pass`
-
-  ``pipe``
-    see :ref:`pipe`
-
-  ``hash``
-    Continue processing the object as a potential candidate for
-    caching. Passes the control over to :ref:`vcl_hash`.
-
-  ``purge``
-    Purge the object and it's variants. Control passes through
-    :ref:`vcl_hash` to :ref:`vcl_purge`.
-
-  ``vcl(label)``
-    Switch to vcl labelled *label*.
-
-    This will roll back the request as if ``std.rollback(req)`` was
-    called and continue vcl processing in :ref:`vcl_recv` of the vcl
-    labelled *label* as if it was the active vcl.
-
-    The ``vcl(label)`` return is only valid while the ``req.restarts``
-    count is zero and if used from the active vcl.
-
-    See the :ref:`ref_cli_vcl_label` command.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``pass``
+  |  see :ref:`pass`
+  |
+  | ``pipe``
+  |  see :ref:`pipe`
+  |
+  | ``hash``
+  |  Continue processing the object as a potential candidate for
+  |  caching. Passes the control over to :ref:`vcl_hash`.
+  |
+  | ``purge``
+  |  Purge the object and it's variants. Control passes through
+  |  :ref:`vcl_hash` to :ref:`vcl_purge`.
+  |
+  | ``vcl(label)``
+  |  Switch to vcl labelled *label*.
+  |
+  |  This will roll back the request as if ``std.rollback(req)`` was
+  |  called and continue vcl processing in :ref:`vcl_recv` of the vcl
+  |  labelled *label* as if it was the active vcl.
+  |
+  |  The ``vcl(label)`` return is only valid while the ``req.restarts``
+  |  count is zero and if used from the active vcl.
+  |
+  |  See the :ref:`ref_cli_vcl_label` command.
 
 .. _vcl_pipe:
 
@@ -98,14 +74,15 @@ other VCL subroutine will ever get called after `vcl_pipe`.
 The `vcl_pipe` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see   :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``pipe``
-    Proceed with pipe mode.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``pipe``
+  |  Proceed with pipe mode.
 
 .. _vcl_pass:
 
@@ -120,17 +97,18 @@ submitted over the same client connection are handled normally.
 The `vcl_pass` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``restart``
-    see  :ref:`restart`
-
-  ``fetch``
-    Proceed with pass mode - initiate a backend request.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``fetch``
+  |  Proceed with pass mode - initiate a backend request.
 
 .. _vcl_hash:
 
@@ -143,26 +121,27 @@ used as a key to look up the object in Varnish.
 The `vcl_hash` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``lookup``
-    Look up the object in cache.
-
-    Control passes to :ref:`vcl_purge` when coming from a ``purge``
-    return in `vcl_recv`.
-
-    Otherwise control passes to the next subroutine depending on the
-    result of the cache lookup:
-
-    * a hit: pass to :ref:`vcl_hit`
-
-    * a miss or a hit on a hit-for-miss object (an object with
-      ``obj.uncacheable == true``): pass to :ref:`vcl_miss`
-
-    * a hit on a hit-for-pass object (for which ``pass(DURATION)`` had been
-      previously returned from ``vcl_backend_response``): pass to
-      :ref:`vcl_pass`
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``lookup``
+  |  Look up the object in cache.
+  |
+  |  Control passes to :ref:`vcl_purge` when coming from a ``purge``
+  |  return in `vcl_recv`.
+  |
+  |  Otherwise control passes to the next subroutine depending on the
+  |  result of the cache lookup:
+  |
+  |  * a hit: pass to :ref:`vcl_hit`
+  |
+  |  * a miss or a hit on a hit-for-miss object (an object with
+  |    ``obj.uncacheable == true``): pass to :ref:`vcl_miss`
+  |
+  |  * a hit on a hit-for-pass object (for which ``pass(DURATION)`` had been
+  |    previously returned from ``vcl_backend_response``): pass to
+  |    :ref:`vcl_pass`
 
 .. _vcl_purge:
 
@@ -174,14 +153,15 @@ Called after the purge has been executed and all its variants have been evicted.
 The `vcl_purge` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``restart``
-    see  :ref:`restart`
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
 
 .. _vcl_miss:
 
@@ -198,21 +178,22 @@ the backend processing side.
 The `vcl_miss` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``restart``
-    see  :ref:`restart`
-
-  ``pass``
-    see  :ref:`pass`
-
-  ``fetch``
-    Retrieve the requested object from the backend. Control will
-    eventually pass to `vcl_backend_fetch`.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``pass``
+  |  see :ref:`pass`
+  |
+  | ``fetch``
+  |  Retrieve the requested object from the backend. Control will
+  |  eventually pass to `vcl_backend_fetch`.
 
 .. _vcl_hit:
 
@@ -226,21 +207,21 @@ stale: It can have a zero or negative `ttl` with only `grace` or
 The `vcl_hit` subroutine may terminate with calling ``return()``
 with one of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``restart``
-    see  :ref:`restart`
-
-  ``pass``
-    see  :ref:`pass`
-
-  ``deliver``
-    Deliver the object. If it is stale, a background fetch to refresh
-    it is triggered.
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``pass``
+  |  see :ref:`pass`
+  |
+  | ``deliver``
+  |  Deliver the object. If it is stale, a background fetch to refresh
+  |  it is triggered.
 
 .. _vcl_deliver:
 
@@ -252,17 +233,18 @@ Called before any object except a `vcl_synth` result is delivered to the client.
 The `vcl_deliver` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``synth(status code, reason)``
-    see  :ref:`synth`
-
-  ``restart``
-    see  :ref:`restart`
-
-  ``deliver``
-    Deliver the object to the client.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``synth(status code, reason)``
+  |  see :ref:`synth`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``deliver``
+  |  Deliver the object to the client.
 
 .. _vcl_synth:
 
@@ -279,18 +261,19 @@ A `vcl_synth` defined object never enters the cache, contrary to a
 The subroutine may terminate with calling ``return()`` with one of the
 following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``restart``
-    see  :ref:`restart`
-
-  ``deliver``
-    Directly deliver the object defined by `vcl_synth` to the client
-    without calling `vcl_deliver`.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``restart``
+  |  see :ref:`restart`
+  |
+  | ``deliver``
+  |  Directly deliver the object defined by `vcl_synth` to the client
+  |  without calling `vcl_deliver`.
 
 Backend Side
-------------
+############
 
 .. _vcl_backend_fetch:
 
@@ -303,19 +286,20 @@ typically alter the request before it gets to the backend.
 The `vcl_backend_fetch` subroutine may terminate with calling
 ``return()`` with one of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``abandon``
-    see  :ref:`abandon`
-
-  ``fetch``
-    Fetch the object from the backend.
-
-  ``error(status code, reason)``
-    Transition to :ref:`vcl_backend_error` with ``beresp.status`` and
-    ``beresp.reason`` being preset to the arguments of ``error()`` if
-    arguments are provided.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``abandon``
+  |  see :ref:`abandon`
+  |
+  | ``fetch``
+  |  Fetch the object from the backend.
+  |
+  | ``error(status code, reason)``
+  |  Transition to :ref:`vcl_backend_error` with ``beresp.status`` and
+  |  ``beresp.reason`` being preset to the arguments of ``error()`` if
+  |  arguments are provided.
 
 Before calling `vcl_backend_fetch`, Varnish core prepares the `bereq`
 backend request as follows:
@@ -356,32 +340,33 @@ the backend.
 The `vcl_backend_response` subroutine may terminate with calling
 ``return()`` with one of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``abandon``
-    see  :ref:`abandon`
-
-  ``deliver``
-    For a 304 response, create an updated cache object.
-    Otherwise, fetch the object body from the backend and initiate
-    delivery to any waiting client requests, possibly in parallel
-    (streaming).
-
-  ``retry``
-    Retry the backend transaction. Increases the `retries` counter.
-    If the number of retries is higher than *max_retries*,
-    control will be passed to :ref:`vcl_backend_error`.
-
-  ``pass(duration)``
-    Mark the object as a hit-for-pass for the given duration. Subsequent
-    lookups hitting this object will be turned into passed transactions,
-    as if ``vcl_recv`` had returned ``pass``.
-
-  ``error(status code, reason)``
-    Transition to :ref:`vcl_backend_error` with ``beresp.status`` and
-    ``beresp.reason`` being preset to the arguments of ``error()`` if
-    arguments are provided.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``abandon``
+  |  see :ref:`abandon`
+  |
+  | ``deliver``
+  |  For a 304 response, create an updated cache object.
+  |  Otherwise, fetch the object body from the backend and initiate
+  |  delivery to any waiting client requests, possibly in parallel
+  |  (streaming).
+  |
+  | ``retry``
+  |  Retry the backend transaction. Increases the `retries` counter.
+  |  If the number of retries is higher than *max_retries*,
+  |  control will be passed to :ref:`vcl_backend_error`.
+  |
+  | ``pass(duration)``
+  |  Mark the object as a hit-for-pass for the given duration. Subsequent
+  |  lookups hitting this object will be turned into passed transactions,
+  |  as if ``vcl_recv`` had returned ``pass``.
+  |
+  | ``error(status code, reason)``
+  |  Transition to :ref:`vcl_backend_error` with ``beresp.status`` and
+  |  ``beresp.reason`` being preset to the arguments of ``error()`` if
+  |  arguments are provided.
 
 .. _vcl_backend_error:
 
@@ -410,25 +395,26 @@ zero in ``vcl_backend_error``.
 The `vcl_backend_error` subroutine may terminate with calling ``return()``
 with one of the following keywords:
 
-  ``fail``
-    see  :ref:`fail`
-
-  ``abandon``
-    see  :ref:`abandon`
-
-  ``deliver``
-    Deliver and possibly cache the object defined in
-    `vcl_backend_error` **as if it was fetched from the backend**, also
-    referred to as a "backend synth".
-
-  ``retry``
-    Retry the backend transaction. Increases the `retries` counter.
-    If the number of retries is higher than *max_retries*,
-    :ref:`vcl_synth` on the client side is called with ``resp.status``
-    preset to 503.
+  |
+  | ``fail``
+  |  see :ref:`fail`
+  |
+  | ``abandon``
+  |  see :ref:`abandon`
+  |
+  | ``deliver``
+  |  Deliver and possibly cache the object defined in
+  |  `vcl_backend_error` **as if it was fetched from the backend**, also
+  |  referred to as a "backend synth".
+  |
+  | ``retry``
+  |  Retry the backend transaction. Increases the `retries` counter.
+  |  If the number of retries is higher than *max_retries*,
+  |  :ref:`vcl_synth` on the client side is called with ``resp.status``
+  |  preset to 503.
 
 During vcl.load / vcl.discard
------------------------------
+#############################
 
 .. _vcl_init:
 
@@ -441,11 +427,12 @@ Typically used to initialize VMODs.
 The `vcl_init` subroutine may terminate with calling ``return()``
 with one of the following keywords:
 
-  ``ok``
-    Normal return, VCL continues loading.
-
-  ``fail``
-    Abort loading of this VCL.
+  |
+  | ``ok``
+  |  Normal return, VCL continues loading.
+  |
+  | ``fail``
+  |  Abort loading of this VCL.
 
 .. _vcl_fini:
 
@@ -458,5 +445,6 @@ Typically used to clean up VMODs.
 The `vcl_fini` subroutine may terminate with calling ``return()``
 with one of the following keywords:
 
-  ``ok``
-    Normal return, VCL will be discarded.
+  |
+  | ``ok``
+  |  Normal return, VCL will be discarded.
diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
index 5eb13c799..b51b1bf92 100644
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ b/doc/sphinx/users-guide/vcl-actions.rst
@@ -14,7 +14,7 @@ processing in varnish continues as shown in :ref:`reference-states`.
 
 Common actions are documented here, while additional actions specific
 to only one or some subroutines are documented in
-:ref:`vcl-built-in-subs` as well as which action can be used from
+:ref:`vcl-steps(7)` as well as which action can be used from
 which built in subroutine.
 
 common actions for the client and backend side
diff --git a/doc/sphinx/users-guide/vcl-built-in-code.rst b/doc/sphinx/users-guide/vcl-built-in-code.rst
index 2bd72ed50..59383e4ef 100644
--- a/doc/sphinx/users-guide/vcl-built-in-code.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-code.rst
@@ -4,7 +4,7 @@ Built-in VCL
 ============
 
 Whenever a VCL program is loaded, the built-in VCL is appended to it. The
-:ref:`vcl-built-in-subs` have a special property, they can appear multiple
+vcl built-in subs (:ref:`vcl_steps`) have a special property, they can appear multiple
 times and the result is concatenation of all built-in subroutines.
 
 For example, let's take the following snippet::
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index 7ef4dd7e7..10748596a 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -99,7 +99,7 @@ Built in subroutines
 
 Varnish has quite a few built-in subroutines that are called for each
 transaction as it flows through Varnish. These built-in subroutines are
-all named ``vcl_*`` and are explained in :ref:`vcl-built-in-subs`.
+all named ``vcl_*`` and are explained in :ref:`vcl_steps`.
 
 Processing in built-in subroutines ends with ``return (<action>)``
 (see :ref:`user-guide-vcl_actions`).
@@ -130,7 +130,7 @@ values.
 
 ``return (<action>)`` (see :ref:`user-guide-vcl_actions`) as shown in
 the example above returns all the way from the top level built in
-subroutine (see :ref:`vcl-built-in-subs`) which, possibly through
+subroutine (see :ref:`vcl_steps`) which, possibly through
 multiple steps, lead to the call of the custom subroutine.
 
 ``return`` without an action resumes execution after the ``call``
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index 831a24071..2647a6491 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -40,7 +40,6 @@ code commented out in the file `builtin.vcl` that ships with Varnish Cache.
    :maxdepth: 1
 
    vcl-syntax
-   vcl-built-in-subs
    vcl-built-in-code
    vcl-variables
    vcl-actions
diff --git a/man/Makefile.am b/man/Makefile.am
index b78dc7cfe..4b89fce68 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -7,6 +7,7 @@ dist_man_MANS = \
 	vcl-backend.7 \
 	vcl-probe.7 \
 	vcl-var.7 \
+	vcl-step.7 \
 	vsl.7 \
 	vsl-query.7 \
 	varnishadm.1 \
@@ -60,6 +61,10 @@ vcl-probe.7: $(top_builddir)/doc/sphinx/reference/vcl-probe.rst
 vcl-var.7: $(top_builddir)/doc/sphinx/reference/vcl-var.rst
 	$(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vcl-var.rst $@
 
+vcl-step.7: $(top_builddir)/doc/sphinx/reference/vcl-step.rst \
+	$(top_builddir)/doc/sphinx/reference/vcl_step.rst
+	$(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vcl-step.rst $@
+
 vsl.7: $(top_builddir)/doc/sphinx/reference/vsl.rst \
 	$(top_builddir)/doc/sphinx/include/vsl-tags.rst
 	$(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vsl.rst $@

From nils.goroll at uplex.de  Mon May 15 13:27:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 15 May 2023 13:27:06 +0000 (UTC)
Subject: [master] 88025e951 man: Fold vcl-actions.rst into vcl-step.rst
Message-ID: <20230515132706.D63C26953@lists.varnish-cache.org>


commit 88025e9515d239a0770be736a9280f2deafe0c83
Author: Walid Boudebouda <walid.boudebouda at gmail.com>
Date:   Thu Mar 30 14:32:51 2023 +0200

    man: Fold vcl-actions.rst into vcl-step.rst
    
    This also belongs in a reference manual and should not be disjoint. It
    also avoids broken references with rst2man that only sphinx can resolve
    across multiple pages.

diff --git a/doc/sphinx/reference/vcl-step.rst b/doc/sphinx/reference/vcl-step.rst
index 011e84e00..6deff971b 100644
--- a/doc/sphinx/reference/vcl-step.rst
+++ b/doc/sphinx/reference/vcl-step.rst
@@ -36,12 +36,97 @@ The behaviour of actions is identical or at least similar across
 subroutines, so differences are only documented where relevant.
 
 Common actions are documented in
-:ref:`user-guide-vcl_actions`. Actions specific to only one or some
-subroutines are documented herein.
+:ref:`vcl_actions` in the next section. Actions specific
+to only one or some subroutines are documented in :ref:`vcl_steps`.
 
 A default behavior is provided for all :ref:`reference-states` in the
 :ref:`vcl-built-in-code` code.
 
+.. _vcl_actions:
+
+VCL Actions
+===========
+
+Actions are used with the ``return(<action>)`` keyword, which returns
+control from subroutines back to varnish. The action determines how
+processing in varnish continues as shown in :ref:`reference-states`.
+
+Common actions are documented here, while additional actions specific
+to only one or some subroutines are documented in the next section
+:ref:`vcl_steps` as well as which action can be used from which built
+in subroutine.
+
+Common actions for the client and backend side
+##############################################
+
+.. _fail:
+
+``fail``
+~~~~~~~~
+
+    Transition to :ref:`vcl_synth` on the client side as for
+    ``return(synth(503, "VCL Failed"))``, but with any request state
+    changes undone as if ``std.rollback()`` was called and forcing a
+    connection close.
+
+    Intended for fatal errors, for which only minimal error handling is
+    possible.
+
+Common actions for the client side
+##################################
+
+.. _synth:
+
+``synth(status code, reason)``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    Transition to :ref:`vcl_synth` with ``resp.status`` and
+    ``resp.reason`` being preset to the arguments of ``synth()``.
+
+.. _pass:
+
+``pass``
+~~~~~~~~
+
+    Switch to pass mode, making the current request not use the cache
+    and not putting its response into it. Control will eventually pass to
+    :ref:`vcl_pass`.
+
+.. _pipe:
+
+``pipe``
+~~~~~~~~
+
+    Switch to pipe mode. Control will eventually pass to
+    :ref:`vcl_pipe`.
+
+.. _restart:
+
+``restart``
+~~~~~~~~~~~
+
+    Restart the transaction. Increases the ``req.restarts`` counter.
+
+    If the number of restarts is higher than the *max_restarts*
+    parameter, control is passed to :ref:`vcl_synth` as for
+    ``return(synth(503, "Too many restarts"))``
+
+    For a restart, all modifications to ``req`` attributes are
+    preserved except for ``req.restarts`` and ``req.xid``, which need
+    to change by design.
+
+Common actions for the backend side
+###################################
+
+.. _abandon:
+
+``abandon``
+~~~~~~~~~~~
+
+    Abandon the backend request. Unless the backend request was a
+    background fetch, control is passed to :ref:`vcl_synth` on the
+    client side with ``resp.status`` preset to 503.
+
 
 .. include:: vcl_step.rst
 
diff --git a/doc/sphinx/reference/vcl_step.rst b/doc/sphinx/reference/vcl_step.rst
index 30d946a9a..ee07f6169 100644
--- a/doc/sphinx/reference/vcl_step.rst
+++ b/doc/sphinx/reference/vcl_step.rst
@@ -25,19 +25,19 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``pass``
-  |  see :ref:`pass`
+  |  see :ref:`pass` section above
   |
   | ``pipe``
-  |  see :ref:`pipe`
+  |  see :ref:`pipe` section above
   |
   | ``hash``
   |  Continue processing the object as a potential candidate for
@@ -57,7 +57,7 @@ of the following keywords:
   |  The ``vcl(label)`` return is only valid while the ``req.restarts``
   |  count is zero and if used from the active vcl.
   |
-  |  See the :ref:`ref_cli_vcl_label` command.
+  |  See the :ref:`ref_cli_vcl_label` command in :ref:`varnish-cli(7)`.
 
 .. _vcl_pipe:
 
@@ -76,10 +76,10 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``pipe``
   |  Proceed with pipe mode.
@@ -99,13 +99,13 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``fetch``
   |  Proceed with pass mode - initiate a backend request.
@@ -123,7 +123,7 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see  :ref:`fail` section above
   |
   | ``lookup``
   |  Look up the object in cache.
@@ -155,13 +155,13 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
 
 .. _vcl_miss:
 
@@ -180,16 +180,16 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``pass``
-  |  see :ref:`pass`
+  |  see :ref:`pass` section above
   |
   | ``fetch``
   |  Retrieve the requested object from the backend. Control will
@@ -208,16 +208,16 @@ The `vcl_hit` subroutine may terminate with calling ``return()``
 with one of the following keywords:
 
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``pass``
-  |  see :ref:`pass`
+  |  see :ref:`pass` section above
   |
   | ``deliver``
   |  Deliver the object. If it is stale, a background fetch to refresh
@@ -235,13 +235,13 @@ of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``synth(status code, reason)``
-  |  see :ref:`synth`
+  |  see :ref:`synth` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``deliver``
   |  Deliver the object to the client.
@@ -263,10 +263,10 @@ following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``restart``
-  |  see :ref:`restart`
+  |  see :ref:`restart` section above
   |
   | ``deliver``
   |  Directly deliver the object defined by `vcl_synth` to the client
@@ -288,10 +288,10 @@ The `vcl_backend_fetch` subroutine may terminate with calling
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``abandon``
-  |  see :ref:`abandon`
+  |  see :ref:`abandon` section above
   |
   | ``fetch``
   |  Fetch the object from the backend.
@@ -342,10 +342,10 @@ The `vcl_backend_response` subroutine may terminate with calling
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``abandon``
-  |  see :ref:`abandon`
+  |  see :ref:`abandon` section above
   |
   | ``deliver``
   |  For a 304 response, create an updated cache object.
@@ -397,10 +397,10 @@ with one of the following keywords:
 
   |
   | ``fail``
-  |  see :ref:`fail`
+  |  see :ref:`fail` section above
   |
   | ``abandon``
-  |  see :ref:`abandon`
+  |  see :ref:`abandon` section above
   |
   | ``deliver``
   |  Deliver and possibly cache the object defined in
diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 8c49e5a8b..88f166204 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -222,7 +222,7 @@ Cookies coming from the backend
 
 If your backend server sets a cookie using the 'Set-Cookie' header
 Varnish will not cache the page when using the default configuration.
-A `hit-for-miss` object (see :ref:`user-guide-vcl_actions`) is
+A `hit-for-miss` object (see :ref:`vcl_actions`) is
 created.  So, if the backend server acts silly and sets unwanted
 cookies just unset the 'Set-Cookie' header and all should be fine.
 
diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
deleted file mode 100644
index b51b1bf92..000000000
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ /dev/null
@@ -1,89 +0,0 @@
-..
-	Copyright (c) 2012-2019 Varnish Software AS
-	SPDX-License-Identifier: BSD-2-Clause
-	See LICENSE file for full text of license
-
-.. _user-guide-vcl_actions:
-
-Actions
-=======
-
-Actions are used with the ``return(<action>)`` keyword, which returns
-control from subroutines back to varnish. The action determines how
-processing in varnish continues as shown in :ref:`reference-states`.
-
-Common actions are documented here, while additional actions specific
-to only one or some subroutines are documented in
-:ref:`vcl-steps(7)` as well as which action can be used from
-which built in subroutine.
-
-common actions for the client and backend side
-----------------------------------------------
-
-.. _fail:
-
-``fail``
-~~~~~~~~
-
-    Transition to :ref:`vcl_synth` on the client side as for
-    ``return(synth(503, "VCL Failed"))``, but with any request state
-    changes undone as if ``std.rollback()`` was called and forcing a
-    connection close.
-
-    Intended for fatal errors, for which only minimal error handling is
-    possible.
-
-common actions for the client side
-----------------------------------
-
-.. _synth:
-
-``synth(status code, reason)``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-    Transition to :ref:`vcl_synth` with ``resp.status`` and
-    ``resp.reason`` being preset to the arguments of ``synth()``.
-
-.. _pass:
-
-``pass``
-~~~~~~~~
-
-    Switch to pass mode, making the current request not use the cache
-    and not putting its response into it. Control will eventually pass to
-    :ref:`vcl_pass`.
-
-.. _pipe:
-
-``pipe``
-~~~~~~~~
-
-    Switch to pipe mode. Control will eventually pass to
-    :ref:`vcl_pipe`.
-
-.. _restart:
-
-``restart``
-~~~~~~~~~~~
-
-    Restart the transaction. Increases the ``req.restarts`` counter.
-
-    If the number of restarts is higher than the *max_restarts*
-    parameter, control is passed to :ref:`vcl_synth` as for
-    ``return(synth(503, "Too many restarts"))``
-
-    For a restart, all modifications to ``req`` attributes are
-    preserved except for ``req.restarts`` and ``req.xid``, which need
-    to change by design.
-
-common actions for the backend side
------------------------------------
-
-.. _abandon:
-
-``abandon``
-~~~~~~~~~~~
-
-    Abandon the backend request. Unless the backend request was a
-    background fetch, control is passed to :ref:`vcl_synth` on the
-    client side with ``resp.status`` preset to 503.
diff --git a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
index 8edf0d193..59bc12bcf 100644
--- a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
+++ b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
@@ -21,4 +21,4 @@ matches certain criteria::
 
 
 We also remove any Set-Cookie headers in order to avoid creation of a
-`hit-for-miss` object. See :ref:`user-guide-vcl_actions`.
+`hit-for-miss` object. See :ref:`vcl_actions`.
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index 10748596a..3d9d819ec 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -102,7 +102,7 @@ transaction as it flows through Varnish. These built-in subroutines are
 all named ``vcl_*`` and are explained in :ref:`vcl_steps`.
 
 Processing in built-in subroutines ends with ``return (<action>)``
-(see :ref:`user-guide-vcl_actions`).
+(see :ref:`vcl_actions`).
 
 The :ref:`vcl-built-in-code` also contains custom assistant subroutines
 called by the built-in subroutines, also prefixed with ``vcl_``.
@@ -128,7 +128,7 @@ subroutine's name::
 Custom subroutines in VCL do not take arguments, nor do they return
 values.
 
-``return (<action>)`` (see :ref:`user-guide-vcl_actions`) as shown in
+``return (<action>)`` (see :ref:`vcl_actions`) as shown in
 the example above returns all the way from the top level built in
 subroutine (see :ref:`vcl_steps`) which, possibly through
 multiple steps, lead to the call of the custom subroutine.
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index 2647a6491..d74898358 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -42,7 +42,6 @@ code commented out in the file `builtin.vcl` that ships with Varnish Cache.
    vcl-syntax
    vcl-built-in-code
    vcl-variables
-   vcl-actions
    vcl-backends
    vcl-hashing
    vcl-grace

From nils.goroll at uplex.de  Mon May 15 13:27:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 15 May 2023 13:27:06 +0000 (UTC)
Subject: [master] 4b8d17603 vcc: Parse VCL steps and return actions from
 vcl_step.rst
Message-ID: <20230515132707.04A3D6958@lists.varnish-cache.org>


commit 4b8d17603659fee69afece028495f3c863f3c210
Author: Walid Boudebouda <walid.boudebouda at gmail.com>
Date:   Tue Mar 28 17:34:51 2023 +0200

    vcc: Parse VCL steps and return actions from vcl_step.rst
    
    With this, the documentation becomes authoritative for the VCL state
    machine, just like VCL variables.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 08187174e..ab35bd7b5 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -83,75 +83,40 @@ tokens = {
 #######################################################################
 # Our methods and actions
 
-returns = (
-    ###############################################################
-    # Client side
-
-    ('recv',
-     "C",
-     ('fail', 'synth', 'restart', 'pass', 'pipe', 'hash', 'purge', 'vcl')
-    ),
-    ('pipe',
-     "C",
-     ('fail', 'synth', 'pipe',)
-    ),
-    ('pass',
-     "C",
-     ('fail', 'synth', 'restart', 'fetch',)
-    ),
-    ('hash',
-     "C",
-     ('fail', 'lookup',)
-    ),
-    ('purge',
-     "C",
-     ('fail', 'synth', 'restart',)
-    ),
-    ('miss',
-     "C",
-     ('fail', 'synth', 'restart', 'pass', 'fetch',)
-    ),
-    ('hit',
-     "C",
-     ('fail', 'synth', 'restart', 'pass', 'deliver',)
-    ),
-    ('deliver',
-     "C",
-     ('fail', 'synth', 'restart', 'deliver',)
-    ),
-    ('synth',
-     "C",
-     ('fail', 'restart', 'deliver',)
-    ),
-
-    ###############################################################
-    # Backend-fetch
-
-    ('backend_fetch',
-     "B",
-     ('fail', 'fetch', 'abandon', 'error')
-    ),
-    ('backend_response',
-     "B",
-     ('fail', 'deliver', 'retry', 'abandon', 'pass', 'error')
-    ),
-    ('backend_error',
-     "B",
-     ('fail', 'deliver', 'retry', 'abandon')
-    ),
-
-    ###############################################################
-    # Housekeeping
-
-    ('init',
-     "H",
-     ('ok', 'fail')
-    ),
-    ('fini',
-     "H",
-     ('ok',)
-    ),
-)
+returns = []
+
+def parse_subs(ln, c):
+    ctx = ['C', 'B', 'H']
+    rets = []
+    for i in ln[2:]:
+        if (len(i) > 4 and i.startswith('  | ``') and i.endswith('``')):
+            rets.append(i[6:-2].split('(')[0])
+    returns.append((ln[0][4:], ctx[c], rets))
+
+def parse_subs_doc(fn):
+    l = []
+    j = -1
+    start = False
+    for i in open(fn):
+        if i.startswith('##########'):
+            j += 1
+            l.append([])
+            start = True
+            continue
+        if start:
+            l[j].append(i.rstrip())
+    for i in range(0, len(l)):
+        for n in range(0, len(l[i])):
+            if len(l[i][n]) < 5 or not l[i][n].startswith("~~~~~"):
+                continue
+            m = n + 1
+            while m < len(l[i]) and (len(l[i][m]) < 5 or not l[i][m].startswith("~~~~~")):
+                m += 1
+            if m != len(l[i]):
+                m -= 2
+            parse_subs(l[i][n-1:m], i)
+
+parse_subs_doc(join(srcroot, "doc/sphinx/reference/vcl_step.rst"))
 
 #######################################################################
 # Variables available in sessions

From dridi.boukelmoune at gmail.com  Mon May 15 14:56:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 14:56:06 +0000 (UTC)
Subject: [master] edc2f1f8b esi: Original typo
Message-ID: <20230515145606.5ECCE100F3E@lists.varnish-cache.org>


commit edc2f1f8b5299d2860f7fd289191b5d497079405
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon May 15 16:53:19 2023 +0200

    esi: Original typo

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 762e42ae2..6ebb063bb 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -645,7 +645,7 @@ ved_gzgz_init(VRT_CTX, struct vdp_ctx *vdc, void **priv, struct objcore *oc)
  * in VDP_bytes(), but ved_bytes() covers it.
  *
  * To avoid unnecessary chunks downstream, it would be nice to re-structure the
- * code to intendify the last block, send VDP_END/VDP_FLUSH for that one and
+ * code to identify the last block, send VDP_END/VDP_FLUSH for that one and
  * VDP_NULL for anything before it.
  */
 

From dridi.boukelmoune at gmail.com  Mon May 15 16:39:07 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:39:07 +0000 (UTC)
Subject: [master] 8937dae96 build: Install vmod_*.vcc files
Message-ID: <20230515163907.181251041B2@lists.varnish-cache.org>


commit 8937dae9629866954f5260861681a363ebba3bad
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Mar 29 16:13:23 2023 +0200

    build: Install vmod_*.vcc files
    
    This change adds a new /usr/share/varnish/vcc (or equivalent) directory
    where the built-in VMOD descriptors are installed. Only vmod_debug.vcc
    is omitted since vmod_debug is not installed. Shipping VCC files can be
    useful to end users, for example, to implement VCL autocompletion in a
    text editor.
    
    This change does not even begin to manage this for third-party VMODs.

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 270e64cb5..884ad4fa5 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -51,6 +51,10 @@ import time
 AMBOILERPLATE = '''\
 # Generated by vmodtool.py --boilerplate.
 
+vmod_XXX_vcc = $(srcdir)/VCC
+
+vmod_vcc_files += $(vmod_XXX_vcc)
+
 vmod_LTLIBRARIES += libvmod_XXX.la
 
 libvmod_XXX_la_SOURCES = \\
@@ -70,7 +74,7 @@ nodist_libvmod_XXX_la_SOURCES = PFX.c PFX.h
 
 EXTRA_libvmod_XXX_la_DEPENDENCIES = $(nodist_libvmod_XXX_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/VCC automake_boilerplate_XXX.am
+EXTRA_DIST += automake_boilerplate_XXX.am
 
 $(libvmod_XXX_la_OBJECTS): PFX.h
 
diff --git a/vmod/Makefile.am b/vmod/Makefile.am
index 197508b28..afe6614a0 100644
--- a/vmod/Makefile.am
+++ b/vmod/Makefile.am
@@ -17,7 +17,10 @@ AM_CPPFLAGS = \
 # Use vmodtool.py generated automake boilerplate
 #
 
+vccdir=$(datarootdir)/$(PACKAGE)/vcc
+
 vmod_LTLIBRARIES =
+vmod_vcc_files =
 
 include $(srcdir)/automake_boilerplate_blob.am
 include $(srcdir)/automake_boilerplate_cookie.am
@@ -51,3 +54,9 @@ vmod_debug_symbols_regex = 'Vmod_.*_Data'
 
 # not --strict
 vmodtoolargs_debug = --boilerplate -o vcc_debug_if
+
+# unregister VCC file installation
+dist_noinst_DATA := $(srcdir)/vmod_debug.vcc
+vmod_debug_vcc =
+
+dist_vcc_DATA = $(vmod_vcc_files)
diff --git a/vmod/automake_boilerplate_blob.am b/vmod/automake_boilerplate_blob.am
index c49d42a6f..1b7c375a5 100644
--- a/vmod/automake_boilerplate_blob.am
+++ b/vmod/automake_boilerplate_blob.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_blob_vcc = $(srcdir)/vmod_blob.vcc
+
+vmod_vcc_files += $(vmod_blob_vcc)
+
 vmod_LTLIBRARIES += libvmod_blob.la
 
 libvmod_blob_la_SOURCES = \
@@ -26,7 +30,7 @@ nodist_libvmod_blob_la_SOURCES = vcc_blob_if.c vcc_blob_if.h
 
 EXTRA_libvmod_blob_la_DEPENDENCIES = $(nodist_libvmod_blob_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_blob.vcc automake_boilerplate_blob.am
+EXTRA_DIST += automake_boilerplate_blob.am
 
 $(libvmod_blob_la_OBJECTS): vcc_blob_if.h
 
diff --git a/vmod/automake_boilerplate_cookie.am b/vmod/automake_boilerplate_cookie.am
index fa3ccfb7f..1cb5c9f46 100644
--- a/vmod/automake_boilerplate_cookie.am
+++ b/vmod/automake_boilerplate_cookie.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_cookie_vcc = $(srcdir)/vmod_cookie.vcc
+
+vmod_vcc_files += $(vmod_cookie_vcc)
+
 vmod_LTLIBRARIES += libvmod_cookie.la
 
 libvmod_cookie_la_SOURCES = \
@@ -19,7 +23,7 @@ nodist_libvmod_cookie_la_SOURCES = vcc_cookie_if.c vcc_cookie_if.h
 
 EXTRA_libvmod_cookie_la_DEPENDENCIES = $(nodist_libvmod_cookie_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_cookie.vcc automake_boilerplate_cookie.am
+EXTRA_DIST += automake_boilerplate_cookie.am
 
 $(libvmod_cookie_la_OBJECTS): vcc_cookie_if.h
 
diff --git a/vmod/automake_boilerplate_debug.am b/vmod/automake_boilerplate_debug.am
index e97ac77da..b5c34f75e 100644
--- a/vmod/automake_boilerplate_debug.am
+++ b/vmod/automake_boilerplate_debug.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_debug_vcc = $(srcdir)/vmod_debug.vcc
+
+vmod_vcc_files += $(vmod_debug_vcc)
+
 vmod_LTLIBRARIES += libvmod_debug.la
 
 libvmod_debug_la_SOURCES = \
@@ -22,7 +26,7 @@ nodist_libvmod_debug_la_SOURCES = vcc_debug_if.c vcc_debug_if.h
 
 EXTRA_libvmod_debug_la_DEPENDENCIES = $(nodist_libvmod_debug_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_debug.vcc automake_boilerplate_debug.am
+EXTRA_DIST += automake_boilerplate_debug.am
 
 $(libvmod_debug_la_OBJECTS): vcc_debug_if.h
 
diff --git a/vmod/automake_boilerplate_directors.am b/vmod/automake_boilerplate_directors.am
index 6cec72ead..ed44678ac 100644
--- a/vmod/automake_boilerplate_directors.am
+++ b/vmod/automake_boilerplate_directors.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_directors_vcc = $(srcdir)/vmod_directors.vcc
+
+vmod_vcc_files += $(vmod_directors_vcc)
+
 vmod_LTLIBRARIES += libvmod_directors.la
 
 libvmod_directors_la_SOURCES = \
@@ -29,7 +33,7 @@ nodist_libvmod_directors_la_SOURCES = vcc_directors_if.c vcc_directors_if.h
 
 EXTRA_libvmod_directors_la_DEPENDENCIES = $(nodist_libvmod_directors_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_directors.vcc automake_boilerplate_directors.am
+EXTRA_DIST += automake_boilerplate_directors.am
 
 $(libvmod_directors_la_OBJECTS): vcc_directors_if.h
 
diff --git a/vmod/automake_boilerplate_proxy.am b/vmod/automake_boilerplate_proxy.am
index 24be960ea..61b54e785 100644
--- a/vmod/automake_boilerplate_proxy.am
+++ b/vmod/automake_boilerplate_proxy.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_proxy_vcc = $(srcdir)/vmod_proxy.vcc
+
+vmod_vcc_files += $(vmod_proxy_vcc)
+
 vmod_LTLIBRARIES += libvmod_proxy.la
 
 libvmod_proxy_la_SOURCES = \
@@ -19,7 +23,7 @@ nodist_libvmod_proxy_la_SOURCES = vcc_proxy_if.c vcc_proxy_if.h
 
 EXTRA_libvmod_proxy_la_DEPENDENCIES = $(nodist_libvmod_proxy_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_proxy.vcc automake_boilerplate_proxy.am
+EXTRA_DIST += automake_boilerplate_proxy.am
 
 $(libvmod_proxy_la_OBJECTS): vcc_proxy_if.h
 
diff --git a/vmod/automake_boilerplate_purge.am b/vmod/automake_boilerplate_purge.am
index 7d2186d11..de5cbc6c4 100644
--- a/vmod/automake_boilerplate_purge.am
+++ b/vmod/automake_boilerplate_purge.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_purge_vcc = $(srcdir)/vmod_purge.vcc
+
+vmod_vcc_files += $(vmod_purge_vcc)
+
 vmod_LTLIBRARIES += libvmod_purge.la
 
 libvmod_purge_la_SOURCES = \
@@ -19,7 +23,7 @@ nodist_libvmod_purge_la_SOURCES = vcc_purge_if.c vcc_purge_if.h
 
 EXTRA_libvmod_purge_la_DEPENDENCIES = $(nodist_libvmod_purge_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_purge.vcc automake_boilerplate_purge.am
+EXTRA_DIST += automake_boilerplate_purge.am
 
 $(libvmod_purge_la_OBJECTS): vcc_purge_if.h
 
diff --git a/vmod/automake_boilerplate_std.am b/vmod/automake_boilerplate_std.am
index 4900e4471..b4f62fde6 100644
--- a/vmod/automake_boilerplate_std.am
+++ b/vmod/automake_boilerplate_std.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_std_vcc = $(srcdir)/vmod_std.vcc
+
+vmod_vcc_files += $(vmod_std_vcc)
+
 vmod_LTLIBRARIES += libvmod_std.la
 
 libvmod_std_la_SOURCES = \
@@ -22,7 +26,7 @@ nodist_libvmod_std_la_SOURCES = vcc_std_if.c vcc_std_if.h
 
 EXTRA_libvmod_std_la_DEPENDENCIES = $(nodist_libvmod_std_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_std.vcc automake_boilerplate_std.am
+EXTRA_DIST += automake_boilerplate_std.am
 
 $(libvmod_std_la_OBJECTS): vcc_std_if.h
 
diff --git a/vmod/automake_boilerplate_unix.am b/vmod/automake_boilerplate_unix.am
index ab9ca4bca..aff2650f4 100644
--- a/vmod/automake_boilerplate_unix.am
+++ b/vmod/automake_boilerplate_unix.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_unix_vcc = $(srcdir)/vmod_unix.vcc
+
+vmod_vcc_files += $(vmod_unix_vcc)
+
 vmod_LTLIBRARIES += libvmod_unix.la
 
 libvmod_unix_la_SOURCES = \
@@ -20,7 +24,7 @@ nodist_libvmod_unix_la_SOURCES = vcc_unix_if.c vcc_unix_if.h
 
 EXTRA_libvmod_unix_la_DEPENDENCIES = $(nodist_libvmod_unix_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_unix.vcc automake_boilerplate_unix.am
+EXTRA_DIST += automake_boilerplate_unix.am
 
 $(libvmod_unix_la_OBJECTS): vcc_unix_if.h
 
diff --git a/vmod/automake_boilerplate_vtc.am b/vmod/automake_boilerplate_vtc.am
index cae60718d..202ad2257 100644
--- a/vmod/automake_boilerplate_vtc.am
+++ b/vmod/automake_boilerplate_vtc.am
@@ -1,5 +1,9 @@
 # Generated by vmodtool.py --boilerplate.
 
+vmod_vtc_vcc = $(srcdir)/vmod_vtc.vcc
+
+vmod_vcc_files += $(vmod_vtc_vcc)
+
 vmod_LTLIBRARIES += libvmod_vtc.la
 
 libvmod_vtc_la_SOURCES = \
@@ -19,7 +23,7 @@ nodist_libvmod_vtc_la_SOURCES = vcc_vtc_if.c vcc_vtc_if.h
 
 EXTRA_libvmod_vtc_la_DEPENDENCIES = $(nodist_libvmod_vtc_la_SOURCES)
 
-EXTRA_DIST += $(srcdir)/vmod_vtc.vcc automake_boilerplate_vtc.am
+EXTRA_DIST += automake_boilerplate_vtc.am
 
 $(libvmod_vtc_la_OBJECTS): vcc_vtc_if.h
 

From dridi.boukelmoune at gmail.com  Mon May 15 16:39:07 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:39:07 +0000 (UTC)
Subject: [master] f32bc1d0e SQUASHME: keyboard accident
Message-ID: <20230515163907.2FE221041B5@lists.varnish-cache.org>


commit f32bc1d0eddab4acd4ab317cb07d703e0a087db9
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Mar 29 18:02:48 2023 +0200

    SQUASHME: keyboard accident

diff --git a/vmod/Makefile.am b/vmod/Makefile.am
index afe6614a0..b50458265 100644
--- a/vmod/Makefile.am
+++ b/vmod/Makefile.am
@@ -56,7 +56,7 @@ vmod_debug_symbols_regex = 'Vmod_.*_Data'
 vmodtoolargs_debug = --boilerplate -o vcc_debug_if
 
 # unregister VCC file installation
-dist_noinst_DATA := $(srcdir)/vmod_debug.vcc
+dist_noinst_DATA = $(srcdir)/vmod_debug.vcc
 vmod_debug_vcc =
 
 dist_vcc_DATA = $(vmod_vcc_files)

From dridi.boukelmoune at gmail.com  Mon May 15 16:42:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:42:06 +0000 (UTC)
Subject: [master] dcaf616c6 build: Centralize suffix .c.vsc make rules
Message-ID: <20230515164206.B3BB410465C@lists.varnish-cache.org>


commit dcaf616c66d93de69735237967cc091fa490bb93
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Mar 29 18:11:54 2023 +0200

    build: Centralize suffix .c.vsc make rules
    
    The new vsc.am file takes care of most of the logic, and include sites
    only need to reference sources as such.

diff --git a/Makefile.am b/Makefile.am
index c0480e143..5e2db4419 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -26,6 +26,7 @@ EXTRA_DIST = \
 	varnishapi.pc.in \
 	varnish.m4 \
 	varnish-legacy.m4 \
+	vsc.am \
 	vtc.am \
 	wflags.py
 
diff --git a/lib/libvsc/Makefile.am b/lib/libvsc/Makefile.am
index 842515148..243f82587 100644
--- a/lib/libvsc/Makefile.am
+++ b/lib/libvsc/Makefile.am
@@ -1,5 +1,7 @@
 #
 
+include $(top_srcdir)/vsc.am
+
 AM_CPPFLAGS = \
 	-I$(top_srcdir)/include \
 	-I$(top_builddir)/include
@@ -14,19 +16,7 @@ VSC_SRC = \
 	VSC_smu.vsc \
 	VSC_vbe.vsc
 
-VSC_GEN_C	= $(VSC_SRC:.vsc=.c)
-VSC_GEN_H	= $(VSC_SRC:.vsc=.h)
-
-$(VSC_GEN_C): vsctool.py
-
-.vsc.c:
-	$(PYTHON) $(srcdir)/vsctool.py -ch $<
-
 noinst_LTLIBRARIES = libvsc.la
 libvsc_la_SOURCES = $(VSC_SRC)
 
-dist_pkgdata_SCRIPTS = \
-	vsctool.py
-
-BUILT_SOURCES	= $(VSC_GEN_C)
-CLEANFILES	= $(VSC_GEN_C) $(VSC_GEN_H)
+dist_pkgdata_SCRIPTS = vsctool.py
diff --git a/vmod/Makefile.am b/vmod/Makefile.am
index b50458265..a485c0372 100644
--- a/vmod/Makefile.am
+++ b/vmod/Makefile.am
@@ -2,6 +2,7 @@
 
 TESTS = @VMOD_TESTS@
 
+include $(top_srcdir)/vsc.am
 include $(top_srcdir)/vtc.am
 
 EXTRA_DIST = $(TESTS)
@@ -36,18 +37,9 @@ include $(srcdir)/automake_boilerplate_vtc.am
 # Post-boilerplate tweaks
 #
 
-.vsc.c: $(top_srcdir)/lib/libvsc/vsctool.py
-	$(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -c $<
+VSC_SRC = VSC_debug.vsc
 
-.vsc.h: $(top_srcdir)/lib/libvsc/vsctool.py
-	$(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -h $<
-
-EXTRA_DIST += VSC_debug.vsc
-nodist_libvmod_debug_la_SOURCES += \
-	VSC_debug.c \
-	VSC_debug.h
-
-vmod_debug.c: VSC_debug.c VSC_debug.h
+libvmod_debug_la_SOURCES += $(VSC_SRC)
 
 # Allow Vmod_wrong*_Data to be exported
 vmod_debug_symbols_regex = 'Vmod_.*_Data'
diff --git a/vsc.am b/vsc.am
new file mode 100644
index 000000000..e199810a4
--- /dev/null
+++ b/vsc.am
@@ -0,0 +1,14 @@
+# Generic rule to generate C code from VSC files. VSC files must be listed
+# in the $(VSC_SRC) variable.
+
+VSC_GEN = $(VSC_SRC:.vsc=.c) $(VSC_SRC:.vsc=.h)
+
+$(VSC_GEN): $(top_srcdir)/lib/libvsc/vsctool.py
+
+.vsc.c:
+	$(AM_V_GEN) $(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -ch $<
+
+clean-local: vsc-clean
+
+vsc-clean:
+	rm -f $(VSC_GEN)
diff --git a/vtc.am b/vtc.am
index f67a4969c..cf29b5c9b 100644
--- a/vtc.am
+++ b/vtc.am
@@ -11,7 +11,7 @@
 VTC_LOG_COMPILER = $(top_builddir)/bin/varnishtest/varnishtest -v -i
 TEST_EXTENSIONS = .vtc
 
-check-am: vtc-check-tests
+check-local: vtc-check-tests
 
 vtc-check-tests:
 	@mkdir -p tests ; \

From dridi.boukelmoune at gmail.com  Mon May 15 16:42:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:42:06 +0000 (UTC)
Subject: [master] 7fda13213 build: Turn the $(vmodtool) make variable uppercase
Message-ID: <20230515164206.D1AAA10465F@lists.varnish-cache.org>


commit 7fda13213009357d2efd24513445e0707c2c47f2
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Apr 3 17:23:56 2023 +0200

    build: Turn the $(vmodtool) make variable uppercase
    
    This is the convention for command line programs used in makefiles, like
    CC, LD and plenty others. This is also what varnish.m4 exposes to out of
    tree projects.

diff --git a/configure.ac b/configure.ac
index 70aa89818..ff51ab9ac 100644
--- a/configure.ac
+++ b/configure.ac
@@ -644,7 +644,7 @@ AC_SUBST(pkgsysconfdir)
 
 # VMOD variables
 AC_SUBST(vmoddir, [$\(pkglibdir\)/vmods])
-AC_SUBST(vmodtool, [$\(top_srcdir\)/lib/libvcc/vmodtool.py])
+AC_SUBST(VMODTOOL, [$\(top_srcdir\)/lib/libvcc/vmodtool.py])
 
 # Check for linker script support
 gl_LD_VERSION_SCRIPT
diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 884ad4fa5..58fbc3041 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -82,8 +82,8 @@ PFX.h vmod_XXX.rst vmod_XXX.man.rst: PFX.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-PFX.c: $(vmodtool) $(srcdir)/VCC
-\t at PYTHON@ $(vmodtool) $(vmodtoolargs_XXX) $(srcdir)/VCC
+PFX.c: $(VMODTOOL) $(srcdir)/VCC
+\t at PYTHON@ $(VMODTOOL) $(vmodtoolargs_XXX) $(srcdir)/VCC
 \ttouch PFX.c
 
 clean-local: clean-vmod-XXX
diff --git a/vmod/automake_boilerplate_blob.am b/vmod/automake_boilerplate_blob.am
index 1b7c375a5..7d7543650 100644
--- a/vmod/automake_boilerplate_blob.am
+++ b/vmod/automake_boilerplate_blob.am
@@ -38,8 +38,8 @@ vcc_blob_if.h vmod_blob.rst vmod_blob.man.rst: vcc_blob_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_blob_if.c: $(vmodtool) $(srcdir)/vmod_blob.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_blob) $(srcdir)/vmod_blob.vcc
+vcc_blob_if.c: $(VMODTOOL) $(srcdir)/vmod_blob.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_blob) $(srcdir)/vmod_blob.vcc
 	touch vcc_blob_if.c
 
 clean-local: clean-vmod-blob
diff --git a/vmod/automake_boilerplate_cookie.am b/vmod/automake_boilerplate_cookie.am
index 1cb5c9f46..50e6c6409 100644
--- a/vmod/automake_boilerplate_cookie.am
+++ b/vmod/automake_boilerplate_cookie.am
@@ -31,8 +31,8 @@ vcc_cookie_if.h vmod_cookie.rst vmod_cookie.man.rst: vcc_cookie_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_cookie_if.c: $(vmodtool) $(srcdir)/vmod_cookie.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_cookie) $(srcdir)/vmod_cookie.vcc
+vcc_cookie_if.c: $(VMODTOOL) $(srcdir)/vmod_cookie.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_cookie) $(srcdir)/vmod_cookie.vcc
 	touch vcc_cookie_if.c
 
 clean-local: clean-vmod-cookie
diff --git a/vmod/automake_boilerplate_debug.am b/vmod/automake_boilerplate_debug.am
index b5c34f75e..69c74062b 100644
--- a/vmod/automake_boilerplate_debug.am
+++ b/vmod/automake_boilerplate_debug.am
@@ -34,8 +34,8 @@ vcc_debug_if.h vmod_debug.rst vmod_debug.man.rst: vcc_debug_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_debug_if.c: $(vmodtool) $(srcdir)/vmod_debug.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_debug) $(srcdir)/vmod_debug.vcc
+vcc_debug_if.c: $(VMODTOOL) $(srcdir)/vmod_debug.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_debug) $(srcdir)/vmod_debug.vcc
 	touch vcc_debug_if.c
 
 clean-local: clean-vmod-debug
diff --git a/vmod/automake_boilerplate_directors.am b/vmod/automake_boilerplate_directors.am
index ed44678ac..c75f0c310 100644
--- a/vmod/automake_boilerplate_directors.am
+++ b/vmod/automake_boilerplate_directors.am
@@ -41,8 +41,8 @@ vcc_directors_if.h vmod_directors.rst vmod_directors.man.rst: vcc_directors_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_directors_if.c: $(vmodtool) $(srcdir)/vmod_directors.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_directors) $(srcdir)/vmod_directors.vcc
+vcc_directors_if.c: $(VMODTOOL) $(srcdir)/vmod_directors.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_directors) $(srcdir)/vmod_directors.vcc
 	touch vcc_directors_if.c
 
 clean-local: clean-vmod-directors
diff --git a/vmod/automake_boilerplate_proxy.am b/vmod/automake_boilerplate_proxy.am
index 61b54e785..03bc9807e 100644
--- a/vmod/automake_boilerplate_proxy.am
+++ b/vmod/automake_boilerplate_proxy.am
@@ -31,8 +31,8 @@ vcc_proxy_if.h vmod_proxy.rst vmod_proxy.man.rst: vcc_proxy_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_proxy_if.c: $(vmodtool) $(srcdir)/vmod_proxy.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_proxy) $(srcdir)/vmod_proxy.vcc
+vcc_proxy_if.c: $(VMODTOOL) $(srcdir)/vmod_proxy.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_proxy) $(srcdir)/vmod_proxy.vcc
 	touch vcc_proxy_if.c
 
 clean-local: clean-vmod-proxy
diff --git a/vmod/automake_boilerplate_purge.am b/vmod/automake_boilerplate_purge.am
index de5cbc6c4..ddf76e60d 100644
--- a/vmod/automake_boilerplate_purge.am
+++ b/vmod/automake_boilerplate_purge.am
@@ -31,8 +31,8 @@ vcc_purge_if.h vmod_purge.rst vmod_purge.man.rst: vcc_purge_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_purge_if.c: $(vmodtool) $(srcdir)/vmod_purge.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_purge) $(srcdir)/vmod_purge.vcc
+vcc_purge_if.c: $(VMODTOOL) $(srcdir)/vmod_purge.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_purge) $(srcdir)/vmod_purge.vcc
 	touch vcc_purge_if.c
 
 clean-local: clean-vmod-purge
diff --git a/vmod/automake_boilerplate_std.am b/vmod/automake_boilerplate_std.am
index b4f62fde6..84ee4466d 100644
--- a/vmod/automake_boilerplate_std.am
+++ b/vmod/automake_boilerplate_std.am
@@ -34,8 +34,8 @@ vcc_std_if.h vmod_std.rst vmod_std.man.rst: vcc_std_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_std_if.c: $(vmodtool) $(srcdir)/vmod_std.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_std) $(srcdir)/vmod_std.vcc
+vcc_std_if.c: $(VMODTOOL) $(srcdir)/vmod_std.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_std) $(srcdir)/vmod_std.vcc
 	touch vcc_std_if.c
 
 clean-local: clean-vmod-std
diff --git a/vmod/automake_boilerplate_unix.am b/vmod/automake_boilerplate_unix.am
index aff2650f4..422492dd4 100644
--- a/vmod/automake_boilerplate_unix.am
+++ b/vmod/automake_boilerplate_unix.am
@@ -32,8 +32,8 @@ vcc_unix_if.h vmod_unix.rst vmod_unix.man.rst: vcc_unix_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_unix_if.c: $(vmodtool) $(srcdir)/vmod_unix.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_unix) $(srcdir)/vmod_unix.vcc
+vcc_unix_if.c: $(VMODTOOL) $(srcdir)/vmod_unix.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_unix) $(srcdir)/vmod_unix.vcc
 	touch vcc_unix_if.c
 
 clean-local: clean-vmod-unix
diff --git a/vmod/automake_boilerplate_vtc.am b/vmod/automake_boilerplate_vtc.am
index 202ad2257..685eb2830 100644
--- a/vmod/automake_boilerplate_vtc.am
+++ b/vmod/automake_boilerplate_vtc.am
@@ -31,8 +31,8 @@ vcc_vtc_if.h vmod_vtc.rst vmod_vtc.man.rst: vcc_vtc_if.c
 
 # A doc-change will not update mtime on the .h and .c files, so a
 # touch(1) is necessary to signal that vmodtool was in fact run.
-vcc_vtc_if.c: $(vmodtool) $(srcdir)/vmod_vtc.vcc
-	@PYTHON@ $(vmodtool) $(vmodtoolargs_vtc) $(srcdir)/vmod_vtc.vcc
+vcc_vtc_if.c: $(VMODTOOL) $(srcdir)/vmod_vtc.vcc
+	@PYTHON@ $(VMODTOOL) $(vmodtoolargs_vtc) $(srcdir)/vmod_vtc.vcc
 	touch vcc_vtc_if.c
 
 clean-local: clean-vmod-vtc

From dridi.boukelmoune at gmail.com  Mon May 15 16:42:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:42:06 +0000 (UTC)
Subject: [master] cff4e9802 build: Internally use a $(VSCTOOL) variable
Message-ID: <20230515164207.015E2104664@lists.varnish-cache.org>


commit cff4e98028539b48d646f03ad4bdf9ac22af724e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Apr 3 17:32:13 2023 +0200

    build: Internally use a $(VSCTOOL) variable
    
    And make vsc.am expect it as one of the input variables.

diff --git a/configure.ac b/configure.ac
index ff51ab9ac..fb0b8d202 100644
--- a/configure.ac
+++ b/configure.ac
@@ -645,6 +645,7 @@ AC_SUBST(pkgsysconfdir)
 # VMOD variables
 AC_SUBST(vmoddir, [$\(pkglibdir\)/vmods])
 AC_SUBST(VMODTOOL, [$\(top_srcdir\)/lib/libvcc/vmodtool.py])
+AC_SUBST(VSCTOOL, [$\(top_srcdir\)/lib/libvsc/vsctool.py])
 
 # Check for linker script support
 gl_LD_VERSION_SCRIPT
diff --git a/vsc.am b/vsc.am
index e199810a4..0a8da842a 100644
--- a/vsc.am
+++ b/vsc.am
@@ -1,12 +1,13 @@
 # Generic rule to generate C code from VSC files. VSC files must be listed
-# in the $(VSC_SRC) variable.
+# in the $(VSC_SRC) variable. The $(VSCTOOL) variable must point to the
+# location of vsctool.py, normally set up by varnish.m4 at configure time.
 
 VSC_GEN = $(VSC_SRC:.vsc=.c) $(VSC_SRC:.vsc=.h)
 
-$(VSC_GEN): $(top_srcdir)/lib/libvsc/vsctool.py
+$(VSC_GEN): $(VSCTOOL)
 
 .vsc.c:
-	$(AM_V_GEN) $(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -ch $<
+	$(AM_V_GEN) $(PYTHON) $(VSCTOOL) -ch $<
 
 clean-local: vsc-clean
 

From dridi at varni.sh  Mon May 15 16:42:38 2023
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Mon, 15 May 2023 16:42:38 +0000
Subject: [master] f32bc1d0e SQUASHME: keyboard accident
In-Reply-To: <20230515163907.2FE221041B5@lists.varnish-cache.org>
References: <20230515163907.2FE221041B5@lists.varnish-cache.org>
Message-ID: <CABoVN9CtkV2YBtO9yWXr0gSXWhnk34mFgJ62v5eOwNa+or=1Uw@mail.gmail.com>

On Mon, May 15, 2023 at 4:39?PM Dridi Boukelmoune
<dridi.boukelmoune at gmail.com> wrote:
>
>
> commit f32bc1d0eddab4acd4ab317cb07d703e0a087db9
> Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
> Date:   Wed Mar 29 18:02:48 2023 +0200
>
>     SQUASHME: keyboard accident

My bad for this not-paying-attention accident.

From phk at FreeBSD.org  Tue May 16 10:34:11 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 16 May 2023 10:34:11 +0000 (UTC)
Subject: [master] 3f8f72058 Dump gzip data in hex
Message-ID: <20230516103411.5CB4D1009F7@lists.varnish-cache.org>


commit 3f8f720581d6a2288524f721cd8ccba4e07e0a4b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 16 10:33:09 2023 +0000

    Dump gzip data in hex

diff --git a/bin/varnishtest/vtc_log.c b/bin/varnishtest/vtc_log.c
index dae3ff0ec..89bbfa5ed 100644
--- a/bin/varnishtest/vtc_log.c
+++ b/bin/varnishtest/vtc_log.c
@@ -224,6 +224,7 @@ void
 vtc_dump(struct vtclog *vl, int lvl, const char *pfx, const char *str, int len)
 {
 	char buf[64];
+	int quote = VSB_QUOTE_UNSAFE | VSB_QUOTE_ESCHEX;
 
 	AN(pfx);
 	GET_VL(vl);
@@ -234,9 +235,12 @@ vtc_dump(struct vtclog *vl, int lvl, const char *pfx, const char *str, int len)
 		    lead[lvl < 0 ? 1: lvl], vl->id, pfx);
 		if (len < 0)
 			len = strlen(str);
+		else if (str[0] == 0x1f && str[1] == 0x8b)
+			quote = VSB_QUOTE_HEX; // Dump gzip data in HEX
 		VSB_quote_pfx(vl->vsb, buf, str,
-		    len > MAX_DUMP ? MAX_DUMP : len,
-		    VSB_QUOTE_UNSAFE | VSB_QUOTE_ESCHEX);
+		    len > MAX_DUMP ? MAX_DUMP : len, quote);
+		if (quote == VSB_QUOTE_HEX)
+			VSB_putc(vl->vsb, '\n');
 		if (len > MAX_DUMP)
 			VSB_printf(vl->vsb, "%s [...] (%d)\n",
 			    buf, len - MAX_DUMP);

From phk at FreeBSD.org  Tue May 16 11:29:05 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 16 May 2023 11:29:05 +0000 (UTC)
Subject: [master] b18568ace Cast for picky compilers
Message-ID: <20230516112905.5F4D710251E@lists.varnish-cache.org>


commit b18568ace231f1354f2af6fffc72c5da8c7201f3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 16 11:28:01 2023 +0000

    Cast for picky compilers

diff --git a/bin/varnishtest/vtc_log.c b/bin/varnishtest/vtc_log.c
index 89bbfa5ed..9c7e33d05 100644
--- a/bin/varnishtest/vtc_log.c
+++ b/bin/varnishtest/vtc_log.c
@@ -235,7 +235,7 @@ vtc_dump(struct vtclog *vl, int lvl, const char *pfx, const char *str, int len)
 		    lead[lvl < 0 ? 1: lvl], vl->id, pfx);
 		if (len < 0)
 			len = strlen(str);
-		else if (str[0] == 0x1f && str[1] == 0x8b)
+		else if (str[0] == 0x1f && (uint8_t)str[1] == 0x8b)
 			quote = VSB_QUOTE_HEX; // Dump gzip data in HEX
 		VSB_quote_pfx(vl->vsb, buf, str,
 		    len > MAX_DUMP ? MAX_DUMP : len, quote);

From nils.goroll at uplex.de  Mon May 22 13:05:14 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 22 May 2023 13:05:14 +0000 (UTC)
Subject: [master] ace59ccfe bereq: Also unset a cached req body
Message-ID: <20230522130514.879071008DD@lists.varnish-cache.org>


commit ace59ccfe8ad99aaa96dc5d9fdcda447086a1225
Author: Lachie <lachie at varnish-software.com>
Date:   Wed Apr 5 12:10:29 2023 +1000

    bereq: Also unset a cached req body

diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index 2bd56b71a..1ec09a74c 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -35,6 +35,7 @@
 #include <stdio.h>
 
 #include "cache_varnishd.h"
+#include "cache_objhead.h"
 #include "cache_transport.h"
 #include "common/heritage.h"
 
@@ -612,6 +613,11 @@ VRT_u_bereq_body(VRT_CTX)
 {
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
+	if (ctx->bo->bereq_body != NULL) {
+		HSH_DerefObjCore(ctx->bo->wrk, &ctx->bo->bereq_body, 0);
+		http_Unset(ctx->bo->bereq, H_Content_Length);
+	}
+
 	if (ctx->bo->req != NULL) {
 		CHECK_OBJ(ctx->bo->req, REQ_MAGIC);
 		ctx->bo->req = NULL;
diff --git a/bin/varnishtest/tests/v00068.vtc b/bin/varnishtest/tests/v00068.vtc
new file mode 100644
index 000000000..4b54a5ba0
--- /dev/null
+++ b/bin/varnishtest/tests/v00068.vtc
@@ -0,0 +1,33 @@
+varnishtest "unset bereq.body with cached req body"
+
+server s1 {
+	rxreq
+	expect req.method == "GET"
+	expect req.http.Content-Length == <undef>
+	txresp
+
+	rxreq
+	expect req.method == "GET"
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	import std;
+
+	sub vcl_recv {
+		std.cache_req_body(2KB);
+	}
+	sub vcl_backend_fetch {
+		unset bereq.body;
+	}
+} -start
+
+client c1 {
+	txreq -body "fine"
+	rxresp
+	expect resp.status == 200
+
+	txreq
+	rxresp
+	expect resp.status == 200
+} -run

From nils.goroll at uplex.de  Mon May 22 13:11:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 22 May 2023 13:11:06 +0000 (UTC)
Subject: [master] 96e87339d vsl: Expose vsl_tag_is_masked
Message-ID: <20230522131106.32FE2100D51@lists.varnish-cache.org>


commit 96e87339d1affb6db94d70ac7d0ec1a528c58c76
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed May 10 16:58:19 2023 +0200

    vsl: Expose vsl_tag_is_masked

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 6e73158f8..b6af25537 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -783,6 +783,7 @@ void VSLbs(struct vsl_log *, enum VSL_tag_e tag, const struct strands *s);
 void VSLb_ts(struct vsl_log *, const char *event, vtim_real first,
     vtim_real *pprev, vtim_real now);
 void VSLb_bin(struct vsl_log *, enum VSL_tag_e, ssize_t, const void*);
+int VSL_tag_is_masked(enum VSL_tag_e tag);
 
 static inline void
 VSLb_ts_req(struct req *req, const char *event, vtim_real now)
diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index b3b2e213d..67150f69a 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -130,6 +130,12 @@ vsl_tag_is_masked(enum VSL_tag_e tag)
 	return (*bm & b);
 }
 
+int
+VSL_tag_is_masked(enum VSL_tag_e tag)
+{
+	return (vsl_tag_is_masked(tag));
+}
+
 /*--------------------------------------------------------------------
  * Lay down a header fields, and return pointer to the next record
  */

From nils.goroll at uplex.de  Mon May 22 13:11:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 22 May 2023 13:11:06 +0000 (UTC)
Subject: [master] 3c09eb39c h2_vsl_frame: Exit early if masked
Message-ID: <20230522131106.4CCEC100D55@lists.varnish-cache.org>


commit 3c09eb39c70788a372a6f75eed34feeaf20a6d46
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Mon May 22 10:57:09 2023 +0200

    h2_vsl_frame: Exit early if masked
    
    The VSB bits in here come with a very significant performance penalty
    for H2 request body processing.

diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c
index c1ced04b6..5b66795b0 100644
--- a/bin/varnishd/http2/cache_http2_proto.c
+++ b/bin/varnishd/http2/cache_http2_proto.c
@@ -243,6 +243,10 @@ h2_vsl_frame(const struct h2_sess *h2, const void *ptr, size_t len)
 	const char *p;
 	unsigned u;
 
+	if (VSL_tag_is_masked(SLT_H2RxHdr) &&
+	    VSL_tag_is_masked(SLT_H2RxBody))
+		return;
+
 	AN(ptr);
 	assert(len >= 9);
 	b = ptr;

From nils.goroll at uplex.de  Mon May 22 13:23:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 22 May 2023 13:23:06 +0000 (UTC)
Subject: [master] b87175a11 Log #3914 change
Message-ID: <20230522132306.E38FC10162E@lists.varnish-cache.org>


commit b87175a1184a0898cb3fef1135ca577f1eb61af8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon May 22 15:15:11 2023 +0200

    Log #3914 change

diff --git a/doc/changes.rst b/doc/changes.rst
index e30c24924..6b7746888 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -38,6 +38,9 @@ Varnish Cache NEXT (2023-09-15)
 .. PLEASE keep this roughly in commit order as shown by git-log / tig
    (new to old)
 
+* A bug has been fixed where ``unset bereq.body`` had no effect when
+  used with a cached body (3914_)
+
 * VCL tracing now needs to be explicitly activated by setting the
   ``req.trace`` or ``bereq.trace`` VCL variables, which are
   initialized from the ``feature +trace`` flag. Only if the trace
@@ -83,6 +86,7 @@ Varnish Cache NEXT (2023-09-15)
 
 .. _3908: https://github.com/varnishcache/varnish-cache/pull/3908
 .. _3911: https://github.com/varnishcache/varnish-cache/issues/3911
+.. _3914: https://github.com/varnishcache/varnish-cache/pull/3914
 
 ================================
 Varnish Cache 7.3.0 (2023-03-15)

From nils.goroll at uplex.de  Mon May 22 15:09:07 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 22 May 2023 15:09:07 +0000 (UTC)
Subject: [master] 8bfc5948c Update vcl-backends.rst
Message-ID: <20230522150907.D2040104BBF@lists.varnish-cache.org>


commit 8bfc5948c5031d4ca83a8ae200977d14fa5f0b75
Author: martin-uplex <martin-uplex at users.noreply.github.com>
Date:   Mon May 22 17:01:06 2023 +0200

    Update vcl-backends.rst
    
    restriction to port 443 in haproxy config inflexible and unnecessary

diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst
index 76ef15e1c..a4708b8b1 100644
--- a/doc/sphinx/users-guide/vcl-backends.rst
+++ b/doc/sphinx/users-guide/vcl-backends.rst
@@ -179,9 +179,9 @@ higher, this snippet can be used as a basis for configuring an
             bind    /path/to/sslon accept-proxy mode 777
             stick-table type ip size 100
             stick   on dst
-            server  s00 0.0.0.0:443 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
-            server  s01 0.0.0.0:443 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
-            server  s02 0.0.0.0:443 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
+            server  s00 0.0.0.0:0 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
+            server  s01 0.0.0.0:0 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
+            server  s02 0.0.0.0:0 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
             # ...
             # A higher number of servers improves TLS session caching