From nils.goroll at uplex.de Fri Sep 1 09:23:09 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 1 Sep 2023 09:23:09 +0000 (UTC) Subject: [master] 990095283 VSB_QUOTE_JSON does not produce JSON. Document what it does in the test Message-ID: <20230901092309.6651DA2B12@lists.varnish-cache.org> commit 9900952837dc96e61f9b00eb55b3f28f8ed8b83a Author: Nils Goroll Date: Fri Sep 1 11:21:02 2023 +0200 VSB_QUOTE_JSON does not produce JSON. Document what it does in the test diff --git a/lib/libvarnish/vsb_test.c b/lib/libvarnish/vsb_test.c index b41512343..373e4bada 100644 --- a/lib/libvarnish/vsb_test.c +++ b/lib/libvarnish/vsb_test.c @@ -83,6 +83,24 @@ static struct tc tcs[] = { 5, "\x00\n\x7e\x7f\xff", "PFX\"\\x00\\n\"\nPFX\"~\\x7f\\xff\"", }, + /* + * VSB_QUOTE_JSON puts >0x7e literally + */ + { + VSB_QUOTE_JSON, + 4, "\xf0\x9f\x90\xb0", + "PFX\xf0\x9f\x90\xb0", + }, + /* + * VSB_QUOTE_JSON encodes <0x20 as \u00XX, which looks like + * unicode codepoints in the JSON standard, but are just + * individual bytes + */ + { + VSB_QUOTE_JSON, + 5, "\"\x01\x02\x03\x04", + "PFX\\\"\\u0001\\u0002\\u0003\\u0004", + }, { VSB_QUOTE_JSON, 5, "\x00\n\x7e\x7f\xff", From dridi.boukelmoune at gmail.com Fri Sep 1 09:36:05 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Fri, 1 Sep 2023 09:36:05 +0000 (UTC) Subject: [master] bbef2afb5 varnishncsa: doc fix for handling and hitmiss Message-ID: <20230901093605.1CBACA3441@lists.varnish-cache.org> commit bbef2afb5c1f916ad230a8a3e61857213ceb5d90 Author: Asad Sajjad Ahmed Date: Fri Aug 25 10:24:38 2023 +0200 varnishncsa: doc fix for handling and hitmiss Signed-off-by: Asad Sajjad Ahmed diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst index 863856505..1585b2ae7 100644 --- a/doc/sphinx/reference/varnishncsa.rst +++ b/doc/sphinx/reference/varnishncsa.rst @@ -180,12 +180,14 @@ Supported formatters are: had been received. Varnish:hitmiss - One of the 'hit' or 'miss' strings, depending on whether the request - was a cache hit or miss. Pipe, pass and synth are considered misses. + In client mode, one of the 'hit' or 'miss' strings, depending on whether + the request was a cache hit or miss. Pipe, pass and synth are considered + misses. In backend mode, this field is blank. Varnish:handling - One of the 'hit', 'miss', 'pass', 'pipe' or 'synth' strings indicating - how the request was handled. + In client mode, one of the 'hit', 'miss', 'pass', 'pipe' or 'synth' strings + indicating how the request was handled. In backend mode, this field is + blank. Varnish:side Backend or client side. One of two values, 'b' or 'c', depending From nils.goroll at uplex.de Fri Sep 1 09:40:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 1 Sep 2023 09:40:06 +0000 (UTC) Subject: [master] f71c2aacc Also sync vsb.h comment with reality Message-ID: <20230901094006.D6399A385E@lists.varnish-cache.org> commit f71c2aaccec2085013eeb83b7e9fae49f11cb52d Author: Nils Goroll Date: Fri Sep 1 11:38:25 2023 +0200 Also sync vsb.h comment with reality diff --git a/include/vsb.h b/include/vsb.h index 537254f4f..37fc1c027 100644 --- a/include/vsb.h +++ b/include/vsb.h @@ -89,9 +89,9 @@ void VSB_destroy(struct vsb **); */ #define VSB_QUOTE_JSON 2 /* - * Output suitable for inclusion between "..." in JSON - * Uses JSON \u%04x quoting. - * Anything above 0x7e had better be UTF-8 + * JSON-like output suitable for inclusion between "..." + * Quotes <0x20 as \u%04x + * Keeps >0x7e unquoted */ #define VSB_QUOTE_HEX 4 /* From dridi.boukelmoune at gmail.com Mon Sep 4 11:32:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 4 Sep 2023 11:32:06 +0000 (UTC) Subject: [master] 5aefbd810 vtc: Fix copy-pasta in c83 comments Message-ID: <20230904113206.244E094B74@lists.varnish-cache.org> commit 5aefbd8107e730c16704299cd110122e12ddd17d Author: Dridi Boukelmoune Date: Mon Sep 4 13:30:23 2023 +0200 vtc: Fix copy-pasta in c83 comments diff --git a/bin/varnishtest/tests/c00083.vtc b/bin/varnishtest/tests/c00083.vtc index d991201be..fe09555e9 100644 --- a/bin/varnishtest/tests/c00083.vtc +++ b/bin/varnishtest/tests/c00083.vtc @@ -119,7 +119,7 @@ process p1 { nlines=`wc -l < ${tmpdir}/v1/_.vsm_mgt/_.index` nminus=`grep -c '^-' ${tmpdir}/v1/_.vsm_mgt/_.index` echo MGT NLINES $nlines NMINUS $nminus - # cat ${tmpdir}/v1/_.vsm_child/_.index + # cat ${tmpdir}/v1/_.vsm_mgt/_.index test $nminus -eq 0 } -run @@ -131,7 +131,7 @@ process p1 { nlines=`wc -l < ${tmpdir}/v1/_.vsm_mgt/_.index` nminus=`grep -c '^-' ${tmpdir}/v1/_.vsm_mgt/_.index` echo MGT NLINES $nlines NMINUS $nminus - # cat ${tmpdir}/v1/_.vsm_child/_.index + # cat ${tmpdir}/v1/_.vsm_mgt/_.index test $nminus -eq 2 } -run From nils.goroll at uplex.de Mon Sep 4 13:50:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 4 Sep 2023 13:50:06 +0000 (UTC) Subject: [6.0] 47bca2475 add bookworm building+packaging Message-ID: <20230904135006.E8BA19C82E@lists.varnish-cache.org> commit 47bca2475223d191c8461f2e9f3068d9436907d0 Author: Guillaume Quintard Date: Fri Sep 1 08:05:32 2023 -0700 add bookworm building+packaging diff --git a/.circleci/config.yml b/.circleci/config.yml index 709916811..0214e77c8 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -403,6 +403,10 @@ workflows: name: build_debian_bullseye dist: debian release: bullseye + - build: + name: build_debian_bookworm + dist: debian + release: bookworm - build: name: build_ubuntu_bionic dist: ubuntu @@ -435,6 +439,7 @@ workflows: - debian:stretch - debian:buster - debian:bullseye + - debian:bookworm - centos:7 - almalinux:8 - almalinux:9 From phk at FreeBSD.org Tue Sep 5 11:27:08 2023 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 5 Sep 2023 11:27:08 +0000 (UTC) Subject: [master] 372093f5a Minor flexelinting Message-ID: <20230905112708.09DD1A4C0C@lists.varnish-cache.org> commit 372093f5a7813031860aaadb0323289037a4235a Author: Poul-Henning Kamp Date: Tue Sep 5 11:26:17 2023 +0000 Minor flexelinting diff --git a/lib/libvarnish/vte.c b/lib/libvarnish/vte.c index 3ebf57086..c97a95441 100644 --- a/lib/libvarnish/vte.c +++ b/lib/libvarnish/vte.c @@ -263,7 +263,7 @@ VTE_format(const struct vte *vte, VTE_format_f *func, void *priv) if (*p == '\v') { if (p - 1 > q) { /* exclude previous separator */ VTE_FORMAT(func, priv, "%.*s%s", - (int)(p - 1 - q), q, sep); + (int)((p - 1) - q), q, sep); } q = ++p; just_left = 1; diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 369c66daa..acd997281 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -456,7 +456,7 @@ char *vcc_Dup_be(const char *b, const char *e); int vcc_Has_vcl_prefix(const char *b); /* vcc_var.c */ -void vcc_Header_Fh(struct vcc *, struct symbol *); +void vcc_Header_Fh(const struct vcc *, const struct symbol *); sym_wildcard_t vcc_Var_Wildcard; /* vcc_vmod.c */ diff --git a/lib/libvcc/vcc_var.c b/lib/libvcc/vcc_var.c index 7292924ae..643af2f1e 100644 --- a/lib/libvcc/vcc_var.c +++ b/lib/libvcc/vcc_var.c @@ -41,7 +41,7 @@ /*--------------------------------------------------------------------*/ void -vcc_Header_Fh(struct vcc *tl, struct symbol *sym) +vcc_Header_Fh(const struct vcc *tl, const struct symbol *sym) { const struct symbol *parent; From phk at FreeBSD.org Tue Sep 5 14:16:06 2023 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 5 Sep 2023 14:16:06 +0000 (UTC) Subject: [master] 5128c660c Dont produce \v when we VSB_quote data. Message-ID: <20230905141606.70502AA252@lists.varnish-cache.org> commit 5128c660c1629d1c3147a35fb0107149d2d67cf5 Author: Poul-Henning Kamp Date: Tue Sep 5 14:15:02 2023 +0000 Dont produce \v when we VSB_quote data. diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c index 30d95a531..e9e153b34 100644 --- a/lib/libvarnish/vsb.c +++ b/lib/libvarnish/vsb.c @@ -626,9 +626,6 @@ VSB_quote_pfx(struct vsb *s, const char *pfx, const void *v, int len, int how) case '\t': VSB_cat(s, "\\t"); break; - case '\v': - VSB_cat(s, "\\v"); - break; default: if (0x20 <= *q && *q <= 0x7e) VSB_putc(s, *q); From nils.goroll at uplex.de Tue Sep 12 15:50:07 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 12 Sep 2023 15:50:07 +0000 (UTC) Subject: [master] cbe1ed892 Changelog TLC Message-ID: <20230912155007.CC6EF73BA@lists.varnish-cache.org> commit cbe1ed8921e6c2e3f167a74ec525ff46732b87d3 Author: Nils Goroll Date: Tue Sep 12 17:47:10 2023 +0200 Changelog TLC If you have feedback, additions or corrections, please prodive/commit them before next Thursday, September 14, when I will continue to work on release documentation unless someone else does it. diff --git a/doc/changes.rst b/doc/changes.rst index 1af371255..1d630bffb 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -1,8 +1,11 @@ .. - Copyright (c) 2011-2021 Varnish Software AS + Copyright (c) 2011-2023 Varnish Software AS + Copyright 2016-2023 UPLEX - Nils Goroll Systemoptimierung SPDX-License-Identifier: BSD-2-Clause See LICENSE file for full text of license +.. role:: ref(emphasis) + =================== About this document =================== @@ -38,15 +41,116 @@ Varnish Cache NEXT (2023-09-15) .. PLEASE keep this roughly in commit order as shown by git-log / tig (new to old) -* The ``VSHA256_*`` functions have been added to libvarnishapi. +* The ``VSB_quote_pfx()`` (and, consequently, ``VSB_quote()``) function + no longer produces ``\v`` for a vertical tab. This improves + compatibility with JSON. + +* The bundled *zlib* has been updated to match *zlib 1.3*. + +* The ``VSHA256_*`` functions have been added to libvarnishapi (3946_). + +* Tabulation of the ``vcl.list`` CLI output has been modified + slightly. + +* VCL now supports "protected headers", which can neither be set nor unset. + +* The ``Content-Length`` and ``Transfer-Encoding`` headers are now + protected. For the common use case of ``unset + xxx.http.Content-Length`` to dismiss a body, ``unset xxx.body`` + should be used. + +* Error handling of numeric literals in exponent notation has been + improved in the VCL compiler (3971_). + +* Finalization of the storage private state of busy objects has been + fixed. This bug could trigger a panic when ``vcl_synth {}`` was used + to replace the object body and storage was changed from one of the + built-in storage engines to a storage engine from an extension (3953_). + +* HTTP/2 header field validation is now more strict with respect to + allowed characters (3952_). + +* A bug has been fixed in the filter handling code which could trigger + a panic when ``resp.filters`` was used from ``vcl_synth {}`` (3968_). + +* The utility macros ``ALLOC_OBJ_EXTRA()`` and ``ALLOC_FLEX_OBJ()`` + have been added to ``miniobj.h`` to simplify allocation of objects + larger than a struct and such with a flexible array. + +* The ``varnishapi`` version has been increased to 3.1 and the + functions ``VENC_Encode_Base64()`` and ``VENC_Decode_Base64()`` are + now exposed. * Two bugs in the ban expression parser have been fixed where one of them could lead to a panic if a ban expression with an empty header name was - issued (3962_) + issued (3962_). + +* The ``v_cold`` macro has been added to add ``__attribute__((cold))`` + on compilers supporting it. It is used for ``VRT_fail()`` to mark + failure code paths as cold. + +* ``varnishtest`` now generates ``User-Agent`` request and ``Server`` + response headers with the respective client and server name by + default. The ``txreq -nouseragent`` and ``txresp -noserver`` options + disable addition of these headers. + +* Error handling of invalid header names has been improved in the VCL + Compiler (3960_). + +* A race condition has been fixed in the backend probe code which + could trigger a panic with dynamic backends (dyn100_). + +* A bug has been fixed in the ESI code which would prevent use of + internal status codes >1000 as their modulus 1000 value (3958_). + +* The ``varnishd_args_prepend`` and ``varnishd_args_append`` macros + have been added to ``varnishtest`` to add arguments to ``varnishd`` + invocations before and after the defaults. + +* A bug has been fixed where ``varnishd`` would hang indefinitely when + the worker process would not come up within ``cli_timeout`` (3940_). + +* The ``startup_timeout`` parameter now specifically replaces + ``cli_timeout`` for the initial startup only (3940_). + +* On Linux, ``close_range()`` is now used if available (3905_). + +* Error reporting has been improved if the working directory + (``varnishd -n`` argument) resides on a file system mounted + ``noexec`` (3943_). + +* The number of backtrace levels in panic reports has been increased + from 10 to 20. + +* The ``PTOK()`` macro has been added to ``vas.h`` to simplify error + checking of ``pthread_*`` POSIX functions. + +* In ``varnishtest``, the basename of the test directory is now + available as the ``vtcid`` macro to serve as a unique string across + concurrently running tests. + +* In ``struct vsmwseg`` and ``struct vsm_fantom``, the ``class`` + member has been renamed to ``category``. + +* ESI ``onerror=abort`` handling has been fixed when ``max_esi_depth`` + is reached (3938_). + +* A spurious *Could not delete 'vcl\_...'* error message has been + removed (3925_). * A bug has been fixed where ``unset bereq.body`` had no effect when used with a cached body (3914_) +* ``.vcc`` files of VMODs are now installed to + ``/usr/share/varnish/vcc`` (or equivalent) to enable re-use by other + tools like code editors. + +* The :ref:`vcl-step(7)` manual page has been added to document the + VCL state machines. + +* ``HSH_Cancel()`` has been moved to ``VDP_Close()`` to enable + transports to keep references to objects. + * VCL tracing now needs to be explicitly activated by setting the ``req.trace`` or ``bereq.trace`` VCL variables, which are initialized from the ``feature +trace`` flag. Only if the trace @@ -80,20 +184,30 @@ Varnish Cache NEXT (2023-09-15) logged under the ``Error`` tag as ``Failed to create object object from %s %s``. -* ``varnishtest`` gained the macro ``varnishd_args`` to globally - append additional arguments to the ``varnishd`` command line. Macros - in this macro's value will be expanded. - * The limit on the size of ``varnishtest`` macros has been raised to 2KB. * The newly introduced abstract socket support was incompatible with other implementations, this has been fixed (3908_). +.. _3905: https://github.com/varnishcache/varnish-cache/issues/3905 .. _3908: https://github.com/varnishcache/varnish-cache/pull/3908 .. _3911: https://github.com/varnishcache/varnish-cache/issues/3911 .. _3914: https://github.com/varnishcache/varnish-cache/pull/3914 +.. _3925: https://github.com/varnishcache/varnish-cache/issues/3925 +.. _3938: https://github.com/varnishcache/varnish-cache/issues/3938 +.. _3940: https://github.com/varnishcache/varnish-cache/issues/3940 +.. _3943: https://github.com/varnishcache/varnish-cache/issues/3943 +.. _3946: https://github.com/varnishcache/varnish-cache/issues/3946 +.. _3952: https://github.com/varnishcache/varnish-cache/issues/3952 +.. _3953: https://github.com/varnishcache/varnish-cache/issues/3953 +.. _3958: https://github.com/varnishcache/varnish-cache/issues/3958 +.. _3960: https://github.com/varnishcache/varnish-cache/issues/3960 .. _3962: https://github.com/varnishcache/varnish-cache/issues/3962 +.. _3968: https://github.com/varnishcache/varnish-cache/issues/3968 +.. _3971: https://github.com/varnishcache/varnish-cache/issues/3971 + +.. _dyn100: https://github.com/nigoroll/libvmod-dynamic/issues/100 ================================ Varnish Cache 7.3.0 (2023-03-15) From nils.goroll at uplex.de Tue Sep 12 16:05:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 12 Sep 2023 16:05:06 +0000 (UTC) Subject: [master] 977430732 Drop a Copyright notice Message-ID: <20230912160506.683F47D4B@lists.varnish-cache.org> commit 9774307323db1a64a17020a9e9a8fdca21d47853 Author: Nils Goroll Date: Tue Sep 12 17:53:11 2023 +0200 Drop a Copyright notice before some lawyer comes along to tell us that we better have had this to not forfeit any claims or lose entitlements. (phk, I know your opinion about this and I hope you are right) diff --git a/configure.ac b/configure.ac index 5627b7aa9..bc38381a5 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,7 @@ AC_PREREQ(2.69) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS -Copyright (c) 2006-2023 Varnish Software]) +Copyright (c) 2006-2023 Varnish Software +Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung]) AC_REVISION([$Id$]) AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) diff --git a/lib/libvarnish/version.c b/lib/libvarnish/version.c index 02c6fb90b..0d09ab748 100644 --- a/lib/libvarnish/version.c +++ b/lib/libvarnish/version.c @@ -77,6 +77,7 @@ VCS_String(const char *which) "\n" "Copyright (c) 2006 Verdens Gang AS\n" "Copyright (c) 2006-2023 Varnish Software\n" + "Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung\n" ); default: WRONG("Wrong argument to VCS_String"); From dridi at varni.sh Tue Sep 12 17:49:36 2023 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 12 Sep 2023 17:49:36 +0000 Subject: [master] 977430732 Drop a Copyright notice In-Reply-To: <20230912160506.683F47D4B@lists.varnish-cache.org> References: <20230912160506.683F47D4B@lists.varnish-cache.org> Message-ID: On Tue, Sep 12, 2023 at 4:05?PM Nils Goroll wrote: > > > commit 9774307323db1a64a17020a9e9a8fdca21d47853 > Author: Nils Goroll > Date: Tue Sep 12 17:53:11 2023 +0200 > > Drop a Copyright notice > > before some lawyer comes along to tell us that we better have had this > to not forfeit any claims or lose entitlements. > > (phk, I know your opinion about this and I hope you are right) > > diff --git a/configure.ac b/configure.ac > index 5627b7aa9..bc38381a5 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -1,6 +1,7 @@ > AC_PREREQ(2.69) > AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS > -Copyright (c) 2006-2023 Varnish Software]) > +Copyright (c) 2006-2023 Varnish Software > +Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung]) > AC_REVISION([$Id$]) > AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) > AC_CONFIG_SRCDIR(include/miniobj.h) > diff --git a/lib/libvarnish/version.c b/lib/libvarnish/version.c > index 02c6fb90b..0d09ab748 100644 > --- a/lib/libvarnish/version.c > +++ b/lib/libvarnish/version.c > @@ -77,6 +77,7 @@ VCS_String(const char *which) > "\n" > "Copyright (c) 2006 Verdens Gang AS\n" > "Copyright (c) 2006-2023 Varnish Software\n" > + "Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung\n" Can we keep the format consistent with the lines above? > ); > default: > WRONG("Wrong argument to VCS_String"); > _______________________________________________ > varnish-commit mailing list > varnish-commit at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit From nils.goroll at uplex.de Wed Sep 13 08:27:57 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Wed, 13 Sep 2023 10:27:57 +0200 Subject: [master] 977430732 Drop a Copyright notice In-Reply-To: References: <20230912160506.683F47D4B@lists.varnish-cache.org> Message-ID: <7f13df6a-7022-af55-52a8-b8e95d97c474@uplex.de> On 12.09.23 19:49, Dridi Boukelmoune wrote: > Can we keep the format consistent with the lines above? I once learned that "Copyright (c)" would somehow be worse than just "Copyright". IIRC it had something to do with the fact that (c) it is not the Copyright symbol ? and that the requirement to use it has been dropped under U.S. law. But I am not qualified, I just applied what I once learned. Nils -- Nils Goroll (he/him) ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg tel +49 40 28805731 mob +49 170 2723133 fax +49 40 42949753 xmpp://slink at jabber.int.uplex.de/ http://uplex.de/ From dridi at varni.sh Wed Sep 13 08:55:29 2023 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 13 Sep 2023 08:55:29 +0000 Subject: [master] 977430732 Drop a Copyright notice In-Reply-To: <7f13df6a-7022-af55-52a8-b8e95d97c474@uplex.de> References: <20230912160506.683F47D4B@lists.varnish-cache.org> <7f13df6a-7022-af55-52a8-b8e95d97c474@uplex.de> Message-ID: On Wed, Sep 13, 2023 at 8:27?AM Nils Goroll wrote: > > On 12.09.23 19:49, Dridi Boukelmoune wrote: > > Can we keep the format consistent with the lines above? > > I once learned that "Copyright (c)" would somehow be worse than just > "Copyright". IIRC it had something to do with the fact that (c) it is not the > Copyright symbol ? and that the requirement to use it has been dropped under > U.S. law. > > But I am not qualified, I just applied what I once learned. I remember having this discussion with you, and I have no opinion in favor or against the (c) part, but I was requesting consistency. From nils.goroll at uplex.de Wed Sep 13 09:10:25 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Wed, 13 Sep 2023 11:10:25 +0200 Subject: [master] 977430732 Drop a Copyright notice In-Reply-To: References: <20230912160506.683F47D4B@lists.varnish-cache.org> <7f13df6a-7022-af55-52a8-b8e95d97c474@uplex.de> Message-ID: <253b558c-cdd8-91eb-661e-99b78f77ca32@uplex.de> On 13.09.23 10:55, Dridi Boukelmoune wrote: > I remember having this discussion with you, and I have no opinion in > favor or against the (c) part, but I was requesting consistency. I would not touch someone else's copyright notice. -- Nils Goroll (he/him) ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg tel +49 40 28805731 mob +49 170 2723133 fax +49 40 42949753 xmpp://slink at jabber.int.uplex.de/ http://uplex.de/ From dridi.boukelmoune at gmail.com Wed Sep 13 12:08:05 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Wed, 13 Sep 2023 12:08:05 +0000 (UTC) Subject: [master] 62272f3a4 vtc.7: s/idex/index/ Message-ID: <20230913120805.B4ED29EA1@lists.varnish-cache.org> commit 62272f3a46997fd4a8a2c7077af6269f6a1237e1 Author: Dridi Boukelmoune Date: Wed Sep 13 14:06:35 2023 +0200 vtc.7: s/idex/index/ diff --git a/bin/varnishtest/vtc_http2.c b/bin/varnishtest/vtc_http2.c index c4fe111c3..025c22340 100644 --- a/bin/varnishtest/vtc_http2.c +++ b/bin/varnishtest/vtc_http2.c @@ -1358,7 +1358,7 @@ cmd_sendhex(CMD_ARGS) * header should be added to the table, shouldn't, or mustn't be * compressed if/when retransmitted. * - * INT is the idex of the header name to use. + * INT is the index of the header name to use. * * The third argument informs about the Huffman encoding: yes (huf) or * no (plain). From nils.goroll at uplex.de Thu Sep 14 18:08:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 14 Sep 2023 18:08:06 +0000 (UTC) Subject: [master] 8695056d0 There is nothing special about releases 7.1 and 7.2 Message-ID: <20230914180806.A9AB010AB16@lists.varnish-cache.org> commit 8695056d05dc0424f3f95af424409e88c1726129 Author: Nils Goroll Date: Thu Sep 14 19:18:05 2023 +0200 There is nothing special about releases 7.1 and 7.2 diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst index 7b4442bd9..b60fd44cd 100644 --- a/doc/sphinx/whats-new/index.rst +++ b/doc/sphinx/whats-new/index.rst @@ -22,7 +22,7 @@ released versions of Varnish, see the chapters listed below.** .. toctree:: :maxdepth: 2 - + changes-trunk upgrading-trunk @@ -35,8 +35,8 @@ Varnish 7.3 changes-7.3 upgrading-7.3 -Varnish **7.2** ---------------- +Varnish 7.2 +----------- .. toctree:: :maxdepth: 2 @@ -44,8 +44,8 @@ Varnish **7.2** changes-7.2 upgrading-7.2 -Varnish **7.1** ---------------- +Varnish 7.1 +----------- .. toctree:: :maxdepth: 2 From nils.goroll at uplex.de Thu Sep 14 18:08:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 14 Sep 2023 18:08:06 +0000 (UTC) Subject: [master] c2ec2e7ee Start skeleton release notes for the next version. Message-ID: <20230914180806.C224810AB19@lists.varnish-cache.org> commit c2ec2e7ee1fb7a831253978ba6f04f24748ef84e Author: Geoff Simmons Date: Tue Sep 25 16:31:17 2018 +0200 Start skeleton release notes for the next version. Restructured so that: * 'Upgrading' is limited to work that has to be done to upgrade from a current deployment to the new version. * 'Changes' is a comprehensive, user-level description of changes and new features. Conflicts: doc/sphinx/whats-new/changes-trunk.rst doc/sphinx/whats-new/index.rst doc/sphinx/whats-new/upgrading-trunk.rst diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index 244eca45e..2070fadd5 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -13,4 +13,57 @@ merged, may be found in the `change log`_. .. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst +varnishd +======== + +Parameters +~~~~~~~~~~ + +**XXX changes in -p parameters** + +Other changes in varnishd +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Changes to VCL +============== + +VCL variables +~~~~~~~~~~~~~ + +**XXX new, deprecated or removed variables, or changed semantics** + +Other changes to VCL +~~~~~~~~~~~~~~~~~~~~ + +VMODs +===== + +**XXX changes in the bundled VMODs** + +varnishlog +========== + +**XXX changes concerning varnishlog(1) and/or vsl(7)** + +varnishadm +========== + +**XXX changes concerning varnishadm(1) and/or varnish-cli(7)** + +varnishstat +=========== + +**XXX changes concerning varnishstat(1) and/or varnish-counters(7)** + +varnishtest +=========== + +**XXX changes concerning varnishtest(1) and/or vtc(7)** + +Changes for developers and VMOD authors +======================================= + +**XXX changes concerning VRT, the public APIs, source code organization, +builds etc.** + *eof* diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst index c0a2a0a3f..6143fde99 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -1,7 +1,7 @@ **Note: This is a working document for a future release, with running updates for changes in the development branch. For changes in the released versions of Varnish, see:** :ref:`whats-new-index` - + .. _whatsnew_upgrading_CURRENT: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -14,4 +14,20 @@ features are listed in "Changes". Explicitly mention what does *not* have to be changed, especially in VCL. May include, but is not limited to:** +* Elements of VCL that have been removed or are deprecated, or whose + semantics have changed. + +* -p parameters that have been removed or are deprecated, or whose + semantics have changed. + +* Changes in the CLI. + +* Changes in the output or interpretation of stats or the log, including + changes affecting varnishncsa/-hist/-top. + +* Changes that may be necessary in VTCs or in the use of varnishtest. + +* Changes in public APIs that may require changes in VMODs or VAPI/VUT + clients. + *eof* From nils.goroll at uplex.de Thu Sep 14 18:08:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 14 Sep 2023 18:08:06 +0000 (UTC) Subject: [master] 2339adbfb Release documentation Message-ID: <20230914180806.E243A10AB1D@lists.varnish-cache.org> commit 2339adbfbba532ead6182b956f7748025f593d97 Author: Nils Goroll Date: Thu Sep 14 20:02:21 2023 +0200 Release documentation diff --git a/doc/changes.rst b/doc/changes.rst index 1d630bffb..9b09a3513 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -145,7 +145,7 @@ Varnish Cache NEXT (2023-09-15) ``/usr/share/varnish/vcc`` (or equivalent) to enable re-use by other tools like code editors. -* The :ref:`vcl-step(7)` manual page has been added to document the +* The :ref:`vcl-steps(7)` manual page has been added to document the VCL state machines. * ``HSH_Cancel()`` has been moved to ``VDP_Close()`` to enable diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index 2070fadd5..7ef34f41e 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -16,54 +16,114 @@ merged, may be found in the `change log`_. varnishd ======== -Parameters -~~~~~~~~~~ +HTTP/2 header field validation is now more strict with respect to +allowed characters. -**XXX changes in -p parameters** +The :ref:`vcl-steps(7)` manual page has been added to document the VCL +state machines. -Other changes in varnishd -~~~~~~~~~~~~~~~~~~~~~~~~~ +VCL Tracing +~~~~~~~~~~~ -Changes to VCL -============== +VCL tracing now needs to be explicitly activated by setting the +``req.trace`` or ``bereq.trace`` VCL variables, which are initialized +from the ``feature +trace`` flag. Only if the trace variables are set +will ``VCL_trace`` log records be generated. + +Consequently, ``VCL_trace`` has been removed from the default +``vsl_mask``, so any trace records will be emitted by +default. ``vsl_mask`` can still be used to filter ``VCL_trace`` +records. -VCL variables -~~~~~~~~~~~~~ +To trace ``vcl_init {}`` and ``vcl_fini {}``, set the ``feature ++trace`` flag while the vcl is loaded/discarded. -**XXX new, deprecated or removed variables, or changed semantics** +Parameters +~~~~~~~~~~ -Other changes to VCL -~~~~~~~~~~~~~~~~~~~~ +The ``startup_timeout`` parameter now specifically replaces +``cli_timeout`` for the initial startup only. -VMODs -===== +Changes to VCL +============== -**XXX changes in the bundled VMODs** +The ``Content-Length`` and ``Transfer-Encoding`` headers are now +protected. For the common use case of ``unset +(be)req.http.Content-Length`` to dismiss a body, ``unset +(be)req.body`` should be used. varnishlog ========== -**XXX changes concerning varnishlog(1) and/or vsl(7)** +Object creation failures by the selected storage engine are now logged +under the ``Error`` tag as ``Failed to create object object from %s +%s``. varnishadm ========== -**XXX changes concerning varnishadm(1) and/or varnish-cli(7)** +Tabulation of the ``vcl.list`` CLI output has been modified slightly. varnishstat =========== -**XXX changes concerning varnishstat(1) and/or varnish-counters(7)** +The counter ``MAIN.http1_iovs_flush`` has been added to track the +number of premature ``writev()`` calls due to an insufficient number +of IO vectors. This number is configured through the ``http1_iovs`` +parameter for client connections and implicitly defined by the amount +of free workspace for backend connections. varnishtest =========== -**XXX changes concerning varnishtest(1) and/or vtc(7)** +The basename of the test directory is now available as the ``vtcid`` +macro to serve as a unique string across concurrently running tests. + +The ``varnishd_args_prepend`` and ``varnishd_args_append`` macros have +been added to allow addition of arguments to ``varnishd`` invocations +before and after those added by ``varnishtest`` by default. + +``User-Agent`` request and ``Server`` response headers are now created +by default, containing the respective client and server name. The +``txreq -nouseragent`` and ``txresp -noserver`` options disable +addition of these headers. Changes for developers and VMOD authors ======================================= -**XXX changes concerning VRT, the public APIs, source code organization, -builds etc.** +Call sites of VMOD functions and methods can now be restricted to +built-in subroutines using the ``$Restrict`` stanza in the VCC file. + +``.vcc`` files of VMODs are now installed to +``/usr/share/varnish/vcc`` (or equivalent) to enable re-use by other +tools like code editors. + +API Changes +~~~~~~~~~~~ + +The ``varnishapi`` version has been increased to 3.1 and the +``VSHA256_*``, ``VENC_Encode_Base64()`` and ``VENC_Decode_Base64()`` +functions are now exposed. + +In ``struct vsmwseg`` and ``struct vsm_fantom``, the ``class`` member +has been renamed to ``category``. + +The ``VSB_quote_pfx()`` (and, consequently, ``VSB_quote()``) function +no longer produces ``\v`` for a vertical tab. This improves +compatibility with JSON. + +Additions to varnish C header files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``PTOK()`` macro has been added to ``vas.h`` to simplify error +checking of ``pthread_*`` POSIX functions. + +The ``v_cold`` macro has been added to add ``__attribute__((cold))`` +on compilers supporting it. It is used for ``VRT_fail()`` to mark +failure code paths as cold. + +The utility macros ``ALLOC_OBJ_EXTRA()`` and ``ALLOC_FLEX_OBJ()`` have +been added to ``miniobj.h`` to simplify allocation of objects larger +than a struct and such with a flexible array. *eof* diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst index 6143fde99..f04a2d933 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -8,26 +8,27 @@ released versions of Varnish, see:** :ref:`whats-new-index` Upgrading to Varnish **$NEXT_RELEASE** %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -**XXX: how to upgrade from previous deployments to this -version. Limited to work that has to be done for an upgrade, new -features are listed in "Changes". Explicitly mention what does *not* -have to be changed, especially in VCL. May include, but is not limited -to:** +Important VCL Changes +===================== -* Elements of VCL that have been removed or are deprecated, or whose - semantics have changed. +When upgrading from Varnish-Cache 7.3, there is only one breaking +change to consider in VCL: -* -p parameters that have been removed or are deprecated, or whose - semantics have changed. +The ``Content-Length`` and ``Transfer-Encoding`` headers are now +*protected*, they can neither be changed nor unset. This change was +implemented to avoid de-sync issues from accidental, inadequate +modifications of these headers. -* Changes in the CLI. +For the common use case of ``unset (be)req.http.Content-Length`` to +dismiss a request body, ``unset (be)req.body`` should be used. -* Changes in the output or interpretation of stats or the log, including - changes affecting varnishncsa/-hist/-top. +Parameter Changes +================= -* Changes that may be necessary in VTCs or in the use of varnishtest. - -* Changes in public APIs that may require changes in VMODs or VAPI/VUT - clients. +The new ``varnishd`` parameter ``startup_timeout`` now specifically +replaces ``cli_timeout`` for the initial startup only. In cases where +``cli_timeout`` was increased specifically to accommodate long startup +times (e.g. for storage engine initialization), ``startup_timeout`` +should be used. *eof* From nils.goroll at uplex.de Thu Sep 14 18:08:07 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 14 Sep 2023 18:08:07 +0000 (UTC) Subject: [master] 2d7c601b4 Polish previous release documents Message-ID: <20230914180807.1BCCD10AB23@lists.varnish-cache.org> commit 2d7c601b43e7c9ce419736b42a99af7fae45b410 Author: Nils Goroll Date: Thu Sep 14 20:05:38 2023 +0200 Polish previous release documents Remove boldface and XXXes which were intended to be removed before release. diff --git a/doc/sphinx/whats-new/changes-7.1.rst b/doc/sphinx/whats-new/changes-7.1.rst index 0b250db85..24ac45e18 100644 --- a/doc/sphinx/whats-new/changes-7.1.rst +++ b/doc/sphinx/whats-new/changes-7.1.rst @@ -1,8 +1,8 @@ .. _whatsnew_changes_7.1: -%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **7.1** -%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish 7.1 +%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the new version, see :ref:`whatsnew_upgrading_7.1`. diff --git a/doc/sphinx/whats-new/changes-7.2.rst b/doc/sphinx/whats-new/changes-7.2.rst index dee76fd9b..c7fbbbf09 100644 --- a/doc/sphinx/whats-new/changes-7.2.rst +++ b/doc/sphinx/whats-new/changes-7.2.rst @@ -1,8 +1,8 @@ .. _whatsnew_changes_7.2: -%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **7.2** -%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish 7.2 +%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the new version, see :ref:`whatsnew_upgrading_7.2`. diff --git a/doc/sphinx/whats-new/changes-7.3.rst b/doc/sphinx/whats-new/changes-7.3.rst index a5877d8d3..b6423edb6 100644 --- a/doc/sphinx/whats-new/changes-7.3.rst +++ b/doc/sphinx/whats-new/changes-7.3.rst @@ -1,8 +1,8 @@ .. _whatsnew_changes_7.3: -%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **7.3** -%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish 7.3 +%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the new version, see :ref:`whatsnew_upgrading_7.3`. @@ -19,8 +19,6 @@ varnishd Parameters ~~~~~~~~~~ -**XXX changes in -p parameters** - There is a new parameter ``transit_buffer`` disabled by default to limit the amount of storage used for uncacheable responses. This is useful in situations where slow clients may consume large but uncacheable objects, to prevent them @@ -76,8 +74,6 @@ Changes to VCL VCL variables ~~~~~~~~~~~~~ -**XXX new, deprecated or removed variables, or changed semantics** - The variables ``req.xid``, ``bereq.xid`` and ``sess.xid`` are now integers instead of strings, but should remain usable without a VCL change in a string context. @@ -119,13 +115,9 @@ after the probe times out. VMODs ===== -**XXX changes in the bundled VMODs** - varnishlog ========== -**XXX changes concerning varnishlog(1) and/or vsl(7)** - The in-memory and on-disk format of VSL records changed to allow 64bit VXID numbers. The new binary format is **not compatible** with previous versions, and log dumps performed with a previous Varnish @@ -141,21 +133,12 @@ find the client IP address. varnishadm ========== -**XXX changes concerning varnishadm(1) and/or varnish-cli(7)** - The ``debug.xid`` command generally used by ``varnishtest`` now sets up the next VXID directly. -varnishstat -=========== - -**XXX changes concerning varnishstat(1) and/or varnish-counters(7)** - varnishtest =========== -**XXX changes concerning varnishtest(1) and/or vtc(7)** - It is now possible to send special keys NPAGE, PPAGE, HOME and END to a process. @@ -168,9 +151,6 @@ available for HTTP/2 checks. Changes for developers and VMOD authors ======================================= -**XXX changes concerning VRT, the public APIs, source code organization, -builds etc.** - There is a new convenience macro ``WS_TASK_ALLOC_OBJ()`` to allocate objects from the current tasks' workspace. diff --git a/doc/sphinx/whats-new/upgrading-7.1.rst b/doc/sphinx/whats-new/upgrading-7.1.rst index 5d458c582..86ec9eb89 100644 --- a/doc/sphinx/whats-new/upgrading-7.1.rst +++ b/doc/sphinx/whats-new/upgrading-7.1.rst @@ -1,8 +1,8 @@ .. _whatsnew_upgrading_7.1: -%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **7.1** -%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish 7.1 +%%%%%%%%%%%%%%%%%%%%%%%% varnishd ======== diff --git a/doc/sphinx/whats-new/upgrading-7.2.rst b/doc/sphinx/whats-new/upgrading-7.2.rst index a1ae38ea3..c7776efc5 100644 --- a/doc/sphinx/whats-new/upgrading-7.2.rst +++ b/doc/sphinx/whats-new/upgrading-7.2.rst @@ -1,8 +1,8 @@ .. _whatsnew_upgrading_7.2: -%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **7.2** -%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish 7.2 +%%%%%%%%%%%%%%%%%%%%%%%% varnishd ======== diff --git a/doc/sphinx/whats-new/upgrading-7.3.rst b/doc/sphinx/whats-new/upgrading-7.3.rst index 1ce10f40a..38ae1cd7d 100644 --- a/doc/sphinx/whats-new/upgrading-7.3.rst +++ b/doc/sphinx/whats-new/upgrading-7.3.rst @@ -1,8 +1,8 @@ .. _whatsnew_upgrading_7.3: -%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **7.3** -%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish 7.3 +%%%%%%%%%%%%%%%%%%%%%%%% New VSL format ============== From nils.goroll at uplex.de Fri Sep 15 07:11:09 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 15 Sep 2023 07:11:09 +0000 (UTC) Subject: [master] cd23b7bce Polish iteration Message-ID: <20230915071109.A58C963D09@lists.varnish-cache.org> commit cd23b7bce239e64de8f1a8def0910b69972ef573 Author: Nils Goroll Date: Fri Sep 15 09:09:50 2023 +0200 Polish iteration Remove an empty section left after 2d7c601b43e7c9ce419736b42a99af7fae45b410 as noticed by Dridi. diff --git a/doc/sphinx/whats-new/changes-7.3.rst b/doc/sphinx/whats-new/changes-7.3.rst index b6423edb6..dc6af79fc 100644 --- a/doc/sphinx/whats-new/changes-7.3.rst +++ b/doc/sphinx/whats-new/changes-7.3.rst @@ -112,9 +112,6 @@ Probes have a new ``.expect_close`` attribute defaulting to ``true``, matching the current behavior. Setting it to ``false`` will defer final checks until after the probe times out. -VMODs -===== - varnishlog ========== From nils.goroll at uplex.de Fri Sep 15 07:11:09 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 15 Sep 2023 07:11:09 +0000 (UTC) Subject: [master] ddd39a7dd Consistently refer to vcl-step.rst as vcl-step Message-ID: <20230915071109.9149063D05@lists.varnish-cache.org> commit ddd39a7dda6d2bda128d70830caa710f75af5048 Author: Nils Goroll Date: Fri Sep 15 09:07:00 2023 +0200 Consistently refer to vcl-step.rst as vcl-step from Dridi via email diff --git a/doc/changes.rst b/doc/changes.rst index 9b09a3513..1d630bffb 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -145,7 +145,7 @@ Varnish Cache NEXT (2023-09-15) ``/usr/share/varnish/vcc`` (or equivalent) to enable re-use by other tools like code editors. -* The :ref:`vcl-steps(7)` manual page has been added to document the +* The :ref:`vcl-step(7)` manual page has been added to document the VCL state machines. * ``HSH_Cancel()`` has been moved to ``VDP_Close()`` to enable diff --git a/doc/sphinx/reference/vcl-step.rst b/doc/sphinx/reference/vcl-step.rst index 6deff971b..fd33764c0 100644 --- a/doc/sphinx/reference/vcl-step.rst +++ b/doc/sphinx/reference/vcl-step.rst @@ -6,7 +6,7 @@ .. role:: ref(emphasis) -.. _vcl-steps(7): +.. _vcl-step(7): ========= VCL-steps diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index 5143f0ff0..b218ae416 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -445,7 +445,7 @@ SEE ALSO * :ref:`varnishd(1)` * :ref:`vcl-backend(7)` * :ref:`vcl-probe(7)` -* :ref:`vcl-steps(7)` +* :ref:`vcl-step(7)` * :ref:`vcl-var(7)` * :ref:`vmod_directors(3)` * :ref:`vmod_std(3)` diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index 7ef34f41e..46982c9c4 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -19,7 +19,7 @@ varnishd HTTP/2 header field validation is now more strict with respect to allowed characters. -The :ref:`vcl-steps(7)` manual page has been added to document the VCL +The :ref:`vcl-step(7)` manual page has been added to document the VCL state machines. VCL Tracing From simon.stridsberg at varnish-software.com Fri Sep 15 14:33:06 2023 From: simon.stridsberg at varnish-software.com (Simon Stridsberg) Date: Fri, 15 Sep 2023 14:33:06 +0000 (UTC) Subject: [master] f07a6d615 Prepare for 7.4.0 Message-ID: <20230915143306.0D4A710D7D3@lists.varnish-cache.org> commit f07a6d615c054cf73c9250f3f09d063ac344dff2 Author: Simon Stridsberg Date: Fri Sep 15 14:31:35 2023 +0200 Prepare for 7.4.0 diff --git a/bin/varnishtest/tests/m00003.vtc b/bin/varnishtest/tests/m00003.vtc index 24f3991b1..1b6a90faa 100644 --- a/bin/varnishtest/tests/m00003.vtc +++ b/bin/varnishtest/tests/m00003.vtc @@ -96,7 +96,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so "\x03" varnish v1 -errvcl {VMOD wants ABI version 1.0} { import wrong; } ############################################################# -# NB: in the tests below "17" should track VRT_MAJOR_VERSION +# NB: in the tests below "18" should track VRT_MAJOR_VERSION filewrite ${tmpdir}/libvmod_wrong.so "VMOD_JSON_SPEC\x02" filewrite -a ${tmpdir}/libvmod_wrong.so { @@ -108,7 +108,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "17", + "18", "0" ], [ "$FOOBAR" @@ -128,7 +128,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "17", + "18", "0" ] ] @@ -146,7 +146,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "17", + "18", "0" ], [ "$CPROTO" @@ -168,7 +168,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_std_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "17", + "18", "0" ], [ "$CPROTO", "/* blabla */" diff --git a/bin/varnishtest/tests/m00055.vtc b/bin/varnishtest/tests/m00055.vtc index 39172c16c..f79182b8a 100644 --- a/bin/varnishtest/tests/m00055.vtc +++ b/bin/varnishtest/tests/m00055.vtc @@ -21,7 +21,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "17", + "18", "0" ], [ diff --git a/configure.ac b/configure.ac index bc38381a5..d72d10dc5 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2023 Varnish Software Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [7.4.0], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/changes.rst b/doc/changes.rst index 1d630bffb..5253888cb 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -34,9 +34,9 @@ http://varnish-cache.org/docs/trunk/whats-new/index.html and via individual releases. These documents are updated as part of the release process. -=============================== -Varnish Cache NEXT (2023-09-15) -=============================== +================================ +Varnish Cache 7.4.0 (2023-09-15) +================================ .. PLEASE keep this roughly in commit order as shown by git-log / tig (new to old) diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-7.4.rst similarity index 97% rename from doc/sphinx/whats-new/changes-trunk.rst rename to doc/sphinx/whats-new/changes-7.4.rst index 46982c9c4..53c56f90f 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-7.4.rst @@ -1,11 +1,11 @@ -.. _whatsnew_changes_CURRENT: +.. _whatsnew_changes_7.4: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **${NEXT_RELEASE}** +Changes in Varnish **7.4** %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the -new version, see :ref:`whatsnew_upgrading_CURRENT`. +new version, see :ref:`whatsnew_upgrading_7.4`. A more detailed and technical account of changes in Varnish, with links to issues that have been fixed and pull requests that have been diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst index b60fd44cd..f3c199b4c 100644 --- a/doc/sphinx/whats-new/index.rst +++ b/doc/sphinx/whats-new/index.rst @@ -13,18 +13,14 @@ This section describes the changes and improvements between different versions of Varnish, and what upgrading between the different versions entail. -Varnish **$NEXT_RELEASE** -------------------------- - -**Note: These are working documents for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see the chapters listed below.** +Varnish 7.4 +----------- .. toctree:: :maxdepth: 2 - changes-trunk - upgrading-trunk + changes-7.4 + upgrading-7.4 Varnish 7.3 ----------- diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-7.4.rst similarity index 70% rename from doc/sphinx/whats-new/upgrading-trunk.rst rename to doc/sphinx/whats-new/upgrading-7.4.rst index f04a2d933..cab2f3824 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-7.4.rst @@ -1,12 +1,8 @@ -**Note: This is a working document for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see:** :ref:`whats-new-index` +.. _whatsnew_upgrading_7.4: -.. _whatsnew_upgrading_CURRENT: - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **$NEXT_RELEASE** -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish 7.4 +%%%%%%%%%%%%%%%%%%%%%%%% Important VCL Changes ===================== diff --git a/include/vrt.h b/include/vrt.h index fedc26da7..36f7081d8 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -46,7 +46,7 @@ # error "include vdef.h before vrt.h" #endif -#define VRT_MAJOR_VERSION 17U +#define VRT_MAJOR_VERSION 18U #define VRT_MINOR_VERSION 0U @@ -57,7 +57,7 @@ * Whenever something is deleted or changed in a way which is not * binary/load-time compatible, increment MAJOR version * - * NEXT (2023-09-15) + * 18.0 (2023-09-15) * [cache_filter.h] struct vdp gained priv1 member * VRT_trace() added * 17.0 (2023-03-15) From simon.stridsberg at varnish-software.com Fri Sep 15 14:33:06 2023 From: simon.stridsberg at varnish-software.com (Simon Stridsberg) Date: Fri, 15 Sep 2023 14:33:06 +0000 (UTC) Subject: [master] d49ac0a23 Merge tag 'varnish-7.4.0' Message-ID: <20230915143306.2860610D7D7@lists.varnish-cache.org> commit d49ac0a239faa8c89a8bdd9af7946aaf5da4ed8e Merge: cd23b7bce f07a6d615 Author: Simon Stridsberg Date: Fri Sep 15 16:32:07 2023 +0200 Merge tag 'varnish-7.4.0' Releasing 7.4.0 From dridi.boukelmoune at gmail.com Mon Sep 18 05:21:08 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 18 Sep 2023 05:21:08 +0000 (UTC) Subject: [master] cb4cad349 build: Back to the future Message-ID: <20230918052108.304CF112E41@lists.varnish-cache.org> commit cb4cad34990178b604238cbf2b0520ae9330c430 Author: Dridi Boukelmoune Date: Mon Sep 18 07:19:46 2023 +0200 build: Back to the future diff --git a/configure.ac b/configure.ac index d72d10dc5..bc38381a5 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2023 Varnish Software Copyright 2010-2023 UPLEX - Nils Goroll Systemoptimierung]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [7.4.0], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) From nils.goroll at uplex.de Mon Sep 18 15:16:06 2023 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Sep 2023 15:16:06 +0000 (UTC) Subject: [master] d19ee66d4 Reflect API rename in docs Message-ID: <20230918151606.17F24654B1@lists.varnish-cache.org> commit d19ee66d49787248367b7751beb35bb96adaf9ab Author: Nils Goroll Date: Mon Sep 18 17:14:45 2023 +0200 Reflect API rename in docs Ref 5df6f8d46cb08ae435f866b8e6c1bb333cebdee3 diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst index 135983ff2..b8a524ce6 100644 --- a/doc/sphinx/reference/vmod.rst +++ b/doc/sphinx/reference/vmod.rst @@ -741,15 +741,15 @@ not be left half-initialized should a failure occur. If your VMOD is running an asynchronous background job you can hold a reference to the VCL to prevent it from going cold too soon and get the same guarantees as backends with ongoing requests for instance. For that, you must acquire the -reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and -later calling ``VRT_rel_vcl`` once the background job is over. Receiving a +reference by calling ``VRT_VCL_Prevent_Discard`` when you receive a ``VCL_EVENT_WARM`` and +later calling ``VRT_VCL_Allow_Discard`` once the background job is over. Receiving a ``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL. You can find an example of VCL references in vmod-debug:: - priv_vcl->vclref = VRT_ref_vcl(ctx, "vmod-debug"); + priv_vcl->vclref = VRT_VCL_Prevent_Discard(ctx, "vmod-debug"); ... - VRT_rel_vcl(&ctx, &priv_vcl->vclref); + VRT_VCL_Allow_Discard(&ctx, &priv_vcl->vclref); In this simplified version, you can see that you need at least a VCL-bound data structure like a ``PRIV_VCL`` or a VMOD object to keep track of the reference From dridi.boukelmoune at gmail.com Tue Sep 19 04:27:07 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 04:27:07 +0000 (UTC) Subject: [master] f8e2f7a28 man: Fix the scope of beresp protected headers Message-ID: <20230919042707.702BDAFAAA@lists.varnish-cache.org> commit f8e2f7a2864c46c9ec2197cf35e53cc9f06b42a0 Author: Dridi Boukelmoune Date: Tue Sep 19 06:22:48 2023 +0200 man: Fix the scope of beresp protected headers Refs #3984 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index b9b74f0e3..392e3efe7 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -1164,7 +1164,7 @@ beresp.http.content-length Type: HEADER - Readable from: client + Readable from: vcl_backend_response, vcl_backend_error The content-length header field is protected, see protected_headers_. @@ -1173,7 +1173,7 @@ beresp.http.transfer-encoding Type: HEADER - Readable from: client + Readable from: vcl_backend_response, vcl_backend_error The transfer-encoding header field is protected, see protected_headers_. From dridi.boukelmoune at gmail.com Tue Sep 19 04:50:07 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 04:50:07 +0000 (UTC) Subject: [master] 00d22b2d9 man: Fix the scope of bereq protected headers Message-ID: <20230919045007.0E1C6B0C14@lists.varnish-cache.org> commit 00d22b2d98ff874dd04114401178deb49eaa34cb Author: Dridi Boukelmoune Date: Tue Sep 19 06:49:30 2023 +0200 man: Fix the scope of bereq protected headers Refs #3984 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index 392e3efe7..faed3aaa9 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -742,7 +742,7 @@ bereq.http.content-length Type: HEADER - Readable from: client + Readable from: backend The content-length header field is protected, see protected_headers_. @@ -751,7 +751,7 @@ bereq.http.transfer-encoding Type: HEADER - Readable from: client + Readable from: backend The transfer-encoding header field is protected, see protected_headers_. From dridi.boukelmoune at gmail.com Tue Sep 19 04:56:04 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 04:56:04 +0000 (UTC) Subject: [7.4] c6afd311f Reflect API rename in docs Message-ID: <20230919045604.C22ECB107F@lists.varnish-cache.org> commit c6afd311f05eb6ff270c36459d50bae0586d1ebb Author: Nils Goroll Date: Mon Sep 18 17:14:45 2023 +0200 Reflect API rename in docs Ref 5df6f8d46cb08ae435f866b8e6c1bb333cebdee3 diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst index 135983ff2..b8a524ce6 100644 --- a/doc/sphinx/reference/vmod.rst +++ b/doc/sphinx/reference/vmod.rst @@ -741,15 +741,15 @@ not be left half-initialized should a failure occur. If your VMOD is running an asynchronous background job you can hold a reference to the VCL to prevent it from going cold too soon and get the same guarantees as backends with ongoing requests for instance. For that, you must acquire the -reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and -later calling ``VRT_rel_vcl`` once the background job is over. Receiving a +reference by calling ``VRT_VCL_Prevent_Discard`` when you receive a ``VCL_EVENT_WARM`` and +later calling ``VRT_VCL_Allow_Discard`` once the background job is over. Receiving a ``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL. You can find an example of VCL references in vmod-debug:: - priv_vcl->vclref = VRT_ref_vcl(ctx, "vmod-debug"); + priv_vcl->vclref = VRT_VCL_Prevent_Discard(ctx, "vmod-debug"); ... - VRT_rel_vcl(&ctx, &priv_vcl->vclref); + VRT_VCL_Allow_Discard(&ctx, &priv_vcl->vclref); In this simplified version, you can see that you need at least a VCL-bound data structure like a ``PRIV_VCL`` or a VMOD object to keep track of the reference From dridi.boukelmoune at gmail.com Tue Sep 19 04:56:04 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 04:56:04 +0000 (UTC) Subject: [7.4] 2e92db9fc man: Fix the scope of beresp protected headers Message-ID: <20230919045605.1CD04B1084@lists.varnish-cache.org> commit 2e92db9fcb9c3f3cab11498179cabd6722e19540 Author: Dridi Boukelmoune Date: Tue Sep 19 06:22:48 2023 +0200 man: Fix the scope of beresp protected headers Refs #3984 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index b9b74f0e3..392e3efe7 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -1164,7 +1164,7 @@ beresp.http.content-length Type: HEADER - Readable from: client + Readable from: vcl_backend_response, vcl_backend_error The content-length header field is protected, see protected_headers_. @@ -1173,7 +1173,7 @@ beresp.http.transfer-encoding Type: HEADER - Readable from: client + Readable from: vcl_backend_response, vcl_backend_error The transfer-encoding header field is protected, see protected_headers_. From dridi.boukelmoune at gmail.com Tue Sep 19 04:56:05 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 04:56:05 +0000 (UTC) Subject: [7.4] 3d8c1800a man: Fix the scope of bereq protected headers Message-ID: <20230919045605.35A4AB1087@lists.varnish-cache.org> commit 3d8c1800aaafb7ba66e91e924a8d9803a131cf4b Author: Dridi Boukelmoune Date: Tue Sep 19 06:49:30 2023 +0200 man: Fix the scope of bereq protected headers Refs #3984 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index 392e3efe7..faed3aaa9 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -742,7 +742,7 @@ bereq.http.content-length Type: HEADER - Readable from: client + Readable from: backend The content-length header field is protected, see protected_headers_. @@ -751,7 +751,7 @@ bereq.http.transfer-encoding Type: HEADER - Readable from: client + Readable from: backend The transfer-encoding header field is protected, see protected_headers_. From dridi.boukelmoune at gmail.com Tue Sep 19 10:04:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 19 Sep 2023 10:04:06 +0000 (UTC) Subject: [7.4] 07c34090c man: Fix the scope of resp protected headers Message-ID: <20230919100406.1343963841@lists.varnish-cache.org> commit 07c34090c01eb7129ec92f4f37ff5e6eb8a77511 Author: Dridi Boukelmoune Date: Tue Sep 19 12:03:12 2023 +0200 man: Fix the scope of resp protected headers Refs #3984 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index faed3aaa9..5d320537f 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -1614,7 +1614,7 @@ resp.http.content-length Type: HEADER - Readable from: client + Readable from: vcl_deliver, vcl_synth The content-length header field is protected, see protected_headers_. @@ -1623,7 +1623,7 @@ resp.http.transfer-encoding Type: HEADER - Readable from: client + Readable from: vcl_deliver, vcl_synth The transfer-encoding header field is protected, see protected_headers_. From dridi.boukelmoune at gmail.com Wed Sep 20 07:06:08 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Wed, 20 Sep 2023 07:06:08 +0000 (UTC) Subject: [7.4] c38de3ea9 vtc: Add coverage for protected headers Message-ID: <20230920070608.2937E65041@lists.varnish-cache.org> commit c38de3ea985e772d8c532ab0e7d8d936700ad5a0 Author: Dridi Boukelmoune Date: Wed Sep 20 09:03:08 2023 +0200 vtc: Add coverage for protected headers Closes #3984 diff --git a/bin/varnishtest/tests/r03984.vtc b/bin/varnishtest/tests/r03984.vtc new file mode 100644 index 000000000..3ce6975ef --- /dev/null +++ b/bin/varnishtest/tests/r03984.vtc @@ -0,0 +1,57 @@ +varnishtest "Access protected headers" + +varnish v1 -vcl { + backend be none; + + sub access_req { + if (req.http.content-length || req.http.content-encoding) {} + } + sub access_resp { + if (resp.http.content-length || resp.http.content-encoding) {} + } + sub access_bereq { + if (bereq.http.content-length || bereq.http.content-encoding) {} + } + sub access_beresp { + if (beresp.http.content-length || beresp.http.content-encoding) {} + } + + sub vcl_recv { + call access_req; + } + sub vcl_hash { + call access_req; + } + sub vcl_purge { + call access_req; + } + sub vcl_miss { + call access_req; + } + sub vcl_pass { + call access_req; + } + sub vcl_hit { + call access_req; + } + sub vcl_synth { + call access_req; + call access_resp; + } + sub vcl_deliver { + call access_req; + call access_resp; + } + + sub vcl_backend_fetch { + call access_bereq; + } + sub vcl_backend_error { + call access_bereq; + call access_beresp; + } + sub vcl_backend_response { + call access_bereq; + call access_beresp; + } +} From dridi.boukelmoune at gmail.com Thu Sep 21 04:54:08 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Thu, 21 Sep 2023 04:54:08 +0000 (UTC) Subject: [7.4] 13ae7d9ed vtc: Check the correct protected headers Message-ID: <20230921045408.6B37C104D28@lists.varnish-cache.org> commit 13ae7d9edcd90ba2e8019ba59a73406c6f441dbd Author: Dridi Boukelmoune Date: Thu Sep 21 06:52:11 2023 +0200 vtc: Check the correct protected headers Refs #3984 diff --git a/bin/varnishtest/tests/r03984.vtc b/bin/varnishtest/tests/r03984.vtc index 3ce6975ef..4285760da 100644 --- a/bin/varnishtest/tests/r03984.vtc +++ b/bin/varnishtest/tests/r03984.vtc @@ -4,16 +4,16 @@ varnish v1 -vcl { backend be none; sub access_req { - if (req.http.content-length || req.http.content-encoding) {} + if (req.http.content-length || req.http.transfer-encoding) {} } sub access_resp { - if (resp.http.content-length || resp.http.content-encoding) {} + if (resp.http.content-length || resp.http.transfer-encoding) {} } sub access_bereq { - if (bereq.http.content-length || bereq.http.content-encoding) {} + if (bereq.http.content-length || bereq.http.transfer-encoding) {} } sub access_beresp { - if (beresp.http.content-length || beresp.http.content-encoding) {} + if (beresp.http.content-length || beresp.http.transfer-encoding) {} } sub vcl_recv { From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:06 +0000 (UTC) Subject: [master] 70a7a70fd hpack: Turn header validation state into an enum Message-ID: <20230926130206.806671118AD@lists.varnish-cache.org> commit 70a7a70fde279c2b8c591b71d1bb6ee98cf8ff7c Author: Walid Boudebouda Date: Fri Sep 8 16:55:18 2023 +0200 hpack: Turn header validation state into an enum Signed-off-by: Dridi Boukelmoune diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c index 6261a2cd1..cc33ec78c 100644 --- a/bin/varnishd/http2/cache_http2_hpack.c +++ b/bin/varnishd/http2/cache_http2_hpack.c @@ -44,6 +44,12 @@ static h2_error h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) { const char *p; + enum { + FLD_NAME_FIRST, + FLD_NAME, + FLD_VALUE_FIRST, + FLD_VALUE + } state; CHECK_OBJ_NOTNULL(hp, HTTP_MAGIC); AN(b); @@ -60,15 +66,15 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) // VSLb(hp->vsl, SLT_Debug, "CHDR [%.*s] [%.*s]", // (int)namelen, b, (int)(len - namelen), b + namelen); - int state = 0; + state = FLD_NAME_FIRST; for (p = b; p < b + namelen - 2; p++) { switch(state) { - case 0: /* First char of field */ - state = 1; + case FLD_NAME_FIRST: + state = FLD_NAME; if (*p == ':') break; /* FALL_THROUGH */ - case 1: /* field name */ + case FLD_NAME: if (*p <= 0x20 || *p >= 0x7f) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field header name (control): %.*s", @@ -93,19 +99,19 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) } } - state = 2; + state = FLD_VALUE_FIRST; for (p = b + namelen; p < b + len; p++) { switch(state) { - case 2: /* First char of field */ + case FLD_VALUE_FIRST: if (*p == ' ' || *p == 0x09) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field value start %.*s", (int)(len > 20 ? 20 : len), b); return (H2SE_PROTOCOL_ERROR); } - state = 3; + state = FLD_VALUE; /* FALL_THROUGH */ - case 3: /* field value character */ + case FLD_VALUE: if (*p != 0x09 && (*p < 0x20 || *p == 0x7f)) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field value (control) %.*s", @@ -117,7 +123,7 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) WRONG("http2 field value validation state"); } } - if (state == 3 && b[len - 1] <= 0x20) { + if (state == FLD_VALUE && b[len - 1] <= 0x20) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field value (end) %.*s", (int)(len > 20 ? 20 : len), b); From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:06 +0000 (UTC) Subject: [master] 48875ccde hpack: Check illegal header blanks with vct_issp() Message-ID: <20230926130206.95AFD1118B0@lists.varnish-cache.org> commit 48875ccde86969aed1641ec1b77ac2c2459203c9 Author: Walid Boudebouda Date: Fri Sep 8 17:25:06 2023 +0200 hpack: Check illegal header blanks with vct_issp() Signed-off-by: Dridi Boukelmoune diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c index cc33ec78c..35610b616 100644 --- a/bin/varnishd/http2/cache_http2_hpack.c +++ b/bin/varnishd/http2/cache_http2_hpack.c @@ -103,7 +103,7 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) for (p = b + namelen; p < b + len; p++) { switch(state) { case FLD_VALUE_FIRST: - if (*p == ' ' || *p == 0x09) { + if (vct_issp(*p)) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field value start %.*s", (int)(len > 20 ? 20 : len), b); @@ -123,7 +123,7 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) WRONG("http2 field value validation state"); } } - if (state == FLD_VALUE && b[len - 1] <= 0x20) { + if (state == FLD_VALUE && vct_issp(b[len - 1])) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field value (end) %.*s", (int)(len > 20 ? 20 : len), b); From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:06 +0000 (UTC) Subject: [master] 0ba31d78d hpack: Validate header values with vct_ishdrval() Message-ID: <20230926130206.AE9DE1118B3@lists.varnish-cache.org> commit 0ba31d78d89168ca888938383472bed0815aeb98 Author: Walid Boudebouda Date: Fri Sep 8 17:37:26 2023 +0200 hpack: Validate header values with vct_ishdrval() Signed-off-by: Dridi Boukelmoune diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c index 35610b616..92befad53 100644 --- a/bin/varnishd/http2/cache_http2_hpack.c +++ b/bin/varnishd/http2/cache_http2_hpack.c @@ -112,9 +112,9 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) state = FLD_VALUE; /* FALL_THROUGH */ case FLD_VALUE: - if (*p != 0x09 && (*p < 0x20 || *p == 0x7f)) { + if (!vct_ishdrval(*p)) { VSLb(hp->vsl, SLT_BogoHeader, - "Illegal field value (control) %.*s", + "Illegal field value %.*s", (int)(len > 20 ? 20 : len), b); return (H2SE_PROTOCOL_ERROR); } From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:06 +0000 (UTC) Subject: [master] 325faac2c hpack: Remove redundant/incorrect header validation Message-ID: <20230926130206.C7D1A1118B9@lists.varnish-cache.org> commit 325faac2cdfb2080f2ecccd340da49b1c41f7fe5 Author: Walid Boudebouda Date: Fri Sep 8 17:13:19 2023 +0200 hpack: Remove redundant/incorrect header validation Control characters will be caught by vct_ishdrval() anyways, but this condition would also reject allowed obs-text non-ASCII characters. Signed-off-by: Dridi Boukelmoune diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c index 92befad53..b5451eb22 100644 --- a/bin/varnishd/http2/cache_http2_hpack.c +++ b/bin/varnishd/http2/cache_http2_hpack.c @@ -75,12 +75,6 @@ h2h_checkhdr(const struct http *hp, const char *b, size_t namelen, size_t len) break; /* FALL_THROUGH */ case FLD_NAME: - if (*p <= 0x20 || *p >= 0x7f) { - VSLb(hp->vsl, SLT_BogoHeader, - "Illegal field header name (control): %.*s", - (int)(len > 20 ? 20 : len), b); - return (H2SE_PROTOCOL_ERROR); - } if (isupper(*p)) { VSLb(hp->vsl, SLT_BogoHeader, "Illegal field header name (upper-case): %.*s", From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:06 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:06 +0000 (UTC) Subject: [master] bc3e4207c vtc: More HPACK header validation coverage Message-ID: <20230926130206.DFEE91118BD@lists.varnish-cache.org> commit bc3e4207c697f6380398138d8836fe4329e57207 Author: Walid Boudebouda Date: Fri Sep 8 17:54:43 2023 +0200 vtc: More HPACK header validation coverage Adds coverage for tab characters at start/end of field value. Regarding the "fo o" " bar" header, it cumulates an error in the name and another in the value, but only one of them will trigger the expected PROTOCOL_ERROR. Only the invalid "fo o" is checked now, and the other error is part of the new coverage. Signed-off-by: Dridi Boukelmoune diff --git a/bin/varnishtest/tests/t02023.vtc b/bin/varnishtest/tests/t02023.vtc index 5f5f434bf..388f3a70a 100644 --- a/bin/varnishtest/tests/t02023.vtc +++ b/bin/varnishtest/tests/t02023.vtc @@ -30,6 +30,7 @@ client c1 { stream 1 { txreq -url "" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -37,6 +38,7 @@ client c1 { stream 1 { txreq -scheme "" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -44,6 +46,7 @@ client c1 { stream 1 { txreq -req "" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -51,8 +54,9 @@ varnish v1 -vsl_catchup client c1 { stream 1 { - txreq -hdr "fo o" " bar" + txreq -hdr "foo" " bar" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -60,6 +64,7 @@ client c1 { stream 1 { txreq -hdr "foo" " " rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -67,6 +72,7 @@ client c1 { stream 1 { txreq -hdr ":foo" "bar" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -74,6 +80,7 @@ client c1 { stream 1 { txreq -hdr "foo" "b\x0car" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -81,6 +88,7 @@ client c1 { stream 1 { txreq -hdr "f o" "bar" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -88,6 +96,7 @@ client c1 { stream 1 { txreq -hdr "f: o" "bar" rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run @@ -95,5 +104,22 @@ client c1 { stream 1 { txreq -hdr "foo" "bar " rxrst + expect rst.err == PROTOCOL_ERROR + } -run +} -run + +client c1 { + stream 1 { + txreq -hdr "foo" " bar" + rxrst + expect rst.err == PROTOCOL_ERROR + } -run +} -run + +client c1 { + stream 1 { + txreq -hdr "foo" "bar " + rxrst + expect rst.err == PROTOCOL_ERROR } -run } -run From dridi.boukelmoune at gmail.com Tue Sep 26 13:02:07 2023 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Tue, 26 Sep 2023 13:02:07 +0000 (UTC) Subject: [master] 90cb8734e vtc: Coverage for h2 empty header in t02023 Message-ID: <20230926130207.068C91118C2@lists.varnish-cache.org> commit 90cb8734e665ad851cee46ab499863010705980f Author: Dridi Boukelmoune Date: Thu Sep 14 12:59:21 2023 +0200 vtc: Coverage for h2 empty header in t02023 diff --git a/bin/varnishtest/tests/t02023.vtc b/bin/varnishtest/tests/t02023.vtc index 388f3a70a..13c4cb445 100644 --- a/bin/varnishtest/tests/t02023.vtc +++ b/bin/varnishtest/tests/t02023.vtc @@ -1,4 +1,4 @@ -varnishtest "Empty pseudo-headers" +varnishtest "Empty and invalid headers" server s1 { rxreq @@ -50,6 +50,14 @@ client c1 { } -run } -run +client c1 { + stream 1 { + txreq -hdr "empty" "" + rxresp + expect resp.status == 200 + } -run +} -run + varnish v1 -vsl_catchup client c1 {