Varnish for multiple magento sites
Riccardo Brunetti
riccardo.brunetti at host.it
Thu Jul 13 06:23:14 UTC 2023
Hi Guillaume,
I'l give it a try.
Thank you very much
Riccardo
12/07/2023, 17:27 Guillaume Quintard ha scritto:
> Hi Riccardo,
>
>
>
> You are right, I assumed that the tags were uuids of some sorts and welp, they're not.
>
>
>
> The best way would be to ask magento to fix it by including a hash of the host or something in each tag, but that's probably not going to happen any time soon.
>
>
>
> The next best way is to teach Varnish to be a bit more selective when banning. This is done in three steps.
>
>
>
> First, we are going to mark the backend response with the host it comes from (maybe Magento2 already does it in some form, in which case you can use that header instead):
>
>
>
> # add this to the beginning of vcl_backend_response
>
> sub vcl_backend_response {
>
> set beresp.http.x-host = bereq.http.host;
>
> ...
>
>
>
> Then, we change the ban() calls to only apply to the responses with the right x-host headers:
>
>
>
> if (req.http.X-Magento-Tags-Pattern) {
> ban("obj.http.x-host == + req.http.host + " && obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
> }
> if (req.http.X-Pool) {
> ban("obj.http.x-host == + req.http.host + "&& obj.http.X-Pool ~ " + req.http.X-Pool);
> }
>
>
> Lastly, we don't need the x-host header to be exposed to the client, so we strip it at the beginning of vcl_deliver:
>
>
>
> sub vcl_deliver {
>
> unset resp.http.x-host;
>
> ...
>
>
>
> disclaimer: This test is completely untested, and it's early in the morning, so you probably shouldn't trust me too much and you should test this before throwing it in prod.
>
>
>
> Technically, you *could* use req.http.host directly, but you don't want to because of the ban-lurker and its performance implications.
>
>
>
> Hope this helps.
>
>
>
> --
>
> Guillaume Quintard
>
>
>
>
>
> On Wed, Jul 12, 2023 at 4:15 AM Riccardo Brunetti <riccardo.brunetti at host.it> wrote:
>
> > Hello Guillaume.
> >
> > Thanks for your answer.
> >
> > The VCL is actually almost identical to that you mentioned in the link (I'm attaching it without references to names and IP anyway)
> >
> >
> >
> > What somehow worries me is that, if I understand, the ban is performed according to some "X-Magento-Tags".
> >
> > Now, if I look at the output of varnishlog and search for "*Tags*", what I get is:
> >
> >
> >
> > 1) while navigating the site:
> >
> > > ....
> > >
> > > - RespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> > >
> > > - RespUnset X-Magento-Tags: NAVIGATIONPRO_MENU_2
> > >
> > > - RespHeader X-Magento-Tags: store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_bottom_strenghts,cms_b
> > >
> > > - RespUnset X-Magento-Tags: store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_bottom_strenghts,cms_b
> > >
> > > - BerespHeader X-Magento-Tags: cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> > >
> > > - BerespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> > >
> > > - RespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> > >
> > > - RespUnset X-Magento-Tags: NAVIGATIONPRO_MENU_2
> > >
> > > - RespHeader X-Magento-Tags: cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> > >
> > > - RespUnset X-Magento-Tags: cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> > >
> > > .....
> >
> > 2) when performing a purge (php bin/magento c:f):
> >
> > > ...
> > >
> > > - ReqHeader X-Magento-Tags-Pattern: .*
> > >
> > > ...
> >
> > In both cases I can't see any specific reference to that particular site.
> >
> >
> >
> > Thanks again.
> >
> > Riccardo
> >
> >
> >
> > 11/07/2023, 17:09 Guillaume Quintard ha scritto:
> >
> > > Hi Ricardo,
> > >
> > >
> > >
> > > Having your VCL (even anonymized) would help here, otherwise debugging is pretty hard. For the moment, I'm going to assume you are using a variation of https://github.com/magento/magento2/blob/13e54e1b28a5d590ab885bd4df9f58877b549052/app/code/Magento/PageCache/etc/varnish6.vcl and deal in generalities.
> > >
> > >
> > >
> > > The way that vcl invalidates content is through bans: https://github.com/magento/magento2/blob/13e54e1b28a5d590ab885bd4df9f58877b549052/app/code/Magento/PageCache/etc/varnish6.vcl#L30-L47 which doesn't need the host header, it's just uses unique tags pushed by the backend in response headers.
> > >
> > > If it was using the actual purge mechanism, then modifying the host should be sufficient because purge acts on the object found in the cache (and if you can get a hit, you can get purged).
> > >
> > >
> > >
> > > Here's a good primer on invalidation: https://docs.varnish-software.com/tutorials/cache-invalidation/
> > >
> > >
> > >
> > > Kind regards,
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > >
> > > Guillaume Quintard
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Jul 11, 2023 at 4:14 AM Riccardo Brunetti <riccardo.brunetti at host.it> wrote:
> > >
> > > > Hello.
> > > >
> > > > I'm new to varnish and I have a question concerning how to manage multiple sites using the same varnish cache frontend.
> > > >
> > > >
> > > >
> > > > More specifically, I need to setup a single varnish cache server for two different Magento2 sites.
> > > >
> > > >
> > > >
> > > > Looking around I found that it is possible to manage different backends using something like:
> > > >
> > > >
> > > >
> > > > > if (req.http.host == "somesite") {
> > > > >
> > > > > set req.backend_hint = somebackend;
> > > > >
> > > > > }
> > > >
> > > > Now, I have two different Magento2 sites and, using the above expression, I can handle the two different backends.
> > > >
> > > > The problem is that I can't understand how to handle the PURGE/BAN of the two independently.
> > > >
> > > >
> > > >
> > > > As far as I understand from the .vcl file that Magento2 itself produces there is nothing inside the "purge" section that specifies which resources must be purged.
> > > >
> > > > It seems to me that is site A performs a purge, than also the cache of site B resources will be cleaned.
> > > >
> > > >
> > > >
> > > > Can you help me with this or point me to some example or tutorials?
> > > >
> > > >
> > > >
> > > > Thanks a lot
> > > >
> > > > Riccardo
> > > >
> > > > _______________________________________________
> > > >
> > > > varnish-misc mailing list
> > > >
> > > > varnish-misc at varnish-cache.org
> > > >
> > > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230713/ee471f16/attachment-0001.html>
More information about the varnish-misc
mailing list