Caching Modified URLs by Varnish instead of the original requested URL

[Uday Kumar]

Hi Guillaume,

*use includes and function calls*
This is great, thank you so much for your help!

Thanks & Regards
Uday Kumar


On Wed, Aug 23, 2023 at 1:32 AM Guillaume Quintard <
guillaume.quintard at gmail.com> wrote:

> Hi Uday,
>
> I'm not exactly sure how to read those diagrams, so I apologize if I'm
> missing the mark or if I'm too broad here.
>
> There are a few points I'd like to attract your attention to. The first
> one is that varnish doesn't cache the request or the URL. The cache is
> essentially a big hashmap/dictionary/database, in which you store the
> response. The request/url is the key for it, so you need to have it in its
> "final" form before you do anything.
>
> From what I read, you are not against it, and you just want to sanitize
> the URL in vcl_recv, but you don't like the idea of making the main file
> too unwieldy. If I got that right, then I have a nice answer for you: use
> includes and function calls.
>
> As an example:
>
> # cat /etc/varnish/url.vcl
> sub sanitize_url {
>   # do whatever modifications you need here
> }
>
> # cat /etc/varnish/default.vcl
> include "./url.vcl";
>
> sub vcl_recvl {
>   call sanitize_url;
> }
>
>
> That should get you going.
>
> Hopefully I didn't miss the mark too much here, let me know if I did.
>
> --
> Guillaume Quintard
>
>
> On Tue, Aug 22, 2023 at 3:45 AM Uday Kumar <uday.polu at indiamart.com>
> wrote:
>
>> Hello All,
>>
>>
>> For our spring boot application, we are using Varnish Caching in a
>> production environment.
>>
>>
>>
>>
>> Requirement: [To utilize cache effectively]
>>
>> Modify the URL (Removal of unnecessary parameters) while caching the user
>> request, so that the modified URL can be cached by varnish which helps
>> improve cache HITS for similar URLs.
>>
>>
>> For Example:
>>
>> Let's consider the below Request URL
>>
>> Url at time t, 1. samplehost.com/search/ims?q=bags&source=android
>> &options.start=0
>>
>>
>> Our Requirement:
>>
>> To make varnish consider URLs with options.start=0 and without
>> options.start parameter as EQUIVALENT, such that a single cached
>> response(Single Key) can be utilized in both cases.
>>
>>
>> *1st URL after modification:*
>>
>> samplehost.com/search/ims?q=bags&source=android
>>
>>
>> *Cached URL at Varnish:*
>>
>> samplehost.com/search/ims?q=bags&source=android
>>
>>
>>
>> Now, Url at time t+1, 2. samplehost.com/search/ims?q=bags&source=android
>>
>>
>> At present, varnish considers the above URL as different from 1st URL
>> and uses a different key while caching the 2nd URL[So, it will be a miss]
>>
>>
>> *So, URL after Modification:*
>>
>> samplehost.com/search/ims?q=bags&source=android
>>
>>
>> Now, 2nd URL will be a HIT at varnish, effectively utilizing the cache.
>>
>>
>>
>> NOTE:
>>
>> We aim to execute this URL Modification without implementing the logic directly
>> within the default.VCL file. Our intention is to maintain a clean and
>> manageable codebase in the VCL.
>>
>>
>>
>> To address this requirement effectively, we have explored two potential
>> Approaches:
>>
>>
>> Approach-1:
>>
>>
>>
>> Approach-2:
>>
>>
>>
>>
>> 1. Please go through the approaches mentioned above and let me know the
>> effective solution.
>>
>> 2. Regarding Approach-2
>>
>> At Step 2:
>>
>> May I know if there is any way to access and execute a custom subroutine
>> from another VCL, for modifying the Request URL? if yes, pls help with
>> details.
>>
>> At Step 3:
>>
>> Tomcat Backend should receive the Original Request URL instead of the
>> Modified URL.
>>
>> 3. Please let us know if there is any better approach that can be
>> implemented.
>>
>>
>>
>> Thanks & Regards
>> Uday Kumar
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230823/92b34478/attachment-0001.html>