Block Unauthorized Requests at Varnish [Code Optimization]

Sun Oct 15 02:02:15 UTC 2023

Hello Uday,

Quick follow-up as I realize that templating can be a bit scary when
confronted for the first time, and you are far from the first one to be
curious about, so I've committed this:
https://github.com/varnish/toolbox/tree/master/gotemplate-example
It probably won't get you very far, but it should at least get you started,
and help understand how templating can make things a tiny be simpler but
splitting data from business logic, for example to add more IPs/ACLs or
source without edit the VCL manually.

Hope that helps.

-- 
Guillaume Quintard

On Thu, Oct 12, 2023 at 12:36 PM Uday Kumar <uday.polu at indiamart.com> wrote:

> > That's mainly how computers work, processing will be linear. You *could*
> create a vmod that packs ACLs into a hashmap to simplify the apparent
> logic, but you will pay that price developing the vmod, and for a very
> modest performance gain. If you have less than 50 sources, or even less
> than 100, I don't think it's worth agonizing over that kind of optimization
> (unless you've actually measured and you did see a performance drop).
>
> Okay, Thanks for your suggestion!
>
> >  I assume that the VCL is currently committed in a repo somewhere and
> gets edited every time you need to add a new IP or source. If so, it's not
> great because editing such repetitive code is error-prone, and therefore
> you should use templating to create the VCL from a simpler, more
> maintainable source.
>
> Sure, will definitely explore!
>
> Thanks & Regards
> Uday Kumar
>
>
> On Fri, Oct 13, 2023 at 12:35 AM Guillaume Quintard <
> guillaume.quintard at gmail.com> wrote:
>
>> > In the above example, if the request URL is source=tablet [for which
>> condition is present at the end], still I have to check all the above
>> conditions.
>>
>> That's mainly how computers work, processing will be linear. You *could*
>> create a vmod that packs ACLs into a hashmap to simplify the apparent
>> logic, but you will pay that price developing the vmod, and for a very
>> modest performance gain. If you have less than 50 sources, or even less
>> than a 100, I don't think it's worth agonizing over that kind of
>> optimization (unless you've actually measured and you did see a
>> performance drop).
>>
>> > One thing I would do though is to generate the VCL from a source file,
>> like a YAML one:
>>
>> All I'm saying is that you should focus on increasing the maintainability
>> of the project before worrying about performance. I assume that the VCL is
>> currently committed in a repo somewhere and gets edited every time you need
>> to add a new IP or source. If so, it's not great because editing such
>> repetitive code is error-prone, and therefore you should use templating to
>> create the VCL from a simpler, more maintainable source.
>>
>> Tools like go templates or jinja can provide that feature and save you
>> from repeating yourself when writing configuration.
>>
>> --
>> Guillaume Quintard
>>
>>
>> On Thu, Oct 12, 2023 at 11:46 AM Uday Kumar <uday.polu at indiamart.com>
>> wrote:
>>
>>> Hi Guillaume,
>>>
>>> I don't think those are redundant checks, from what you are showing,
>>> they are all justified. Sure, there may be a bunch of them, but you have to
>>> go through to them.
>>>
>>> By redundant I meant, I have to write multiple checks for each source
>>> and list of IPs associated with it. [which would be *worse *if the
>>> number of sources are huge]
>>>
>>> *Example:*
>>>
>>> If(
>>>
>>> (req.url ~ "source=mobile" && client.ip != mobile_source) ||
>>>
>>> (req.url ~ "source=desktop" && client.ip != desktop_source) ||
>>>
>>> (req.url ~ "source=laptop" && client.ip != laptop_source) ||
>>>
>>> (req.url ~ "source=tablet" && client.ip != tablet_source)
>>>
>>> ){
>>>
>>>            return(Synth(403, "access denied!"))
>>>
>>> }
>>>
>>>
>>> In the above example, if the request URL is source=tablet *[for which
>>> condition is present at the end]*, still I have to check all the above
>>> conditions.
>>>
>>>
>>>
>>>
>>>
>>> One thing I would do though is to generate the VCL from a source file,
>>> like a YAML one:
>>>
>>> Didn't understand, can you please elaborate?
>>>
>>> Thanks & Regards
>>> Uday Kumar
>>>
>>>
>>> On Thu, Oct 12, 2023 at 11:11 PM Guillaume Quintard <
>>> guillaume.quintard at gmail.com> wrote:
>>>
>>>> Hi Uday,
>>>>
>>>> I don't think those are redundant checks, from what you are showing,
>>>> they are all justified. Sure, there may be a bunch of them, but you have to
>>>> go through to them.
>>>>
>>>> One thing I would do though is to generate the VCL from a source file,
>>>> like a YAML one:
>>>>
>>>> mobile:
>>>>   - IP1
>>>>   - IP2
>>>>   - IP3
>>>> desktop:
>>>>   - IP4
>>>>   - IP5
>>>>   - IP6
>>>>
>>>>
>>>> From that, you can build the VCL without having to manually write
>>>> "client.ip" or "(req.url ~ "source=" every time.
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>>
>>>> On Thu, Oct 12, 2023 at 10:17 AM Uday Kumar <uday.polu at indiamart.com>
>>>> wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> We use varnish in our production environment for caching content.
>>>>>
>>>>> Our Requirement:
>>>>>
>>>>> We are trying to block unauthorized requests at varnish based on the
>>>>> source parameter in the URL and the client IP in the request header.
>>>>>
>>>>> For example:
>>>>>
>>>>> Sample URL:
>>>>>
>>>>> www.hostname:port/path?source=mobile&keyword= bags
>>>>>
>>>>> Let's assume there are 3 IPs [which are allowed to access varnish]
>>>>> associated with the above request of mobile source.
>>>>>
>>>>> i.e *IP1, IP2, IP3*
>>>>>
>>>>> So if any request comes with the source as *mobile *and client-ip as
>>>>> *IP4*, it's treated as an unauthorized request and should be blocked
>>>>> at varnish.
>>>>>
>>>>>
>>>>> What we have done for blocking?
>>>>>
>>>>> *Sample URL:*
>>>>> www.hostname:port/path?source=mobile&keyword= bags
>>>>>
>>>>> Created a map using ACL as below:
>>>>>
>>>>> acl mobile_source{
>>>>>
>>>>>               "IP1";
>>>>>
>>>>>               "IP2";
>>>>>
>>>>>               "IP3";
>>>>>
>>>>> }
>>>>>
>>>>> If(req.url ~ "source=mobile" && client.ip !~ mobile_source) {
>>>>>
>>>>>            return(Synth(403, "varnish access denied!"))
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> The problem we are facing:
>>>>>
>>>>> The source parameter can have different values like mobile, desktop,
>>>>> laptop, tablet, etc. and each value can have different IPs associated with
>>>>> it.
>>>>>
>>>>> ACL Rules will be as below:
>>>>>
>>>>> acl mobile_source{
>>>>>
>>>>>               "IP1";
>>>>>
>>>>>               "IP2";
>>>>>
>>>>>               "IP3";
>>>>>
>>>>> }
>>>>>
>>>>> acl desktop_source{
>>>>>
>>>>>               "IP4";
>>>>>
>>>>>               "IP5";
>>>>>
>>>>>               "IP6";
>>>>>
>>>>> }
>>>>>
>>>>> and so on,
>>>>>
>>>>>
>>>>> If we wanted to block unauthorized access from different source vs IP
>>>>> combinations, we would have to add that many conditions as below.
>>>>>
>>>>> If(
>>>>>
>>>>> (req.url ~ "source=mobile" && client.ip != mobile_source) ||
>>>>>
>>>>> (req.url ~ "source=desktop" && client.ip != desktop_source) ||
>>>>>
>>>>> (req.url ~ "source=laptop" && client.ip != laptop_source) ||
>>>>>
>>>>> (req.url ~ "source=tablet" && client.ip != tablet_source)
>>>>>
>>>>> ){
>>>>>
>>>>>            return(Synth(403, "access denied!"))
>>>>>
>>>>> }
>>>>>
>>>>> This becomes worse, if we have 10's or 20's of source values.
>>>>>
>>>>> Our question:
>>>>>
>>>>> We would like to know if there is any way to optimize the code by
>>>>> removing redundant checks so that we can scale it even if we have many
>>>>> sources vs IP combinations.
>>>>>
>>>>>
>>>>> Thanks & Regards
>>>>> Uday Kumar
>>>>> _______________________________________________
>>>>> varnish-misc mailing list
>>>>> varnish-misc at varnish-cache.org
>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20231014/b753dfbd/attachment-0001.html>