New challenge for Varnish!

[zabrane Mikael]

Thanks Rob. Let me give it a try!

2010/7/8 Rob S <rtshilston at gmail.com>

> zabrane Mikael wrote:
>
>> Hi Rob,
>>
>>    I don't think that's going to be possible with a pure VCL
>>    approach.  Can you give us a better idea of exactly what you're
>>    trying to achieve, as there might be a simpler / more
>>    Varnish-friendly way of achieving what you're after.
>>
>>
>> To make it simple, I'd like to store for a document A, all its raw headers
>> (both request and response) for later processing.
>> A cron job will collect them (i.e these stored files) and send them to a
>> complex analysis system.
>>
>>    You can certainly use varnishlog to see the full headers for a
>>    given UUID
>>
>>
>> I'll give it try!
>>
>>    but the body content is more difficult.
>>
>>
>> Is there another way to achieve that?
>> Maybe by using some C to implement a callback for handling raw "request"
>> and "response".
>>
> You could use some C, but I don't know enough to comment about race
> conditions between concurrent requests if the same UUID came in at about the
> same time, or how to work gracefully if Varnish queued some concurrent
> requests to the same resource.
> If the UUID isn't that important, I'd consider doing the following:
>
> Set up additional backends, on 127.0.0.2, called 'loggedsquid' and
> 'loggedcustomerserver'.  Then, approximately have:
>
>
>
>  // set default backend
>  set req.backend = squid;
>
>  if (req.http.X-RAWDATA != "") {
>       set req.backend = customserver;
>  }
>
> if (req.http.X-STORE!= "") {
>    set req.backend = loggedsquid;
>
>
>    if (req.http.X-RAWDATA != "") {
>       set req.backend = loggedcustomserver;
>    }
>  }
>
> THEN, on the varnish server, run (approximately) " tcpdump -s 0 -w
> loggedtraffic.pcap -d lo ip and host 127.0.0.2 "
>
> Finally, on the cron, you could parse the pcap file, as that'd have the
> full headers and body, all in one coherent place.  Indeed, if the UUID is
> essential, there's no reason why you couldn't parse it out of the pcap file,
> work out the tcp port number for that request, then reparse the pcap to
> write out the packet contents to a specific file for that UUID.
>
> I suppose it all depends on how regularly this logging would happen.
>
>
> Rob
>



-- 
Regards
Zabrane
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.varnish-cache.org/pipermail/varnish-misc/attachments/20100708/2a51912a/attachment.html>