bitrate and memory allocated
Miguel González
miguel_3_gonzalez at yahoo.es
Sat Sep 3 12:14:39 CEST 2016
Why I don´t get a copy of my messages and I do get the rest of emails? I
have just double checked my settings and I should get a copy of my messages
Miguel
On 09/03/16 11:59 AM, Miguel González wrote:
> Dear all,
>
> I have Varnish 4.0.3 in front of a WHM/Cpanel hosting around 40
> Wordpress sites. Only 4-5 sites are quite visited but the rest are blogs
> that don´t have many visits.
>
> Varnish seems to work fine, I have increased the RAM of the VPS from 7
> to 21 Gb of RAM because from time to time the RAM of the system got
> exhausted and I had to restart the OS.
>
> I have signed up in datadoghq to get a handle of how well Varnish is
> doing and try to squeeze the performance of it. Mostly because I have
> realized that when I test any site with a tool like pingdom, the first
> test gets a worse performance than consecutive tests (when I assume
> Varnish starts caching).
>
> I have assigned in malloc 6 Gb of RAM but I have been checking the
> numbers provided by varnishstat and after 12 hours I don´t even get
> close to 1 Gb of RAM (I assume c_bytes is what stands for) used by varnish:
>
> varnishstat -1 |grep SMA
> SMA.s0.c_req 33969 0.76 Allocator
> requests
> SMA.s0.c_fail 0 0.00 Allocator
> failures
> SMA.s0.c_bytes 984052835 21896.06 Bytes allocated
> SMA.s0.c_freed 395844358 8807.89 Bytes freed
> SMA.s0.g_alloc 19833 . Allocations
> outstanding
> SMA.s0.g_bytes 588208477 . Bytes
> outstanding
> SMA.s0.g_space 5854242467 . Bytes available
>
> I had a ttl of 2 hours and now I have changed it to 24 hours, maybe I
> will increase it if I see the RAM allocated to Varnish is not entirely
> used. The bitrate is erratic, reaching almost 1 and then dropping to
> less than 0.5.
>
> Are my assumptions right?
>
> Another question, I know that running varnishtop I get the current most
> visited site, but how do I aggregate that info over the hours? I would
> like to get a picture of which are the most used sites (I´m considering
> separating storages and dedicate only RAM for the most used and have
> disk storage for the rest).
>
> Thanks in advance
>
> Miguel
>
> ps: Here is my default.vcl
>
> #
> # This is an example VCL file for Varnish.
> #
> # It does not do anything by default, delegating control to the
> # builtin VCL. The builtin VCL is called when there is no explicit
> # return statement.
> #
> # See the VCL chapters in the Users Guide at
> https://www.varnish-cache.org/docs/
> # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.
>
> # Marker to tell the VCL compiler that this VCL has been adapted to the
> # new 4.0 format.
> vcl 4.0;
>
> import std;
>
> # Default backend definition. Set this to point to your content server.
> backend default {
> .host = "MYIP";
> .port = "82";
> .connect_timeout = 600s;
> .first_byte_timeout = 600s;
> .between_bytes_timeout = 600s;
>
>
> }
>
> acl purge {
> "localhost";
> "127.0.0.1";
> "MYIP";
> }
>
> # This function is used when a request is send by a HTTP client (Browser)
> sub vcl_recv {
>
> # remove ?ver=xxxxx strings from urls so css and js files are
> cached.
> # Watch out when upgrading WordPress, need to restart Varnish or
> flush cache.
> set req.url = regsub(req.url, "\?ver=.*$", "");
>
> # Remove "replytocom" from requests to make caching better.
> set req.url = regsub(req.url, "\?replytocom=.*$", "");
>
> #We pass real IP to the backend
>
> if (req.restarts == 0) {
> if (req.http.X-Forwarded-For) {
> set req.http.X-Forwarded-For = req.http.X-Forwarded-For +
> ", " + client.ip;
> } else {
> set req.http.X-Forwarded-For = client.ip;
> }
> }
>
>
> # Normalize the header, remove the port (in case you're testing
> this on various TCP ports)
>
> set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
>
> # Remove has_js and CloudFlare/Google Analytics __* cookies.
> set req.http.Cookie = regsuball(req.http.Cookie,
> "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
> # Remove a ";" prefix, if present.
> set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
>
>
> # Allow purging from ACL
> if (req.method == "PURGE") {
> # If not allowed then a error 405 is returned
> if (!client.ip ~ purge) {
> return(synth(405, "This IP is not allowed to
> send PURGE requests."));
> }
> # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
> return (purge);
> }
>
> # Post requests will not be cached
> #if (req.http.Authorization || req.method == "POST") {
> # return (pass);
> #}
>
> # Pass anything other than GET and HEAD directly.
> if (req.method != "GET" && req.method != "HEAD") {
> return( pass );
> } /* We only deal with GET and HEAD by default */
>
> # --- WordPress specific configuration
>
> # Did not cache the admin and login pages
> if (req.url ~
> "nocache|cart|my-account|checkout|addons|tienda|carro|wp-admin|wp-(comments-post|login|signup|activate|mail|cron)\.php|preview\=true|admin-ajax\.php|xmlrpc\.php|bb-admin|whm-server-status|server-status|control\.php|bb-login\.php|bb-reset-password\.php|register\.php")
> {
> return (pass);
> }
>
> if (req.url ~ "(ajax|dynamic|custom)") {
> return(pass);
> }
>
> # Remove the "has_js" cookie
> set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(;
> )?", "");
>
> # Remove any Google Analytics based cookies
> set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(;
> )?", "");
>
> # Remove the Quant Capital cookies (added by some plugin, all __qca)
> set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(;
> )?", "");
>
> # Remove the wp-settings-1 cookie
> set req.http.Cookie = regsuball(req.http.Cookie,
> "wp-settings-1=[^;]+(; )?", "");
>
> # Remove the wp-settings-time-1 cookie
> set req.http.Cookie = regsuball(req.http.Cookie,
> "wp-settings-time-1=[^;]+(; )?", "");
>
> # Remove the wp test cookie
> set req.http.Cookie = regsuball(req.http.Cookie,
> "wordpress_test_cookie=[^;]+(; )?", "");
>
> # Are there cookies left with only spaces or that are empty?
> if (req.http.cookie ~ "^ *$") {
> unset req.http.cookie;
> }
>
> # Cache the following files extensions
> if (req.url ~ "\.(txt|css|js|png|gif|jp(e)?g|swf|ico)") {
> unset req.http.cookie;
> }
>
> # Normalize Accept-Encoding header and compression
> # https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
> if (req.http.Accept-Encoding) {
> # Do no compress compressed files...
> if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
> unset req.http.Accept-Encoding;
> } elsif (req.http.Accept-Encoding ~ "gzip") {
> set req.http.Accept-Encoding = "gzip";
> } elsif (req.http.Accept-Encoding ~ "deflate") {
> set req.http.Accept-Encoding = "deflate";
> } else {
> unset req.http.Accept-Encoding;
> }
> }
>
> # Check the cookies for wordpress-specific items
> if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~
> "comment_") {
> return (pass);
> }
> if (!req.http.cookie) {
> unset req.http.cookie;
> }
>
> # --- End of WordPress specific configuration
>
> # Did not cache HTTP authentication and HTTP Cookie
> if (req.http.Authorization || req.http.Cookie) {
> # Not cacheable by default
> return (pass);
> }
>
> # Cache all others requests
> return (hash);
> }
>
> sub vcl_pipe {
> return (pipe);
> }
>
> sub vcl_pass {
> return (fetch);
> }
>
> # The data on which the hashing will take place
> sub vcl_hash {
> hash_data(req.url);
> if (req.http.host) {
> hash_data(req.http.host);
> } else {
> hash_data(server.ip);
> }
>
> # If the client supports compression, keep that in a different cache
> if (req.http.Accept-Encoding) {
> hash_data(req.http.Accept-Encoding);
> }
>
> return (lookup);
> }
>
> # This function is used when a request is sent by our backend (Nginx server)
> sub vcl_backend_response {
> # Remove some headers we never want to see
> unset beresp.http.Server;
> unset beresp.http.X-Powered-By;
>
> # For static content strip all backend cookies
> if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
> unset beresp.http.cookie;
> }
> # Don't store backend
> if (bereq.url ~ "wp-(login|admin)" || bereq.url ~ "preview=true") {
> set beresp.uncacheable = true;
> set beresp.ttl = 30s;
> return (deliver);
> }
>
> # Only allow cookies to be set if we're in admin area
> if (!(bereq.url ~
> "(wp-login|cart|my-account|checkout|addons|tienda|carro|wp-admin|preview=true)"))
> {
> unset beresp.http.set-cookie;
> }
>
> # don't cache response to posted requests or those with basic auth
> if ( bereq.method == "POST" || bereq.http.Authorization ) {
> set beresp.uncacheable = true;
> set beresp.ttl = 120s;
> return (deliver);
> }
>
> # don't cache search results
> if ( bereq.url ~ "\?s=" ){
> set beresp.uncacheable = true;
> set beresp.ttl = 120s;
> return (deliver);
> }
>
> # only cache status ok
> if ( beresp.status != 200 ) {
> set beresp.uncacheable = true;
> set beresp.ttl = 120s;
> return (deliver);
> }
>
> # A TTL of 24h
> set beresp.ttl = 24h;
> # Define the default grace period to serve cached content
> #set beresp.grace = 30s;
> set beresp.grace = 1h;
>
> return (deliver);
> }
>
> # The routine when we deliver the HTTP request to the user
> # Last chance to modify headers that are sent to the client
> sub vcl_deliver {
> if (obj.hits > 0) {
> set resp.http.X-Cache = "cached";
> } else {
> set resp.http.x-Cache = "uncached";
> }
>
> # Remove some headers: PHP version
> unset resp.http.X-Powered-By;
>
> # Remove some headers: Apache version & OS
> unset resp.http.Server;
>
> # Remove some heanders: Varnish
> unset resp.http.Via;
> unset resp.http.X-Varnish;
>
> return (deliver);
> }
>
> sub vcl_init {
> return (ok);
> }
>
> sub vcl_fini {
> return (ok);
> }
>
>
>
>
>
>
>
>
More information about the varnish-misc
mailing list