A Varnish threads story

Varnish is a (now) well-known HTTP caching reverse-proxy. It has been written primarily by Poul-Henning Kamp, a famous FreeBSD developer. Varnish is very BSDish: simple, versatile and powerful.

(Yet, configuring it may be pretty tough because HTTP is a complex protocol with regard to caching (RFC 2616 mentions client-side proxies but not server-side ones). Besides, applications living on top of it are often written without any caching consideration in mind. For instance by default Varnish doesn't cache response from requests containing cookies, not it caches responses with a Set-Cookie header, for obvious reasons. Unfortunately PHP applications make heavy use of the PHPSESSID cookie simply because the session_start() function, which is part of the PHP library, is very handy for developers.

Varnish uses a pool of threads to serve requests, with a configurable minimum and maximum values as well as a timeout value (with the -w command-line option). Much like what Apache does with processes when used with the MPM prefork module. Additionally, Varnish enforces a configurable delay between thread creation (parameter's name is thread_pool_add_delay, you can configure it with the -p command-line option).

For some reason, one Varnish instance on a preproduction server here was configured with silly values regarding thread limits: only one thread at minimum. Given the server was often unused, threads timed out and were removed down to one. The problem was that when a developer wanted to test the websites, there was only one thread available and the aforementioned delay between thread creation prevented from spawning them all at one. Albeit being a very powerful server, the website was felt very sluggish.

It took me some time to find out this problem. When I modified the configuration, the website was really, really fast.

Portmaster options combo to upgrade FreeBSD ports

Some years ago I was using the famous Portupgrade to maintain my ports. This software is mature, very powerful and easy to use. Unfortunately its dependency on Ruby makes it really cumbersome, especially because I have many jails.

Therefore when Doug Barton began Portmaster, which is written is shell and does more or less the same thing (well, actually less, but I can live with it), I was quite eager to use it. One thing I didn't like from the beginning with Postmaster was that it is not able to work alone: it is constantly asking things. Of course there are options to disable this, but this leads to me the second problem: they are not intuitive! (at least for me...)

After some struggle, I finally managed to find the options I always want to use and I'm writing them as a reminder and in the hope to help someone else in the same hassle:

# portmaster -dBGm BATCH=1 --no-confirm --delete-packages -a

Here are the details:

• I'm using portconf to configure the ports' build knobs, so I don't want to run the configuration or to be asked something about it. Just use the defaults unless I told otherwise: -G -m BATCH=1;
• Don't create a backup package, I'm not running any financial application: -B;
• Don't ask me if the distfiles must be cleaned, just do it: -d;
• Don't ask me if I really want to upgrade my ports, I already executed the command proving it: --no-confirm;
• Remove packages once installed: --delete-packages;
• Upgrade everything: -a, but you might not want to ugprade everything at once so you can replace this with one or more port name.