Compiling Varnish modules

Varnish is a powerful tool and although simple to get running initially can have a steep learning curve beyond the initial install.

In this post I look at compiling a Varnish module and some issues you may face and how to overcome them.

When implementing Varnish, sometimes you want to cache pages for a 'test' site or a site that isn't open to the public yet.

Typically you would use basic authentication on the Apache server and have Varnish not cache pages that require authentication.

But if you do want Varnish to cache pages for a 'closed' site then you have to move authentication to Varnish.

There are two modules that support this -

• basic authentication: which works like basic authentication in Apache and can use the same htpasswd files - https://www.varnish-cache.org/vmod/basicauth
• authentication - that uses hases of user:passwrd in the vcl itself. - https://www.varnish-cache.org/vmod/authentication

I recently evaluated the basic authentication module:   https://www.varnish-cache.org/vmod/basicauth -  source: http://git.gnu.org.ua/cgit/vmod-basicauth.git/

The first problem I faced was the scant documentation about how to use Varnish modules.

I came across a detailed explanation of how to do this here : http://stackoverflow.com/questions/28163273/varnish-4-basic-authentication

In order to be able to use a Varnish module you need clone the module from source and compile it against the Varnish source.

To do this you have to get the Varnish source (and many dependencies)  and build it (although you don't have to install it if you have already done so via your package manager).

$agt-get source varnish

To build Varnish from source follow the instructions here: https://www.varnish-cache.org/docs/trunk/installation/install.html and make sure you have the dependencies listed for your platform.

However the issue I faced was that the Varnish source would not compile.

This was due to the automake functionality not recursing into subdirectories.

warning: source file <some source file> is in a subdirectory,
but option 'subdir-objects' is disabled

To solve this you have to add the subdir-options option to the automake in line 18 of the configure.ac:

$vi /usr/src/varnish-4.0.3/configure.ac

AM_INIT_AUTOMAKE([1.11 foreign color-tests parallel-tests subdir-options])

However with this option there is an issue (on Debian at least) whereby variable expansion fails in the make files in the bin directory for the Varnish programs (varnishadm and varnishhist). (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=402727).

To solve this you have to edit the Makefile.am files in the varnishadm and varnishhist bin folders:

$vi bin/varnishadm/Makefile.am

varnishadm_SOURCES = \
	varnishadm.c \
	$(top_srcdir)/lib/libvarnish/vas.c \
	$(top_srcdir)/lib/libvarnish/vsa.c \
	$(top_srcdir)/lib/libvarnish/vtcp.c \
	$(top_srcdir)/lib/libvarnish/vss.c

varnishadm_SOURCES = \
	varnishadm.c \
	../../lib/libvarnish/vas.c \
	../../lib/libvarnish/vsa.c \
	../../lib/libvarnish/vtcp.c \
	../../lib/libvarnish/vss.c

Otherwise when you run the make command, the build will fail and you will have subfolders in the bin/varnishadm folder called $(top_srcdir).

This only appears to affect the SOURCES element. - see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=402727

Once that is done - you can build the varnish source and then compile and install your module as outlined in http://stackoverflow.com/questions/28163273/varnish-4-basic-authentication