Forcing Rats To Eat Selenium!

As I've mentioned before we are currently in the run up to a big software release at work. Not only are we making a major release of GATE but we are also updating the website and making minor releases of other bits of software developed by the group.

As part of this I've been fixing bugs and looking at security issues in our wiki/CMS system -- GATEWiki. Testing a web application automatically and reliably can be really tricky, but we are using Selenium to test GATEWiki. Without going into too many details a Selenium test case is a script which is used to control a web browser (in our case Firefox) in the same way a normal user would interact with the application. We have Hudson setup to run a whole bunch of Selenium tests every time someone checks in changes (as well as running normal unit and integration tests), and anyone can check on the latest build results.

Whilst we have have been careful about security during the development of GATEWiki we want to make sure that there are no gaping security holes. One interesting approach to looking for security issues is to use ratproxy. As it's name suggest ratproxy is a web proxy that monitors requests for security issues. It is designed to watch normal user behaviour and so I decided that using it to monitor the Selenium tests being run would be a clever way of reviewing potential security problems in a repeated and reproducible fashion. Configuring this was actually quite straightforward (if you are interested I created a simple Firefox profile template directory with the necessary prefs.js file).

One obvious security issue was that we were not enforcing the use of HTTPS during login and hence passwords were being sent in the clear. GATEWiki is built using Grails which uses an embedded Jetty web server and it was easy to configure this to send all requests that would involve sensitive data via HTTPS. Unfortunately this caused Hudson to be unable to run the Selenium tests via ratproxy.

The problem is that when secure requests are sent via ratproxy, ratproxy throws away the original security certificate and re-signs the requests using it's own certificate. Due to the way Firefox now handles untrusted certificates (lots and lots of big scary warnings) there is no way I could find to make Firefox remember that it should trust pages signed by ratproxy and hence it would just sit there waiting for the user to accept the certificate. This is fine for local testing, but not much help for unattended testing via Hudson. Fortunately the fix is actually quite easy, although it took me an age to figure out.

If you look in the ratproxy directory you will find a file called keyfile.pem which contains the security certificates used to sign any secure page. All you need to do is replace this file with one that a) ratproxy can handle and b) Firefox can be made to remember should be trusted.

If you are on a Linux box or have cygwin installed under Windows then you probably have a copy of the OpenSSL package available and this can be used to produce a new set of certificates which will do the job. The keyfile we are trying to produce actually contains both a private key and a public certificate so we need to create both of these and then combine them into a single file.

Let's start with the private key as we need this to create the certificate. Simply issue the command:

openssl genrsa -out keyfile.key

Dead easy! The thing to note is that this key isn't password protected, which may not be ideal but is required otherwise ratproxy won't be able to use it.

So we have the private key let's now create the certificate. This is a bit more complex but start by issuing the command:

openssl req -new -x509 -key keyfile.key -out keyfile.crt

Now this will ask you for a whole bunch of information. Fortunately we don't have to provide many answers. Simply leave all the fields blank (by answering .) except the 'Common Name' field. This must be set to the hostname of the ratproxy server. Most of the time this will simply be 'localhost' but if you are running the proxy on a remote machine then you will need to set this value appropriately.

The final step is to combine the key and certificate into a single file that you can use to replace the default ratproxy keyfile.pem file with. Just issue the command:

cat keyfile.key keyfile.crt > keyfile.pem

And that is it. You now have a file that ratproxy can use and that you can tell Firefox to trust permanently.

I know it probably isn't an overly interesting blog post (I certainly doubt it lived up to it's title) but hopefully it will save someone else an afternoon of trying to figure out what's wrong with the ratproxy certificate and how to replace it.