Introduction
Our OpenPHIN application is heavily tested using rspec and cucumber. At first, our tests ran fairly quickly to a magnitude of less than 3 minutes. Now that we are up to 217 scenarios and 3133 step definitions, we have reached a critical point where we choose to run our tests less during daily development which is leading to more commits without tests passing. This can start to become a real problem, expecially if any kind of continuous integration is used. We were also seeing a wide range of cucumber processing times across all our developers between 5 minutes on a quad 3.33Ghz to 12 minutes on an older Macbook. We had heard about testjour but found there was completely zero documentation or blog posts about getting testjour working and others hadn't been able to get it to work in their environment. Regardless, we decided it was time to invest in getting testjour working now to save a lot more future development time and improve commited code quality.
Testjour is a ruby gem written by Bryan Helmkamp which is used for distributing cucumber tests across multiple systems. Since ruby/cucumber are not threaded, Testjour can be useful for distributing tests across a single or multiple systems. We used one dual quad-core 2.83Ghz Xeon, 16Gb RAM blade server running VMWare ESXi and set up 8 virtual machines for our Testjour environment. Each virtual machine was a single processor installation and has its processor affinity set to its own core since Testjour is more CPU bound than disk intensive. We found that it was most efficient to set up the first virtual machine exactly as needed, then copied the virtual machine to a new virtual machine, change the hostname, IP, sshd key, and user ssh key. With this kind of set up we are able to scale very quickly with more slaves as our test suite continues to grow, not to mention that now all our developers see the same build time for our test suite.
Testjour uses rsync to synchronize the local repository with its slaves. Since our hardware was not located at the same location as our development machines, we also decided to create a head server (as one of the 8 virtual machines) which would also serve as our Testjour master. We were losing about 1.5 minutes running our tests via Testjour due to the rsync from our development machine to our Testjour master. Rather than using rsync to the Testjour master, we instead set up a git repository on our master and added it as a remote to our local repositories. This allowed us a much more efficient process for transfering the code we are currently working on to the testing environment. We will also in the future be working on both pushing git changes to the master, then using rsync to push any further changes that haven't been committed. Currently we must commit to our local branch and then push to the Testjour master remote for the changes to be tested.
Since we currently use Capistrano for deployment, we decided to automate our testing experience with Capistrano as well. This also played well in to using a master testing git repository. Excerpts of our Capistrano recipes are included in part 3.
Testjour was actually missing one critical thing for our project so I actually created my own fork of Testjour which is located at Dishwasha/testjour. I would recommend using this version when you follow this article. I also added in a couple other things which may not be necessary for your environment.
Wherever testjour is run from (i.e. the master), it must be running a redis server. The redis server is used for master/slave communication and must run on the master server. It is not necessary for the slaves. I would recommend copying redis.conf to /etc, change the daemonize line to yes, and add "redis-server /etc/redis.conf" it to your rc.local. You'll also need to open TCP port 6379 or whatever port you configure in your redis.conf to your master's firewall configuration to allow slaves to communicate with the master. Both the master and all slaves must have the redis gem installed as well. Run a "gem install redis" or get it from ezmobius/redis. You'll need to resolve any other dependencies as well. Once you have that done, you're ready to run testjour.
Continue to part 2
No comments:
Post a Comment