How we made our Rails test suite about 75% faster
One of my recent tasks has been to take the current test suite for a large-ish Rails app and speed it up as much as possible. This is the story of how I managed to do so without compromising the integrity of the tests.
Our current suite
At the time of starting, our test suite contained 1347 assertions, and took approximately 12 minutes to run on a 2018 Macbook Pro.
The setup is as follows:
- The testing framework is
MiniTest, which comes set up by default in new Rails applications.
- The database is populated using thoughtbot’s
- Feature tests are done using
- Browser driver used for Capybara is
Timing of tests
I was able to get a breakdown for how long each test took by running the following command:
bundle exec rake test TESTOPTS="-v"
The output then looked as the following example:
Displaying orders for today::Given the company does not allows soup orders#test_Soup_orders_being_hidden = 1.02 s = .
With this, I was able to figure out which tests were taking longer than others.
Watching feature tests
Normally, our tests are run using headless Firefox, as mentioned above, with
geckodriver. However, I was able to run the tests with an attached window and
monitor which steps were taking particularly long.
Monitoring the database
I entered the following line into
Rails.logger.level = Logger::DEBUG
I then could watch SQL commands during my tests with the following:
RAILS_ENABLE_TEST_LOG=1 bundle exec rails test
Speeding up Capybara tests
Our biggest bottleneck when starting out was the feature tests. I was able to fix a lot of these by looking deeper into the Capybara documentation.
For example, when checking that a piece of text wasn’t visible, I’d run the following:
Which does work! It turns out, however, that this was causing Capybara to look
for the text on the page until its timeout, and then return
having lots of these calls in a test, it adds up quickly.
A much faster way is to do the following:
There are lots of examples for this, such as
#has_no_select?, and even
Going through our tests and replacing these checks sped things up greatly!
Replacing (some) factories with fixtures
Don’t get me wrong,
FactoryBot is an incredibly helpful gem. The problem was
with the way I was using it!
See, a lot, if not all, of the tests were using a similar database setup in terms of data.
What I was doing previously was using
FactoryBot to create a series of sample
data used by tests.
This created a huge overload in terms of setting up the database. There are things that can be done, such as using transactional fixtures to dump the database between tests, but setting up the stage still takes a lot of time.
I then re-took a look at Rails fixtures. With these, I can set up a series of sample data that can be loaded for every test.
What I didn’t know, though, is that using fixtures significantly lowers the
number of SQL
INSERT statements, as opposed to a single one for each factory,
reducing the amount of time needed to set the stage!
All’s well that keeps going
These are the two biggest factors that contributed to a faster suite. Got it down from 12 minutes to 3!
This doesn’t by any means indicate that I’m done, but definitely a great set of steps forward.
There are certainly other paths I could take to speed up our test suite, such as:
- Re-considering the exhaustive nature of certain feature tests. Do I really need to check certain aspects in such detail?
- Parallel testing. We’re not on Rails 6 yet but would be nice to have!
Yo, do you have any favourite techniques to make your Rails tests faster? Do please let me know!