Attempts to move from PhantomJS/Poltergeist to Selenium/Headless Chrome and the Pros and Cons

  • Created on 29 August 18 by Matthew Rider
  • Last updated over 1 year ago
  • Votes: 2

Recently, I was moved to migrate away from PhantomJS and capybara-webkit to Headless Chrome for a few motives. Firstly, we, at AgileVentures, were having quite a few issues with getting people set up locally on their MacOS for our flagship site WebsiteOne. The issue is due to capybara-webkit’s reliance on qt, and there are some problems with compatibility issues for the latest versions of qt5, which no longer installs qmake with brew install qt5. There are also issues if someone has updated their Xcode. Secondly, PhantomJS is no longer being maintained. After the announcement that Chrome had released headless testing capabilities after Chrome 59, Vitaly, the PhantomJS maintainer, tweeted, “This is the end- 2.5 will not be released.” That was more than a year ago! Recent blockers in making this transition has made me question this move, at least partially.

I found two blogs and it seemed like a rather straightforward process, so I began, full of optimism. The first, and seemingly most important, part was to start with getting rid of capybara-webkit, since that was the one that was actually causing issues and corresponding time spent debugging. The link to the blog is I had Chrome 68 and chromedriver 2.37 already installed; although, later I would realize that I needed to update my chromedriver to a later release. I skimmed through that part, and started by replacing capybara-webkit with capybara-selenium. I then copied and pasted the configuration, but placed it in my spec_helper.rb file as their in no rails_helper.rb in WebsiteOne. The configuration registers chrome as a driver, sets the Selenium driver to use the chrome browser, and also registers headless chrome-setting it as the JavaScript driver. This was enough to run all the rspec and cucumber tests without failures. Feeling victorious, I ate lunch and got ready for what turned out be the real challenge.

What I hadn’t taken into account was that I hadn’t actually configured Cucumber to use headless chrome. The second blog, which details GitLab’s migration from PhantomJS/Poltergeist can be found here: The first part of the blog is similar, as it just walks through the configuration of Selenium and chrome. I went through the code and started removing all traces of PhanotmJS and Poltergeist feeling like I was ridding myself of sand after a day on the beach. This is when I noticed that there is a cabybara.rb file in the support directory in the features directory, which is where the set up for phantomjs lived, so I added the config there. I ran the tests, I was getting some weird failures, which I would spend most of the rest of the day and late into the night on. The aforementioned blog from GitLab walked through a lot of the changes, but didn’t make any reference to my issue, which was described as a ChildProcess::Error in the failure message and also seemed to stem from an ArgumentError from trying to call to_io on nil. I’ll spare you the details, but after hours of agonizing over possible solutions, I finally threw up my hands and decided to ask for help on the tech_talk slack channel! :O Need to get better at managing my time by asking for help earlier?

This has led me to question my motives for wanting to make this move in the first place. So what are the pros and cons of each action? Well, there are maybe three courses of action that could be taken, possibly infinitely more? One, is to see if, having rid the project of capybara-webkit, that is a small victory to revel in and test more. Two, would be to debug more and realize that making a change this drastic to a larger project is not supposed to go without a hitch. Three, as the failures are all related to testing the JavaScript on the project, someone brought to my attention that maybe the sane thing to do is to have a specific JavaScript testing framework to just handle that part of the project.

The first course of action would be the least work, the way I see it. It would involve testing further to understand where capybara-webkit was actually being used previously and how, if any impact, my replacing it had on the test-suite. It would also potentially be punting the issue of phantomjs not having a maintainer down the line for someone else to deal with. Things can change however, and maybe tackling an issue before it becomes an actual issue is not the wisest thing to do? The second option would have the satisfaction at a job completed, and also provide a great learning experience in how to debug, but would also potentially be a time sink? The third would most certainly involve much more work, as a lot of tests in Cucumber would need to be modified, if not done away entirely, but could arguably lead to more reliability in testing as a JS test framework would handle JS better? No matter the route chosen, lessons can be learned from this experience, and hopefully the code base will have been improved by the time we get through with it.

comments powered by Disqus