Watir Revisited

A few months back I wrote about our team’s experience with User Acceptance Testing (UATs) tools (WatiN, Watir and Selenium Reviewed). I did not give Watir very high marks.


Bret’s ResponseWithin a day of the review, Bret Pettichord (creator of Watir) left the following comment.

We would appreciate any leads you can give about where you found reliability problems with Watir. Our apologies for your troubles.

This is great customer service. Bret saw that our team was having problems and responded thoughtfully. I sent the following email reply.

Thanks for your comment on my blog.We were using Watir 1.4.1. We also tried upgrading to 1.5? and thesame problem existed.We saw that accessing an element while the browser was busy caused thetest to hang. This happened frequently for us as our site has lotsof ajax calls.There are two differences between Watir and WatiN that really made thedifference for our team.

  1. Problem described above, WatiN does not seem to have this problem.May be because WatiN uses STA – check this link out for moreinformation: http://watin.sourceforge.net/apartmentstateinfo.html.
  2. WatiN waits and tries again if it cannot find the element on thepage. This makes it much easier to code tests.

I am not saying that WatiN is perfect, but these two things help uskeep our tests repeatable/reliable.

The above email prompted Bret to send this email to the Watir development mailing list.

Adam Esterline recently reported some stability issues with Watir on hisblog.http://adamesterline.com/2007/04/23/watin-watir-and-selenium-reviewed/Because of them, he decided to use Watin instead, which did not have theseproblems. I asked him for more information, and he provided the following.This is something that I would like to understand better.


Watir RevisitedGiven all of the above, I decided to revisit Watir. In order to re-evaluate Watir, I did the following:

  1. I downloaded the latest development build of Watir (1.5.1.1192).
  2. I used the same set of tests from the original review.

ResultsI ran the tests several times over the last several weeks. The problems described in my email to Bret seem to be completely fixed. I even removed all of the sleeps that were originally used to help with timing issues and the tests still passed. As far as I can tell, Watir has caught up with WatiN.Will we switch back to Watir?We have considered it. It is much easier in Ruby to create a multi-threaded test suite.Today, speed is our biggest problem with our UATs. We think they may run faster if they were run in parallel. I will keep you posted on what we decide. If James has his way, we will switch to Selenium 😉


Lesson’s Learned

  1. There is a great team of developers behind Watir that really care about their product.
  2. The internet is smaller than I realized. I certainly did not think that my post would get a response from Bret and the Watir team.

Questions/Comments are always welcome.

Advertisements

6 Responses

  1. I trust you know about ie.set_fast_speed.

  2. Yes. I have used ie.set_fast_speed, but I still wish they were faster.

  3. […] Bret’s comment was a pretty simple one, but it engaged Adam. Enough so to prompt Adam to revisit Watir, and try again, this time with much better results. Even more so, it was enough to get Adam to post a follow-up praising Bret: […]

  4. Why not try WET?

  5. Thanks for exposing me to WET. I will have to take a closer look.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: