Checking parts of content in webpages using cucumber

I’ve updated my tests of external webpages using cucumber. If you like to test the amount of results of the google search we tested last time you can get the content of a special tag on the result of the query using the following step

Then /^I should get more than (d+) results$/ do |arg1|
  page.find('#resultStats').text.sub('.', '').scan(/d+/)[0].to_i.should be > arg1.to_i
end

See the files on github

I also prepared the cucumber steps for using the ruby debugger if something goes wrong.

Here you have use the ruby gem ruby-debug19 and start the debugger in the step with the debugger command. First you have to require the gem in env.rb

require 'ruby-debug'

See the files on github

 

Using Cucumber with PHP and without Ruby on Rails

Cucumber is a tool for running behaviour driven development. It is mostly used during the development of Ruby On Rails applications.

But I tried it for using on arbitray web applications programmed with other programming languages such as php or java. So I googled a lot but just found one page about cucumber testing php pages. But it ends with an error so you can’t really use it.

So I was digging into the source code. First of all the above page used webrat as the simulated browser. But I wanted to use cabybara which is a further development of webrat.

The problem mentioned on the other page lies in capybara-mechanize, a gem to fit cabybara to mechanize. So I forked it on github and fixed the part which was causing the trouble in a special branch.

So now let’s go step by step to a simple example:

Setting up the environment

First create a directory for this test:

mkdir demo

Now create and use a special gemset using rvm

rvm gemset create demo
rvm gemset use demo

Install bundler

gem install bundler

Create a Gemfile file with the following gems:

source :rubygems

gem 'capybara'
gem 'capybara-mechanize', :git => 'git://github.com/JerryWho/capybara-mechanize.git', :branch => 'relative-redirects'
gem 'rspec'
gem 'cucumber'
gem 'launchy'

And install them

bundle

Create a directory structure

mkdir -p tests/features/support
mkdir -p tests/features/step_definitions

Create the env.rb file with the following content in tests/features/support/env.rb

require "bundler/setup"

require 'capybara/mechanize'
require 'capybara/cucumber'

Capybara.app = "http://www.google.de"
Capybara.run_server = false
Capybara.app_host = 'http://www.google.de'
Capybara.default_selector = :css
Capybara.default_driver = :mechanize
# Capybara.default_driver = :selenium

The last line should be used if you want to use Firefox via selenium as testing browser instead of mechanize.

Features

Now it’s time to setup the features. Open a new file google.feature in tests/features/
[gherkin]
Feature: Search
In order to find information
As an user
I want to use the search function
Scenario: Simple search and check for result
Given I am on the Google search page
When I look for "Gummibärchen"
Then I should see "Gummibärchen" in search result
And I should not see "foobar"
[/gherkin]

Step definitions

If you run it

cucumber tests

the output directs you what to put into a google_steps.rb file in tests/features/support

Feature: Search
  In order to find information
  As an user
  I want to use the search function

  Scenario: Simple search and check for result        # tests/features/google.feature:5
    Given I am on the Google search page              # tests/features/google.feature:6
    When I look for "Gummibärchen"                    # tests/features/google.feature:7
    Then I should see "Gummibärchen" in search result # tests/features/google.feature:8
    And I should not see "foobar"                     # tests/features/google.feature:9

1 scenario (1 undefined)
4 steps (4 undefined)
0m0.025s

You can implement step definitions for undefined steps with these snippets:

Given /^I am on the Google search page$/ do
  pending # express the regexp above with the code you wish you had
end

When /^I look for "([^"]*)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

Then /^I should see "([^"]*)" in search result$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

Then /^I should not see "([^"]*)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

So let’s fill in these steps:

Given /^I am on the Google search page$/ do
  visit 'http://www.google.de'
end

When /^I look for "([^"]*)"$/ do |arg1|
  fill_in "lst-ib", :with => arg1
  click_button "Google-Suche"
end

Then /^I should see "([^"]*)" in search result$/ do |arg1|
  page.should have_selector('li.g')
  page.find('em').should have_content(arg1)
end

Then /^I should not see "([^"]*)"$/ do |arg1|
  page.should have_no_content(arg1)
end

and run the tests again:

Feature: Search
  In order to find information
  As an user
  I want to use the search function

  Scenario: Simple search and check for result        # tests/features/google.feature:5
    Given I am on the Google search page              # tests/features/step_definitions/google_steps.rb:1
    When I look for "Gummibärchen"                    # tests/features/step_definitions/google_steps.rb:5
    Then I should see "Gummibärchen" in search result # tests/features/step_definitions/google_steps.rb:10
    And I should not see "foobar"                     # tests/features/step_definitions/google_steps.rb:15

1 scenario (1 passed)
4 steps (4 passed)
0m0.637s

That’s it.
You can clone the files from github:

git clone http://github.com/JerryWho/cucumber-demo.git

A problem still exists, though. If you want to run the tests using selenium instead of mechanzie you’ll get an error. There’s a problem with the way google puts in the submit-button. Actually I don’t see a workaround but maybe I’ll update this post.

Update: see comment for the solution

Optional features

Adding a .rvmrc file

rvm --rvmrc --create 1.9.2-p180@demo

Now every time you change into this directory the right gemset and ruby version is used by rvm.

Skipping a warning

If you see the warning

warning: already initialized constant WFKV_

when running the features you can get rid of it using an older version of rack (see for details)

Usually I blog in German. But I thought that this article may be interesting for other people, too. So I decided to write it in English.

Folgen der Ladezeitoptimierung

Nachdem ich Ende August ein paar Optimierungen an meiner Website vornahm, die ich aufgrund von Googles Page Speed Online-Tests vornahm, habe ich nun einen Blick auf Googles Webmaster Tools geworfen und musste feststellen, dass das Crawlen meiner Site sich doch drastisch verändert hat:

So hat sich angeblich die Ladezeit erhöht, obwohl Sinn und Zweck ja eigentlich das Senken der Ladezeit war.

Ladezeit

 

 

 

 

 

 

Die Anzahl der pro Tag heruntergeladenen Kilobytes hat sich drastisch gesenkt. Okay, das ist plausibel, da ich die Inhalte ja nun vor dem Ausliefern zippe.

Downloadgröße

 

 

 

 

 

 

Doch kann dies auch dadurch erklärt werden, dass sich die Anzahl der pro Tag gecrawlten Seiten drastisch reduziert hat.

Anzahl der gecrawlten Seiten

 

 

 

 

 

 

Ob dieses Verhalten nun daran liegt, dass ich auch am Cache-Verhalten geschraubt habe, weiß ich nicht. Ich habe nun zunächst erstmal mod_gzip deaktiviert und verzichte auf das zippen der ausgelieferten Daten.

In diesem Zeitraum habe ich allerdings nur einen geringen Rückgang an Besuchen beobachtet, den ich vor allem darauf zurückführe, dass ich in den letzten Wochen eher selten etwas geschrieben habe und der Blog daher (aus Google Sicht) an Attraktivität verlor.

Ich bin gespannt, wie sich die Graphen bei den Webmastertools nun weiterentwickeln werden. Das übertragen Volumen sollte ja in jedem Fall ansteigen, auch wenn die Anzahl der gecrawlten Seiten niedrig bleibt.