Ideally you want to run your tests in all different browsers on all platforms. It can be achieved with Selenium Grid. Setting up Selenium Grid might be an time-consuming task and you have to deal with license costs. Therefore, some companies providing cloud-based solutions.

TestingBot offers a cloud-based solution with many extra functionalities, such as: taking screenshots per command, video recording of the test execution, integrated proxy and advanced reporting.

TestingBot can be used with all kinds of programming languages. In this tutorial you will see how easy it is to run your tests in the cloud.

How to to it

1. Create an account on TestingBot and try it for free. (or choose a sufficient plan)
2. Download the required library from: https://github.com/testingbot
3. Change the remote address, so the tests will be run at the remote Grid.

import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestingBotTest {
    private WebDriver driver;

    @BeforeClass
    public void setUp() throws Exception {
        DesiredCapabilities capabillities = DesiredCapabilities.firefox();
        capabillities.setCapability("version", "11");
        capabillities.setCapability("platform", Platform.WINDOWS);
        capabillities.setCapability("name", "Testing Selenium 2");

        driver = new RemoteWebDriver(
                new URL(
                        "http://ClientKey:ClientSecret@hub.testingbot.com:4444/wd/hub"),
                capabillities);
    }

    @Test
    public void testSimple() throws Exception {
        driver.get("http://selenium.polteq.com/prestashop/");
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys(
                "ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']"))
                .click();
        String searchHeader = driver.findElement(By.cssSelector("H1"))
                .getText().toLowerCase();

        Assert.assertTrue(searchHeader.contains("ipod nano"));
    }

    @AfterClass
    public void tearDown() throws Exception {
        driver.quit();
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestingBotTest {
    private WebDriver driver;

    @BeforeClass
    public void setUp() throws Exception {
        DesiredCapabilities capabillities = DesiredCapabilities.firefox();
        capabillities.setCapability("version", "11");
        capabillities.setCapability("platform", Platform.WINDOWS);
        capabillities.setCapability("name", "Testing Selenium 2");

        driver = new RemoteWebDriver(
                new URL(
                        "http://ClientKey:ClientSecret@hub.testingbot.com:4444/wd/hub"),
                capabillities);
    }

    @Test
    public void testSimple() throws Exception {
        driver.get("http://selenium.polteq.com/prestashop/");
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys(
                "ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']"))
                .click();
        String searchHeader = driver.findElement(By.cssSelector("H1"))
                .getText().toLowerCase();

        Assert.assertTrue(searchHeader.contains("ipod nano"));
    }

    @AfterClass
    public void tearDown() throws Exception {
        driver.quit();
    }
}

Additional Test Options

Screenshots

Capture screenshots at every step of the test.

capabillities.setCapability("screenshot", true);

1

capabillities.setCapability("screenshot", true);

Video

Record a video of your test, which is accessible in the member area.

capabillities.setCapability("screenrecorder", true);

1

capabillities.setCapability("screenrecorder", true);

Test Name

Add a name to this test, which will show up in our member area.

capabillities.setCapability("name", "Test Script Name");

1

capabillities.setCapability("name", "Test Script Name");

Custom data

Send along custom data, for example your build number, release, server, commit hash, etc…

capabillities.setCapability("extra", "release 1.2.3");

1

capabillities.setCapability("extra", "release 1.2.3");

Problem

Imagine all the development/QA/CI machines will push their test scripts to the HUB. They will be queued and executed according to the FIFO principles (First In – First Out).

But also the Continuous Integration (CI) environment will push the test scripts to the HUB. Although they have a higher priority (because we want to keep the feedback cycle as short as possible), they will be queued according to the same principles. We can implement the code described in this recipes to make an distinction based on priority.

Solution

1. Create the prioritizer
2. Create a config file (hub.jsonJavaScript Object Notation)
3. Launch the HUB with the custom prioritizer
4. Modify the test scripts

Create the prioritizer

import java.util.Map;

import org.openqa.grid.internal.listeners.Prioritizer;

public class CustomPrioritizer implements Prioritizer {

    public int compareTo(Map<String, Object> a, Map<String, Object> b) {
        boolean aImportant = a.get("_important") == null ? false : Boolean
                .parseBoolean(a.get("_important").toString());
        boolean bImportant = b.get("_important") == null ? false : Boolean
                .parseBoolean(b.get("_important").toString());
        if (aImportant == bImportant) {
            return 0;
        }
        if (aImportant && !bImportant) {
            return -1;
        } else {
            return 1;
        }
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import java.util.Map;

import org.openqa.grid.internal.listeners.Prioritizer;

public class CustomPrioritizer implements Prioritizer {

    public int compareTo(Map<String, Object> a, Map<String, Object> b) {
        boolean aImportant = a.get("_important") == null ? false : Boolean
                .parseBoolean(a.get("_important").toString());
        boolean bImportant = b.get("_important") == null ? false : Boolean
                .parseBoolean(b.get("_important").toString());
        if (aImportant == bImportant) {
            return 0;
        }
        if (aImportant && !bImportant) {
            return -1;
        } else {
            return 1;
        }
    }
}

Create a config file (hub.jsonJavaScript Object Notation)

{
  "host": null,
  "port": 4444,
  "newSessionWaitTimeout": -1,
  "servlets" : [],
  "prioritizer": myPriotizer.customPrioritizer,
  "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
  "throwOnCapabilityNotPresent": true,
  "nodePolling": 5000,

  "cleanUpCycle": 5000,
  "timeout": 300000,
  "maxSession": 20
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

{
  "host": null,
  "port": 4444,
  "newSessionWaitTimeout": -1,
  "servlets" : [],
  "prioritizer": myPriotizer.customPrioritizer,
  "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
  "throwOnCapabilityNotPresent": true,
  "nodePolling": 5000,

  "cleanUpCycle": 5000,
  "timeout": 300000,
  "maxSession": 20
}

Launch the HUB with the custom prioritizer

java -cp selenium-server-standalone-<version>.jar:prioritizer.jar org.openqa.grid.selenium.GridLauncher -role hub -hubConfig hub.jsonJavaScript Object Notation

1

java -cp selenium-server-standalone-<version>.jar:prioritizer.jar org.openqa.grid.selenium.GridLauncher -role hub -hubConfig hub.jsonJavaScript Object Notation

Modify the test scripts

import java.net.MalformedURLException;
import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.Test;

public class PrioritizerExample {
    
    @Test
    public void importantTest() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setCapability("_important", true);
        
        RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
        
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
        
        driver.close();
        driver.quit();
    }
    
    @Test
    public void lessImportantTest() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setCapability("_important", false);
        
        RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
        
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
        
        driver.close();
        driver.quit();
    }

}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

import java.net.MalformedURLException;
import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.Test;

public class PrioritizerExample {
    
    @Test
    public void importantTest() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setCapability("_important", true);
        
        RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
        
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
        
        driver.close();
        driver.quit();
    }
    
    @Test
    public void lessImportantTest() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setCapability("_important", false);
        
        RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
        
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
        
        driver.close();
        driver.quit();
    }

}

Problem

We can set parameters to the node in order to configure them. We can set for example the maximum number of concurrent tests, platform, browserName and so on. We can use those values in our tests to specify the corresponding node.

Prerequisites

Open the command prompt and navigate to the directory where we put the selenium-server-standalone file.

Solution

We can add the -browser parameters to the command line with the following values to launch 5 FireFox 3.6 instances on Linux.

-browser browserName=firefox,version=3.6,maxInstances=5,platform=LINUX

There is more…

We can also load a JSONJavaScript Object Notation file while launching the node, instead of specifying the browser parameter on the command line. We need to add the following parameter to the test: -nodeConfig node.jsonJavaScript Object Notation The contents of node.jsonJavaScript Object Notation may look like this:

{
  "capabilities":
      [
        {
          "browserName": "firefox",
          "version": "3.6",
          "platform": "WINDOWS",
          "maxInstances": 1
        },
        {
          "browserName": "internet explorer",
          "version": "8",
          "platform": "WINDOWS",
          "maxInstances": 1
        }
      ],
    "configuration":
        {
        "nodeTimeout":120,
        "port":5555,

        "hubPort":4444,
        "hubHost":"localhost",

        "nodePolling":2000,

        "registerCycle":10000,
        "register":true,
        "cleanUpCycle":2000,
        "timeout":30000,
        "maxSession":1,
        }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

{
  "capabilities":
      [
        {
          "browserName": "firefox",
          "version": "3.6",
          "platform": "WINDOWS",
          "maxInstances": 1
        },
        {
          "browserName": "internet explorer",
          "version": "8",
          "platform": "WINDOWS",
          "maxInstances": 1
        }
      ],
    "configuration":
        {
        "nodeTimeout":120,
        "port":5555,

        "hubPort":4444,
        "hubHost":"localhost",

        "nodePolling":2000,

        "registerCycle":10000,
        "register":true,
        "cleanUpCycle":2000,
        "timeout":30000,
        "maxSession":1,
        }
}

It makes it possible to use a firefox and Internet Explorer browser on a windows platform.

Command line script to start a Selenium Node with JSONJavaScript Object Notation configuration file:

java -jar selenium-server-standalone-<version>.jar -role node -hub http://localhost:4444/grid/register -nodeConfig defaultNodeConfig.jsonJavaScript Object Notation

Problem

We have to make minor changes to the tests in order to run our tests against the Grid. This recipes will show us how we can define capabilities to match a node.

Prerequisites

Make sure the desired capabilities are in the Grid otherwise the test can not be routed to the node.

Solution

The code below will match the following nodes: browserName=firefox,version=3.6,platform=LINUX

import java.net.MalformedURLException;
import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class GridExample {
    private static RemoteWebDriver driver;
    
    @BeforeClass
    public void setUp() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setBrowserName("firefox");
        capability.setPlatform(Platform.LINUX);
        capability.setVersion("3.6");
        driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
    }
    
    @Test
    public void measurePerformance() {
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import java.net.MalformedURLException;
import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class GridExample {
    private static RemoteWebDriver driver;
    
    @BeforeClass
    public void setUp() throws MalformedURLException {
        DesiredCapabilities capability = DesiredCapabilities.firefox();
        capability.setBrowserName("firefox");
        capability.setPlatform(Platform.LINUX);
        capability.setVersion("3.6");
        driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
        driver.get("http://selenium.polteq.com/prestashop/");
    }
    
    @Test
    public void measurePerformance() {
        driver.findElement(By.cssSelector("input#search_query_top")).sendKeys("ipod nano");
        driver.findElement(By.cssSelector("input[name='submit_search']")).click();
        String searchHeader = driver.findElement(By.cssSelector("H1")).getText().toLowerCase();
        
        Assert.assertTrue(searchHeader.contains("ipod nano"));
    }
}

Problem

We can add nodes to the Selenium Grid Hub, now the hub is up and running. This recipe will describe how to launch Selenium instances and register them to the hub so it starts forming the grid op Selenium instances that we are expecting.

Prerequisites

Open the command prompt and navigate to the directory where we put the selenium-server-standalone file.

Solution

Enter something like the following command in the command prompt: java -jar selenium-server-standalone-.jar -role node -hub http://localhost:4444/grid/register -port 5556 We should see the following output in the console:

What has been done

The command above will register a node to the hub, which is fully backwards compatible with Selenium 1. We can see this node in the Hub Console:

We can add multiple operating systems to our Selenium Grid. This allows us to check, whether different versions of Firefox on different operating systems are showing the same behavior.

Problem

We want to start Selenium HUB, which is the central point in the Grid that will receive all Selenium commands and route them to the right node.

Prerequisites

Download the latest version of selenium-server-standalone.jar from the Selenium website. (http://code.google.com/p/selenium/downloads/list) Open the command prompt and navigate to the directory where we put the selenium-server-standalone file.

Solution

Enter the following command in the command prompt to start the hub: java -jar selenium-server-standalone-.jar -role hub The output in the console will look like this:

What has been done

The hub will use port 4444 by default. We can see the Selenium instances that are connected to the hub and the status of the nodes if we put http://ip-of-hub:4444/grid/console in a browser window, where ip-of-hub is the name of the machine with the hub. If it is on the local machine then you can place http://localhost:4444/grid/console. We can see that in the next screenshot:

Tip: We can also specify a different port to run the hub on. We can give an extra parameter to the command, like: -port # (where # is the port-number)

Selenium Grid allows us to have multiple Selenium instances on multiple machines and then have one point to send Selenium commands to. We can specify the operating system / browser and browser version where we want to run the test on. The hub which is the central point of the Grid will route all the Selenium command to the requested node.

In this section of the blog we will see how to set up Selenium Grid and how to make use of some common properties.