Het meten van performance – Introductie

<

!–:nl–>

We kunnen met Selenium WebDriver performance metingen doen op de website. Er zijn veel manieren om de performance van een website te verbeteren, maar eerst moet je weten waar de knelpunten zijn. We willen bijvoorbeeld weten wat de totale laadtijd van de pagina is, want dat geeft informatie over hoe lang het duurt voordat het onload event wordt getriggerd. Daarnaast kunnen we geïnteresseerd zijn in de laadtijd per item. Met deze informatie kunnen we bijvoorbeeld thirdparty content of langzaam ladende afbeeldingen mijden. Ook kunnen we meten hoe lang het duurt voordat elementen zichtbaar zijn in de DOM.

Implementing code timings

Problem

The easiest way to get some performance related metrics is to implement timings in your testcode. This will work in every browser which is the big advantage. The disadvantage is probably that we have to enrich the testcode with timers. A lot of websites will use AJAX or other JavaScript functionality, these days. It can be interesting to see how long it takes before an element appears in the HTML DOM. In this recipe we will see how we can implement a timing strategy in the testcode itself.

Prerequisites

Think of the objectives you want to measure, because the outcome really depends on the implementation.

Solution

The script within a testmethod will be executed hierarchically, from top till down. The moment the script hits the line with long startTime, the start-time will be stored in a local variable. The moment the script hits the line with long endTime, the end-time will be stored in a local variable. After that the time difference is calculated, with long totalTime = endTime – startTime;.


@Test
public void codeTimings() {
    driver.get("/");
    long startTime = System.currentTimeMillis();
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    long endTime = System.currentTimeMillis();
    long totalTime = endTime - startTime;
    System.out.println("Totaltime: " + totalTime + "milliseconds");
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
@Test
public void codeTimings() {
    driver.get("/");
    long startTime = System.currentTimeMillis();
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    long endTime = System.currentTimeMillis();
    long totalTime = endTime - startTime;
    System.out.println("Totaltime: " + totalTime + "milliseconds");
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    
}

There’s more…

We have to enrich our testscripts with a lot of extra code, following the above principles. What will happen if the way we calculated the time difference will change or we want to store the time difference in a database? We have to adjust the code on multiple places. A solution for this challenge is to extract the code to a separate class. We can use the class below to calculate the timings.


public class Timer {
    private long startTime = 0;
    private long endTime = 0;

    public void start() {
        this.startTime = System.currentTimeMillis();
    }

    public void end() {
        this.endTime = System.currentTimeMillis();
    }

    public long getStartTime() {
        return this.startTime;
    }

    public long getEndTime() {
        return this.endTime;
    }

    public long getTotalTime() {
        return this.endTime - this.startTime;
    }
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
public class Timer {
    private long startTime = 0;
    private long endTime = 0;

    public void start() {
        this.startTime = System.currentTimeMillis();
    }

    public void end() {
        this.endTime = System.currentTimeMillis();
    }

    public long getStartTime() {
        return this.startTime;
    }

    public long getEndTime() {
        return this.endTime;
    }

    public long getTotalTime() {
        return this.endTime - this.startTime;
    }
}

We have to initialize a new object if we want to use the functions. We can use the code below for implementing the timer class.


@Test
public void measureWebTimings() {
    driver.get("/");
    Timer timer = new Timer();
    timer.start();
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    timer.end();
    System.out.println("Totaltime: " + timer.getTotalTime()
                + " milliseconds");
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
@Test
public void measureWebTimings() {
    driver.get("/");
    Timer timer = new Timer();
    timer.start();
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
    timer.end();
    System.out.println("Totaltime: " + timer.getTotalTime()
                + " milliseconds");
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText()
                .contains("Selenium"));
}

Using NetExport to export Firebug’s NET panel

Problem

Firebug comes with a create feature which is visible in the NET panel. The NET panel gives use the per item download time in a breakdown structure. This structure is build up from a HAR file. This recipe shows us how we can export the HAR file from a code perspective.

Prerequisites

We have to download three firefox add-ons, before we can start implementing this. It is not mandatory to install the add-ons, we can also store them locally. The following three files are available from http://getfirebug.com/releases/ : firebug, NetExport and firestarter.

Solution

The following code can be used in the setup method to implement the NetExport measurements:


@BeforeMethod
public void setUp() {
    FirefoxProfile profile = new FirefoxProfile();
    try {
        profile.addExtension(new File("c:/firebug-1.7.2.xpi"));
        profile.addExtension(new File("c:/netExport-0.8b16.xpi"));
        profile.addExtension(new File("c:/fireStarter-0.1a6.xpi"));
    } catch (IOException e) {
        throw new RuntimeException("Could not load required extensions, did you download them to the above location? ", e);
    }
        profile.setPreference("extensions.firebug.currentVersion", "9.99");
        profile.setPreference("extensions.firebug.DBG_NETEXPORT", false);
        profile.setPreference("extensions.firebug.onByDefault", true);
        profile.setPreference("extensions.firebug.defaultPanelName", "net");
        profile.setPreference("extensions.firebug.net.enableSites", true);
        profile.setPreference("extensions.firebug.net.persistent", true);
        profile.setPreference("extensions.firebug.netexport.alwaysEnableAutoExport", true);
        profile.setPreference("extensions.firebug.netexport.autoExportToFile", true);
        profile.setPreference("extensions.firebug.netexport.autoExportToServer", false);
        profile.setPreference("extensions.firebug.netexport.defaultLogDir", "C:\\temp");
        profile.setPreference("extensions.firebug.netexport.showPreview", true); // preview.
        profile.setPreference("extensions.firebug.netexport.sendToConfirmation", false);
        profile.setPreference("extensions.firebug.netexport.pageLoadedTimeout", 1500);
        profile.setPreference("extensions.firebug.netexport.Automation", true);
    driver = new FirefoxDriver(profile);
    selenium = new WebDriverBackedSelenium(driver, "http://www.google.com");
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
@BeforeMethod
public void setUp() {
    FirefoxProfile profile = new FirefoxProfile();
    try {
        profile.addExtension(new File("c:/firebug-1.7.2.xpi"));
        profile.addExtension(new File("c:/netExport-0.8b16.xpi"));
        profile.addExtension(new File("c:/fireStarter-0.1a6.xpi"));
    } catch (IOException e) {
        throw new RuntimeException("Could not load required extensions, did you download them to the above location? ", e);
    }
        profile.setPreference("extensions.firebug.currentVersion", "9.99");
        profile.setPreference("extensions.firebug.DBG_NETEXPORT", false);
        profile.setPreference("extensions.firebug.onByDefault", true);
        profile.setPreference("extensions.firebug.defaultPanelName", "net");
        profile.setPreference("extensions.firebug.net.enableSites", true);
        profile.setPreference("extensions.firebug.net.persistent", true);
        profile.setPreference("extensions.firebug.netexport.alwaysEnableAutoExport", true);
        profile.setPreference("extensions.firebug.netexport.autoExportToFile", true);
        profile.setPreference("extensions.firebug.netexport.autoExportToServer", false);
        profile.setPreference("extensions.firebug.netexport.defaultLogDir", "C:\\temp");
        profile.setPreference("extensions.firebug.netexport.showPreview", true); // preview.
        profile.setPreference("extensions.firebug.netexport.sendToConfirmation", false);
        profile.setPreference("extensions.firebug.netexport.pageLoadedTimeout", 1500);
        profile.setPreference("extensions.firebug.netexport.Automation", true);
    driver = new FirefoxDriver(profile);
    selenium = new WebDriverBackedSelenium(driver, "http://www.google.com");
}

Do not forget to update the paths to the latest version of the extensions. The code above is tested with Mozilla Firefox 3.6.

What has been done

The script will load three add-ons while launching the Firefox browser. Some parameters are set, like the default logging folder.

Using Dynatrace to measure the performance

Problem

DynaTrace Ajax Edition is a free tool for optimizing web 2.0 applications. It supports Microsoft Internet Explorer as well as Mozilla Firefox. Dynatrace finds problems and provides a complete picture of what is going on in the browser and it automatically recommend optimizations. This recipe will show how dynaTrace can capture the network traffic from the activity generated by your testing tool.

Prerequistes

We have to download and install dynaTrace AJAX Edition from http://ajax.dynatrace.com/, before we can start measuring our websites performance.

Solution

  1. We have to set an environment variable to the Selenium process that launches the browser, before we can capture a performance metrics. We can do this in eclipse by opening the context menu on the project and select Run As > Run Configurations…
     Click on the TestNG test class in the left panel, then click on the environment tab in the right panel. In this tab we can add our environment variables that will launch with the TestNG test, by clicking New…
  2. Start the installed dynaTrace AJAX edition application.
  3. Run the Selenium tests.
  4. The results are visible in dynaTrace AJAX edition, after finishing the tests.

You have to start dynaTrace AJAX Edition before running your test. The Browser that you launch via Selenium will not start the AJAX Edition Client but will just try to connect to an existing instance.

What has been done

Selenium starts a browser with the predefined environment variables and will pass all performance related metric through dynaTrace AJAX Edition. Testresults can be analyzed in dynaTrace AJAX Edition after finishing the testscripts.

There is more…

Allow thirdparty add-ons

We might not see any results in dynaTrace AJAX edition due Microsoft Internet Explorer security restrictions. In this case we need to modify the browser settings using the following actions.

  1. Open Microsoft Internet Explorer
  2. Go to the manage add-ons window by clicking Tools > Internet Options and select Programs > Manage add-ons
  3. Enable the dynaTrace AJAX edition Toolbar and Agent, by selecting the add-on and click on Enable

Environments variables

We can set more environment variables to the Selenium process. We can implement the environment variables in the tables below on the same way as described above.

Variable Description
 DT_AE_AGENTACTIVE  True will activates the browser add-on Agent.
 DT_AE_AGENTNAME  Name of the recorded session.
 DT_AE_SERVER  (optional) Allows you to connect to a dynaTrace AJAX Client on a different port (only localhost connections are allowed). Default: localhost:9988
 DT_AE_CLEARCACHE  (optional) Allows you to clear the browser cache before browsing through your sites. Default: false
 DT_AE_STARTURL  (optional) Allows you to specify an initial Start Url. Default: about:blank

Debugging

We might want to know which environment variables are set for debugging purposes. We can download Microsoft Process Explorer from the following location:  http://technet.microsoft.com/en-us/sysinternals/bb896653
We can check the browser process (iexplorer.exe or firefox.exe) and check which environment variables get passed. In the screenshot below we can see the environment variables that get passed by dynaTrace AJAX Edition Client to the browser when launching a browser from the Client.

 

Viewing the HAR format

Problem

We have seen the HAR (Http ARchive) format as the output from the BrowserMob proxy implementation and the NetExport implementation. The HAR file contains a log with requests and responses to the server. The file can be displayed as a waterfall or as an expandable tree. This recipe shows a couple of ways to view the HAR format.

 

Prerequisites

Make sure you gathered a HAR file from the application under test.

Solution

We will see how to view a HAR file by using an online viewer.

  1. Open the online HAR viewer, http://www.softwareishard.com/har/viewer/
  2. Copy and paste the contents of the HAR file into the textbox. We can also drag and drop the file from our file-system to the website.
  3. Press Preview, in case you copied the contents of the HAR file.
  4. The result will look like the image below:

What has been done

The online HAR viewer reads the HAR file and builds the waterfall diagram.

There’s more…

There are more ways to view the HAR files. This section will cover a couple of frequently used ways.

Fiddler

We have to follow a couple of steps before we can view a HAR file in Fiddler. The steps are described below:

  1. Download fiddler from the following website: http://www.fiddler2.com
  2. Install Fiddler on your local machine
  3. Click on File → Import Sessions… and select your import format
  4. Select the existing HAR file from your filesystem.
  5. All the details from the sessions are visible in Fiddler.

Showslow.com

NetExport gives us the functionality to upload the HAR file to http://www.showslow.com/beacon/har/. The HAR file is now accessible from the internet and can be easily shared with others. It is even possible to install ShowSlow on a private server to limit the risk.

Implementing web.Timings

Problem

A new embedded feature in some browsers is the webtimings. This feature works only in Internet Explorer 9 and Google Chrome, by the time this book is published. We can also combine it with predefined Javascript variables. We can use that if we want to start the timings after a specific element is present in the HTML DOM. The disadvantage of web.Timings is that it does not give timings per item downloaded. In this recipe we will see how we can make use of the new standard.

Solution

We have to execute some JavaScript through Selenium, before we get performance related values. The script, including the JavaScript calls, will look like this:


@Test
public void measureWebTimings() throws InterruptedException {
    driver.get("/");
    JavascriptExecutor js = (JavascriptExecutor) driver;
    Double loadEventEnt = (Double) js.executeScript("return window.performance.timing.loadEventEnd;");
    Double navigationStart = (Double) js.executeScript("return window.performance.timing.navigationStart;");
    System.out.println("Page Load Time = " + (loadEventEnt - navigationStart));
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText().contains("Selenium"));
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText().contains("Selenium"));
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
@Test
public void measureWebTimings() throws InterruptedException {
    driver.get("/");
    JavascriptExecutor js = (JavascriptExecutor) driver;
    Double loadEventEnt = (Double) js.executeScript("return window.performance.timing.loadEventEnd;");
    Double navigationStart = (Double) js.executeScript("return window.performance.timing.navigationStart;");
    System.out.println("Page Load Time = " + (loadEventEnt - navigationStart));
    WebElement search = driver.findElement(By.id("q"));
    search.sendKeys("Selenium");
    search.submit();
    assertTrue(driver.findElement(By.tagName("body")).getText().contains("Selenium"));
    WebElement result = driver.findElement(By.linkText("Selenium"));
    result.click();
    assertTrue(driver.findElement(By.tagName("body")).getText().contains("Selenium"));
}

What has been done

The script is taking some values from the browser and pass them through back to the test.