Automated web testing with Java, Selenium RC, LoggingSelenium, HtmlUnit and TestNG

I used the Selenium web automation framework extensively for a suite of websites I had tested. Over the course of a year I had ended up with a complete automation framework for a suite of websites. This resulted in lots of project specific code. Now that I need to use Selenium for other sites I thought I would write about setting things up at a basic level which includes using HtmlUnit, Selenium, LoggingSelenium, starting the Selenium server through java and using TestNG.

Getting Setup
Download Selenium RC and Selenium IDE from http://seleniumhq.org/download/. Selenium IDE will install right into Firefox as an add-on. I am using Selenium RC 1.0.3.

Download LoggingSelenium from http://sourceforge.net/projects/loggingselenium/files/

Download TestNG from http://testng.org/doc/index.html

Extract the files into your projects library folder and add the following 4 jars to your project:

1. selenium-server.jar
2. selenium-java-client-driver.jar
3. logging-selenium-1.2.jar
4. testng.jar

Configuring Selenium using a properties file

I use a properties file for all my Selenium settings. I like controlling things from a GUI interface and the properties file makes it easy to add a Swing dialog. Below is a simple class which facilitates this.

import java.io.File;
import java.io.FileInputStream;
import java.util.Properties;

public class Config {

    /**
     * Singleton instance.
     */
    private static Config instance;

    /**
     * Location of properties file.
     */
    public static final String PROPERTIES_FILE = "config/config.properties";

    /**
     * Properties file var for selenium browser.
     */
    public static final String SELENIUM_BROWSER = "selenium.browser";

    /**
     * Properties file var for selenium server port.
     */
    public static final String SELENIUM_SERVER_PORT = "selenium.server.port";

    /**
     * Properties file var for location of the Firefox Profile
     */
    public static final String SELENIUM_FIREFOX_PROFILE = "selenium.firefox.profile";

    /**
     * Properties file var for selenium server timeout setting.
     */
    public static final String SELENIUM_SERVER_TIMEOUT_SEC = "selenium.server.timout_in_sec";

    /**
     * Properties file var for encoding
     */
    public static final String ENCODING = "encoding";

    /**
     * Properties file var for the selenium server host.
     */
    public static final String SELENIUM_SERVER_HOST = "selenium.server.host";

    /**
     * Properties file var for the reports output path
     */
    public static final String SELENIUM_REPORTS_PATH = "selenium.reports.output_path";
    
    /**
     * Properties file var for the Selenium base url.
     */
    public static final String BASE_URL = "selenium.base_url";

    /**
     * Properties file var for thread count.
     */
    public static final String THREAD_COUNT = "thread.count";

    /**
     * Properties file var for thread pool size.
     */
    public static final String THREAD_POOL_SIZE = "thread.pool.size";

    private Properties properties;

    /**
     * Constructor
     * Loads the Properties file.
     */
    private Config() {
        properties = new Properties();
        File file = new File(Config.PROPERTIES_FILE);
        try {
            properties.load(new FileInputStream(file));
        } catch (Exception ex) {
            System.err.println("Could not load properties file: " + file.getAbsolutePath());
            ex.printStackTrace();
        }
    }

    /**
     * Singleton access to this class
     * @return Config
     */
    public synchronized static Config getInstance() {
        if (instance == null) {
            instance = new Config();
        }
        return instance;
    }

    /**
     * Proxy to properties getProperty
     */
    public String getProperty(String key) {
        return properties.getProperty(key);
    }

    /**
     * Proxy to properties getProperty
     */
    public String getProperty(String key, String defaultValue) {
        return properties.getProperty(key, defaultValue);
    }

    /**
     * Proxy to properties setProperty
     */
    public void setProperty(String key, String value) {
        properties.setProperty(key, value);
    }
}

Next create the Properties file.

selenium.firefox.profile=config\\firefox-profiles
selenium.server.port=4444
selenium.browser=*firefox3 c:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe
selenium.server.timout_in_sec=60
encoding=UTF8
selenium.server.host=localhost
selenium.reports.output_path=C:\\
selenium.base_url=localhost
thread.pool.size=2
thread.count=2

In the properties file I set the browser to Firefox. On Windows if I do this, Selenium complains that it can't find Firefox so I include the path afterwords.

There is also a setting for Firefox profile. If you are testing a site with a self signed certificate, you may get stuck accessing the website because Selenium launches an anonymous Firefox profile without your settings. When the web page loads, Selenium tests fail on the "This connection is Untrusted" page. If you create a profile specific to Selenium and create an exception for the self-signed certificate you can get past this untrusted connection warning:

1. Launch Firefox from the command line with: firefox.exe -ProfileManager
2. Create a new profile and specify the location of the profile. I named my profile Selenium and specified c:\selenium as the profile location for ease.
3. Launch the profile, navigate to the page with the self signed cert. Click, "I understand the risks", "Add Exception" and add the page exception.
4. Next go to c:\selenium and find cert.db and cert_override.txt
5. I copy these files to a location in my project folder and specify that location in the properties file key: selenium.firefox.profile.

Creating the Selenium Server
You can start the Selenium Server from the command line by navigating to the selenium-server.jar and entering: java -jar selenium-server.jar using options specified on SeleniumHq however I prefer to save myself the hassle of having to start and stop the server every time I want to run tests and do this through code.

import java.io.File;
import org.openqa.selenium.server.RemoteControlConfiguration;
import org.openqa.selenium.server.SeleniumServer;

public class MySeleniumServer {

    /**
     * The Selenium server instance.
     */
    private SeleniumServer seleniumServer;

    /**
     * Starts Selenium Server
     */
    public void start() {

        Config config = Config.getInstance();

        RemoteControlConfiguration rcc = new RemoteControlConfiguration();
        
        String browser = config.getProperty(Config.SELENIUM_BROWSER, "*chrome");
        if (browser.contentEquals("*chrome") || browser.contentEquals("*firefox") ||
            browser.contentEquals("*firefox2") || browser.contentEquals("*firefox3")) {
            String path = config.getProperty(Config.SELENIUM_FIREFOX_PROFILE);
            rcc.setFirefoxProfileTemplate(new File(path).getAbsoluteFile());
        }

        String sTimeout = config.getProperty(Config.SELENIUM_SERVER_TIMEOUT_SEC, "60");
        int timeout = Integer.parseInt(sTimeout);
        rcc.setTimeoutInSeconds(timeout);

        int port = Integer.parseInt(config.getProperty(Config.SELENIUM_SERVER_PORT, "4444"));
        rcc.setPort(port);

        rcc.setSingleWindow(true);

        try {
            seleniumServer = new SeleniumServer(false, rcc);
            seleniumServer.start();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
   
    /**
     * Stops Selenium Server
     */
    public void stop() {
        if (seleniumServer != null) {
            seleniumServer.stop();
        }
    }

The above class initializes the Selenium server with the remote control configuration options from the properties file.

Creating a Selenium base test class
Now we can create a base test case class that initializes LoggingSelenium (Selenium with reporting capabilites).

import com.thoughtworks.selenium.HttpCommandProcessor;
import com.thoughtworks.selenium.SeleneseTestCase;
import java.io.BufferedWriter;
import com.unitedinternet.portal.selenium.utils.logging.HtmlResultFormatter;
import com.unitedinternet.portal.selenium.utils.logging.LoggingCommandProcessor;
import com.unitedinternet.portal.selenium.utils.logging.LoggingDefaultSelenium;
import com.unitedinternet.portal.selenium.utils.logging.LoggingResultsFormatter;
import com.unitedinternet.portal.selenium.utils.logging.LoggingUtils;
import java.io.File;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.GregorianCalendar;

public class MySeleniumTestCase {

    /**
     * The results writer
     */
    private BufferedWriter loggingWriter;

    /**
     * The Config instance
     */
    private Config config = Config.getInstance();

    /**
     * The LoggingSelenium writer
     */
    protected LoggingDefaultSelenium selenium;

    /**
     * Instance of SeleneseTestCase.
     * Did not subclass because may use JUnit4 or TestNG
     */
    protected SeleneseTestCase selenese = new SeleneseTestCase();

    /**
     * Initializes Selenium
     * @param baseUrl The baseUrl of the website to test
     * @param browserString The browser to test with.
     * @throws Exception
     */
    public void setUp(String baseUrl, String browserString) throws Exception {
        String encoding = config.getProperty(Config.ENCODING, "UTF8");
        String host = config.getProperty(Config.SELENIUM_SERVER_HOST, "localhost");
        String output = config.getProperty(Config.SELENIUM_REPORTS_PATH);
        int port = Integer.parseInt(config.getProperty(Config.SELENIUM_SERVER_PORT, "4444"));

        File logger = new File(output + File.separator +
                getResultsName(this.getClass().getName()) + ".html");
        loggingWriter = LoggingUtils.createWriter(logger.getAbsolutePath(), encoding, true);

        LoggingResultsFormatter htmlFormatter = new HtmlResultFormatter(loggingWriter, encoding);
        htmlFormatter.setScreenShotBaseUri("");
        htmlFormatter.setAutomaticScreenshotPath(output);

        LoggingCommandProcessor htmlProcessor =
                new LoggingCommandProcessor(
                new HttpCommandProcessor(host, port, browserString, baseUrl), htmlFormatter);

        selenium = new LoggingDefaultSelenium(htmlProcessor);
        selenium.start();
    }

    public void setUp(String url) throws Exception {
        String browser = config.getProperty(Config.SELENIUM_BROWSER, "*chrome");
        setUp(url, browser);
    }

    public void setUp() throws Exception {
        setUp(config.getProperty(Config.BASE_URL));
    }

    /**
     * Takes a screenshot of the Desktop
     */
    public void takeScreenshot() {
        selenium.captureScreenshot(config.getProperty(Config.SELENIUM_REPORTS_PATH) + File.separator +
                getResultsName(this.getClass().getName()) + ".png");
    }

    /**
     * Tears down selenium and closes the logging file.
     * @throws Exception
     */
    public void tearDown() throws Exception {
        try {
            selenese.checkForVerificationErrors();
        } finally {
            selenium.stop();
            if (loggingWriter != null) {
                loggingWriter.close();
            }
         }
    }

   /**
    * A datetime to use as a unique filename.
    * Filenames are unique per second. 
    * @param filename The output file name.
    * @return The unique filename. Exmample: 2011-01-03-19-20-11-filename
    */
    public String getResultsName(String filename) {
        String name = filename.substring(filename.lastIndexOf(".") + 1, filename.length());
        Calendar cal = new GregorianCalendar();
        cal.setTime(new Date());
        DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd-HH-mm-ss");
        String date = dateFormat.format(cal.getTime());
        return date + "-" + name;
    }

}

The bulk of the above class is the setUp method which pulls values from the properties file in order to configure an instance of LoggingSelenium. Notice how the class has a protected member: selenese which is an instance of SeleneseTestCase. This is a class that subclasses JUnit 3. Since I may want to use JUnit 4 I did not subclass SeleneseTestCase. SeleneseTestCase contains many useful testing helper functions including "verify" functions. Instead of AssertTrue, or AssertEquals, SeleneseTestCase has verifyTrue or VerifyEquals. This allows tests to continue even if they fail. When tests fail, SeleneseTestCase adds the failure exception messages to an array and then throws an Exception only when SeleneseTestCase.tearDown() or SeleneseTestCase.checkForVerificationErrors() is called. Therefore JUnit won't catch the test failure until one of these methods is called. Be sure to pay particular attention that one of these methods is called. (tearDown calls checkForValidationErrors).

Creating Tests
Finally, we can create our tests.

import com.gargoylesoftware.htmlunit.IncorrectnessListener;
import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController;
import com.gargoylesoftware.htmlunit.SilentCssErrorHandler;
import com.gargoylesoftware.htmlunit.WebClient;
import java.lang.annotation.Annotation;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;
import org.testng.Assert;
import org.testng.IAnnotationTransformer;
import org.testng.TestNG;
import org.testng.annotations.AfterGroups;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.ITestAnnotation;
import org.testng.annotations.Test;

public class SampleSeleniumTest extends SeleniumTestCase {

    protected MySeleniumServer server;

    @BeforeGroups(groups = "Selenium")
    public void startServer() {
        server = new MySeleniumServer();
        server.start();
    }

    @AfterGroups(groups = "Selenium")
    public void stopServer() {
        server.stop();
    }

    @Test(groups = "Selenium")
    public void googleSearch() throws Exception {
        setUp("http://google.com");
        selenium.open("/");
        selenium.type("q", "test");
        takeScreenshot();
        selenium.click("btnG");
        selenese.verifyTrue(selenium.isTextPresent("Images"));
        takeScreenshot();
        tearDown();
    }

    @Test(groups = {"HtmlUnit", "Concurrent"})
    public void webUnit() throws Exception {
        WebDriver driver = new HtmlUnitDriver(true) {

            @Override
            protected WebClient modifyWebClient(WebClient client) {
                client.setCssErrorHandler(new SilentCssErrorHandler());
                client.setAjaxController(new NicelyResynchronizingAjaxController());
                client.setThrowExceptionOnScriptError(true);
                client.setCssEnabled(true);
                client.setIncorrectnessListener(new IncorrectnessListener() {

                    public void notify(String arg0, Object arg1) {
                        //ignore messages
                    }
                });
                return client;
            }
        };
        driver.get("http://www.google.com");
        WebElement element = driver.findElement(By.name("q"));
        element.sendKeys("Test");
        element.submit();
        Assert.assertEquals("Test - Google Search", driver.getTitle());
        driver.close();
    }

    public static void main(String[] args) {
        TestNG ng = new TestNG();
        Config config = Config.getInstance();
        int threadPoolSize = Integer.parseInt(config.getProperty(Config.THREAD_POOL_SIZE, "1"));
        int threadCount =Integer.parseInt(config.getProperty(Config.THREAD_COUNT, "1"));
        ConcurrentConfig concurrentConfig = new ConcurrentConfig();
        concurrentConfig.setInvocationCount(threadCount);
        concurrentConfig.setThreadPoolSize(threadPoolSize);
        ng.setAnnotationTransformer(concurrentConfig);
        ng.setTestClasses(new Class[]{SampleSeleniumTest.class});
        ng.setGroups("Selenium, HtmlUnit");
        ng.run();
    }
}

class ConcurrentConfig implements IAnnotationTransformer {

    private int invocationCount = 1;
    private int threadPoolSize = 1;

    public void setInvocationCount(int invocationCount) {
        this.invocationCount = invocationCount;
    }

    public void setThreadPoolSize(int threadPoolSize) {
        this.threadPoolSize = threadPoolSize;
    }

    public void transform(ITestAnnotation annotation, Class arg1, Constructor arg2, Method testMethod) {
        Annotation[] methodAnnotations = testMethod.getAnnotations();

        for (Annotation methodAnnotation : methodAnnotations) {

            if (methodAnnotation instanceof Test) {
                Test test = (Test) methodAnnotation;

                for (String group : test.groups()) {

                    if (group.contentEquals("Concurrent")) {
                        annotation.setInvocationCount(invocationCount);
                        annotation.setThreadPoolSize(threadPoolSize);
                    }
                }
            }
        }
    }
}

The above example runs two very simple tests. One uses Selenium and the other uses HtmlUnit.

The Selenium test will launch the browser, execute the tests and output an html report with screenshots of the test.

The HtmlUnit test will launch an embedded java browser and run a test without launching a heavy weight browser. I often times fall back on HtmlUnit for stress testing. For example, I can set the TestNG annotations for thread pool and thread count to 50 and simulate 50 users accessing a web site at the same time.

I included the ConcurrentConfig class in the above code snipped just because it is boilerplate code that is useful in order to control the thread pool and count from a properties file. Aside from that the above class is mainly setup the way it is for demonstration purposes.