Arquillian Blog · Arquillian

The Arquillian team is proud to announce the 1.0.0 release of the RushEye component!

We are introducing the very first release of the Arquillian RushEye extension.

Arquillian RushEye

RushEye is a tool for automated bulk or on-the-fly inspecting of browser screen images and comparison with predefined image suite.

It enables automated visual testing (using Selenium) and minimizes efforts for manual visual testing of applications.

How to use it?

Getting rusheye.jar

Get the rusheye-dist jar from Maven central or:

Obtain the source from

https://github.com/arquillian/arquillian-rusheye

and build it using

cd arquillian-rusheye/
mvn clean install

You can find the rusheye.jar necessary to run the test suite install

arquillian-rusheye/rusheye-dist/target/rusheye.jar

Creating test suite

Let’s assume we have two directories, screenshots1 and screenshots2.

The screenshots1 is the directory with sample images (patterns), while screenshots2 is a directory with new images which we would like to compare with patterns.

Directory structure

The crawl mechanism expects the flat structure of crawled directory, e.g.:

screenshots1/test1.png
screenshots1/test2.png

Crawling directory

We can create an XML descriptor of RushEye test suite by calling:

java -jar rusheye.jar crawl screenshots1/ -O suite.xml

A file suite.xml with content similar to following one will be created:

<?xml version="1.0" encoding="UTF-8"?>
<visual-suite xmlns="http://www.jboss.org/rusheye/visual-suite"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.jboss.org/rusheye/visual-suite http://fryc.eu/rusheye/visual-suite.xsd">

  <global-configuration>
    <listener type="org.jboss.rusheye.parser.listener.CompareListener">
      ...
    </listener>
    <pattern-retriever type="org.jboss.rusheye.retriever.FileRetriever"/>
    <mask-retriever type="org.jboss.rusheye.retriever.FileRetriever"/>
    <sample-retriever type="org.jboss.rusheye.retriever.sample.FileSampleRetriever"/>
    <perception/>
  </global-configuration>

  <test name="test1">
    <pattern name="test1" source="test1.png"/>
  </test>
  ...

Running test suite

Once we created suite.xml from the screenshots1 directory structure, we can use RushEye to run the test suite against images in the screenshots2 directory.

java -jar rusheye.jar parse suite.xml  \
    -D result-output-file=result.xml  \
    -D samples-directory=screenshots2  \
    -D patterns-directory=screenshots1  \
    -D file-storage-directory=diffs

On the standard output, you can see following lines:

[ DIFFER ] SeparatorTestCase.testThirdSeparator
[ DIFFER ] SimpleEditorTestCase.testEditorSource
[ DIFFER ] SpinnerTestCase.testLowerClickUp
  ...
[ DIFFER ] TabPanelTestCase.testTabPanelExample
[ DIFFER ] OrderingListTestCase.testUpMultipleSongsShift
=====================
  Overall Statistics:
  DIFFER: 391
=====================

The directory diffs will be created and an image comparing pattern with sample will be created for every test where pattern and sample image differs. This image highlights changes between the two compared images.
Adding mask to test suite

Since they are a lot of differences in images, we can define masks to hide the unimportant differences.

There is currently only one type of mask:
Selective-Alpha

There are predefined masks in masks directory:
masks/masks-selective-alpha

Remove or clear the diffs directory and run the crawl command again with masks directory specified and use -f to override existing suite.xml.

java -jar rusheye.jar crawl screenshots1/ -O suite.xml -m masks/ -f

Now you can run the comparison script again:

java -jar rusheye.jar parse suite.xml  \
    -D result-output-file=result.xml  \
    -D samples-directory=screenshots2  \
    -D patterns-directory=screenshots1  \
    -D file-storage-directory=diffs \
    -D masks-directory=masks

Now, you are getting much more stable results:

[ SAME ] KeepAliveTestCase.testUsingIncorrectWay
[ SAME ] PanelCustomizationTestCase.testThirdPanel
[ SAME ] TooltipDataTableTestCase.testIterateThroughTable
[ SAME ] ColorPickerTestCase.testPageSource
[ SAME ] SimpleEditorTestCase.testStrikethroughButton
[ SAME ] SeparatorTestCase.testFirstSeparator
[ DIFFER ] TableSortingTestCase.testExternalSorting
[ SAME ] TooltipTestCase.testDefaultToolTip
  ...
[ SAME ] StyleTestCase.testBackgroundColor
=====================
  Overall Statistics:
  SAME: 318
  DIFFER: 76
=====================

What is Arquillian?

Arquillian is open source software that empowers you to test JVM-based applications more effectively. Created to defend the software galaxy from bugs, Arquillian brings your test to the runtime so you can focus on testing your application's behavior rather than managing the runtime. Using Arquillian, you can develop a comprehensive suite of tests from the convenience of your IDE and run them in any IDE, build tool or continuous integration environment.

Release details

Component	RushEye
Version	1.0.0 view tag
Release date	2017-05-03
Released by	Matous Jobanek

Published artifacts org.arquillian.rusheye

org.arquillian.rusheye » rusheye-api jar javadoc pom
org.arquillian.rusheye » rusheye-impl jar javadoc pom

Thanks to the following list of contributors: Lukas Fryc, Matous Jobanek, Bartosz Majsak, Selvaraj, Vinoth