Smart Testing 0.0.1 Released · Arquillian Blog

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.1 release of the Smart Testing component!

First release of Smart Testing is here!

It all started as a Google Summer of Code project last year with great ground work of our student Dimcho Karpachev.

Flash forward to this summer, after 222 commits, 69 pull requests merged and insane amount of CI builds we are super excited to announce a new addition to Arquillian Universe we’ve been working so vigorously on.

This time it’s not a new extension to Arquillian Testing Platform, but a brand new tool which we hope will bring a breath of fresh air into your builds, let it be on your local machine or the CI server.

You probably know it best – you write some code and tests, run the build, wait few minutes to only see at the later stage that your changes are breaking it. Of course you can go and grab a coffee, but if you commit early and often that might mean caffeine overdose in a very short time. Running tests before pushing is a good practice, but if it takes too long it simply slows you down.

Moreover having long lasting builds on your CI server makes your CI/CD process not really that continuous and leads to Pull Requests piling up. Bringing new features to production with confidence should be seamless. And for this we need to have faster feedback loops.

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

That’s why we created this cure for you!

Highlights of this release

With this release we provide following strategies which you can use to optimize your build execution:

new – gives higher priority to newly added tests
changed – gives higher priority to changed tests (e.g. new test methods or refactorings of components under tests which imply changes in the tests)
affected – based on changes in your business logic, gives higher priority to the tests which are exercising them
failed – gives higher priority to the tests which failed in the previous (local) build

In addition you can either decide to run the whole test suite (ordering mode) or only those tests which fall into the selected categories (selecting, which is also a default mode).

new, changed and affected rely on SCM information. For this release we only support Git.

How to get started

If you are using Maven 3.3.x or newer (and you definitely should!) adding Smart Testing to your build is very easy. Create a file .mvn/extensions.xml in the root of project
with the following content:

extensions.xml

<?xml version="1.0" encoding="UTF-8"?>
<extensions>
  <extension>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>maven-lifecycle-extension</artifactId>
    <version>0.0.1</version>
  </extension>
</extensions>

and you are all set!

Executing the build with Smart Testing enabled is as simple as adding one system property with strategies of your choice:

$ mvn clean verify -Dsmart.testing=new,changed,affected

This will execute the build running only those tests which are falling into selected categories, based on your local changes.

For the CI environment, such as Jenkins, you can pass commit hashes using environment variables. For example:
$ mvn verify -Dsmart.testing=affected -Dscm.range.head=GIT_COMMIT -Dscm.range.tail=GIT_PREVIOUS_COMMIT
This will optimize tests based on the changes between the current commit and the one against which the previous build was run.

For more details head over to our documentation backed by awesome Asciidoctor!

We are very excited about Smart Testing but we need you to help us make it even more awesome. We spent tremendous amount of effort building it with the developer experience in mind, but without the community feedback we can only get that far. So give it a try today and tell us what you think! Maven is a first stop for us. Stay tuned! There’s more to come!

Thanks to the whole team for making it happen! You are real heroes!

Also big kudos to Infinitest team for the inspiration! Give it a spin if you are Eclipse or IDEA user.

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component	Smart Testing
Version	0.0.1 view tag
Release date	2017-09-14
Released by	Bartosz Majsak
Compiled against	Arquillian Spacelift – 1.0.2 ShrinkWrap Resolver – 3.0.0-alpha-4

Published artifacts org.arquillian.smart.testing

org.arquillian.smart.testing » core jar javadoc pom
org.arquillian.smart.testing » surefire-provider jar javadoc pom
org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
org.arquillian.smart.testing » strategy-affected jar javadoc pom
org.arquillian.smart.testing » strategy-changed jar javadoc pom
org.arquillian.smart.testing » strategy-failed jar javadoc pom
org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
org.arquillian.smart.testing » git-rules jar javadoc pom
org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 30

Initial release with Maven integration.

Component: Core

Component: Test Bed

Component: Maven

Component: Documentation

#74 Smart Testing User Guide

Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar, Hemani