Smart Testing 0.0.5 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.5 release of the Smart Testing component!

Highlights of this release

In this release we ship several API improvements making integration with 3rd party tools much easier.

Smart Testing API

The Smart Testing tool consists of three parts:

  • Core
  • Maven extension
  • Surefire provider integration

Both the Surefire provider integration and the Maven extension are implemented only for the usage in Maven builds. They take care of the integration and invoke Smart Testing API provided by Core.

The Core contains the main logic that is responsible for selecting/ordering and applying corresponding strategies. This logic is exposed by Smart Testing API. If you want to use the logic in your environment or as an integration with some third-party library, you need to have the dependency of Smart Testing Core on your classpath:

<dependency>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>core</artifactId>
    <version>${smart.testing.version}</version>
</dependency>

plus all dependencies of strategies you want to use for the prioritization:

<dependency>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>strategy-${strategy.name}</artifactId>
    <version>${smart.testing.version}</version>
</dependency>

Having these dependencies specified, you can start using the API. The starting point is SmartTesting class that provides you a fluent API:

SmartTesting
    .with(className -> isTest(className), configuration)
    .in("path/to/my/project")
    .applyOnClasses(suite);

The first method with takes as the first parameter a function that says which class is a test and which not. The second parameter is a configuration that should be used. The configuration file can be loaded using:

ConfigurationLoader.load(projectDir)

The second method in sets path to the project where the tests will be executed in.
Last method is either applyOnClasses or applyOnNames. It invokes the prioritization process and returns a prioritized set of TestSelection classes. This class contains class name and list of strategies that are applied to it. If you want to get only the names or classes then use the method SmartTesting.getNames(selection) or SmartTesting.getClasses(selection) respectively.

With it, you should be able to use the Smart Testing logic in your test suites without being limited to the Maven builds.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.5 view tag
Release date 2017-11-28
Released by Matous Jobanek
Compiled against

Published artifacts org.arquillian.smart.testing

Release notes and resolved issues 4

Component: Core

Thanks to the following list of contributors: Matous Jobanek, Bartosz Majsak

Smart Testing 0.0.4 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.4 release of the Smart Testing component!

Highlights of this release

In this release we shipped two key new features – support for JUnit 5 and declarative way of defining relations between tests and business code. And bunch of fixes.

Annotations for affected tests

Starting from this release you are able to use declarative way of defining relationship between test and business code. By default affected strategy uses imports declared in the tests to build the graph of collaborators. This approach is fine for unit tests (white box tests) but might not work for higher level tests (black box test).

Using annotation like below you can define classes or packages which particular test covers.

@ComponentUnderTest(packages = "org.acme.main.superbiz.*")
public class AcmeServiceRestTest { ... }

Check our detailed documentation on how to use it.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.4 view tag
Release date 2017-11-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

Release notes and resolved issues 13

Component: Test Bed
Component: Maven
Other
Component: Core
Component: Selection

Thanks to the following list of contributors: Hemani, Dipak Pawar, Alex Soto, Matous Jobanek, Bartosz Majsak

Smart Testing 0.0.3 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.3 release of the Smart Testing component!

Highlights of this release

After one month of hard work we bring you the third release of Smart Testing tool!
Apart from several small improvements, component upgrades, and bug fixes, this release contains one killer feature and one convenient feature that will make your life easier.

Configuration file

In previous releases, there was only one way of configuring the tool – through system properties. As we started introducing more options, we quickly realized that this is definitely not the most pleasant way of setting up the tool. From now on you can also use YAML configuration file.
The name of the file should be either smart-testing.yml or smart-testing.yaml and should be located in the root of the directory the build is executed from. Here’s an example:

strategies: new, changed, affected
mode: ordering
applyTo: surefire
debug: true
report:
    enable: true
scm:
    range:
      head: HEAD
      tail: HEAD~2

We are keeping the names as it is done for the system properties, just without the prefix smart.testing. This means that when you want to activate a set of strategies on a command line, then you should use the system property smart.testing.strategies. As you can see in the example, in the configuration file it is just strategies. Analogically for other parameters.

The system properties are not going away. When both system properties and the configuration file are used, the values defined by system properties have the precedence. This can be helpful when your Smart Testing setup is shared in your scm but you want to have special settings for different stages of CI pipeline.

Autocorrection of mistyped strategy names

It might happen that you misspell a strategy when executing the build. For example:

$ mvn clean verify -Dsmart.testing=new,change,afected

It can take you a second or two to notice why this will lead to a failing build with some hints in the error message. To save time (and headaches) you can activate the autocorrection feature by setting smart.testing.autocorrect=true on a command line or just autocorrect: true in the config file. If it is set, the mistyped name is automatically corrected and the affected strategy is used. In this case afected should be affected.

What’s next

We are working hard on the next release which will bring JUnit 5 support, a possibility of configuring custom strategies and many more cool features! Stay tuned!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.3 view tag
Release date 2017-10-23
Released by Matous Jobanek
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 16

Component: Core
Component: Maven
Component: Test Bed
Component: Selection

Thanks to the following list of contributors: Matous Jobanek, Dipak Pawar, Hemani, Bartosz Majsak, Alex Soto

Smart Testing 0.0.2 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.2 release of the Smart Testing component!

Highlights of this release

After a week since our very first release we have a brand new one! This time with several improvements and bug fixes, namely:

  • simplified installation process
  • hinting available strategies when misspelled in the configuration
  • ability to narrow search for affected tests
  • improved documentation

Simplified getting started

Even though installation process is very simple, we are huge fans of automation. So we created one liner which will take care of adding Smart Testing to your project.

Simply execute following snippet and you are ready to go.

$ curl -sSL https://git.io/v5jy6 | bash

Hinting available strategies

It might happen that you misspell a strategy when executing the build. For example:

$ mvn clean verify -Dsmart.testing=new,change,afected

This will lead to a failing build, but instead of leaving you with not just an exception thrown at your face and documentation to dig into, we are now hinting to matching strategies. With 0.0.2 you will see instead:

Unable to find strategy [afected]. Did you mean [affected]?
Unable to find strategy [change]. Did you mean [changed]?

Affected tests

Affected is one of the strategies in Smart Testing which let you find tests related to business code you have just changed. By default we transitively look up all related tests, but this can lead to a lot of tests being prioritized. Especially if you have deep hierarchies in your project.

To find only immediate related tests we have introduced a configuration option smart.testing.affected.transitivity which you can set to false.

Asciidoctor Extensions

We’ve created two Asciidoctor extensions to make documentation more resilient to changes and also to simplify the copy-paste process from documentation to your terminal.

First extension allows you to reference constants in the documentation from source code. For example to get the value represented by SMART_TESTING_MODE constant, add const macro in your document:

const:core/src/main/java/org/arquillian/smart/testing/Configuration.java[name="SMART_TESTING_MODE"]

Second extension brings copy-to-clipboard button in the rendered docs. To add it, simply use copyToClipboard macro:

[[singleTest]]
`mvn test -Dtest=SampleTest`  copyToClipboard:singleTest[]

What’s next

We are working hard on the next release which will bring simplified configuration and many more cool features! Stay tuned!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.2 view tag
Release date 2017-09-26
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 21

Component: Test Bed
Component: Core
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar

Smart Testing 0.0.1 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.1 release of the Smart Testing component!

First release of Smart Testing is here!

It all started as a Google Summer of Code project last year with great ground work of our student Dimcho Karpachev.

Flash forward to this summer, after 222 commits, 69 pull requests merged and insane amount of CI builds we are super excited to announce a new addition to Arquillian Universe we’ve been working so vigorously on.

This time it’s not a new extension to Arquillian Testing Platform, but a brand new tool which we hope will bring a breath of fresh air into your builds, let it be on your local machine or the CI server.

You probably know it best – you write some code and tests, run the build, wait few minutes to only see at the later stage that your changes are breaking it. Of course you can go and grab a coffee, but if you commit early and often that might mean caffeine overdose in a very short time. Running tests before pushing is a good practice, but if it takes too long it simply slows you down.

Moreover having long lasting builds on your CI server makes your CI/CD process not really that continuous and leads to Pull Requests piling up. Bringing new features to production with confidence should be seamless. And for this we need to have faster feedback loops.

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

That’s why we created this cure for you!

Highlights of this release

With this release we provide following strategies which you can use to optimize your build execution:

  • new – gives higher priority to newly added tests
  • changed – gives higher priority to changed tests (e.g. new test methods or refactorings of components under tests which imply changes in the tests)
  • affected – based on changes in your business logic, gives higher priority to the tests which are exercising them
  • failed – gives higher priority to the tests which failed in the previous (local) build

In addition you can either decide to run the whole test suite (ordering mode) or only those tests which fall into the selected categories (selecting, which is also a default mode).

new, changed and affected rely on SCM information. For this release we only support Git.

How to get started

If you are using Maven 3.3.x or newer (and you definitely should!) adding Smart Testing to your build is very easy. Create a file .mvn/extensions.xml in the root of project
with the following content:

extensions.xml
<?xml version="1.0" encoding="UTF-8"?>
<extensions>
  <extension>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>maven-lifecycle-extension</artifactId>
    <version>0.0.1</version>
  </extension>
</extensions>

and you are all set!

Executing the build with Smart Testing enabled is as simple as adding one system property with strategies of your choice:

$ mvn clean verify -Dsmart.testing=new,changed,affected

This will execute the build running only those tests which are falling into selected categories, based on your local changes.

For the CI environment, such as Jenkins, you can pass commit hashes using environment variables. For example:
$ mvn verify -Dsmart.testing=affected -Dscm.range.head=GIT_COMMIT -Dscm.range.tail=GIT_PREVIOUS_COMMIT
This will optimize tests based on the changes between the current commit and the one against which the previous build was run.

For more details head over to our documentation backed by awesome Asciidoctor!

We are very excited about Smart Testing but we need you to help us make it even more awesome. We spent tremendous amount of effort building it with the developer experience in mind, but without the community feedback we can only get that far. So give it a try today and tell us what you think! Maven is a first stop for us. Stay tuned! There’s more to come!

Thanks to the whole team for making it happen! You are real heroes!

Also big kudos to Infinitest team for the inspiration! Give it a spin if you are Eclipse or IDEA user.

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.1 view tag
Release date 2017-09-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 30

Initial release with Maven integration.

Component: Core
Component: Test Bed
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar, Hemani