Smart Testing 0.0.2 Released

The Arquillian team is proud to announce the 0.0.2 release of the Smart Testing component!

Highlights of this release

After a week since our very first release we have a brand new one! This time with several improvements and bug fixes, namely:

  • simplified installation process
  • hinting available strategies when misspelled in the configuration
  • ability to narrow search for affected tests
  • improved documentation

Simplified getting started

Even though installation process is very simple, we are huge fans of automation. So we created one liner which will take care of adding Smart Testing to your project.

Simply execute following snippet and you are ready to go.

$ curl -sSL https://git.io/v5jy6 | bash

Hinting available strategies

It might happen that you misspell a strategy when executing the build. For example:

$ mvn clean verify -Dsmart.testing=new,change,afected

This will lead to a failing build, but instead of leaving you with not just an exception thrown at your face and documentation to dig into, we are now hinting to matching strategies. With 0.0.2 you will see instead:

Unable to find strategy [afected]. Did you mean [affected]?
Unable to find strategy [change]. Did you mean [changed]?

Affected tests

Affected is one of the strategies in Smart Testing which let you find tests related to business code you have just changed. By default we transitively look up all related tests, but this can lead to a lot of tests being prioritized. Especially if you have deep hierarchies in your project.

To find only immediate related tests we have introduced a configuration option smart.testing.affected.transitivity which you can set to false.

Asciidoctor Extensions

We’ve created two Asciidoctor extensions to make documentation more resilient to changes and also to simplify the copy-paste process from documentation to your terminal.

First extension allows you to reference constants in the documentation from source code. For example to get the value represented by SMART_TESTING_MODE constant, add const macro in your document:

const:core/src/main/java/org/arquillian/smart/testing/Configuration.java[name="SMART_TESTING_MODE"]

Second extension brings copy-to-clipboard button in the rendered docs. To add it, simply use copyToClipboard macro:

[[singleTest]]
`mvn test -Dtest=SampleTest`  copyToClipboard:singleTest[]

What’s next

We are working hard on the next release which will bring simplified configuration and many more cool features! Stay tuned!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.2 view tag
Release date 2017-09-26
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 21

Component: Test Bed
Component: Core
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar

Smart Testing 0.0.1 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.1 release of the Smart Testing component!

First release of Smart Testing is here!

It all started as a Google Summer of Code project last year with great ground work of our student Dimcho Karpachev.

Flash forward to this summer, after 222 commits, 69 pull requests merged and insane amount of CI builds we are super excited to announce a new addition to Arquillian Universe we’ve been working so vigorously on.

This time it’s not a new extension to Arquillian Testing Platform, but a brand new tool which we hope will bring a breath of fresh air into your builds, let it be on your local machine or the CI server.

You probably know it best – you write some code and tests, run the build, wait few minutes to only see at the later stage that your changes are breaking it. Of course you can go and grab a coffee, but if you commit early and often that might mean caffeine overdose in a very short time. Running tests before pushing is a good practice, but if it takes too long it simply slows you down.

Moreover having long lasting builds on your CI server makes your CI/CD process not really that continuous and leads to Pull Requests piling up. Bringing new features to production with confidence should be seamless. And for this we need to have faster feedback loops.

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

That’s why we created this cure for you!

Highlights of this release

With this release we provide following strategies which you can use to optimize your build execution:

  • new – gives higher priority to newly added tests
  • changed – gives higher priority to changed tests (e.g. new test methods or refactorings of components under tests which imply changes in the tests)
  • affected – based on changes in your business logic, gives higher priority to the tests which are exercising them
  • failed – gives higher priority to the tests which failed in the previous (local) build

In addition you can either decide to run the whole test suite (ordering mode) or only those tests which fall into the selected categories (selecting, which is also a default mode).

new, changed and affected rely on SCM information. For this release we only support Git.

How to get started

If you are using Maven 3.3.x or newer (and you definitely should!) adding Smart Testing to your build is very easy. Create a file .mvn/extensions.xml in the root of project
with the following content:

extensions.xml
<?xml version="1.0" encoding="UTF-8"?>
<extensions>
  <extension>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>maven-lifecycle-extension</artifactId>
    <version>0.0.1</version>
  </extension>
</extensions>

and you are all set!

Executing the build with Smart Testing enabled is as simple as adding one system property with strategies of your choice:

$ mvn clean verify -Dsmart.testing=new,changed,affected

This will execute the build running only those tests which are falling into selected categories, based on your local changes.

For the CI environment, such as Jenkins, you can pass commit hashes using environment variables. For example:
$ mvn verify -Dsmart.testing=affected -Dscm.range.head=GIT_COMMIT -Dscm.range.tail=GIT_PREVIOUS_COMMIT
This will optimize tests based on the changes between the current commit and the one against which the previous build was run.

For more details head over to our documentation backed by awesome Asciidoctor!

We are very excited about Smart Testing but we need you to help us make it even more awesome. We spent tremendous amount of effort building it with the developer experience in mind, but without the community feedback we can only get that far. So give it a try today and tell us what you think! Maven is a first stop for us. Stay tuned! There’s more to come!

Thanks to the whole team for making it happen! You are real heroes!

Also big kudos to Infinitest team for the inspiration! Give it a spin if you are Eclipse or IDEA user.

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.1 view tag
Release date 2017-09-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 30

Initial release with Maven integration.

Component: Core
Component: Test Bed
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar, Hemanik