Smart Testing 0.0.4 Released

The Arquillian team is proud to announce the 0.0.4 release of the Smart Testing component!

Highlights of this release

In this release we shipped two key new features – support for JUnit 5 and declarative way of defining relations between tests and business code. And bunch of fixes.

Annotations for affected tests

Starting from this release you are able to use declarative way of defining relationship between test and business code. By default affected strategy uses imports declared in the tests to build the graph of collaborators. This approach is fine for unit tests (white box tests) but might not work for higher level tests (black box test).

Using annotation like below you can define classes or packages which particular test covers.

@ComponentUnderTest(packages = "org.acme.main.superbiz.*")
public class AcmeServiceRestTest { ... }

Check our detailed documentation on how to use it.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.4 view tag
Release date 2017-11-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

Release notes and resolved issues 11

Component: Test Bed
Component: Maven
Other
Component: Core
Component: Selection

Thanks to the following list of contributors: Hemanik, Dipak Pawar, Alex Soto, Matous Jobanek, Bartosz Majsak

Smart Testing 0.0.3 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.3 release of the Smart Testing component!

Highlights of this release

After one month of hard work we bring you the third release of Smart Testing tool!
Apart from several small improvements, component upgrades, and bug fixes, this release contains one killer feature and one convenient feature that will make your life easier.

Configuration file

In previous releases, there was only one way of configuring the tool – through system properties. As we started introducing more options, we quickly realized that this is definitely not the most pleasant way of setting up the tool. From now on you can also use YAML configuration file.
The name of the file should be either smart-testing.yml or smart-testing.yaml and should be located in the root of the directory the build is executed from. Here’s an example:

strategies: new, changed, affected
mode: ordering
applyTo: surefire
debug: true
report:
    enable: true
scm:
    range:
      head: HEAD
      tail: HEAD~2

We are keeping the names as it is done for the system properties, just without the prefix smart.testing. This means that when you want to activate a set of strategies on a command line, then you should use the system property smart.testing.strategies. As you can see in the example, in the configuration file it is just strategies. Analogically for other parameters.

The system properties are not going away. When both system properties and the configuration file are used, the values defined by system properties have the precedence. This can be helpful when your Smart Testing setup is shared in your scm but you want to have special settings for different stages of CI pipeline.

Autocorrection of mistyped strategy names

It might happen that you misspell a strategy when executing the build. For example:

$ mvn clean verify -Dsmart.testing=new,change,afected

It can take you a second or two to notice why this will lead to a failing build with some hints in the error message. To save time (and headaches) you can activate the autocorrection feature by setting smart.testing.autocorrect=true on a command line or just autocorrect: true in the config file. If it is set, the mistyped name is automatically corrected and the affected strategy is used. In this case afected should be affected.

What’s next

We are working hard on the next release which will bring JUnit 5 support, a possibility of configuring custom strategies and many more cool features! Stay tuned!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.3 view tag
Release date 2017-10-23
Released by Matous Jobanek
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 16

Component: Core
Component: Maven
Component: Test Bed
Component: Selection

Thanks to the following list of contributors: Matous Jobanek, Dipak Pawar, Hemanik, Bartosz Majsak, Alex Soto

Smart Testing 0.0.2 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.2 release of the Smart Testing component!

Highlights of this release

After a week since our very first release we have a brand new one! This time with several improvements and bug fixes, namely:

  • simplified installation process
  • hinting available strategies when misspelled in the configuration
  • ability to narrow search for affected tests
  • improved documentation

Simplified getting started

Even though installation process is very simple, we are huge fans of automation. So we created one liner which will take care of adding Smart Testing to your project.

Simply execute following snippet and you are ready to go.

$ curl -sSL https://git.io/v5jy6 | bash

Hinting available strategies

It might happen that you misspell a strategy when executing the build. For example:

$ mvn clean verify -Dsmart.testing=new,change,afected

This will lead to a failing build, but instead of leaving you with not just an exception thrown at your face and documentation to dig into, we are now hinting to matching strategies. With 0.0.2 you will see instead:

Unable to find strategy [afected]. Did you mean [affected]?
Unable to find strategy [change]. Did you mean [changed]?

Affected tests

Affected is one of the strategies in Smart Testing which let you find tests related to business code you have just changed. By default we transitively look up all related tests, but this can lead to a lot of tests being prioritized. Especially if you have deep hierarchies in your project.

To find only immediate related tests we have introduced a configuration option smart.testing.affected.transitivity which you can set to false.

Asciidoctor Extensions

We’ve created two Asciidoctor extensions to make documentation more resilient to changes and also to simplify the copy-paste process from documentation to your terminal.

First extension allows you to reference constants in the documentation from source code. For example to get the value represented by SMART_TESTING_MODE constant, add const macro in your document:

const:core/src/main/java/org/arquillian/smart/testing/Configuration.java[name="SMART_TESTING_MODE"]

Second extension brings copy-to-clipboard button in the rendered docs. To add it, simply use copyToClipboard macro:

[[singleTest]]
`mvn test -Dtest=SampleTest`  copyToClipboard:singleTest[]

What’s next

We are working hard on the next release which will bring simplified configuration and many more cool features! Stay tuned!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.2 view tag
Release date 2017-09-26
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 21

Component: Test Bed
Component: Core
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar

Smart Testing 0.0.1 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.1 release of the Smart Testing component!

First release of Smart Testing is here!

It all started as a Google Summer of Code project last year with great ground work of our student Dimcho Karpachev.

Flash forward to this summer, after 222 commits, 69 pull requests merged and insane amount of CI builds we are super excited to announce a new addition to Arquillian Universe we’ve been working so vigorously on.

This time it’s not a new extension to Arquillian Testing Platform, but a brand new tool which we hope will bring a breath of fresh air into your builds, let it be on your local machine or the CI server.

You probably know it best – you write some code and tests, run the build, wait few minutes to only see at the later stage that your changes are breaking it. Of course you can go and grab a coffee, but if you commit early and often that might mean caffeine overdose in a very short time. Running tests before pushing is a good practice, but if it takes too long it simply slows you down.

Moreover having long lasting builds on your CI server makes your CI/CD process not really that continuous and leads to Pull Requests piling up. Bringing new features to production with confidence should be seamless. And for this we need to have faster feedback loops.

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

That’s why we created this cure for you!

Highlights of this release

With this release we provide following strategies which you can use to optimize your build execution:

  • new – gives higher priority to newly added tests
  • changed – gives higher priority to changed tests (e.g. new test methods or refactorings of components under tests which imply changes in the tests)
  • affected – based on changes in your business logic, gives higher priority to the tests which are exercising them
  • failed – gives higher priority to the tests which failed in the previous (local) build

In addition you can either decide to run the whole test suite (ordering mode) or only those tests which fall into the selected categories (selecting, which is also a default mode).

new, changed and affected rely on SCM information. For this release we only support Git.

How to get started

If you are using Maven 3.3.x or newer (and you definitely should!) adding Smart Testing to your build is very easy. Create a file .mvn/extensions.xml in the root of project
with the following content:

extensions.xml
<?xml version="1.0" encoding="UTF-8"?>
<extensions>
  <extension>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>maven-lifecycle-extension</artifactId>
    <version>0.0.1</version>
  </extension>
</extensions>

and you are all set!

Executing the build with Smart Testing enabled is as simple as adding one system property with strategies of your choice:

$ mvn clean verify -Dsmart.testing=new,changed,affected

This will execute the build running only those tests which are falling into selected categories, based on your local changes.

For the CI environment, such as Jenkins, you can pass commit hashes using environment variables. For example:
$ mvn verify -Dsmart.testing=affected -Dscm.range.head=GIT_COMMIT -Dscm.range.tail=GIT_PREVIOUS_COMMIT
This will optimize tests based on the changes between the current commit and the one against which the previous build was run.

For more details head over to our documentation backed by awesome Asciidoctor!

We are very excited about Smart Testing but we need you to help us make it even more awesome. We spent tremendous amount of effort building it with the developer experience in mind, but without the community feedback we can only get that far. So give it a try today and tell us what you think! Maven is a first stop for us. Stay tuned! There’s more to come!

Thanks to the whole team for making it happen! You are real heroes!

Also big kudos to Infinitest team for the inspiration! Give it a spin if you are Eclipse or IDEA user.

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.1 view tag
Release date 2017-09-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom

Release notes and resolved issues 30

Initial release with Maven integration.

Component: Core
Component: Test Bed
Component: Maven
Component: Documentation
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Alex Soto, Matous Jobanek, Dipak Pawar, Hemanik

Testing with Aliens; How to test a JPA type converter with Arquillian

JPA type converters provide an easy way to define how an entity attribute gets persisted to the database. You can use them to implement lots of different features, e.g. to encrypt your data as I showed in a previous post: How to use a JPA Type Converter to encrypt your data
But writing the type converter is not enough. We also need to make sure, that it is working correctly.

In general, there are two ways to test a type converter. We could write a unit test to check, if the conversion works correctly. But a unit test performs a test of the isolated class without putting it into the real execution environment. That means that we will still not know, if the converter works in one of our applications. If everything is set up correctly, the persistence provider will call the converter before writing to and after reading from the database. So we also need to check if the type converter gets called by the persistence provider and if everything works fine under that condition. We need to test the converter inside of the container we want to use for our application.

We will have a look at how this can be done with Arquillian and its persistence extension.

Something about Arqillian

If you are already familiar with Arquillian, you can skip this part. For all of you who have never worked with Arquillian so far, I just want to give some basic information. You can find a more detailed description at the Arquillian Getting Started guide.

Arquillian is a test framework for in container testing. The idea is to do not mock the container you want to use but to test your code inside of it. This provides the advantage, that you can test if your code will also work in your execution environment and not only in your mocked up test scenario. Arquillian provides lots of functionality to manage the container, inject required resources like EJBs or an EntityManager and make your live much easier.

The Arquillian tests are executed by junit. This is great, because you can use them everywhere, where you can execute junit tests. And that means in your IDE, as part of your build process, on your CI server, simply everywhere.

Object under test

The following code snippet shows the object under test for this example. It is a type converter that encrypts and decrypts a String attribute. The converter gets called by the persistence provider before writing to and after reading from the database. If you want to read more about how this type converter works, check my posting about it.

@Converter
public class CryptoConverter implements AttributeConverter<String, String> {

    private static final String ALGORITHM = "AES/ECB/PKCS5Padding";
    private static final byte[] KEY = "MySuperSecretKey".getBytes();

    @Override
    public String convertToDatabaseColumn(String ccNumber) {
      // do some encryption
      Key key = new SecretKeySpec(KEY, "AES");
      try {
         Cipher c = Cipher.getInstance(ALGORITHM);
         c.init(Cipher.ENCRYPT_MODE, key);
         return Base64.encodeBytes(c.doFinal(ccNumber.getBytes()));
      } catch (Exception e) {
         throw new RuntimeException(e);
      }
    }

    @Override
    public String convertToEntityAttribute(String dbData) {
      // do some decryption
      Key key = new SecretKeySpec(KEY, "AES");
      try {
        Cipher c = Cipher.getInstance(ALGORITHM);
        c.init(Cipher.DECRYPT_MODE, key);
        return new String(c.doFinal(Base64.decode(dbData)));
      } catch (Exception e) {
        throw new RuntimeException(e);
      }
    }
}

Setting it up

Before we can start to write our tests, we need to define a few dependencies. I will only show how to configure the dependencies we need for this example. If you have not already set up arquillian tests for your project, you will have to do a little bit more. Please check the Getting Started guide to learn how to setup arquillian for your project. Don’t be afraid, there is not too much to do.

As you can see in the following snippet, we will use JUnit 4.11, Arquillian 1.1.3.Final, the Arquillian Persistence Extension 1.0.0.Alpha7 and the WildFly Application Server 8.1.0.Final.

<?xml version="1.0" encoding="UTF-8"?>
<project
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
    xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    
    ...

    <properties>
        <version.junit>4.11</version.junit>
        <version.arquillian>1.1.3.Final</version.arquillian>
        <version.arquillian_persistence>1.0.0.Alpha7</version.arquillian_persistence>
        <version.wildfly>8.1.0.Final</version.wildfly>
    </properties>

    <dependencyManagement>
        <dependencies>
            ...
            <dependency>
                <groupId>org.jboss.arquillian</groupId>
                <artifactId>arquillian-bom</artifactId>
                <version>${version.arquillian}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        ...
        <dependency>
            <groupId>javax.enterprise</groupId>
            <artifactId>cdi-api</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${version.junit}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.junit</groupId>
            <artifactId>arquillian-junit-container</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-persistence-dbunit</artifactId>
            <version>${version.arquillian_persistence}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

Writing the tests

There are two things we need to do to setup our test environment. At first, we need to tell junit that this test shall be executed as a junit test. This is done by @RunWith(Arquillian.class).

Additionally, we need to create the test deployment, that will be deployed to the container. Therefore we need to implement at least one method and annotate it with @Deployment. As you can see in the following code snippet, we use ShrinkWrap to create a jar archive deployment. The archive contains the CreditCard entity, the CryptoConverter type converter and the test class. There is no need to include any EJBs or other classes which implement business logic. We can inject the EntityManager into our test case and use it directly to persist and read entities. We will have a more detailed look at it later on.

Additionally, we need to add some manifest resources to create a persistence unit, register the type converter and add an empty beans.xml to activate CDI. Please check the getting started guide to get more information on ShrinkWrap and creating deployments.

@RunWith(Arquillian.class)
public class TestCryptoConverter {

    @Deployment
    public static JavaArchive createDeployment() {
        return ShrinkWrap
                .create(JavaArchive.class)
                .addClasses(CreditCard.class, CryptoConverter.class,
                        TestCryptoConverter.class)
                .addAsManifestResource("META-INF/persistence.xml",
                        "persistence.xml")
                .addAsManifestResource("META-INF/orm.xml", "orm.xml")
                .addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
    }

After this is done, we can start to write the test cases. At first, we will persist a CreditCard entity and check, if the credit card number gets encrypted by the CryptoConverter. Therefore we inject the EntityManager, create a CreditCard entity and pass it to the persist method of the EntityManager. The validation of the persisted data, is done by the Arquillian persistence extension. We just need to define the data we expect to be written to the database. The expected data is defined in the cc.yml file, which is referenced in the @ShouldMatchDataSet annotation. Because the id attribute is generated by the database, we want to exclude it from the validation. This can be done by referencing it in the excludeColumns attribute of the annotation.

    @PersistenceContext
    private EntityManager em;

    @Test
    @ShouldMatchDataSet(value = "data/cc.yml", excludeColumns = "id")
    public void testEncryption() {
        CreditCard cc = new CreditCard();
        cc.setName("My Name");
        cc.setCcNumber("123456789");

        this.em.persist(cc);
    }

The cc.yml contains the following information.

CreditCard:
  - id: 1
    name: My Name
    ccNumber: egFfkhd8cRh82tvsh3VVUg==

In the second test, we will check if we can search the database for a CreditCard entity with a given credit card number. Therefore we use the @UsingDataSet annotation to seed the database with data defined in the cc.yml file. Now we can use the injected EntityManager to call a named query to search for CreditCard entity with the given number.

    @Test
    @UsingDataSet("data/cc.yml")
    public void testRead() {
        CreditCard cc = this.em
                .createNamedQuery(CreditCard.BY_NUMBER, CreditCard.class)
                .setParameter("number", "123456789").getSingleResult();
        Assert.assertEquals("My Name", cc.getName());
    }

Conclusion

We used Arquillian and the Arquillian Persistence extension to test a JPA type converter. Therefore we injected the EntityManager and used the annotations @ShouldMatchData and @UsingDataSet to validate and seed the database with a yml file.
If you want to try it yourself, you can find the sources on github.
You can run it by calling: git clone https://github.com/thjanssen/JPA2.1.git && cd JPA2.1/CryptoConverter && mvn test

What are your experiences with testing your Java EE application with Arquillian? Please write a comment about it.

Want to learn more about Arquillian, see the Arquillian Guides: http://arquillian.org/guides/

About the author

Thorben Janssen is a senior developer with more than 10 years of experience in Java EE development and architecture. During these years he acted as developer, architect, project and technical lead to create high available, clustered mobile billing solutions and laboratory information management systems.
Visit his blog: http://www.thoughts-on-java.org or follow him on twitter and google+ to read more about Java EE7 and Arquillian.