Smart Testing 0.0.7 Released

The Arquillian team is proud to announce the 0.0.7 release of the Smart Testing component!

Highlights of this release

In this release we ship more support to JUnit 5. Concretely we have added support for @org.junit.jupiter.api.Tag and @org.junit.jupiter.api.Tags annotations.

JUnit 5 tag support

In this realease we have added support for @org.junit.jupiter.api.Tag and @org.junit.jupiter.api.Tags annotations in categorized strategy.
Tags is the way to categorize tests in JUnit 5 and a categorized test looks like:

@Tag("first")
@Tag("second")
public class FirstAndSecondTaggedClass {
}

Then you need to configure the strategy by setting categories property with tag value.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.7 view tag
Release date 2018-01-17
Released by Matous Jobanek
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » api jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » strategy-categorized jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom
  • org.arquillian.smart.testing » known-surefire-providers jar javadoc pom

Release notes and resolved issues 4

Component: Selection
Component: Test Bed
Component: Maven
Component: Core

Thanks to the following list of contributors: Dipak Pawar, Alex Soto, Matous Jobanek, Hemanik, Bartosz Majsak

Smart Testing 0.0.6 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.6 release of the Smart Testing component!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.6 view tag
Release date 2017-12-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

  • org.arquillian.smart.testing » core jar javadoc pom
  • org.arquillian.smart.testing » api jar javadoc pom
  • org.arquillian.smart.testing » surefire-provider jar javadoc pom
  • org.arquillian.smart.testing » junit-test-result-parser jar javadoc pom
  • org.arquillian.smart.testing » strategy-affected jar javadoc pom
  • org.arquillian.smart.testing » strategy-changed jar javadoc pom
  • org.arquillian.smart.testing » strategy-failed jar javadoc pom
  • org.arquillian.smart.testing » strategy-categorized jar javadoc pom
  • org.arquillian.smart.testing » maven-lifecycle-extension jar javadoc pom
  • org.arquillian.smart.testing » git-rules jar javadoc pom
  • org.arquillian.smart.testing » smart-testing-test-bed jar javadoc pom
  • org.arquillian.smart.testing » known-surefire-providers jar javadoc pom

Release notes and resolved issues 7

Component: Core
Priority: High
Component: Maven
Component: Selection

Thanks to the following list of contributors: Bartosz Majsak, Matous Jobanek, Dipak Pawar, Alex Soto

End-To-End Integration Testing For Kubernetes and OpenShift

Does the title leave you wondering, why you need integration tests for your deployments ?

If so, let me give you a few examples. Consider you decide to upgrade Kubernetes, deploy a new Zapier
integration or add some authentication capability. In order to ensure your platform works every single
time, all of the above scenarios need to be tested with robust integration tests.

With the advent of microservices and cloud native applications, adding integration or
end-to-end tests to your Kubernetes or OpenShift projects might be challenging and cumbersome especially
when you want to set up the test infrastructure to be as close as possible to production.

Arquillian Community understands this pain, and continuously strive to provide users a seamless
experience for writing integration tests with ease by bringing their tests to the real environment and
deploy with confidence like never before.

With this blog post, we aim at showcasing integration tests for your Kubernetes or OpenShift clusters using just Arquillian Cube. Moving a step further, we also demonstrate building and deploying from scratch, applications with no deployment configuration, leveraging the power of Fabric8 Maven Plugin along with the Arquillian Cube Extension.

The key point here is that if OpenShift or Kubernetes is used as deployable platform in production, your tests are executed in a the same environment as it will be in production, so your tests are even more real than before.

Further, the test cases are meant to consume and test the provided services and assert that the environment is in the expected state.

Deployment Testing Recipes

If, you wish to read no more and see it all in action, head straight to our specially curated
examples to help you get started with ease.

Example 1

Deploying a sample PHP Guestbook application with Redis on Kubernetes from the resource descriptor
manifest file and testing it using Arquillian Cube Extension for Kubernetes and Kubernetes custom assertions.

Source: arquillian-testing-microservices/kubernetes-deployment-testing

Example 2

Deploying a Wordpress and My SQL application to OpenShift from a Template file and testing it using Arquillian
Cube Extension for OpenShift and Fabric8 OpenShift Client.

Source: arquillian-testing-microservices/openshift-deployment-testing

Example 3

Building and deploying a sample SpringBoot GuestBook application with zero deployment configuration using
Fabric8 Maven Plugin and Arquillian Cube Extension .

Fabric8 Maven Plugin aids in building Docker images and creating Kubernetes and OpenShift resource
descriptors for the application that allows for a quick ramp-up with some opinionated defaults and Arquillian
Cube Extension deploys the application from the generated resource descriptors and then executes deployment tests.

Source: arquillian-testing-microservices/zero-config-deployment-test

Step By Step Guide To Writing Deployment Tests

For a more in-depth step by step coverage of deployment testing, continue reading below.

Step 1: Setting Up Kubernetes/OpenShift Cluster

One of the pre-requisites for Arquillian Kube Extension, is to have Kubernetes or OpenShift cluster running on
your host machine.

An easy way to setup Kubernetes or OpenShift cluster locally is using Minikube and Minishift respectively.

Step 2: Adding Arquillian Kube Dependencies to your project

Arquillian Kube Extension provides a black box approach to testing your deployment that neither
mutates the containers (by deploying, reconfiguring etc) nor the Kubernetes/OpenShift resources.

It is used for immutable infrastructure and integration testing, wherein the test cases are meant to,
consume and test the provided services and assert that the environment is in the expected state,
providing you with the confidence that your application will work correctly when deployed on
Kubernetes/OpenShift cluster.

Before we can start writing our tests, we need to define a few dependencies as shown below:

Arquillian Cube BOM – Unified Dependencies

<properties>
    <version.arquillian_cube>${latest_released_version}</version.arquillian_cube>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.arquillian.cube</groupId>
            <artifactId>arquillian-cube-bom</artifactId>
            <version>${version.arquillian_cube}</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>

Arquillian Cube Requirement

Arquillian optionally provides an `ArquillianConditionalRunner` to control situations under which your tests should be executed by letting you specify the requirements using annotations.
For example, you can use annotations like `@RequiresOpenshift` to skip test when the environment is not prepared to run the tests.

To configure these requirements for your tests, enable the dependency to the following module for your project.

<dependency>
    <groupId>org.arquillian.cube</groupId>
    <artifactId>arquillian-cube-requirement</artifactId>
    <scope>test</scope>
</dependency>

Arquillian Cube Kubernetes (For Kubernetes Deployment)

<dependency>
    <groupId>org.arquillian.cube</groupId>
    <artifactId>arquillian-cube-kubernetes</artifactId>
    <scope>test</scope>
</dependency>

Arquillian Cube OpenShift (For OpenShift Deployment)

<dependency>
    <groupId>org.arquillian.cube</groupId>
    <artifactId>arquillian-cube-openshift</artifactId>
    <scope>test</scope>
</dependency>

Arquillian JUnit

<dependency>
    <groupId>org.jboss.arquillian.junit</groupId>
    <artifactId>arquillian-junit-standalone</artifactId>
    <version>${latest_released_version}</version>
    <scope>test</scope>
</dependency>

Fabric8 OpenShift Client

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>openshift-client</artifactId>
    <version>${latest_released_version}</version>
</dependency>

For Fabric8 OpenShift Client, include the above dependency in the pom.xml

Step 3: Writing Deployment Tests

Arquillian Cube extension provides out of the box functionality to create and manage a temporary namespace
per test suite for your tests and then applies all the required kubernetes/openshift resources as defined
in the resource descriptors generated by fabric8 maven plugin for your environment.

Kubernetes/OpenShift resources can then be made accessible within the Test Cases by injecting them using
Arquillian’s @ArquillianResources annotation (see example test below).

@RunWith(Arquillian.class)    (1)
public class ExampleTest {

    @Named("dummy")           (2)
    @PortForward
    @ArquillianResource
    Service dummyService;

    @ArquillianResource       (3)
    OpenShiftClient client;

    @RouteURL("application")  (4)
    @AwaitRoute
    private URL route;

    @Test
    public void service_instance_should_not_be_null() throws Exception {
        assertThat(service).isNotNull();
    }

    @Test
    public void test_at_least_one_pod() throws Exception {
       assertThat(client).pods().runningStatus().filterNamespace(session.getNamespace()).hasSize(1); (5)
    }

    @Test
    public void verify_route_is_configured_and_service_is_accessible() throws IOException {
        assertThat(route).isNotNull();
        OkHttpClient okHttpClient = new OkHttpClient();
        Request request = new Request.Builder().get().url(route).build();
        Response response = okHttpClient.newCall(request).execute();

        assertThat(response).isNotNull();
        assertThat(response.code()).isEqualTo(200);
    }
}

Explained below are the steps for the above snippet of a sample deployment test.

  1. Configuring Arquillian Test Runner
    To setup our test environment, we need to tell junit that this test shall be executed as a arquillian junit
    test. This is done by @RunWith(Arquillian.class).
  2. Injecting Deployment Resources within Test Cases
    Kubernetes/OpenShift resources can then be made accessible within the Test Cases by injecting them
    using Arquillian’s @ArquillianResources annotation.
    The resource providers available, can be used to inject to your test cases the following resources:
    • A kubernetes client as an instance of KubernetesClient.
    • Session object that contains information (e.g. the namespace) or the uuid of the test session.
    • Services (by id or as a list of all services created during the session, optionally filtered by label)
    • Deployments (by id or as a list of all deployments created during the session, optionally filtered by label)
    • Pods (by id or as a list of all pods created during the session, optionally filtered by label)
    • Replication Controllers (by id or as a list of all replication controllers created during the session,
      optionally filtered by label)
    • Replica Sets (by id or as a list of all replica sets created during the session, optionally filtered by label)
  3. The OpenShift extension also provides:
    • An openshift client as an instance of OpenShiftClient.
    • Deployment Configs (by id or as a list of all deployment configs created during the session)
    • Resources can be injected into test cases by id or as a list of all deployments created during the
      session, optionally filtered by label.
  4. Injecting Container Resources to access the deployed service
    Test Enrichers like @RouteURL further aid in injection of container resources like route to the deployed service.
    For farbric8 maven plugin to identify the route, @RouteURL should be set to artifactId of the project by
    default, or explicity configured otherwise.
  5. Adding Test Cases that assert environment is in the expected state.

Step 4: Using Assertion Libraries

Optionally, using Fabric8 Kubernetes Assertions , a nice library based on assert4j, aids in performing
meaningful and expressive assertions on top of the Kubernetes/OpenShift model.

To enable Fabric8 Kubernetes Assertions in your test, include the following dependency in the pom.xml

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>kubernetes-assertions</artifactId>
    <version>${latest_released_version}</version>
    <scope>test</scope>
</dependency>

Once everything is ready, Arquillian Kube runs your tests, enriched with resources required to access service
and finally cleaning up everything after the testing is over.

For more details and available configuration options check arquillian kube documentation.

Building Docker Images and creating Resource Descriptors.

Optionally, if you just have a Java application that you wish you to bring to Kubernetes or OpenShift,
use Fabric8 Maven Plugin to create docker images and resource descriptors for you.

Fabric8 Maven Plugin makes Kubernetes/OpenShift look and feel like an application server to a Java
developer by letting you build and deploy your application from maven just like you would with other
maven plugins.

To enable fabric8 on your existing maven project just type fabric8:setup command which adds the
fabric8-maven-plugin to your pom.xml.

Alternatively, you can manually add the following plugin definition to your pom.xml file:

<plugin>
    <groupId>io.fabric8</groupId>
    <artifactId>fabric8-maven-plugin</artifactId>
    <version>3.5.32</version>

    <!-- Connect fabric8:resource and fabric8:build to lifecycle phases -->
    <executions>
        <execution>
            <id>fmp</id>
            <phase>package</phase>
               <goals>
                    <goal>resource</goal>
                    <goal>build</goal>
                </goals>
        </execution>
    </executions>
</plugin>

For more and complete configuration details check out fabric8 maven plugin documentation.

Summary

With some basic features, simple steps and specially crafted examples as highlighted above, you can easily get started with Arquillian Cube extension to test your applications in real production environments, be it Kubernetes or OpenShift clusters, without worrying about setting up the infrastructure and only focusing on the core test logic.

Smart Testing 0.0.5 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.5 release of the Smart Testing component!

Highlights of this release

In this release we ship several API improvements making integration with 3rd party tools much easier.

Smart Testing API

The Smart Testing tool consists of three parts:

  • Core
  • Maven extension
  • Surefire provider integration

Both the Surefire provider integration and the Maven extension are implemented only for the usage in Maven builds. They take care of the integration and invoke Smart Testing API provided by Core.

The Core contains the main logic that is responsible for selecting/ordering and applying corresponding strategies. This logic is exposed by Smart Testing API. If you want to use the logic in your environment or as an integration with some third-party library, you need to have the dependency of Smart Testing Core on your classpath:

<dependency>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>core</artifactId>
    <version>${smart.testing.version}</version>
</dependency>

plus all dependencies of strategies you want to use for the prioritization:

<dependency>
    <groupId>org.arquillian.smart.testing</groupId>
    <artifactId>strategy-${strategy.name}</artifactId>
    <version>${smart.testing.version}</version>
</dependency>

Having these dependencies specified, you can start using the API. The starting point is SmartTesting class that provides you a fluent API:

SmartTesting
    .with(className -> isTest(className), configuration)
    .in("path/to/my/project")
    .applyOnClasses(suite);

The first method with takes as the first parameter a function that says which class is a test and which not. The second parameter is a configuration that should be used. The configuration file can be loaded using:

ConfigurationLoader.load(projectDir)

The second method in sets path to the project where the tests will be executed in.
Last method is either applyOnClasses or applyOnNames. It invokes the prioritization process and returns a prioritized set of TestSelection classes. This class contains class name and list of strategies that are applied to it. If you want to get only the names or classes then use the method SmartTesting.getNames(selection) or SmartTesting.getClasses(selection) respectively.

With it, you should be able to use the Smart Testing logic in your test suites without being limited to the Maven builds.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.5 view tag
Release date 2017-11-28
Released by Matous Jobanek
Compiled against

Published artifacts org.arquillian.smart.testing

Release notes and resolved issues 4

Component: Core

Thanks to the following list of contributors: Matous Jobanek, Bartosz Majsak

Smart Testing 0.0.4 Released

Since we wrote this post we didn't laze around. Check our latest announcement.

The Arquillian team is proud to announce the 0.0.4 release of the Smart Testing component!

Highlights of this release

In this release we shipped two key new features – support for JUnit 5 and declarative way of defining relations between tests and business code. And bunch of fixes.

Annotations for affected tests

Starting from this release you are able to use declarative way of defining relationship between test and business code. By default affected strategy uses imports declared in the tests to build the graph of collaborators. This approach is fine for unit tests (white box tests) but might not work for higher level tests (black box test).

Using annotation like below you can define classes or packages which particular test covers.

@ComponentUnderTest(packages = "org.acme.main.superbiz.*")
public class AcmeServiceRestTest { ... }

Check our detailed documentation on how to use it.

What’s next

We continue improving the tool so watch out for more!

What is Smart Testing?

Smart Testing is a tool that speeds up the test running phase by reordering test execution plan to increase a probability of fail-fast execution and thus give you faster feedback about your project’s health.

Release details

Component Smart Testing
Version 0.0.4 view tag
Release date 2017-11-14
Released by Bartosz Majsak
Compiled against

Published artifacts org.arquillian.smart.testing

Release notes and resolved issues 13

Component: Test Bed
Component: Maven
Other
Component: Core
Component: Selection

Thanks to the following list of contributors: Hemanik, Dipak Pawar, Alex Soto, Matous Jobanek, Bartosz Majsak