Arquillian Blog

Hackfest: DevConf 2013, Czech Republic, Brno

Posted by Aslak Knutsen on Feb 16, 2013
Feb 16
| Comments

In about a weeks time DevConf 2013 in Brno, Czech Republic will kick of for the 5. year in a row.
This year, as last, we will take part in the HackFest spanning from Saturday 23. to Sunday 24. of February. And for the eager hackers even Monday 25. February.

Introducing the candidates

Without any more fuzz, let’s jump straight into some of the ideas we have for this hackfest. Of course, you’re free to bring your own as well!

Gradle Container Controller

One of the hacks that was started during the Devoxx 2012 Hackergarten: was the Arquillian Gradle integration. The basic idea is to leverage Arquillian’s Container support to control Containers and Deployments within the Gradle build system, much like we do for Maven via the Arquillian Maven Plugin. This in particular is a very interesting integration since it opens up Arquillian to a new type of usage, as a pure Container Controller.

Why would Gradle use Arquillian for this? It’s a simple matter of focus. Gradle care about build systems and don’t have the resources to implement support for all the Containers. Arquillian on the other hand care very much of the Containers and already support much more than Gradle currently do in this area. Perfect combination of two tools with different capabilities.

You can read up on some of the discussions and see look at the early prototype.

The end goal would be to match as much as possible of the Gradle Design docs and have Arquillian be the internal runner.

Arquillian Forge Plugin – Extensions

JBoss Forge is best described by them selves.

A core framework and next-generation shell for tooling and automation at a command line level; with APIs for integration in IDEs, extending built in functionality with plugins, and scripting for automating repetitive tasks, Forge is a tool every open-source developer should be looking at.

A core framework for rapid-application development in a standards-based environment. Plugins / incremental project enhancement for Java EE, and more.

There exists already an Arquillian Plugin for Forge which handles setup of Arquillian Core and Container dependencies, Container configuration and simple TestClass generation.

But the missing bit is set up and support for the Arquillian Extensions. This hack should be able to answer, by using Forge, questions like:

  • How do I setup Drone, Persistence, Transactions?
  • How do I configure Drone?
  • Which configuration options does Persistence support?

Arquillian Warp

Arquillian Warp fills the void between client-side and server-side testing. Using Warp, you can initiate an HTTP request using a client-side testing tool such as WebDriver and, in the same request cycle, execute in-container server-side tests. This powerful combination lets you cover integration across client and server.

SeamTest

Warp leverages any HTTP client (such as httpclient or WebDriver), to produce testable requests and provides server-side hooks which allows you to execute arbitrary logic in any part of the request lifecycle. However the fact that developer needs to provide HTTP request in order to test is limiting – sometimes you need to test using any request type: such as conversations. The goal is designing an API and implementation for testing sequence of requests without providing an HTTP client logic. (This extension should provide a migration layer for SeamTest )

CDI events

Warp plays nicely with IoC principles – your test does not need to tightly couple with container, it will bring all the necessary dependencies to the test. In Java EE, Warp leverages CDI to inject user-defined beans into the test. Warp has the potential to use much more from the concepts brought by CDI: observe CDI events, produce beans or intercept calls to existing beans. The idea is to provide hooks which would make testing of client request interaction with CDI beans as easy as defining Warp’s lifecycle hooks.

Arquillian Google Web Toolkit

The Google Web Toolkit is a development tool for building complex browser-based applications written in pure Java. The new Arquillian GWT extension brings true integration testing to GWT. This means that GWT integration tests can break free and execute in the actual runtime container instead of being tied to GWT’s embedded Jetty server. Further it is now possible to combine GWT client-side and standard in-container tests in the same test class. This allows to test features without having to worry about the client-server bridge and should pave the way for future support of Arquillian Warp and Drone in GWT integration tests.

Remove the need for GwtArchive

Currently the Arquillian GWT extension uses a custom utility (GwtArchive) to enrich WebArchives for GWT deployments. This causes the API for Arquillian GWT tests to deviate from standard Arquillian tests. It should be possible to automatically enhance the deployment archive to include all dependencies and configurations that are relevant to the GWT deployment, possibly by leveraging an Arquillian packaging SPI’s. As a result of this, GWT integration tests will use the same API as all other Arquillian tests.

Leverage Arquillian Warp in GWT integration tests

Warp enables tests to assert on both client-side and server-side logic and is an extremely useful tool for writing compact integration tests. The focus will be on brainstorming to come up with a battle plan for how Warp could be integrated with Arquillian GWT.

Arquillian Distribution Extension

This is probably the hardest and largest of them all. But even more interesting, right?. As Arquillian work today it will allow you to interact with local and remote Containers. But the big assumption with Remote Containers is that the Container is already up and running and we only need to connect to it. As we’re growing out to support more and more advanced use cases we’re starting to see the need for the ability to control and distribute infrastructure. Infrastructure being servers, containers or clouds.

A very brief description of what this extension should do:

  • Install Container A on Server B (copy source and configure)
  • Install Container X on Server Y (copy source and configure)
  • Configure Arquillian to use Container A on Server B
  • Configure Arquillian to use Container X on Server Y

From here on Arquillian Test takes over and run tests as described by the Test Suite. Next steps would be:

  • Uninstall Container X on Server Y
  • Uninstall Container A on Server B

This is particularly useful for the upcoming Performance Extension to allow it to install a full environment, including clients and targets, for performance testing.

What is Arquillian?

Arquillian is open source software that empowers you to test JVM-based applications more effectively. Created to defend the software galaxy from bugs, Arquillian brings your test to the runtime so you can focus on testing your application’s behavior rather than managing the runtime. Using Arquillian, you can develop a comprehensive suite of tests from the convenience of your IDE and run them in any IDE, build tool or continuous integration environment.

When and where are we hacking?

We are going to use the free Hackfest at DevConf 2013 as an opportunity to put our heads together to make progress on these ideas. Here are the details:

Of course, anyone is welcome to hack on these ideas at anytime from anywhere. This is open source!

Who will be there hacking?

Happy hacking!

permalink | tagged as hack-idea 2013 nonrelease

Hack Idea: Fully Automate ATDD by Integrating Thucydides with Arquillian

Posted by Dan Allen on Nov 05, 2012
Nov 05
| Comments

Imagine being able to organize and structure your Arquillian tests as human-readable specifications by example that describe at a high-level how the application should work, then being able to present illustrated documentation generated from a test run that gives all parties an account of the features under development and tracks feature sign-off through automation of the acceptance criteria.

This goal can be achieved by Thucydides and Arquillian joining forces. Acceptance tests will not only be agile and illustrated, but fully automated. With a proof of concept in hand, we plan to advance this combination, including substituting Arquillian Drone to manage WebDriver, at the Devoxx 2012 Hackergarten.

Introducing the candidates

What is Thucydides?

Thucydides is a testing tool that encourages developers write more flexible and reusable end-to-end acceptance tests. In the style of Acceptance Test-Driven Development, Thucydides introduces a narrative for tests that describes the high-level requirements and automates the acceptance criteria using the tests. Developers then iterate using a test-driven approach to complete the requirements (i.e., stories), as depicted in the diagram below.

By analyzing the test metadata and test results, Thucydides generates illustrated documentation that describes how the application is used in the form of high-level requirements, acceptance criteria and screenshots, and records the progress of development.

What is Arquillian?

Arquillian is open source software that empowers you to test JVM-based applications more effectively. Created to defend the software galaxy from bugs, Arquillian brings your test to the runtime so you can focus on testing your application’s behavior rather than managing the runtime. Using Arquillian, you can develop a comprehensive suite of tests from the convenience of your IDE and run them in any IDE, build tool or continuous integration environment.

What is Arquillian Drone?

Arquillian Drone is an extension to Arquillian that manages the lifecycle of browsers for the purpose of controlling them from a test, thus simplifying automated functional / acceptance testing. Drone bootstraps the tooling necessary to send commands to the browser (e.g. Selenium Server, WebDriver, etc), spawns an instance of a browser, injects the browser API into the test class to allow tests to control that instance and, finally, properly disposes of the browser instance after the test is complete.

Integration benefits

Arquillian and Thucydides as yin and yang

While Thucydides handles automation of the browser—via the WebDriver library—it does not handle deploying the application being tested, nor does it manage the lifecycle of the container to which the application is deployed. In other words, it doesn’t automate everything. That’s where Arquillian steps in.

Thucydides brings the ATDD methodology and rich test reports to Arquillian, both new features to the Arquillian ecosystem. Arquillian complements Thucydides by managing the lifecycle of the container and deploying the application (as a whole or micro-deployment) to it. These services would otherwise need to be handled by the build when using Thucydides alone.

Beyond their combined core feature set, the integration brings all of the Arquillian extensions to Thucydides tests, opening the door to services such as loading seed data into the database, measuring performance and even managing browser automation. Speaking of managing browser automation…

More powerful browser automation with Drone

Bringing Arquillian Drone into the fold provides additional benefit by introducing a more powerful and flexible choice for controlling the browser in a Thucydides test, including simplified configuration, alternatives to WebDriver and extensions (e.g., Graphene). This secondary integration between Arquillian and Thucydides also unifies the developer experience with regard to how the browser is controlled across vanilla Arquillian Drone tests and Thucydides tests (embracing the concept that each tool should focus on doing one thing well).

Basic integration between Arquillian and Thucydides is a first step and getting Drone integrated is an optional (but valuable) bonus.

How this integration helps test writers

  • Arquillian tests become contract deliverables by leveraging Thucydides’ language for ATDD
  • Test segments of the webapp in isolation (using micro-deployments)
    • Accelerates acceptance testing because entire application doesn’t have to be built and deployed
  • Skip the build (e.g., Run As > XYZ Test in the IDE)
    • automatic container lifecycle management
    • deploy test archive defined in test class
    • inject application (initial) URL into test
  • Support for multiple target environments (e.g., local, dev, qa, staging)
  • Write Thucydides tests using other test runners, such as TestNG and Spock
  • Write Thucydides tests using any Arquillian-supported programming model (CDI, Spring, etc)
  • Rich reports summarizing test results generated by Thucydides, something Arquillian doesn’t currently provide
  • Use additional Arquillian extensions in Thucydides tests
  • Integrated configuration (arquillian.xml) for containers and browsers*
  • Use alternate browser automation libraries supported by Drone*
  • Use Graphene page fragments to reduce boilerplate code common in acceptance tests*

* requires Drone integration

Balancing responsibility

Thucydides’ strong suit is promoting ATDD through its programming model for writing acceptance tests as specifications by example and its reporting pipeline that summarizes test results as user stories. We believe Thucydides could move faster by focusing on tools and metrics needed to facilitate ATDD, allowing Arquillian to address the need of general purpose test extensions such as alternate test runners, data loading, browser automation, programming model integration, etc. In other words, Thucydides has more freedom to focus on its core competency, as stated on its website:

Organize and structure acceptance tests, associate tests with the user stories or features that they test and generate illustrated documentation describing how the application is used based on the stories described by the tests.

Hacking on the integration

We plan to hack on this idea during the Hackergarten at Devoxx 2012. Keep in mind there is a proof of concept we can reference as a starting point, described later in this section.

John Smart, the author of Thucydides, has offered to host the integration in the Thucydides code base, which is fantastic.

When and where are we hacking?

We are going to use the free Hackergarten day at Devoxx 2012 as an opportunity to put our heads together to make progress on this integration. Here are the details:

We’ll also be hacking throughout the week, either at the long white “hacking” tables on the second floor of the venue or after hours at a bar. Of course, anyone is welcome to hack on this idea at anytime from anywhere. This is open source!

Who will be there hacking?

Karel Piwko (Arquillian Drone Lead) and Vineet Reynolds (Arquillian Core and Thucydides integration) will be participating, time permitting, from afar. Feel free to reach out to them on IRC or the forums.

What we want to achieve

The following two tests are proposed milestones we want to achieve during the hack. They contrast what integration looks like without Drone to the integration with Drone. Thinking through these scenarios helps reveal integartion points that need to be exposed or refined.

The tests in this section apply to the Beer Advisor sample application.

Sample test without Drone integration

@RunWith(ThucydidesRunner.class) // or @RunWith(Arquillian.class)
@Story(BeerAdvisorFeatures.SearchingBeers.class)
public class SearchingBeersStory {

    // Thucydides members

    @Managed(uniqueSession = true)
    public WebDriver browser;

    @ManagedPages
    public Pages pages;

    @Steps
    public SearchingSteps searchingSteps;

    // Arquillian members

    @Deployment(testable = false)
    public static WebArchive createTestArchive() {
       return ShrinkWrap.create(WebArchive.class, "searching-beers-story.war")...;
    }

    @ArquillianResource
    URL deploymentUrl;

    // Tests

    @Before
    public void before_tests() {
        pages.setDefaultBaseUrl(deploymentUrl.toExternalForm());
    }

    @Test
    public void should_find_all_usa_beers() {
        searchingSteps.on_main_beer_advisor_page();
        searchingSteps.search_for("from usa");
        searchingSteps.should_contain_beers(new Beer("90 minute IPA"), new Beer("Mirror Pond"));
    }
}

The following steps outline the interplay between Arquillian and Thucydides while the test executes in this scenario.

  • Either Arquillian or Thucydides takes over test execution and activate both frameworks
  • Responding to the test suite start event, Arquillian and configures and starts (or connects to) the container
  • Arquillian deploys test archive(s) to container
  • Arquillian injects the application URL into the @ArquillianResource URL inject point of the test
  • Thucydides spawns a browser instance using WebDriver
  • Thucydides executes tests, sending browser commands to WebDriver
  • Thucydides takes screenshots using WebDriver as the browser is interacting with the web application
  • Thucydides destoys the browser instance using WebDriver
  • Arquillian undeploys the test archive
  • Arquillian stops the container
  • Thucydides generates reports from the high-level summarizes, screenshots and test results

Any Arquillian extensions that are loaded should also work as normal.

Sample test with Drone integration

@RunWith(ThucydidesRunner.class) // or @RunWith(Arquillian.class)
@Story(BeerAdvisorFeatures.SearchingBeers.class)
public class SearchingBeersStory {

    // Thucydides members

    @Steps
    public SearchingSteps searchingSteps;

    // Arquillian members

    @Deployment(testable = false)
    public static WebArchive createTestArchive() {
       return ShrinkWrap.create(WebArchive.class, "searching-beers-story.war")...;
    }

    @Drone // may get moved to the page objects
    public WebDriver browser;

    // Tests

    @Test
    public void should_find_all_usa_beers() {
        searchingSteps.on_main_beer_advisor_page();
        searchingSteps.search_for("from usa");
        searchingSteps.should_contain_beers(new Beer("90 minute IPA"), new Beer("Mirror Pond"));
    }
}

The following steps outline the interplay between Arquillian and Thucydides while the test executes in this scenario.

  • Either Arquillian or Thucydides takes over test execution and activate both frameworks
  • Arquillian configures and starts (or connects to) the container
  • Arquillian deploys test archive(s) to container
  • Arquillian injects the application URL into the @ArquillianResource URL injection point of the test
  • Arquillian Drone spawns a browser instance using the configured browser automation library (e.g., WebDriver)
  • Arquillian Drone sets the default base URL on the managed pages
  • Thucydides executes tests, sending browser commands to the browser automation library
  • Arquillian Drone takes screenshots as the browser is interacting with the web application
  • Arquillian Drone destoys the browser instance using the configured browser automation library
  • Arquillian undeploys the test archive
  • Arquillian stops the container
  • Thucydides generates reports from the high-level summarizes, screenshots and test results

Any Arquillian Drone extensions, such as Graphene, that are loaded should also work as normal.

Sample code to use for exploring the integration

Prior work

Vineet Reynolds has been experimenting with an integration between Thucydides and Arquillian Core. Vineet’s integration code uses a Thucydides StepListener implementation to weave the Arquillian lifecycle into the test. Tests are run using the Thucydides testrunner, while Arquillian manages the container lifecycle and performs the deployment of archives defined in the tests. The integration does not incorporate Arquillian Drone, opting to leave management of WebDriver up to Thucydides.

Aslak and Bartosz improved upon the integration and incorporated it into the Beer Advisor sample app for their JavaZone talk in September 2012. They introduced a JUnit Rule that activates the Arquillian services that run before and after each test method, such as test enrichment. A JUnit Rule is currently necessary due to insufficient hooks in Thucydides, but it gets the job done. Eventually we want to phase out the JUnit Rule because it exposes integration plumbing in the test.

Engineering challenges

Challenge #1: Unified test runner

Thucydides needs a startup API that isn’t coupled with JUnit so that Arquillian is able to perform the startup in an extension. At the moment there is a lot of setup code in the ThucydidesRunner to reproduce. The alternative is that Thucydides provides enough hooks in its StepEventBus for Arquillian to hook its own startup into the Thucydides lifecycle. In the integration prototype, a JUnit rule is used to activate Arquillian in a Thucydides test, but this is not the most elegant approach.

Ideally, the @RunWith(Arquillian.class) annotation should be enough for Arquillian to drive Thucydides if Arquillian detects Thucydides is also in use (e.g., the @Story annotation is present on the test class). This is the preferred approach since it simplifies the experience for the user across the test suite (a single @RunWith for all integration tests) and it may make the integration more compatible with extensions since Arquillian has a more sophisticated event bus.

Challenge #2: Drone integration

When Drone is integrated, the user has an option to use the browser automation library (directly or via the page object pattern) as with other Drone tests or use the page object support provided by Thucydides. Drone should be able to pass Thucydides the WebDriver instance so that Thucydides can operate its own page objects.

Ideally, the integration with Drone should cut down on as many redundant declarations in the class as possible to simplify test authoring. That means that the @Drone injection point is used to access the browser instance and and @ArquillianResource URL injection point is used to access the initial (base) url.

Karel Piwko mentioned that Drone needs a better SPI for improved integration with Arquillian Graphene, slated for 1.2.0.Final. He believes the Drone SPI will also help facilitate Thucydides integration. It also opens the door for Drone to be integrated into Thucydides on its own.

Happy hacking!

Resources

permalink | tagged as hack-idea thucydides drone atdd 2012 nonrelease