The Danger of Embedded Containers

In the Arquillian project, we’re all about real tests. There’s a good reason for this philosophy. If you use mocks or a substitute container when testing code that uses a programming model (i.e., CDI), all you can be sure of is that you’ve faked out enough functionality to get the test to work. You can’t be certain that your code works—really, truly works.

Obviously, we provide adapters for embedded containers (i.e., substitute containers). The reason is, they often do a decent enough job of simultating the real environment that we are willing to make the trade-off for its variations when we are working in development. I would expect, though, that you are also running the same tests in CI using a real container…just to get that peace of mind.

How do you know when not to use an embedded container?

If you write a test that you are positive should work, but instead throws some wacky exception, immediately stop and try the same test in a real container. For example, if you are using Embedded GlassFish, I would advise running the test in managed or remote GlassFish. The latter two adapters both use an authentic GlassFish instance. If the test passes, then you have answered the question of this section.

When the embedded container begins to act inconsistently, drop it.

As with all things, there are exceptions to the rule. If the code really should work on the embedded container, then it may be worth exploring why it isn’t. That may lead you to the discovery of a bug in the library you’re using. If debugging the library is your main concern, chase it down. If you are just trying to get a test written for your application, stop using the embedded container (or save debugging it for a rainy day).

Examples

See this thread in the Seam 3 forums for an example where the Embedded GlassFish container was acting more of a hindrence than a help. The thread includes the steps I took to get a green bar using a different GlassFish container adapter.

Markus Eisele identifies another hazard. He points out in a recent blog post that testing JPA using Embedded GlassFish is problematic because JPA weaving (in EclipseLink) does not occur:

[Weaving] only works when the entity classes to be weaved only exist in the application classloader. The combination of Embedded GlassFish, Arquillian and the Maven Surefire Plugin mix this up a bit and the end of the story is, that exactly none of your entities are enhanced at all.

Again, testing on Embedded GlassFish is not serving us well in this scenario. Graduate.

Closing thoughts

Code that uses Solder works fine on GlassFish 3.1.2, but it does not play very nicely with Embedded GlassFish—it’s just a hostile environment for CDI because of the visibility issues that come with starting an embedded container inside an existing Java process. The same goes for JPA. You end up in a minefield of class visibility and access problems, and far from making any progress writing tests (your ultimate goal).

If you want the container to start automatically like with embedded, then switch to a managed container. It works almost exactly the same as the remote container, except that Arquillian will start the standalone process at the beginning of the test execution and stop it at the end. In other words:

managed >>>> embedded

It’s clean. It’s true.

For speed of development:

remote > managed

With remote, you don’t have to wait for the container to start…and you can start it in debug mode very easily and keep it there. That’s why I say that the best container adapters for development are the remote adapters and the best adapters for CI are the managed adapters. I personally don’t like the embedded container adapters much (with the exception of embedded CDI since that’s mostly a pure standalone CDI environment).

Keep that advice in mind.

Death to all bugs! Arquillian testing platform reaches first stable release

Red Hat, Inc. and the JBoss Community today announced the 1.0.0.Final release of Arquillian, its award-winning testing platform built to run on the Java Virtual Machine (JVM). Arquillian substantially reduces the effort required to write and execute Java middleware integration and functional tests. It even enables test engineers to address scenarios previously considered untestable or too expensive to test.

The Arquillian project is led by Aslak Knutsen and has received contributions from over 100 contributors and community members (between Arquillian and ShrinkWrap combined). At the JavaOne 2011 conference, Arquillian received the Duke’s Choice Award for innovation in integration testing.

The 1.0.0.Final release of Arquillian Drone, a key add-on to the platform, is included in this release. Final versions of select container adapters will be released later in the week. ShrinkWrap, a central component of Arquillian, announced its 1.0.0.Final release last week.

Mission and History

Arquillian adheres to three core principles:

  • Tests should be portable to any supported container
  • Tests should be executable from the IDE to eliminate the need for an explicit build step and to simplify debugging
  • The platform should unify the Java testing ecosystem by extending or integrating with existing test frameworks

By focusing on these principles, Arquillian makes integration and functional tests as simple to write and execute as unit tests.

Arquillian originated from the test harness developed for the CDI 1.0 (JSR-299) specification in 2009. It spun off as an independent project and has evolved into an extensible testing platform. Coming full circle, the test suite in CDI 1.1 (JSR-346), the next iteration of the CDI specification, has migrated to Arquillian. Other specifications are expected to follow. Arquillian is also used by numerous open source projects, including Hibernate, JBoss AS 7, Drools, RHQ, JClouds and Apache DeltaSpike.

Functionality

Arquillian brings test execution to the target runtime, alleviating the burden on the developer of managing the runtime from within the test or project build. To invert this control, Arquillian wraps a lifecycle around test execution that does the following:

  • Manages the lifecycle of one or more containers
  • Bundles the test case, dependent classes and resources as ShrinkWrap archives
  • Deploys the archives to the containers
  • Enriches the test case with dependency injection and other declarative services
  • Executes the tests inside (or against) the containers
  • Returns the results to the test runner for reporting

Arquillian runs with Java 1.5 and above, integrates seamlessly with familiar testing frameworks such as JUnit and TestNG and allows tests to be launched using existing IDE, Ant and Maven test plugins.

Loving quotes about Arquillian

…using Arquillian, we were able to cut the setup needed to run a plugin in-container by 90% and we were able to introduce a number of convenience annotations from which you can get a variety of data injected into your tests.”

— Lukáš Krejčí, RHQ core developer

Arquillian is a really great integration testing tool full of potential. It’s just great that the JBoss guys are aiming to provide support for almost all widely used application servers and web containers. If you are writing an application for the Java EE 6 stack, not using Arquillian is a serious mistake!”

— Bartosz Majsak, Cambridge Technology Partners

[Arquillian] reminds me of the old Cactus project back in the day, but done much, much better.”

— Laird Nelson

Newest features

Arquillian can manage more than a dozen container vendors, including JBoss AS, GlassFish and Tomcat, and supports running tests in cloud services. The container support allows developers to target a variety of technology platforms, including Java EE 5 and 6, Servlet environments, OSGi, Embedded EJB and standalone CDI.

Additional new features include:

  • Orchestration of multiple deployments across multiple containers in a single test
  • Support for multiple protocol contexts within a single deployment
  • Descriptor deployment
  • Assertions for deployment exceptions
  • A new configuration schema that supports multiple configurations per container
  • EL-like evaluation in properties and configuration overrides via Java properties
  • Explicit ordering of test methods
  • Control over when the container is started and stopped

Arquillian’s extensibility is reflected in its growing ecosystem of extensions. The most mature extension, Arquillian Drone, is included in today’s release. Drone is an abstraction over browser controllers such as Selenium and WebDriver that enables the developer to write browser-based tests without having to fuss with the typical setup and plumbing. Other extensions under active development include an Android test controller, DBUnit integration, a SeamTest replacement for testing Seam 2, BDD runners (Spock and JBehave), performance metrics, code coverage (Jacoco) and Arquillian Graphene (a type-safe Selenium API). Expect more extensions to emerge now that the platform has reached a stable release.

Availability

The Arquillian platform and extensions are available in the Maven Central and JBoss Community artifact repositories. The Arquillian libraries are typically added to the test suite of a project using a dependency management tool such as Apache Maven or Apache Ivy. Instructions for setting up Arquillian in your project and writing Arquillian tests are covered in the newly-minted Arquillian Guides.

Arquillian is released under the Apache License, v2.0, an OSI-approved open source software license.

For more information and updates about the Arquilian project, follow the Arquillian project blog and circle the Arquillian project on Google+.

arquillian.org: The Bridge of the Arquillian Project

The short version

Arquillian now has guides! If you want to know how to get started with Arquillian, head over to arquillian.org to check them out…in one of 8 languages! If you want to make them better, fork the website project and get involved.

The slightly longer version

When I thought of how to present arquillian.org, naturally images of alien spacecraft came to mind. These days, there’s a lot of activity flowing in an out of the Arquillian mothership. To manage and make sense of all that information, we were in need of a central view of everything. In other words, we needed to get to “the bridge” of the ship. From there we can monitor activity, publish the lastest information, collect and exchange ideas and decide which actions to take.

Well, we’ve made it, finally. Welcome to arquillian.org. As Bob McWhirter likes to say, “You’re soaking in it.”

The full story

We’ve been thrilled to witness the impact Arquillian has made over these last two years and the agility it’s brought to the Java ecosystem. There’s no question, the Arquillian invasion is in full force.

But even as we stood on stage at JavaOne to receive a Duke’s Choice Award for Arquillian on behalf of our awesome community, I kept thinking about how much more successful it could be. I mean, if developers have been able to figure out how to use and build on Arquillian so far, just imagine how many more developers would use it—and how much easier it would be for them—if the project actually had some decent guides to get started.

We felt your pain. We heard your voice.

That’s why we decided at JavaOne it was time to get serious about closing this gap by writing several step-by-step guides. We didn’t leave off there, though. We recognized the whole experience of discovering Arquillian for the first time (or coming back to it after a break) needs to be extremely easy and rewarding. So after we drafted the guides, we molded them into a visually appealing website with a call to action button you can’t possibly miss. We then called on our noble community members to volunteer to translate those guides into as many languages as possible. And they came through!

The long road

Ever since then, we have been trying to get these guides into your hands. It’s been a long and rather ridiculous journey. I describe it as ridiculous for contrasting reasons.

On one hand, the reason we couldn’t make it available was ridiculous. But that’s a long and boring story. On a positive note, it was ridiculous to see how much effort the community has contributed to driving this vision to make Arquillian easy to adopt and use—to a website that wasn’t even being published, no less. It certainly demonstrated a lot of spirit. That’s why we kept pressing against that first ridiculous reason to get it launched :)

The final stretch

Over the past several weeks, I’ve been working with Sarah and Aslak (with blood, sweat and tears) to get this website ready to publish. We have curated as much information as we can find—at least so far—and organized it into a website that’s going to give you a view into the Arquillian project like you’ve never seen before. Heck, even while building the website, I was stunned to finally see a list of modules, releases and changelogs in one place for the first time. And now, it’s finally out there.

A new beginning

This is just the beginning. By no means do I consider the website to be set in stone for you just to look at (though I need a few days off to sleep). Quite the opposite. arquillian.org is an open book. It’s a Creative Commons (CC BY-SA 3.0) open source project hosted on, you guessed it, github. Clone (or fork) the repository named arquillian.github.com and checkout the README. The project comes complete with an issue tracker and wiki.

It’s important to recognize that the website needs to be a part of the Arquillian project just like any other module. After all, the project is only as good as the information we put out there. I’m envisioning have contributors that join the website team much the same as they might join a module. The group will be a cross section of design, outreach, messaging, documentation and, of course, automation.

We got it spun up (nearly writing it twice) because we knew Arquillian was severely lacking in good documentation and reference material, but we want you to be as involved in it as you like.

The website reflects the quality of the software—especially in the eyes of new users—and therefore it represents all of our work. That’s why my favorite page on the site is the list of contributors ;) It’s all about you, the nobles.

What’s inside

Before I close, I want to mention some key highlights of the website. Then, I’ll leave you to explore and discover the rest :)

  • All content managed in git and hosted on Github
  • Baked with Awestruct, a static site generator
  • A data curating pipeline written in Ruby (JRuby) as Awestruct extensions
  • Pages written in Haml (with a mix of Textile)
  • Easy to read, step-by-step guides (written in Textile, changelog retrieved via git)
  • Blogs written in Textile (or Markdown)
  • Guest blogs and improvements integrated via pull requests
  • Layout and components built on Twitter Bootstrap and jQuery
  • CSS3 goodness and cross-browser support compiled by SASS
  • Designed to be responsive for mobile phones and tablets (give it a try!)
  • Module, release and commit information mined from git repositories
  • Auto-generated, in-depth blog entry plus custom notes for each release (example)
  • Detailed summary page for each module (repository, versions, dependencies, contributors, etc)
  • Contributor information backed by Github and Gravatar
  • Dynamic information retrieved via json-p (tunneled through jgfeed when necessary)
  • Upcoming talks on Arquillian syndicated from Lanyrd
  • Blogs and articles about Arquillian syndicated from Diigo
  • Under the watchful eye of Ike (as you can see in the upper-left hand corner)

If you are curious about more details, check out the Information Architecture page I drafted on the wiki about the site. Take a look at the README if you want to build the site locally.

The vision

Our vision is to automate as much as the documentation as we can possibly accomplish. It ain’t (just) because we’re lazy. It’s because we know it’s the only way to get you the most accurate, up to date information about Arquillian and to reasonably manage the growing Arquillian ecosystem. In other words, we need to be in “the bridge”. And now we have it. Just check out the module pages. What a start!

Use it. Enjoy it. Pass it on. And fork it if you want it to be better.

If words could do

I want to give a huge thanks to Sarah White, who has stuck with me through trying times to get this website looking visually stunning, organized and logical (and for just putting up with my intense focus on this project).

I also want to recognize the following translators, who have brought the guides to 8 languages in total: Markus Eisele (German), Antoine Sabot-Durand and Badr El Houari (French), Takayuki Konishi (Japanese), Hantsy Bai (Simplified Chinese), Bartosz Majsak (Polish), Jose Rodolfo Freitas (Portuguese) and Tommy Tynja (Swedish). Arquillian is forever changed because of your effort.

Finally, I want to thank all of the contributors who have embraced writing release blog entries—Vineet Reynolds, Bartosz Majsak, Karel Piwko and Lukas Fryc—and contributing guides—Karel Piwko, Paul Bakker, Lincoln Baxter and Marek Schmidt.

That’s exactly the participation I envision will make this website a key communication hub of the Arquillian project. Let’s keep the invasion strong!

Goodbye, Ajocado - Welcome, Graphene!

Goodbye, Ajocado

Wait, the Ajocado project is not leaving us!
It is just undergoing rename.
Why such a change?

Ajocado has always been a proud member of Arquillian family, however its name didn’t tell very well what’s its purpose is in the true alien world. In other words, the name didn’t fit the story.

Moreover, we have experienced many troubles explaining how Ajocado should be even pronounced.

I must admit it was unlucky choice.

Choosing the new name…

The project is like a diamond – very strong in its basis:

  • a strongly-typed, chaining API
  • efficient jQuery-based locators
  • proven to handle AJAX successfully

But on the other hand, diamonds can’t be simply manufactured.

Let’s look what more could Ajocado offer us:

  • rapid test development
  • readability
  • high-level abstractions

What can be as strong as diamond, but be much more flexible?

Welcome, Graphene

Graphene is stronger and stiffer than diamond, yet can be stretched by a quarter of its length, like rubber.

Let me introduce the Graphene – the strong and flexible UI automation tool – the old/new member of the alien crew, on the way to save the world!

What’s next?

The rename is already done, waiting in Maven repositories as Graphene 1.0.0.CR3.

And what about migration? It’s pretty straight-forward, as you can see in my previous post.

Thank you all who participated in choosing new name, renaming and migration testing (Vlasta Elias, Dan Allen, Jan Papousek, Karel Piwko)!

Project Links

Very large, active development team

According to ohloh, Arquillian has a very large, active development team. Here’s a summary of the contributor stats that ohloh curated.

Over the past twelve months, 43 developers contributed new code to Arquillian.

This is one of the largest open-source teams in the world, and is in the top 2% of all project teams on Ohloh.

For this measurement, Ohloh considered only recent changes to the code. Over the entire history of the project, 49 developers have contributed.

Thank you all for making this project so successful and fun! Arquillian has a bright future. Above all other awards and recognitions, it’s the number and diversity of contributors that’s the most significant. The smartest one amongst us is all of us, working together.

Cheers!