In the Arquillian project, we’re all about real tests. There’s a good reason for this philosophy. If you use mocks or a substitute container when testing code that uses a programming model (i.e., CDI), all you can be sure of is that you’ve faked out enough functionality to get the test to work. You can’t be certain that your code works—really, truly works.
Obviously, we provide adapters for embedded containers (i.e., substitute containers). The reason is, they often do a decent enough job of simultating the real environment that we are willing to make the trade-off for its variations when we are working in development. I would expect, though, that you are also running the same tests in CI using a real container…just to get that peace of mind.
How do you know when not to use an embedded container?
If you write a test that you are positive should work, but instead throws some wacky exception, immediately stop and try the same test in a real container. For example, if you are using Embedded GlassFish, I would advise running the test in managed or remote GlassFish. The latter two adapters both use an authentic GlassFish instance. If the test passes, then you have answered the question of this section.
When the embedded container begins to act inconsistently, drop it.
As with all things, there are exceptions to the rule. If the code really should work on the embedded container, then it may be worth exploring why it isn’t. That may lead you to the discovery of a bug in the library you’re using. If debugging the library is your main concern, chase it down. If you are just trying to get a test written for your application, stop using the embedded container (or save debugging it for a rainy day).
Examples
See this thread in the Seam 3 forums for an example where the Embedded GlassFish container was acting more of a hindrence than a help. The thread includes the steps I took to get a green bar using a different GlassFish container adapter.
Markus Eisele identifies another hazard. He points out in a recent blog post that testing JPA using Embedded GlassFish is problematic because JPA weaving (in EclipseLink) does not occur:
[Weaving] only works when the entity classes to be weaved only exist in the application classloader. The combination of Embedded GlassFish, Arquillian and the Maven Surefire Plugin mix this up a bit and the end of the story is, that exactly none of your entities are enhanced at all.
Again, testing on Embedded GlassFish is not serving us well in this scenario. Graduate.
Closing thoughts
Code that uses Solder works fine on GlassFish 3.1.2, but it does not play very nicely with Embedded GlassFish—it’s just a hostile environment for CDI because of the visibility issues that come with starting an embedded container inside an existing Java process. The same goes for JPA. You end up in a minefield of class visibility and access problems, and far from making any progress writing tests (your ultimate goal).
If you want the container to start automatically like with embedded, then switch to a managed container. It works almost exactly the same as the remote container, except that Arquillian will start the standalone process at the beginning of the test execution and stop it at the end. In other words:
managed >>>> embedded
It’s clean. It’s true.
For speed of development:
remote > managed
With remote, you don’t have to wait for the container to start…and you can start it in debug mode very easily and keep it there. That’s why I say that the best container adapters for development are the remote adapters and the best adapters for CI are the managed adapters. I personally don’t like the embedded container adapters much (with the exception of embedded CDI since that’s mostly a pure standalone CDI environment).
Keep that advice in mind.
