Recently I was converting some test cases so that they would run as integration tests instead of unit tests. The maven-failsafe-plugin instead of maven-surefire-plugin.
The initial bits of the conversion were running smoothly until I hit a few complex components with lots of dependencies. Then, suddenly I was getting to a point where tests that were running just fine with Surefire would halt, crash and catch fire with Failsafe.
Running the tests with mvn --debug reveals really strange errors:
- TestEngine with ID ‘junit-jupiter’ failed to discover tests
- Tests failing with java.lang.ClassNotFoundException
- Tests failing with java.lang.NullPointerException
Some failsafe users might know that it does manipulate the classpath so one of the first reactions when facing these problems would be to useSystemClassLoader or useManifestOnlyJar options.
Unfortunately this may not resolve the issue with version 3.0.0-M5. It looks like a bug on the failsafe plugin. More specifically, it appears to be the SUREFIRE-1831 bug.
So, what to do if you are having that problem?
When facing this issue there are at least 2 possibilities to work-around it: the easiest one is to downgrade to 3.0.0-M4 and wait for M6. If that’s not possible, using forkCount=0 seems to solve some the problem at the cost of a stern warning message.
Update 20/jun/2021: one of the Surefire developers commented about the work-around of using forkCount=0 and provided a suggestion about it. There are links to additional discussions which are relevant to the issue. Long story short is that we should avoid it and I strongly advise following his suggestion.
4 replies on “Test woes with Maven Failsafe 3.0.0-M5”
I am a developer of Surefire and I would not advice our community to use forkCount=0. Please read https://issues.apache.org/jira/browse/SUREFIRE-1809 and use the workaround with false.
useModulePath=false
Thank you for sharing this! I will edit the post to highlight your suggestion.
The article and the comment helped me. Thanks !!