Robert Hanson's Tech Log: Testing Servlets with JUnit

Robert Hanson's (mostly) tech log.

Testing Servlets with JUnit

Thursday, December 20, 2007 by Robert Hanson. «Permalink»

In my day-to-day job I write a lot of unit tests, and I try to test everything I can. Recently though I had written a service that I was unable to test completely. One of the methods in the service had the job of sending an HTTP request to a remote server and responding with the results. For my project I used commons-httpclient to send the request.

@SuppressWarnings("unchecked")
public String sendHttpPost (String url, String queryString)
   throws Exception
{
    String result = null;
    
    try {
        HttpClient client = new HttpClient();
        PostMethod post = new PostMethod(url);
        post.setQueryString(queryString);
        client.executeMethod(post);
        
        result = post.getResponseBodyAsString();
        post.releaseConnection();
    }
    catch (Exception e) {
        throw new Exception("post failed", e);
    }
    
    return result;
}

If you have used commons-httpclient, this is about as simple as it gets, but I still wanted to have a unit test for it. So the problem became, "how can I test this method when it requires that I hit a web site". After some searching I found that the Jetty servlet-container has a ServletTester class just for this purpose.

In my Maven 2 I included the ServletTester using the following repository and dependency information. I always use Maven 2 as allows other developers to quickly set up their IDE and download all required JARs. If you don't use Maven, you will need to manually download all of the dependencies.

<repositories>
  <repository>
    <id>codehaus-release-repo</id>
    <name>Codehaus Release Repo</name>
    <url>http://repository.codehaus.org</url>
  </repository>
</repositories>

...

<dependency>
  <groupId>org.mortbay.jetty</groupId>
  <artifactId>jetty-servlet-tester</artifactId>
  <version>6.1.6</version>
  <scope>test</scope>
</dependency>

With this in place the next step was writing the JUnit test cases. For me I wanted to initialize the servlet-container once, then run a set of tests against it. In JUnit 4 you can use the @BeforeClass and @AfterClass annotations to mark methods that should be executed before and after all of the tests.

public class HttpPostServiceTest
{
    private static ServletTester tester;
    private static String baseUrl;
    

    /**
     * This kicks off an instance of the Jetty
     * servlet container so that we can hit it.
     * We register an echo service that simply
     * returns the parameters passed to it.
     */
    @BeforeClass
    public static void initServletContainer () throws Exception
    {
        tester = new ServletTester();
        tester.setContextPath("/");
        tester.addServlet(EchoServlet.class, "/echo");
        baseUrl = tester.createSocketConnector(true);
        tester.start();
    }

    /**
     * Stops the Jetty container.
     */
    @AfterClass
    public static void cleanupServletContainer () throws Exception
    {
        tester.stop();
    }
}

The code highlighted in blue is where we start an instance of the server. I created a new instance of the ServletTester, setting the context path and adding a servlet mapping. This alone does not bind the server to a port, for that you need to call createSocketConnector(true), which binds the server to a local port and returns the URL. The port used will be a high unused port. I save an instance of the ServletTester so that I can stop the service in the @AfterClass block, and I save the baseUrl so that I can target it in my tests.

The servlet I added to the container I called EchoServlet. This servlet simply echos the parameters passed to it.

public class EchoServlet extends GenericServlet
{
    
    @SuppressWarnings("unchecked")
    @Override
    public void service (ServletRequest request, ServletResponse response)
        throws ServletException, IOException
    {
        PrintWriter out = response.getWriter();
        Map<String, String[]> params = 
            new TreeMap<String, String[]>(request.getParameterMap());

        out.println("SIZE=" + params.size());
        for (Entry<String, String[]> entry : params.entrySet()) {
            out.println(entry.getKey() + ":::"
                + StringUtils.join(entry.getValue(), ","));
        }
    }
}

The servlet takes the parameter map and creates a TreeMap out of it. The TreeMap is needed so that the parameter names are returned in sorted order. Being able to predict the order of the returned keys is required in order to test against the output of the servlet.

To finish things up, I just needed to write a unit test.

@Test
public void testPost () throws Exception
{
    PostService svc = new PostService();
    
    String res = svc.sendHttpPost(baseUrl + "/echo",
        "foo=bar&baz=%25%26%3D%2F");
    String[] resList = StringUtils.split(res, "\n");
    
    assertEquals(3, resList.length);
    assertEquals("SIZE=2", resList[0].trim());
    assertEquals("foo:::bar", resList[1].trim());
    assertEquals("baz:::%&=/", resList[2].trim());
}

After that, run the test and watch it work.

Labels: Java, Programming, testing

19 comments. Links to this post.

Comments.

19 Comments:

At January 03, 2008 12:31 PM, Ajanta said...

Your instructions worked beautifully for me. I know there is a way to use the servlet filters as well though i was not able to get it working. Would appreciate if you could provide any insight on that too..
Thanks!

At January 03, 2008 12:42 PM, Robert Hanson said...

Ajanta, you can use Spring-Mock for that... even if you aren't using Spring.

Filter filter = new MyFilter();

MockHttpServletRequest req = new MockHttpServletRequest("GET", "/");
MockHttpServletResponse res = new MockHttpServletResponse();
MockFilterChain chain = new MockFilterChain();

filter.doFilter(req, res, chain);

After the filter executed you can use the methods on the mock request/response objects to see verify that the filter did what it was supposed to do.

I am not sure where this library is listed on the Spring site, but you can download it from the Maven repository here, http://repo1.maven.org/maven2/org/springframework/spring-mock/.

At January 03, 2008 3:05 PM, Ajanta said...

Hey, thanks for your quick response, I think ServletTester also provides a addFilter method and I tried using that but it didn't work. Have you used that?

At January 03, 2008 3:20 PM, Robert Hanson said...

No, I haven't tried the addFilter for ServletTester (yet), but I expect that would work well.

In my case though it is easier/better to use mock object for filter testing because I can set/inspect things in the mock objects that I couldn't do without bending over backwards in a server environment.

At February 20, 2008 3:53 AM, Erik Vonderheid said...

Have you ever tried the apache cactus framework?

At February 21, 2008 10:12 AM, Robert Hanson said...

Erik, nope, never used Cactus. I have heard about it, and probably looked at the site once or twice, but never got around to actually trying it.

I just took a look at the Quick Start guide and see one thing that immediately turns me off of using it, the part where it asks me to install Tomcat.

*I* don't have a problem installing Tomcat, but requiring that my developers do this adds time to getting them up to speed on a project... And asking my clients to do this is too much to ask.

My goal is that any developer or client can take the source or a project and run "mvn test" to automatically download and run the tests. I want Maven to be the only tool that needs to be required to test, compile, and package the app.

Of course... I don't know much about Cactus, so maybe I am making an incorrect assumption

At March 04, 2008 6:34 AM, Damian said...

Hi, I'm looking at using the new Jetty ServletTester utilities for unit testing servlets. I'm stuck attempting to mock out a HttpSession for any servlets that require one. Any ideas?

At March 04, 2008 7:27 AM, Robert Hanson said...

Damian,

You are looking for ideas on how to mock HTTPSession without using a servlet container?

Check out spring-mock, it has all of that stuff and is well maintained.

JARs from Maven repo:
http://repo1.maven.org/maven2/org/springframework/spring-mock/

Random article:
http://www.devx.com/Java/Article/30067

At June 04, 2008 1:37 PM, Mike Kaufman said...

Robert, Damian,

A shameless plug... another way you can now "mock" Servlet API objects for out-of-container testing is using my own "ObMimic" library.

This provides plain-Java classes that cover the whole Servlet API for precisely this purpose. It covers all of the Servlet API's interfaces and abstract classes (including HttpSession), and completely implements all of their methods. It does this with plain Java objects that are fully configurable, fully inspectable, and have some additional features to help in testing (e.g. record/examine the Servlet API calls made to each object, report any ambiguous or questionable Servlet API calls, switch between Servlet 2.3, 2.4, 2.5).

Because there are no unsupported or incomplete Servlet API methods it can handle even complex use of the API (e.g. request-dispatcher "forwards" and "includes", listener notifications, servlet mappings, filter chains etc), and as a result you can use it not only for pure "unit" testing but also for testing more extensive paths through your code, framework code, and other libraries.

As of May/June 2008 this is in private "beta" release, but if you want to take a look at it just let me know and we'll set you up. For more details, see my recent blog posting.

At June 05, 2008 8:03 AM, Robert Hanson said...

Mike,

> A shameless plug

Shameless plugs are welcome... just as long as it is on topic.

> As of May/June 2008 this is in
> private "beta" release, but if you
> want to take a look at it just let
> me know

I am interested, but don't currently have any projects on my plate right now that need this. If something does come up, you can definitely expect to hear from me.

Thanks for letting me(us) know about this.

At June 16, 2008 9:24 AM, toupil said...

This post has been removed by a blog administrator.

At March 02, 2009 9:55 AM, Himanshu said...

Hi, a very nice tutorial.Please correct the font color of the code. Its appears nearly invisible as font is of white color in white background. Thanks

At March 02, 2009 11:19 AM, Anonymous said...

Hi Robert. Thanks for sharing your experience.

Basically, the ServletTester works well, but I faced the following problem and maybe you can help me:

Can you tell me which values you take for the contextPath and the DocumentBase? For both there are setters and getters, but using a contextpath like "/webapp" doesn't works when resources are to be loaded. For example, when using servlet-methods like "getServletContext().getRealPath("WEB-INF/classes")", which work when used inside a "stand-alone" container, return null when used with the ServletTester.

Thanks and best regards, Reza (from germany ;-) ).

At March 03, 2009 8:40 AM, Robert Hanson said...

Himanshu, it should be better now. Thanks for mentioning it.

At March 03, 2009 12:35 PM, Himanshu said...

That's good now n legible. Well despite repeated attempts your tutorial never worked for me and I,ve to satisfy with spring mock objects

At March 07, 2009 9:22 AM, W-Mark Kubacki said...

Indeed, saved me some time. Thanks!

But, the blue font color is not optimal, too. With black background it appears blurry.

At March 31, 2009 8:13 AM, Robert Hanson said...

> Anonymous said...
> using a contextpath like
> "/webapp" doesn't works when
> resources are to be loaded.

I really don't know. This example wasn't meant to load resources. If I needed to do that, for instance if I needed to pass a test XML file back to the caller, I would just use File to open/read the file then simply print it to the response object.

At July 02, 2009 6:59 PM, Nima said...

Hi Robert, The tutorial is great. I just wonder if you know of a way by which I can add a servlet instance to ServletTester rather than passing the servlet class. My servlet object should receive a db storage object in its constructor and thus I need to instantiate the servlet before passing it to ServletTester.

However, it seems like when testing a servlet you can only pass the servlet class to ServletTester and let it instantiate the servlet. This causes null pointer exceptions in my case which is causing problem.

Do you have any idea or suggestion on how this can be solved?

thanks a lot

At July 03, 2009 11:04 AM, Robert Hanson said...

Nima, sorry, I don't know how you might do that without either changing the ServletTester or the servlet.