May 102015
 

EDIT – This post was written for mocking a Netty 3.x server. For mocking a Netty 4.0 server, see this post.

While working on an app with my current job, I wound up touching some code that didn’t have any unit tests associated with it. Since we’re a small team (but growing), any automation in testing really helps (not to mention just being a good thing to do). The issue was the code was all in a request handler for a Netty server, which meant I needed a way of either running a Netty server during the Maven build process, or I needed to simulate 1 via some type of mocking library. Ultimately, I settled on the latter. Here’s how I did, and the things I learned along the way.

As a little background, Netty is a handy framework that we use for web servers. They work great, but aren’t nearly as simple and useful for testing. With other parts of the application, which I had written, I had broken out all the actual processing into separate classes that I could then instantiate and test in JUnit. So now the code I had written had handlers listening for web requests from Netty, another class doing something with that request and returning any information back to the request listener so it could form the proper response (hey, more classes means more better Java, right?).

The code I was looking at had been written by another developer, and refactoring it to fit the way things were being done everywhere else, while probably the best overall solution, was outside the scope of the work I had been doing, so that was off the table. Since we had been discussing using mocking libraries to help test our code anyhow, this seemed like a good time to try it out.

Setting up the server

As with all great software endeavors, I started by Googling to see if someone had already figured this out for me. After all, Maven has a Jetty plugin, maybe I could find one for Netty. Turns out I couldn’t, but I did stumble across MockServer. While MockServer looked like it would be a great fit for almost everything we would need to test, but using MockServer involves mocking the response from the server, which was precisely the thing I was trying to test. Not only that, but there were other objects that I wanted to mock, such as a user’s login session information as well as some account information related to various objects under test. That made Mockito the clear winner here – I could use it to mock any external web service I was using as well as mock the other objects the code I’m testing depends on.

Now, I didn’t immediately launch into “let me mock a server!” As I said, there were other objects we were using in our tests that were nothing more than dummy placeholders for references that had to be present for 1 little piece of the code being unit tested to function properly. 1 of those was user login session information. This I had originally solved with a dummy object, which had getters for the 2 member variables actually used. The problem was, this same dummy object was in the unit tests for every module of the project. I mocked it first. That being a mocked object with a couple of getters set, I was hardly impressed by any of this mocking stuff, but it let me delete a file that had no purpose other than letting me actually test code.

Next up was related account data I was shoving in the database before every test ran. This data was used by a couple of methods I was testing, and would be present in the database when the code would run in production, but naturally isn’t there when I’m testing a separate module. So, the next order of business was to mock up this account data, and then have my code use that mocked object instead. That was about as simple and straightforward as mocking up the login, and at this point I could start to see why people were into this idea of mocking up data, although I still wasn’t convinced that adding another library and degree of complication was really that much better than just whipping up some dummy objects.

Now we get to the point where I’m trying to decide what to do about mocking up this server, and I’ve come to the realization that situations like this is what Mockito was meant for. If I could simulate a server, I could avoid a refactoring of the code that served no real purpose other than to jury-rig it for testing. Plus, I’d learn something useful and I could run a set of tests that completely mirrored how the code was going to be called. After mocking not only the webserver for the code I was testing, but also the third-party APIs being used by that code and the rest of the application (which both sped up testing and helped me better test error conditions), I’m finally sold on the idea of using Mockito during unit testing.

To actually get the Mockito version of Netty running, I had to rely on a couple of code demos I found online. First, to actually have a Netty server listening for requests, I leaned pretty heavily on the code posted here. Even with that as a good starting block, it still took me 2 days of stepping though the code with a debugger to get Netty server mocked to the point where tests weren’t throwing errors. That said, it was worth it in the end. I not only spent some time getting familiar with how Netty works, but I also got my mocked server working. Next up was to figure out how to get this working for all unit tests.

Since I was going to need to change the URLs being called between tests, I created 2 HttpRequest objects, getRequest and postRequest, to finish up the mocking in each test case. At the start of each case, I’d stub out the getUri() method to return the URL of the endpoint I wanted to test. For example:

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import java.net.InetAddress;
import java.net.InetSocketAddress;

import org.jboss.netty.bootstrap.ClientBootstrap;
import org.jboss.netty.buffer.ChannelBuffer;
import org.jboss.netty.buffer.ChannelBuffers;
import org.jboss.netty.channel.Channel;
import org.jboss.netty.channel.ChannelFuture;
import org.jboss.netty.channel.ChannelHandlerContext;
import org.jboss.netty.channel.ChannelPipeline;
import org.jboss.netty.channel.ChannelStateEvent;
import org.jboss.netty.channel.MessageEvent;
import org.jboss.netty.handler.codec.http.DefaultHttpHeaders;
import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
import org.jboss.netty.handler.codec.http.HttpHeaders;
import org.jboss.netty.handler.codec.http.HttpMethod;
import org.jboss.netty.handler.codec.http.HttpRequest;
import org.jboss.netty.handler.codec.http.HttpResponse;
import org.jboss.netty.handler.codec.http.HttpResponseStatus;
import org.jboss.netty.handler.codec.http.HttpVersion;

public class TestClass {

    private Channel channel;
    private ChannelFuture success;
    private ChannelStateEvent channelEvent;
    private ClientBootstrap httpClient;
    private HttpRequest getRequest;
    private HttpRequest postRequest;
    private MessageEvent event;

    @Before
    public void before() throws Exception {
        
        channel = mock(Channel.class);
        channelEvent = mock(ChannelStateEvent.class);
        event = mock(MessageEvent.class);
        getRequest = mock(HttpRequest.class);
        httpClient = mock(ChannelBootstrap.class);
        postRequest = mock(HttpRequest.class);
        success = mock(ChannelFuture.class);

        ChannelHandlerContext context = mock(ChannelHandlerContext.class);
        ChannelPipeline channelPipeline = mock(ChannelPipeline.class);

        /*
         * Stub out the behavior needed to return the InstagramRequestHandler
         */
        when(channel.getRemoteAddress()).thenReturn(
                new InetSocketAddress(InetAddress.getLoopbackAddress(), 0));
        when(event.getChannel()).thenReturn(channel);
        when(channelEvent.getChannel()).thenReturn(channel);
        when(success.getChannel()).thenReturn(channel);
        when(getRequest.getMethod()).thenReturn(HttpMethod.GET);
        when(postRequest.getMethod()).thenReturn(HttpMethod.POST);
        when(getRequest.getProtocolVersion()).thenReturn(HttpVersion.HTTP_1_1);
        when(getRequest.headers()).thenReturn(new DefaultHttpHeaders());
        when(httpClient.getPipeline()).thenReturn(channelPipeline);
    }

    @Test
    public void testTheThing()  {
        when(getRequest.getUri()).thenReturn("/endpoint/being/tested");
        when(event.getMessage()).thenReturn(getRequest);

        /* This is the call to the Netty server. */
        handler.messageReceived(context, event);
    }
}

It’s a lot of code, and it doesn’t even cover sending an HTTP POST request with parameters, which involves adding a block like this to your test case:

StringBuilder params = new StringBuilder();
params
    .append("param1=stringValue1")
    .append("&param2=stringValue2")
    .append("&param3=stringValue3");
            
ChannelBuffer buffer = ChannelBuffers.copiedBuffer(
params.toString(), CharsetUtil.UTF_8);
            
HttpHeaders.addHeader(postRequest, HttpHeaders.Names.CONTENT_LENGTH, 
    buffer.readableBytes());
HttpHeaders.addHeader(postRequest, HttpHeaders.Names.CONTENT_TYPE,
    "application/x-www-form-urlencoded");
when(postRequest.getContent()).thenReturn(buffer);

There’s something I left out of the above example (yes, I actually did leave parts of this out for brevity) – if your service requires certain headers be present in the requests, you’ll need to set those when you initialize your HttpRequest in your @Before block.

As you may have noticed, there’s nothing in the sample code that actually reads a result from our call to the Netty server. Netty, being asynchronous, isn’t going to block until there’s a response to return. It’s just going to write the response to the same channel it heard the request on, and whoever called the server can listen for it and react accordingly. That means we’re going to have to add some type of listener to our test cases to get the response from the server. That’s right boys and girls, we need to add more code!

Thanks to the the code found in this GitHub repository, we have the answer to our woes – Mockito’s ArgumentCaptor. The ArgumentCaptor can listen for the response coming from the Netty server and store it for you to compare to what you were expecting. Now our code will look something like this:

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import java.net.InetAddress;
import java.net.InetSocketAddress;

import org.jboss.netty.bootstrap.ClientBootstrap;
import org.jboss.netty.buffer.ChannelBuffer;
import org.jboss.netty.buffer.ChannelBuffers;
import org.jboss.netty.channel.Channel;
import org.jboss.netty.channel.ChannelFuture;
import org.jboss.netty.channel.ChannelHandlerContext;
import org.jboss.netty.channel.ChannelPipeline;
import org.jboss.netty.channel.ChannelStateEvent;
import org.jboss.netty.channel.MessageEvent;
import org.jboss.netty.handler.codec.http.DefaultHttpHeaders;
import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
import org.jboss.netty.handler.codec.http.HttpHeaders;
import org.jboss.netty.handler.codec.http.HttpMethod;
import org.jboss.netty.handler.codec.http.HttpRequest;
import org.jboss.netty.handler.codec.http.HttpResponse;
import org.jboss.netty.handler.codec.http.HttpResponseStatus;
import org.jboss.netty.handler.codec.http.HttpVersion;

import org.mockito.ArgumentCaptor;

public class TestClass {

    private ArgumentCaptor<Object> captor;
    private Channel channel;
    private ChannelFuture success;
    private ChannelStateEvent channelEvent;
    private ClientBootstrap httpClient;
    private HttpRequest getRequest;
    private MessageEvent event;

    @Before
    public void before() throws Exception {

        captor = ArgumentCaptor.forClass(Object.class);
        channel = mock(Channel.class);
        channelEvent = mock(ChannelStateEvent.class);
        event = mock(MessageEvent.class);
        getRequest = mock(HttpRequest.class);
        httpClient = mock(ChannelBootstrap.class);
        success = mock(ChannelFuture.class);

        ChannelHandlerContext context = mock(ChannelHandlerContext.class);
        ChannelPipeline channelPipeline = mock(ChannelPipeline.class);

        /*
         * Stub out the behavior needed to return the InstagramRequestHandler
         */
        when(channel.write(captor.capture())).thenReturn(success);
        when(channel.getRemoteAddress()).thenReturn(
                new InetSocketAddress(InetAddress.getLoopbackAddress(), 0));
        when(event.getChannel()).thenReturn(channel);
        when(channelEvent.getChannel()).thenReturn(channel);
        when(success.getChannel()).thenReturn(channel);
        when(getRequest.getMethod()).thenReturn(HttpMethod.GET);
        when(getRequest.getProtocolVersion()).thenReturn(HttpVersion.HTTP_1_1);
        when(getRequest.headers()).thenReturn(new DefaultHttpHeaders());
        when(httpClient.getPipeline()).thenReturn(channelPipeline);
    }

    @Test
    public void testTheThing()  {
        when(getRequest.getUri()).thenReturn("/endpoint/being/tested");
        when(event.getMessage()).thenReturn(getRequest);

        /* This is the call to the Netty server. */
        handler.messageReceived(context, event);

        Object captured = captor.getValue();

        HttpResponse response = (HttpResponse) captured;
        String data = response.getContent().toString(CharsetUtil.UTF_8);
    }
}

If your RESTful call returns JSON, then add this code block immediately after the existing testTheThing() code:

ObjectMapper mapper = new ObjectMapper();
Map<String, Object> jsonData = mapper.readValue(data.getBytes(),
    new TypeReference<Map<String, Object>>() {});

If your RESTful endpoint returns a JSON list, just change the Map<String, Object> above to a List<Map<String, Object>>, and bam, you now have actual results from your RESTful service you can compare to some expected results to make sure your code is actually working. The TypeReference is from the com.fasterxml.jackson.core.type package, since we were already using Jackson for JSON parsing elsewhere in our code.

Other things worth noting while I’m on the subject

Now that we have a Netty server powered by Mockito, we’re ready to start running some tests! We can even keep using Mockito to help stub out supporting objects or libraries as needed. Which begs the question – when should you use Mockito to stub out an object vs. just creating an object with a bunch of dummy data? My general rule of thumb is that I use Mockito to stub any external services (Facebook and Instagram API libraries, AWS libraries, etc.) so my code isn’t sitting around waiting on some other web service to answer me. It also lets me control the results of the calls so I can inject errors to make sure those code blocks are working like I expect.

Outside of that rule, I generally go with the path of least resistance. More often than not, that’s creating a dummy object and saving it into a local test database that I drop after each test. Occasionally, as with our login session information, there’s an object that’s just easier to mock than create, but I’ve found those situations to be pretty rare.

Once you start using Mockito heavily, I found that it was really easy to start getting mock-happy, to the point where I almost started mocking a few parts of my code that I should have been testing (the very thing I was trying to avoid when I chose not to use MockServer). Be on the lookout for that impulse and make sure you’re avoiding it, otherwise you’ll end up defeating the purpose of your unit tests.

Lastly, you may find yourself where you just want to stub a subset of a real object’s behavior. Mockito’s spy wrapper is perfect for these situations, but there’s a trick to stubbing out behavior when spy() is used. Specifically, the when().thenReturn() pattern you’ve been used to doesn’t work properly on spied objects. As specified in this wonderful Mockito guide, you’ll want to use the doReturn().when() pattern instead.

Mocking a Netty server is entirely possible, albeit not concise. While I think a lot of the time it took me to get my mocked server up and running was due to trying to only stub out a bare minimum of stuff, so I didn’t follow the code I was using as an example very closely. Just assume you’re going to need most of that code. Hopefully, this  post can take something that took me a couple of days and get you up and testing in a matter of hours – if not less. My adventures with Mockito aren’t over – I want to abstract this stubbed Netty server out into its own object that I can just pull into my other unit tests. For now though, this is a good start to better testing.

 Posted by at 4:20 PM