Unit Testing REST Clients

I frequently seem to end up writing code that calls REST services – on pretty much every project I work on anymore.  Since I seem to be often writing the same code over and over, I’m working on a library (well a couple libraries) to simplify things and consolidate all that I typically come across.

A challenge in writing a REST client is how to easily unit test it.  I don’t want to have to stand up a server somewhere just to run simple tests, especially if the server needs to return valid and invalid data that needs to change with the test.

So I create a really simple, yet really useful class I’m calling SimpleServer and it’s available as a gist.  It basically allows you to define a URI and then set callback methods for the common HTTP verbs (GET, PUT, POST and DELETE).

Let’s say you want to test out a GET operation.  In the example below the RestConnector encapsulates the goo required to make the HTTP call, but the cool part is how it uses the SimpleServer to test an expected response to a GET at the API root.

        public void GetNoCreds()
            var uri = "http://localhost:8080/";
            var response = "Hello!";

            using (var svr = new SimpleServer(uri, (request) =>
                return response;

                using (var rc = new RestConnector(uri))
                    var result = rc.GetString("/");

                    Assert.AreEqual(response, result);

Here I’m using the default handler parameter for a GET, but you can individually pass in Func’s for GET, PUT, POST and DELETE which makes testing even fairly complex scenarios both quick and really, really easy.

Should You Develop Software for Free?

I just read an excellent opinion piece over on the New York Times site. I’d recommend you read it, but if you’ve not got the time, or are generally just too lazy, let me summarize. The author is, not unexpected, a writer. Often he is asked to do writing or give speeches and offered $0.00 in return. Basically people want him to give away the output of his “craft” yet those same people would never ask or expect “a keychain or a Twizzler” for free. What gives? How do you tell them to pound sand, but in a polite way? And should you?

Well it’s not so greatly different here in the computer world. Many people feel software should be free. After all, it’s just electrons, and I can shuffle my feet across the carpet and get electron for free, why should you buy electrons from someone else? In fact the idea is so popular that there’s an entire sect that firmly believes that all software should be free. Amazingly, these people even tend to be software developers.

Yes, I’m somewhat of a contributor to this. I have written a boatload of software that I give away. I even encourage people to go get for free. But let’s be clear, that software was paid for. It was developed over many years and many projects. The customers whom I was working for at the time paid for me to solve their problem, and part of that solution involved adding to or fixing some of those core libraries. Still, it was paid for. I simply reserved a small portion of the work as a part of a “free” library that carried over to the next project.

And here’s another dirty little secret. I didn’t do it out of altruism. No, I didn’t do it for “exposure”. That’s a line that someone trying to wheedle free stuff from you would use. I kept those libraries “free” so that a customer can’t say “hey, you did that work on my dime, I own it!” No, sir, you own the solution to your problem part of which uses a set of free libraries.  You’ve gained from work done for countless other projects in the past. Those libraries allowed me to solve you problem much faster than if I had to start them from scratch. So the reason I provide free software is to save me from doing repetitive work and to, generally, keep a set of base libraries that I’m free to use anywhere, on any project.

So why do I also give those libraries away? I don’t know, maybe I am an altruist. Maybe I have the delusion that some day when I’m called in to fix a broken project they’ll already be using my libraries and it will save me the headache and frustration of coming up to speed. Maybe I just hate seeing people reinvent the wheel. Maybe I’m just crazy. Maybe it’s a bit of all of the above, but let’s be clear: I don’t do work for free.

I don’t feel that anyone has a right to ask for software that is not paid for. You may be able to sway me somewhat with the argument about you being free to run and modify the software you paid for, but you’ll have a harder time on the copy and distribute side of the argument.  We, as software developers, still have to pay our bills.  We’ve spent years honing our skills to be able to solve these problems (admittedly, some have done a better job of honing than others) and it takes actual time and work solving problems.  Sometimes I work on problems or with customers where it, most decidedly, is work.  Even drudgery.  So no, I’m not a believer that software, music, writing, photographs or really any other form of “art” should be free.  You wouldn’t expect an electrician to work for free, don’t expect it of a “content creator” either.

Stop Using “The Cloud”

We’ve all been in a meeting, conference or seminar where the talk goes something along these lines:

“We collect the data at the device, then we send it up to The Cloud where we can do analysis and other things on it.”

This is usually accompanies by the vague graphic of a line – sometimes solid, sometimes dotted – pointing to the generic PowerPoint cloud image.

I hate this statement.  I hate this cloud.  This is usually a huge red flag that the speaker or author really has no idea what they’re talking about.  It a sales or marketing cop-out that is the technical equivalent of the popular “then a miracle happens” cartoon.

What does “send it to The Cloud” even mean?  What cloud?  How is the data going to get there?  How will I get it back out? I’ve got other questions about “the cloud” that I’ll address in another post, but these three are the biggies that you should always raise your hand and ask.

1. What is “The Cloud?”

Here’s the little secret – there is no single “cloud”.  There are a boatload of them, and the list seems to be continually growing as it gets cheaper to stand up servers with a lot of storage and as more and more use-cases make use of off-device storage.  “The Cloud” can mean a whole panoply of things, including (but definitely not limited to):

Again there are plenty of others, these are just most of the ones I’ve dealt with in the past year or so.

And bear in mind that not all customers are going to be amenable to all clouds.  Some customers aren’t so comfortable putting their data on servers they don’t control.  Some aren’t comfortable putting their data on servers in particular countries where governmental agencies are known to mine them.  Some customers simply have predispositions to different services for different reasons.

Maybe they like Azure because it provides a simple interface for their .NET enterprise apps.  Maybe they like Amazon’s scale.  Maybe they like Brand X just because they do.  The point is that if you have more than one customer, you’re probably going to need to look at more than one cloud provider.  We’ve got customers that use multiple clouds due to the benefits each provides for certain types of data, data retention policies or total cost of use.

2. How do I get my data into “The Cloud”?

Yeeaaahhh, about that…since there are a boatload of cloud services, there are a boatload of ways that an app gets data into them.  Network databases might use ODBC or a proprietary/specific API set.  Services on the web typically use a web service interface.  Maybe REST, maybe OData, maybe something else.  The point here is none of them are the same.  None of them.

So that means that if you have to support multiple clouds, and you will have to support multiple API sets.  Of course you can abstract them all back to a common interface – I’d certainly recommend it, it’s what we do using the OpenNETCF ORM – but there’s still work to be done to actually implement each.

3. How do I get my data back out of “The Cloud”?

Just like putting the data in, getting it out requires an API, and again, they are all different.  Another thing to consider in the “getting it out” part of the equation is how you actually use it.

Some clouds allow you to run services and/or applications on the servers as well and data access is direct.  Sometimes you have to pull it out back to another server.  Once again, this means more work from a development perspective.  And again, you’ve got to multiply that by the clouds you need to support.

And how about data retention? Some clouds are not “forever” storage.  The data gets purged on a time basis.  If you need to keep the data archived for later data mining then add that work onto your plate too.

So the next time you see that slide with the cloud on it and the speaker says “just send the data to the cloud” raise your hand.  Take them to task over it.  We build software, and while that seems like magic to some, we don’t do miracles.