OpenNETCF MQTT Updated to .NET Standard

I’ve refactored the OpenNETCF MQTT library, which is a simple MQTT Client, to ne a .NET Standard 1.3 Library.  I also added a few async methods to bring it a little more up-to-date.

Source is in GitHub
Library is available via NuGet (as package ID ‘OpenNETCF.MQTT’)

Here’s a test method that generally shows how it can be used:

    public void ClientReceiveTest()
        var receivedMessage = false;

        // create the client
        var client = new MQTTClient("", 1883);

        // hook up the MessageReceived event with a handler
        client.MessageReceived += (topic, qos, payload) =>
            Debug.WriteLine("RX: " + topic);
            receivedMessage = true;

        var i = 0;
        // connect to the MQTT server
        // wait for the connection to complete
        while (!client.IsConnected)

            if (i++ > 10) Assert.Fail();


        // add a subscription
        client.Subscriptions.Add(new Subscription("OpenNETCF/#"));

        i = 0;
        while (true)
            if (receivedMessage) break;


            // publish on our own subscribed topic to see if we hear what we send
            client.Publish("OpenNETCF/Test", "Hello", QoS.FireAndForget, false);

            if (i++ > 10) break;


Using iBeacons and Eddystone in Xamarin Forms

I’m working on a project where I need to get temperature from a wireless sensor into my phone app.  The exact sensor doesn’t matter – it just needs to be inexpensive.  Decent battery life would also be nice.  And so began yet another typical “searching for a sensor” trip for me.

After a little research, I ordered a SensePeanut.  Looked great.  Decent prototype cost. Available off-the-shelf and in a package.  Just what I need!  Except I got it in hand only to find out that you can get data from it only using their app, which they don’t give out the source for.  Soon they’ll have a “cloud SDK” that will let me get the data, but for a whole host of reasons, that just isn’t an option.

So I did a bit more research.  I needed a sensor that provided a “direct to the sensor” SDK.  I ended up ordering some proximity beacons from Estimote.  Proximity isn’t what I’m after, but they also can transmit temperature so it seemed promising.

Well last weekend I finally got time to start writing code.  Turns out that the SDKs they provide are, at least as far as I can tell, native binary SDKs for iOS and Android.  I’m a C# dev this week and I really want this to be cross platform, but they don’t have a Xamarin SDK.  Someone at Xamarin did create an SDK for the devices a few years back, but that project doesn’t actually work with the latest Android SDK – discovery returns nothing.  Evidently that’s not uncommon when using a library built against an older Android SDK.

Well it’s turns out that the existing Xamarin library is really just a bindings library that wraps the Java SDK.  So let me get the latest Estimote SDK and recompile.  Well, evidently the Estimote SDK is now deployed through JCenter (think NuGet for Java).  That appears to require that I install Java tools and I really have no desire to pollute my working machine with a toolchain I don’t intend to use.  No idea what can of worms that might open.

They don’t appear to have an AAR download, nor do they have the source for the SDK available – only source for some sample apps.

A little reverse-engineering with JCenter, however, let me to find the URL to direct-download the AAR.  I pulled it down and created a Xamarin Bindings Library project targeting it.  This is my first attempt at a Binding Library, but how hard could it be, right?  Turns out that the project generates a load of error, and this is not at all uncommon for a Bindings Library. Auto-generating wrapper code is hard, I realize that.  But I really don’t have the desire, or especially the time at this point, to climb the learning curve for Bindings Libraries, especially since I just want to read a simple temperature.

Maybe the existing Xamarin project can be used against the new Estimote AAR?  Surely the original dev had to solve these errors, right?  I loaded up the existing project and stuffed in the newer AAR.  Build and…still a bunch of errors.  Admittedly only about half the errors than the new project had, so it’s progress, but still isn’t getting me data.

I then spent a few hours reading and working on the project.  It’s a steep learning curve with not a whole lot of “Bindings-Libraries-for-Dummies” tutorials out there.  I got rid of maybe half the errors (the easy ones) and then got stuck.

Time to pivot again.  There’s got to be a way to do this.  Back to researching.  I found a Windows 10 C# library that can read data from iBeacons and Eddystone.  The Estimote can be configured to transmit the temperature in an Eddystone telemetry packet so that seems at least somewhat promising.  At least I have full source for the library.

I ran the sample app on my laptop and, lo-and-behold, I see temperature data which is hugely promising. Again, it’s a Windows 10 library but I’ve been making a career out of making desktop code mobile for a long time so it’s certainly in my wheelhouse.

First task was to port the library to .NET Standard so I’d have some expectation I’d be able to use it over on Android and iOS.  That required extracting the actual Bluetooth packet discovery out to an interface that was platform-specific.  The starting code was pretty well thought out and constructed so the task was not too hard and I actually had a UWP app receiving and displaying the data in just a couple hours.

The next job was to create the Android implementation of the Bluetooth discovery bits.  The UWP platform provided the Bluetooth data in a really nice format.  I’m sure it looks that way since the starting code I was working with was built specifically for it.  Fitting the reality of the packets that we get in Android into that structure wasn’t straightforward, but since Bluetooth and Eddystone are well documented it also wasn’t overly difficult.  It took me about a day of working with it to understand the spec well enough to get the incoming packets integrated into library.

I got a hold of Andreas Jakl, the original library author, let him know about my fork, and requested collaborator status.  He was gracious and added me to the project so my changes have all be back-integrated into the original tree, as well as fixes to the original sample non-Xamarin UWP app to use the new libraries.  We’re in teh process of getting the new stuff out as NuGet packages.

Getting BLE temperature data into my app certainly wasn’t as straightforward as I had hoped when I first ordered a sensor.  In fact, the journey to get there was nothing like what I’d hoped, or even expected once I started trying to write code for it.  In the end, however, I got it working (well in UWP and Android anyway, iOS will have to wait until I start porting the app that started this whole thing) and the work is public and open source.  I still have to manage the beacons with vendor-specific apps, which still irritates me, but at least we all now have a generic way to receive data from BLE beacons in our .NET apps.


Unit Testing REST Clients

I frequently seem to end up writing code that calls REST services – on pretty much every project I work on anymore.  Since I seem to be often writing the same code over and over, I’m working on a library (well a couple libraries) to simplify things and consolidate all that I typically come across.

A challenge in writing a REST client is how to easily unit test it.  I don’t want to have to stand up a server somewhere just to run simple tests, especially if the server needs to return valid and invalid data that needs to change with the test.

So I create a really simple, yet really useful class I’m calling SimpleServer and it’s available as a gist.  It basically allows you to define a URI and then set callback methods for the common HTTP verbs (GET, PUT, POST and DELETE).

Let’s say you want to test out a GET operation.  In the example below the RestConnector encapsulates the goo required to make the HTTP call, but the cool part is how it uses the SimpleServer to test an expected response to a GET at the API root.

        public void GetNoCreds()
            var uri = "http://localhost:8080/";
            var response = "Hello!";

            using (var svr = new SimpleServer(uri, (request) =>
                return response;

                using (var rc = new RestConnector(uri))
                    var result = rc.GetString("/");

                    Assert.AreEqual(response, result);

Here I’m using the default handler parameter for a GET, but you can individually pass in Func’s for GET, PUT, POST and DELETE which makes testing even fairly complex scenarios both quick and really, really easy.

Should You Develop Software for Free?

I just read an excellent opinion piece over on the New York Times site. I’d recommend you read it, but if you’ve not got the time, or are generally just too lazy, let me summarize. The author is, not unexpected, a writer. Often he is asked to do writing or give speeches and offered $0.00 in return. Basically people want him to give away the output of his “craft” yet those same people would never ask or expect “a keychain or a Twizzler” for free. What gives? How do you tell them to pound sand, but in a polite way? And should you?

Well it’s not so greatly different here in the computer world. Many people feel software should be free. After all, it’s just electrons, and I can shuffle my feet across the carpet and get electron for free, why should you buy electrons from someone else? In fact the idea is so popular that there’s an entire sect that firmly believes that all software should be free. Amazingly, these people even tend to be software developers.

Yes, I’m somewhat of a contributor to this. I have written a boatload of software that I give away. I even encourage people to go get for free. But let’s be clear, that software was paid for. It was developed over many years and many projects. The customers whom I was working for at the time paid for me to solve their problem, and part of that solution involved adding to or fixing some of those core libraries. Still, it was paid for. I simply reserved a small portion of the work as a part of a “free” library that carried over to the next project.

And here’s another dirty little secret. I didn’t do it out of altruism. No, I didn’t do it for “exposure”. That’s a line that someone trying to wheedle free stuff from you would use. I kept those libraries “free” so that a customer can’t say “hey, you did that work on my dime, I own it!” No, sir, you own the solution to your problem part of which uses a set of free libraries.  You’ve gained from work done for countless other projects in the past. Those libraries allowed me to solve you problem much faster than if I had to start them from scratch. So the reason I provide free software is to save me from doing repetitive work and to, generally, keep a set of base libraries that I’m free to use anywhere, on any project.

So why do I also give those libraries away? I don’t know, maybe I am an altruist. Maybe I have the delusion that some day when I’m called in to fix a broken project they’ll already be using my libraries and it will save me the headache and frustration of coming up to speed. Maybe I just hate seeing people reinvent the wheel. Maybe I’m just crazy. Maybe it’s a bit of all of the above, but let’s be clear: I don’t do work for free.

I don’t feel that anyone has a right to ask for software that is not paid for. You may be able to sway me somewhat with the argument about you being free to run and modify the software you paid for, but you’ll have a harder time on the copy and distribute side of the argument.  We, as software developers, still have to pay our bills.  We’ve spent years honing our skills to be able to solve these problems (admittedly, some have done a better job of honing than others) and it takes actual time and work solving problems.  Sometimes I work on problems or with customers where it, most decidedly, is work.  Even drudgery.  So no, I’m not a believer that software, music, writing, photographs or really any other form of “art” should be free.  You wouldn’t expect an electrician to work for free, don’t expect it of a “content creator” either.

Stop Using “The Cloud”

We’ve all been in a meeting, conference or seminar where the talk goes something along these lines:

“We collect the data at the device, then we send it up to The Cloud where we can do analysis and other things on it.”

This is usually accompanies by the vague graphic of a line – sometimes solid, sometimes dotted – pointing to the generic PowerPoint cloud image.

I hate this statement.  I hate this cloud.  This is usually a huge red flag that the speaker or author really has no idea what they’re talking about.  It a sales or marketing cop-out that is the technical equivalent of the popular “then a miracle happens” cartoon.

What does “send it to The Cloud” even mean?  What cloud?  How is the data going to get there?  How will I get it back out? I’ve got other questions about “the cloud” that I’ll address in another post, but these three are the biggies that you should always raise your hand and ask.

1. What is “The Cloud?”

Here’s the little secret – there is no single “cloud”.  There are a boatload of them, and the list seems to be continually growing as it gets cheaper to stand up servers with a lot of storage and as more and more use-cases make use of off-device storage.  “The Cloud” can mean a whole panoply of things, including (but definitely not limited to):

Again there are plenty of others, these are just most of the ones I’ve dealt with in the past year or so.

And bear in mind that not all customers are going to be amenable to all clouds.  Some customers aren’t so comfortable putting their data on servers they don’t control.  Some aren’t comfortable putting their data on servers in particular countries where governmental agencies are known to mine them.  Some customers simply have predispositions to different services for different reasons.

Maybe they like Azure because it provides a simple interface for their .NET enterprise apps.  Maybe they like Amazon’s scale.  Maybe they like Brand X just because they do.  The point is that if you have more than one customer, you’re probably going to need to look at more than one cloud provider.  We’ve got customers that use multiple clouds due to the benefits each provides for certain types of data, data retention policies or total cost of use.

2. How do I get my data into “The Cloud”?

Yeeaaahhh, about that…since there are a boatload of cloud services, there are a boatload of ways that an app gets data into them.  Network databases might use ODBC or a proprietary/specific API set.  Services on the web typically use a web service interface.  Maybe REST, maybe OData, maybe something else.  The point here is none of them are the same.  None of them.

So that means that if you have to support multiple clouds, and you will have to support multiple API sets.  Of course you can abstract them all back to a common interface – I’d certainly recommend it, it’s what we do using the OpenNETCF ORM – but there’s still work to be done to actually implement each.

3. How do I get my data back out of “The Cloud”?

Just like putting the data in, getting it out requires an API, and again, they are all different.  Another thing to consider in the “getting it out” part of the equation is how you actually use it.

Some clouds allow you to run services and/or applications on the servers as well and data access is direct.  Sometimes you have to pull it out back to another server.  Once again, this means more work from a development perspective.  And again, you’ve got to multiply that by the clouds you need to support.

And how about data retention? Some clouds are not “forever” storage.  The data gets purged on a time basis.  If you need to keep the data archived for later data mining then add that work onto your plate too.

So the next time you see that slide with the cloud on it and the speaker says “just send the data to the cloud” raise your hand.  Take them to task over it.  We build software, and while that seems like magic to some, we don’t do miracles.