New Open Source Projects

I’m working on a new application that’s going to be open source.  Once it’s a bit further along I’ll provide more details, but essentially it’s a paperless forms solution that encompasses a WPF desktop app, a self-hosted ASP.NET WebAPI service and a Xamarin Forms mobile app all working together.

In the process of getting the foundation up and running, I’m moving a lot of stuff over to Github, but more importantly I’m updating, extending, and creating whole new libraries that can be used for other stuff.  These, too, are open source.

What’s being updated?

    I’m primamrily working with SQLite, but I’ve already uncovered and fixed some issues around one-to-many entity relationships
  • OpenNETCF.IoC
    Work and updates to make it more Xamarin/PCL friendly
  • OpenNETCF Extensions
    This thing has been stable and in heavy production use for years.  Support continues for it with some minor changes and improvements in the Validation namespace so far.

What’s New?

  • OpenNETCF.Xamarin.Forms
    Adding things that I’m finding missing or hard in Xamarin’s BCL.  Things like “hey, scale this label to the right size depending on my screen resolution, on any platform”.  Thanks go out to Peter Foot for pointing me in the right direction there.
    No idea why Xamarin didn’t provide a basic navigation framework.  I created one.  Is it awesome?  I don’t know – but it works for my use case (plus 3 other apps I’ve done with it).
  • OpenNETCF.Google.Analytics (moving to it’s own repo soon)
    Again, seems like low-hanging fruit here.  Why isn’t it in the box?  I don’t know, but there’s now a simple, open source library for it.

One note – these are all in active development, so don’t expect a NuGet package updates on them for at least a little while (end of May?).  I’d like to get features in and stable before rolling them out.

Feedback welcome. Testers wanted.  Enjoy.

ORM Migration to Github

With the announced mothballing of Codeplex, I’m working to migrate at least some of the open source projects I have to Github.

It turns out I own 29 projects over on Codeplex. Some of them, like the barcode decoding library, were simply learning exercises.  Some of them were idea that I never found time to actually work on.  Many are for technologies that are now dead (I’m looking at you Compact Framework!). But some are actually useful and I still use quite regularly.

As I pull them into Github, I’m also taking the time to merge in local forks I have from doing other projects as well as doing NuGet releases.  Turns out that this is a fairly large undertaking, but it’s forcing me to do some cleanup that’s long overdue.

So yesterday I pulled over the OpenNETCF ORM.  For those who don’t know, it’s a lightweight ORM (way, way, way lighter than Entity Framework) that supports code first, data first *and* I don’t know my data structure until run time, which no other ORM I’ve seen supports.  It has support for MySQL, SQLite, SQL Server, SQL Compact, Oracle, Azure Table Service and Dream Factory out of the box.  Adding new implementations, especially anything that had an ADO provider already, is really easy.

So, if you’re a current ORM user, or looking for a simple ORM for a .NET project, take a look.

Another ORM Update

One of the laws of software development is “A better way of finding bugs than testing is to release” and sure enough, that happened.  I rolled a release of the ORM yesterday, and no sooner than I did, someone hit a bug with the new connection pooling in the Compact Framework.  It turns out that the SqlCeConnection exposes a Disposed event in the CF, but that event never fires.  The connection pooling implementation relied on that event to know when to remove connections from the pool, and voila – a bug is born.  Is it a bug in the ORM, or a bug in the SqlCeConnection?  I’d say that’s debatable both ways, but since I only have control over one of them I made the fix where I could.

There’s a workaround checked into the source tree over on Codeplex (change set 107904).  This fix is *not* yet rolled into a release package (sure would be nice if that were automated).

ORM Updates

Generally speaking, the OpenNETCF ORM library gets worked on offline.  We use it extensively in our Solution Engine product and so the “working” copy of the code resides on a separate source tree.  The good part of this is that it’s always getting tested, extended and fixed.  The down side is that I have to manually merge those changes back to the public source tree so it doesn’t happen probably as often as it should.

At any rate, I did a large merge this morning and there are some cool new features that have been added, along with a long list of bug fixes.

The general bug fixes and enhancements are largely SQL Server related, since we did a large SQL Server push for a customer this year, and the ORM benefitted from that.

The new features are twofold:

1. I added some rudimentary Connection Pool support.  The ORM itself can now keep a pool of connections to your target data store and it will re-use them in a round-robin fashion.  It’s nothing overly complex, but if you’ve got a few different threads doing different work on the database, it improves performance.

2. I added what I call a “Recovery Service” because I couldn’t come up with a better name.  This doesn’t restore data, but instead it attempt to recover from failures when it can. It’s really useful for remotely connected data stores.  For example, if you are using a SQL Server or DreamFactory DSP and the server or network goes down, your data actions are going to fail and you have to add in a load of handling logic for that.  The Recovery Service is designed to help you out here.  It will look for those types of failure and cache your operations until the store comes back online, at which time it will then apply the changes.  Sort of a “store-and-forward” type of capability, but at the command level and in RAM.

The new release is over on CodePlex in source and binary format.

Row filtering in the ORM

For a while now we’ve had an unwanted behavior in our Solution Engine product. The larger the on-device database got, the longer the app took to load. To the point that some devices in the field were taking nearly 5 minutes to boot (up from roughly 1 minute under normal circumstances). This morning we decided to go figure out what was causing it.

First, we pulled a database file from a device that is slow to boot. It turns out that the database was largely empty except to about 50k rows in a log table where we record general boot/run information on the device for diagnostics.

At startup the logging service pulls the last hour of log information and outputs it to the console, which has proven to be very helpful in diagnosing bad behaviors and crashes. Looking at the code that gets that last hour of data, we saw the following:

var lastHourEntries = m_store.Select<SFTraceEntry>(a => a.TimeStamp >= selectFromTime);

Now let’s look at this call in the context of having 50k rows in the table. What it effectively says is “Retrieve every row from the SFTraceEntry Table, hydrate a SFTraceEntry class for each row, then walk through that entire list checking the TimeStamp field. If the TimeStamp is less that an hour old, then copy that item to a new list and when you’re done, return the filtered list.” Ouch. This falls into the category of “using a tool wrong”. The ORM supports FilterConditions that, depending on the backing store, will attempt to decipher into a SQL statement, index walk or something more efficient than “return all rows”. In this case, the change was as simple as this:

var dateFilter = new FilterCondition("TimeStamp", selectFromTime, FilterCondition.FilterOperator.GreaterThan);
var lastHourEntries = m_store.Select<SFTraceEntry>(new FilterCondition[] { dateFilter });

OpenNETCF ORM Updates: Dream Factory and Azure Tables

We’ve been busy lately.  Very, very busy with lots of IoT work.  A significant amount of that work has been using the Dream Factory DSP for cloud storage, and as such we’ve done a lot of work to make the Dream Factory implementation of the OpenNETCF ORM more solid and reliable (as well as a pretty robust, stand-along .NET SDK for the Dream Factory DSP as a byproduct) .  It also shook out a few more bugs and added a few more features to the ORM core itself.

I’ve pushed a set of code updates (though not an official release yet) up to the ORM Codeplex project that includes these changes, plus an older Azure Table Service implementation I had been working on a while back in case anyone is interested and wanted to play with it, use it or extend it.  The interesting thing about the Azure implementation is that it includes an Azure Table Service SDK that is Compact Framework-compatible.

As always, feel free to provide feedback, suggestions, patches or whatever over on the project site.

New ORM Release: v1.0.14007

I’ve finally gotten around to wrapping up all of the changes I’ve made in the last year (has it really been that long since the last release?) to the OpenNETCF ORM library.  The changes have always been availble in the change set browser, but I actually have them as binary and source downloads now.  I probably should find the time to create a NuGet package for it (and IoC) now.