Padarn: Dynamic Authentication

Our Padarn Web Server largely uses a subset of the IIS object model, so for many features and capabilities how you do things in IIS is paralleled in Padarn. Authentication follows that rule, but with a couple exceptions.

For good old-fashioned, built-into-the-browser Basic or Digest authentication it follows the same steps. Set the Authentication in your app.config file and the user will get a browser-provided authentication popup.

But what if we want to do authentication ourselves against our own user store? In this case Padarn has a few “custom” extensions to make things easier.

The first mechanism is to add users to the underlying user cache when you start up the Padarn Server. Something along these lines:

var server = new WebServer();
server.Configuration.Authentication.Users.Add(
    new OpenNETCF.Web.Configuration.User { Name = "username", Password = "password" }
);

You could also hard-code the user identities right into the app.config file.

This isn’t all that friendly, though – at least in my thinking. Another, more robust, way is to use Padarn’s AuthenticationCallback. This allows you to send a delegate into Padarn and have it call your customer authentication algorithm every time it needs to do authentication. It’s slightly more complex, but really it’s still not too terrible, and it supports Basic or Digest authentication.

// somewhere in startup
var server = new WebServer();
server.Configuration.Authentication.AuthenticationCallback = VerifyUsername;

...
// then somewhere else - in this case in the same class
bool VerifyUsername(IAuthenticationCallbackInfo info)
{
    // no username is invalid
    if (string.IsNullOrEmpty(info.UserName)) return false;

    // first do a lookup of the password - this might come from a database, file, etc
    string password = GetPasswordForUser(info.UserName);
    if (password == null) return false;

    // determine the authentication type
    var basic = info as BasicAuthInfo;
    if (basic != null)
    {
        // we're using basic auth
        return (basic.Password == password);
    }

    // it wasn't basic, so it must be digest (the only two supported options)
    var digest = info as DigestAuthInfo;
    // let the DigestAuthInfo check the password for us (it's hashed)
    return digest.MatchCredentials(password);
}

Uploading Large Files to a Padarn Server

About a year ago I posted some code that showed how to use our Padarn Web Server to serve up large files from a device to a connected client browser. Late last week I got a request for the corollary operation – how do I send large file from a client browser to my Padarn server?  It’s pretty straightforward. First, you need to render a simple page that allows the user to select a file.  For simplicity I’ll show an HTML version, though you could just as easily use jQuery or whatever you like.

protected override void Render(HtmlTextWriter writer)
{
    writer
        .Html()
            .Body()
                .Form(t => t
                    [HtmlTextWriterAttribute.Id, "Form"]
                    // submit back to ourselves
                    ["action", string.Empty]
                    // use a POST method on submit
                    ["method", "post"]
                    // set the POST data mime encoding for a file
                    ["enctype", "multipart/form-data"]
                    )

                    .P()
                        .Text("File:")

                        // render a file input button
                        // this allows the user to browse for a file
                        .Tag(HtmlTextWriterTag.Input, t => t
                            [HtmlTextWriterAttribute.Type, "file"]
                            [HtmlTextWriterAttribute.Size, "80"]
                            [HtmlTextWriterAttribute.Id, "filename"]
                            [HtmlTextWriterAttribute.Name, "filename"])
                        .EndTag() // input

                    .EndTag() // p

                    // render a submit button
                    // when clicked it will send the file
                    .Tag(HtmlTextWriterTag.Input, t => t
                        [HtmlTextWriterAttribute.Type, "submit"]
                        [HtmlTextWriterAttribute.Id, "upload"]
                        [HtmlTextWriterAttribute.Name, "upload"]
                        [HtmlTextWriterAttribute.Value, "Upload File"])
                    .EndTag()

                .EndTag() // form
            .EndTag() // body
        .EndTag(); // html

}

This will get a page that looks like this (yes, it’s a bit ugly without CSS): Now we simply need to handle incoming file data. For simplicity I have the Form above just posting back to itself, so in the same page, in Page_Load, I handle incoming data:

Side Note: WordPress isn’t properly showing my Media Uploads, so if it’s  broken image below, I know about it and am working to resolve it

padarn_upload

protected void Page_Load(object sender, EventArgs e)
{
    if (Request.Files.Count > 0)
    {
        // NOTE: You may have to change the config file to allow large 
        // files. The maxRequestLength is in KB, so make sure it is in 
        // line with expectations.
        // For example, this allows up to 300MB files to be uploaded
        //    <httpRuntime
        //      maxRequestLength="300000"
        //      requestLengthDiskThreshold="256"
        //    />

        // this might take a while, set the session.timeout to 120 minutes
        var origTimeout = Session.Timeout;
        Session.Timeout = 120; // 120 minutes;

        string error = "";
        // file upload
        try
        {
            // make sure the target local folder exists
            var path = Path.GetFullPath(".\uploads");
            if(!Directory.Exists(path))
            {
                Directory.CreateDirectory(path);
            }

            // now save off the files
            for(int i = 0 ; i < Request.Files.Count ; i++)
            {
                path = Path.Combine(path, Request.Files[i].FileName);

                Request.Files[i].SaveAs(path);
            }
        }
        catch (Exception ex)
        {
            Debug.WriteLine(ex.Message);
            error = ex.Message;
        }

        // put the timeout back to what it was
        Session.Timeout = origTimeout;
    }
}

Sending large files to a client in Padarn

A customer recently contacted me with a Padarn Web Server problem.  They wanted users to be able to download data files from their device to a client browser, but they were running out of device memory when the file got large (38MB in their case, but the size would be completely hardware and environment dependent).

What you have to understand is that Padarn caches your Page’s response in memory until such time as you either tell it to Flush explicitly, or until the Page has full rendered (in Padarn’s view) at which point it will call Flush for you.

For a large file, it’s going to be bad to try to hold the entire thing in memory before pushing it across the wire, so the strategy would be to read the local file in packets, then send those packets across to the client using Response.BinaryWrite.

Now BinaryWrite simply puts the data into the memory cache, so you have to actually tell Padarn to send it across the wire by calling Flush.  The issue there, though, is that on the first Flush, Padarn has to assemble and send the payload header to the client, and part of that header is the total length of the content.  If you haven’t explicitly set the content length, Padarn has no idea how big it might be, so it defaults to the length what it has in the buffer, and sends the header.  The browser then decodes that header and expects a length of only that first packet.

The solution is to set the Response.ContentLength property before you send any data.  Padarn will use your override value when it sends it to the client, and all will be well.

Here’s a quick example of a page that I used to send a 1.1GB media file from a page request with no problem.  You might want to tune the packet size based on device memory and speed, but 64k worked pretty well for my test.

class LargeFileDownload : Page
{
    protected override void Render(HtmlTextWriter writer)
    {
        var largeFilePath = @"D:MediaMoviesshort_circuit.mp4";

        var packetSize = 0x10000; // 64k packets

        using(var stream = new FileStream(
            largeFilePath,
            FileMode.Open,
            FileAccess.Read))
        {
            var toSend = stream.Length;
            var packet = new byte[packetSize];

            // set the response content length
            Response.ContentLength = toSend;
            // set the content type as a binary stream
            Response.ContentType = "application/octet-stream";
            // set the filename so the browser doesn't try to render it
            Response.AppendHeader("Content-Disposition",
                string.Format("attachment; filename={0}",
                Path.GetFileName(largeFilePath)));

            // send the content in packets
            while (toSend > 0)
            {
                var actual = stream.Read(packet, 0, packetSize);

                if (!Response.IsClientConnected)
                {
                    Debug.WriteLine(
                        "Client disconnected.  Aborting file send.");
                    return;
                }
                else
                {
                    Debug.WriteLine(
                        string.Format("Sending {0} bytes. {1} to go.",
                        actual, toSend));
                }

                if (actual == packetSize)
                {
                    Response.BinaryWrite(packet);
                }
                else
                {
                    // partial packet, so crop
                    var last = new byte[actual];
                    Array.Copy(packet, last, actual);
                    Response.BinaryWrite(packet);
                }

                // send the packet acrtoss the wire
                Response.Flush();

                toSend -= actual;
            }
        }
    }
}

On Software Development

This post is part of my “Software Development” series.  The TOC for the entire series can be found here.






Developing good software is hard.  Really hard.  Sure, anyone can buy a book on writing software or pull up some code samples and get something that compiles and runs, but that’s not’s really developing software.  A lot of code in the wild – I’d bet a vast majority of it – just plain sucks.


It’s hard to point out where the blame lies.  It seems that most developers are environmentally or institutionally destined to write bad code. Schools teach how to write code, but not how to architect it or to follow reasonable design practices.  In the zeal for clarity, publishers churn out books, blogs and samples that show bad practices (when is it ever a good idea to access a data model from your UI event handler?).  Managers and customers alike push hard to get things done now, not necessarily done right – only to find months or years later that doing it right would have saved a boatload of time and money.  And let’s face it – many developers are simply showing up to work to pull a paycheck.  You know who they are.  You’ve no doubt worked with them in the past.  You’re probably working with them now.


I was watching Gordon Ramsay the other day and it occurred to me that he and I are alike in our own peculiar way.  I’m not saying that I see myself as the “Gordon Ramsay of Software Development” – hardly –   but we share a common trait.  Just as Gordon gets angry and starts spewing colorful language when he walks into a crap kitchen, it bothers the hell out of me to see complete idiots in my chosen field out there just making a mess of things.  When I see bad code – not necessarily minor errors, or code that could be refactored and made better – but just outright shit code that should not have occurred to a developer in the first place it pisses me off.  By the nature of my work, often getting called in only when the project is off the rails, I see it all the time. Code that, on review, a peer or mentor should have seen and said “Whoa!  There’s no way that’s going into our code base”.  Code that just makes it harder for the next person to do their job.


In an effort to simplify things for my own code, for my customers’ code as well as anyone who is willing to listen to my ravings, I’ve spent a lot of time building, testing, fixing and extending tools and frameworks that many of which I turn around and give away.  This isn’t out of altruism, no, it’s largely because I’m a lazy developer.  I hate writing the same thing twice.  When I start a project, I don’t want to spend large amounts of time building up the same infrastructure that every project needs. Building up a framework for handling UI navigation isn’t what I’d call interesting, but just about every project needs it.  Handling object dependencies and events is common.  Writing a DAL for serializing and deserializing entities is not just drudgery, I find it’s highly susceptible to errors because you end up doing a lot of copy and paste.


I have these cool tools and frameworks that I use in literally every project I work on now.  That’s great for me, but it doesn’t really help others, right?  Without reasonable documentation or explanation, only a small handful of people are going to go through the effort of getting the tools and trying to understand them – even if they are deceptively simple and could potentially save you weeks of effort. 


So I’ve decided to put together a series of blogs over the coming weeks and months that explain, hopefully in simple terms, what these frameworks do, how to use them, and most importantly, why they are worth using.  There’s nothing groundbreaking here.  I didn’t invent some new way to do things.  I’ve simply appropriated other peoples’ ideas and extended them to work in the environments that I work.


Generally I’ll be covering the following topics and frameworks:



  • Dependency Injection and Inversion of Control (using OpenNETCF IoC)
  • Event Aggregation (using OpenNETCF IoC)
  • Plug-in Architectures and interface-based programming (using OpenNETCF IoC)
  • Software features as services (using OpenNETCF IoC)
  • Data Access through an ORM (using OpenNETCF ORM)
  • Parameter Checking (using OpenNETCF Extensions)
  • Exposing data services over HTTP (using Padarn)
  • Whatever else I think of

If there’s a topic you’d like me to talk about, feel free to send me an email.  I may turn on comments here and let you post ideas, but I find that when I enable comments on my blog, I start getting more comment spam than I really want to deal with, so if comments are turned off just drop me a line.

Padarn for XPe/Desktop Windows


IIS is a robust web server.  I’d never try to argue that it isn’t or that Padarn could in any way try to replace it.  There are instances, however, where IIS just isn’t a very good fit, and Padarn really is.  Some examples might be:



  • You want a simple web-based configuration interface for an embedded device
  • You want to expose REST APIs from your device in a small footprint
  • You want to expose data from your device but don’t want users to have to fight configuring IIS
  • You actually have a size limitation and IIS makes it tough to achieve
  • You’d like your WebRequests to not all be in their own AppDomain
  • You’d like to share code assets between Windows CE and Embedded Standard/XPe

Padarn fits all of these scenarios well, and we continue to improve both performance and feature sets.


We just published a set of evaluation binaries compatible with desktop OSes (XP, Vista and Windows 7).  Make sure you’re registered and logged in to see the downloads.

Improved Rendering in Padarn

Every time we start a new project using our Padarn Web Server, I always like to see if we can improve the server’s capabilities to make our job a little easier.  On the last project that turned into adding custom IHttpHandler support so we could easily serve up REST-based web services.


Well we started a new project last week, and unlike the last one, this one is UI-focused (the last one used Padarn strictly for services).  With that in mind, I went about looking at how to improve and simply building pages.


Of course I came back to our lack of support for server controls.  I did a little more playing and decided once again that it’s a gigantic amount of work and it requires a boatload of RegEx and string manipulation – not necessarily a recipe for success on a low-memory, slow-CPU Windows CE device.


Still, there had to be other thing that full-blown ASP.NET offers that would be reasonable to add and that would help.  The existing samples we’d created in the past used overrode Page_Load method and then went about doing Response.Writes to emit page code.  The first time I did it I found it painful, so I created a class library that had objects for a lot of the basic HTML constructs and used that to at least help in generating well-formed code.


Still, I didn’t really like the object model – it was still way too verbose – and it wasn’t really like what anyone does on the desktop.  Additionally, rending in Page_Load really didn’t feel right.


So I started looking around and found that the ASP.NET Page class provides a Render method.  That sounds a lot more like where I should be, oh maybe *rendering* a page.  Of course it required implementing the HtmlTextWriter, but that wasn’t too terrible.


Once I implemented the Render method and started using the HtmlTextWriter, I quickly concluded that it, too, really sucked for generating page code.  It’s just crazy verbose, it doesn’t read anything like the page you’re attempting to render and it’s got things that just feel plain wrong (like having to add attributes before the tag they apply to? WTF is that?).


So I started looking at how ASP.NET developers who render their own controls with this monstrosity got around in.  It turns out that there is a pretty nice fluent interface to the HtmlTextWriter available on CodePlex.  It was missing a lot of stuff that I needed (and had a load of non-HtmlTextWriter stuff I didn’t need), but it was a great start.  So after expanding the object model there I can now render a page in something that doesn’t give me quite so much heartburn.


What originally looked like this:


protected override void Page_Load(object sender, EventArgs e)
{
  Response.Write(DoctypeTags.XHTML1);

  Response.Write(“<html xmlns=”http://www.w3.org/1999/xhtml”>&#8221;);
  Response.Write(“<head>”);
  Response.Write(“<link rel=”stylesheet” type=”text/css” href=”css/mystyle.css” />”);
  Response.Write(“</head>”);

  Response.Write(“<body>”);

  Response.Write(“<h1>”);
  Response.Write(“Hello World!”);
  Response.Write(“</h1>”);

  Response.Write(“</body>”);
  Response.Write(“</html>”);
  Response.Flush();
}


Ends up looking more like this:


protected override void Render2(HtmlTextWriter writer)
{
  writer.Write(DoctypeTags.XHTML1);

  writer
    .Tag(HtmlTextWriterTag.Html, t => t[“xmlns”, http://www.w3.org/1999/xhtml&#8221;])

      .Tag(HtmlTextWriterTag.Head)
        .StylesheetLink(“/css/mystyle.css”)
      .EndTag() // head

      .Tag(HtmlTextWriterTag.Body)
        .H1(“Hello World!”)
      .EndTag() // body
    .EndTag(); // html
}


And it gets much cleaner looking the larger the page is.  I also find I’m doing a lot less searching for where I malformed my HTML.


I think the next Item I’ll look into is the __doPostBack Javascript method that IIS injects into pretty much every page.  It’s probably the next incremental step toward better productivity.

Dynamic Authentication with Padarn

I’ve just rolled a new Padarn release – version 1.4.10126 – to add a new Authentication feature.  In previous versions of Padarn, you had to put your username/password pairs into the WebServer’s configuration file.  This isn’t too useful if you want to do run-time checking of credentials against something like a database.


Currently Padarn using what would be an analog to IIS authentication – that is to say that the user credentials are checked before any page code is run.  This means that, for now anyway, Forms authentication is not an option.  Instead I opted to allow you to set up the server to give you a callback whenever an authentication request occurs whether you’re using Basic or Digest authentication.  For simplicity, the simple act of setting the callback tells the server to no longer look in the user list in the config file.


Setting up for dynamic authentication looks something like this.  First, you create your server instance and then simply set the authentication callback:


m_ws = new WebServer();
m_ws.Configuration.Authentication.AuthenticationCallback = VerifyUsername;


And then you might define you callback like this (note that I’ve desinged it to handle both Basic and Digest – you probably would pick just one):


bool VerifyUsername(IAuthenticationCallbackInfo info)
{
  // no username is invalid
  if (string.IsNullOrEmpty(info.UserName)) return false;

  // first do a lookup of the password – this might come from a database, file, etc
  string password = GetPasswordForUser(info.UserName);
  if (password == null) return false;

  // determine the type
  BasicAuthInfo basic = info as BasicAuthInfo;
  if (basic != null)
  {
    // we’re using basic auth
    return (basic.Password == password);
  }

  // it wasn’t basic, so it must be digest
  DigestAuthInfo digest = info as DigestAuthInfo;
  return digest.MatchCredentials(password);
}


The GetPasswordForUser function would look up a password given a username.