I just finished upgrading the last of our computers at home to Windows 7 and now that I have Windows Live Writer installed I figured I’d do my yearly retrospective - see what’s gone on this year and recap.

In January I found that they were ending Google Notebook so I had to move all of my notes over to PBworks. That was sort of painful, but I’m really enjoying PBworks now and I have a ton of stuff in there. I also released version 2.2.1.0 of CR_Documentor, a bug fix release.

In February I showed you how to upgrade your Windows Home Server capacity with an eSATA port replicator and upgraded my main TV to a 52” Samsung LN52A750 (still great, and still recommended).

March was pretty eventful. I upgraded my blog to Subtest 2.1.0.5. I reflected a bit on why it’s a good idea to keep a cool head in a tough situation. I ran into some User Account Control issues in Windows Server 2008 and wrote about some power toys that will help you out with that as well as providing my own Visual Studio related elevation toy. I went to MIX09 and blogged all three days (1, 2, 3). My MIX trip then spun off some travel luggage recommendations.

At the beginning of April I found that most of my network configuration problems were due to Verizon updating my FiOS router automatically, requiring me to restore my router to factory defaults more than once since then. I found that an HDMI switchbox can help solve issues where your TV loses the HDMI signal when your computer goes to sleep. Jenn and I took a three-day vacation to Vegas, which is always a good time. I released version 2.3.0.1 of CR_Documentor and put out a solicitation for input (with not much response, to be honest). I wrote about some challenges I’ve faced while trying to write multi-tenant ASP.NET apps. I also wrote about some challenges I faced with my Blackberry due to the Facebook application - if you’re having trouble synchronizing, check your default services.

May found me offering some tips on proper use for bullet lists and finishing off the ripping of all of my DVDs. (Did you know an average DVD is 6.7GB?) I wrote a script to automatically set the album artist on your iTunes tracks. I reviewed ASP.NET 3.5 Enterprise Application Development with Visual Studio 2008. I explained how to get iTunes music playing in Windows Media Center, though Windows 7 fixes a lot of the issues there. Plus I showed you how to use the MSBuild engine in your programs and take advantage of the file finding functionality in there.

June started out where I showed you how to use Typemock Isolator to skip generic constructors. I got an HD camcorder and struggled with the file formats. I reflected a bit on blogs that rip off content and publish it as their own. I talked about defragmenting your Windows Home Server drives, then I went to see the B-52s in concert. June ended on a huge down note, though, as my Grandma Jeanne passed away at 86.

July started out with a nifty trick - I showed how to change Windows Service runtime behavior using Typemock Isolator. I also showed you how to get the Windows OS version from inside MSBuild. I talked about the SQL Server installer constantly requiring a reboot (which always seems to be the case for me) and how to fix it. I updated my custom NAnt tasks to .NET 2.0. On a personal note, my birthday was awesome and I went to see Tears for Fears in concert. The biggest thing in July, though, was Jenn and I running our own fireworks shoot in Sandy. Scary and exhilarating.

August started out brilliantly with one of our famous 24 marathons. I discovered the coolness that is Asset UPnP on Windows Home Server and showed you how to back Windows Home Server up to MozyHome. I showed you how to write Firefox extensions using Visual Studio and I even released one of my own - Firefox NTLMAuth, a plugin to help you with Windows pass-through authentication in Firefox.

Jenn and I started September with a trip to Victoria, BC, Canada - good times. I found a little gotcha when using the Windows Vista DVD burner and it tells you there aren’t any files to burn. I updated my iTunes metadata copying program for the latest iTunes. I reviewed Professional ASP.NET MVC 1.0. Jenn and I went to see The Killers in concert (awesome!). I posted a couple of ASP.NET AJAX tips - using ASP.NET AJAX String.format() in jQuery validation and parsing currency values. Finally, I created a quick DXCore plugin that helps other plugin writers explore contexts.

In October I started out with another jQuery/ASP.NET AJAX tip - converting relative to absolute paths. I did a one-year retrospective with my Windows Home Server. I discovered the hugest gotcha with the “COMPLUS_Version” environment variable and the .NET runtime

• I wish that was more clearly documented. And, finally, this year we had 243 trick-or-treaters on Halloween.

In November I had to manually uninstall a Windows Home Server add-in and showed how to upgrade PerfectDisk for Windows Home Server. I ran into a weird edge case with XML serialization while debugging a Visual Studio add-in. I did some work on my Media Center and figured out the Windows 7 supported media formats, how to fix that one-pixel line in Windows Media Center playback, and talked a bit about metadata and artist names on music files. I showed how to create icons for your context menu items in DXCore and how to put your log4net.config outside your app.config/web.config file. November ended, as it always does, with my least favorite holiday ever.

In December I updated my Subtext database maintenance page for Subtext 2.1.0.5, but I did two other programming things that I was more proud of: I released a DXCore plugin, CR_CodeTweet, that lets you tweet code snippets from inside Visual Studio; and I released a Windows Live Writer plugin that lets you upload images to ImageShack. As part of that ImageShack upload plugin, I figured out how to post multipart/form-data using .NET WebRequest.

Overall, the year was decent, but not great. Like everyone else, we’ve had our challenges with the economy. There have been a couple of fairly difficult personal issues to face as well. On the bright side, Jenn and I are both healthy, and we’ve got a nice place to live, we both have jobs, and we’re otherwise doing well, so I can’t say we’re in a bad spot. I’ll be glad to see 2009 past and I look forward to seeing what 2010 holds.

While making my ImageShack plugin for Windows Live Writer I had to figure out how to make a web request that posts data to an endpoint in “multipart/form-data” format. That’s a lot different than URL-encoding everything into a querystring format and sticking that in the body of the POST request. If you’re uploading one or more files, you need to format things differently.

I looked all over for some built-in function that does this, but it doesn’t seem to exist in the .NET base class library. Surprising, but also not surprising. I found a question on StackOverflow that talked about it and an article with some tiny code snippets in it, but that didn’t tell me what was really going on in the request and the code I found was either incomplete, confusing, or both.

So, in the spirit of “Not Invented Here,” I decided to write my own. :)

First was figuring out what the format looked like in the request. I found an article that showed just that. Not a very thorough article, but it got me started. Let me boil it down in terms that [at least I] understand.

• Generate a “boundary.” A boundary is a unique string that serves as a delimiter between each of the form values you’ll be sending in your request. Usually these boundaries look something like

  ---------------------------7d01ecf406a6

  with a bunch of dashes and a unique value.

• Set the request content type to multipart/form-data; boundary= and your boundary, like:

  multipart/form-data; boundary=---------------------------7d01ecf406a6

• Any time you write a standard form value to the request stream, you’ll write:
  • Two dashes.
  • Your boundary.
  • One CRLF (\r\n).

  • A content-disposition header that tells the name of the form field you’ll be inserting. That looks like:

    Content-Disposition: form-data; name="yourformfieldname"

  • Two CRLFs.
  • The value of the form field - not URL encoded.
  • One CRLF.
• Any time you write a fileto the request stream (for upload), you’ll write:
  • Two dashes.
  • Your boundary.
  • One CRLF (\r\n).

  • A content-disposition header that tells the name of the form field corresponding to the file and the name of the file. That looks like:

    Content-Disposition: form-data; name="yourformfieldname"; filename="somefile.jpg"

  • One CRLF.

  • A content-type header that says what the MIME type of the file is. That looks like:

    Content-Type: image/jpg

  • Two CRLFs.
  • The entire contents of the file, byte for byte. It’s OK to include binary content here. Don’t base-64 encode it or anything, just stream it on in.
  • One CRLF.
• At the end of your request, after writing all of your fields and files to the request, you’ll write:
  • Two dashes.
  • Your boundary.
  • Two more dashes.

That’s how it works. The problem is that it’s not built-in to the HttpWebRequest class, so you have some manual work to do. Lucky for you, I’ve got some snippets that do a lot of heavy lifting.

First, here’s a method you can use to create the boundary:

/// <summary>
/// Creates a multipart/form-data boundary.
/// </summary>
/// <returns>
/// A dynamically generated form boundary for use in posting multipart/form-data requests.
/// </returns>
private static string CreateFormDataBoundary()
{
  return "---------------------------" + DateTime.Now.Ticks.ToString("x");
}

Next, here’s an extension method I wrote for generic dictionaries with string names and string values, sort of like you’d have if you were generating a querystring or a regular post request. This method takes the dictionary and writes the name/value pairs out to a supplied stream in the proper format.

using System;
using System.Collections.Generic;
using System.IO;
using System.Web;

namespace MultipartFormData
{
  /// <summary>
  /// Extension methods for generic dictionaries.
  /// </summary>
  public static class DictionaryExtensions
  {
    /// <summary>
    /// Template for a multipart/form-data item.
    /// </summary>
    public const string FormDataTemplate = "--{0}\r\nContent-Disposition: form-data; name=\"{1}\"\r\n\r\n{2}\r\n";

    /// <summary>
    /// Writes a dictionary to a stream as a multipart/form-data set.
    /// </summary>
    /// <param name="dictionary">The dictionary of form values to write to the stream.</param>
    /// <param name="stream">The stream to which the form data should be written.</param>
    /// <param name="mimeBoundary">The MIME multipart form boundary string.</param>
    /// <exception cref="System.ArgumentNullException">
    /// Thrown if <paramref name="stream" /> or <paramref name="mimeBoundary" /> is <see langword="null" />.
    /// </exception>
    /// <exception cref="System.ArgumentException">
    /// Thrown if <paramref name="mimeBoundary" /> is empty.
    /// </exception>
    /// <remarks>
    /// If <paramref name="dictionary" /> is <see langword="null" /> or empty,
    /// nothing wil be written to the stream.
    /// </remarks>
    public static void WriteMultipartFormData(
      this Dictionary<string, string> dictionary,
      Stream stream,
      string mimeBoundary)
    {
      if (dictionary == null || dictionary.Count == 0)
      {
        return;
      }
      if (stream == null)
      {
        throw new ArgumentNullException("stream");
      }
      if (mimeBoundary == null)
      {
        throw new ArgumentNullException("mimeBoundary");
      }
      if (mimeBoundary.Length == 0)
      {
        throw new ArgumentException("MIME boundary may not be empty.", "mimeBoundary");
      }
      foreach (string key in dictionary.Keys)
      {
        string item = String.Format(FormDataTemplate, mimeBoundary, key, dictionary[key]);
        byte[] itemBytes = System.Text.Encoding.UTF8.GetBytes(item);
        stream.Write(itemBytes, 0, itemBytes.Length);
      }
    }
  }
}

That takes care of regular form fields, but what about files? Here’s an extension method for FileInfo objects that will write the file out to a supplied stream, also in the proper format:

using System;
using System.IO;
using System.Text;

namespace MultipartFormData
{
  /// <summary>
  /// Extension methods for <see cref="System.IO.FileInfo"/>.
  /// </summary>
  public static class FileInfoExtensions
  {
    /// <summary>
    /// Template for a file item in multipart/form-data format.
    /// </summary>
    public const string HeaderTemplate = "--{0}\r\nContent-Disposition: form-data; name=\"{1}\"; filename=\"{2}\"\r\nContent-Type: {3}\r\n\r\n";

    /// <summary>
    /// Writes a file to a stream in multipart/form-data format.
    /// </summary>
    /// <param name="file">The file that should be written.</param>
    /// <param name="stream">The stream to which the file should be written.</param>
    /// <param name="mimeBoundary">The MIME multipart form boundary string.</param>
    /// <param name="mimeType">The MIME type of the file.</param>
    /// <param name="formKey">The name of the form parameter corresponding to the file upload.</param>
    /// <exception cref="System.ArgumentNullException">
    /// Thrown if any parameter is <see langword="null" />.
    /// </exception>
    /// <exception cref="System.ArgumentException">
    /// Thrown if <paramref name="mimeBoundary" />, <paramref name="mimeType" />,
    /// or <paramref name="formKey" /> is empty.
    /// </exception>
    /// <exception cref="System.IO.FileNotFoundException">
    /// Thrown if <paramref name="file" /> does not exist.
    /// </exception>
    public static void WriteMultipartFormData(
      this FileInfo file,
      Stream stream,
      string mimeBoundary,
      string mimeType,
      string formKey)
    {
      if (file == null)
      {
        throw new ArgumentNullException("file");
      }
      if (!file.Exists)
      {
        throw new FileNotFoundException("Unable to find file to write to stream.", file.FullName);
      }
      if (stream == null)
      {
        throw new ArgumentNullException("stream");
      }
      if (mimeBoundary == null)
      {
        throw new ArgumentNullException("mimeBoundary");
      }
      if (mimeBoundary.Length == 0)
      {
        throw new ArgumentException("MIME boundary may not be empty.", "mimeBoundary");
      }
      if (mimeType == null)
      {
        throw new ArgumentNullException("mimeType");
      }
      if (mimeType.Length == 0)
      {
        throw new ArgumentException("MIME type may not be empty.", "mimeType");
      }
      if (formKey == null)
      {
        throw new ArgumentNullException("formKey");
      }
      if (formKey.Length == 0)
      {
        throw new ArgumentException("Form key may not be empty.", "formKey");
      }
      string header = String.Format(HeaderTemplate, mimeBoundary, formKey, file.Name, mimeType);
      byte[] headerbytes = Encoding.UTF8.GetBytes(header);
      stream.Write(headerbytes, 0, headerbytes.Length);
      using (FileStream fileStream = new FileStream(file.FullName, FileMode.Open, FileAccess.Read))
      {
        byte[] buffer = new byte[1024];
        int bytesRead = 0;
        while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
        {
          stream.Write(buffer, 0, bytesRead);
        }
        fileStream.Close();
      }
      byte[] newlineBytes = Encoding.UTF8.GetBytes("\r\n");
      stream.Write(newlineBytes, 0, newlineBytes.Length);
    }
  }
}

Finally, you need to put all of that together when you make your web request. That means setting the content type on your request, writing the parameters to the request stream using the extension methods, and getting the response back. An example of what this might look like is here:

public string ExecutePostRequest(
  Uri url,
  Dictionary<string, string> postData,
  FileInfo fileToUpload,
  string fileMimeType,
  string fileFormKey
){
  HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url.AbsoluteUri);
  request.Method = "POST";
  request.KeepAlive = true;
  string boundary = CreateFormDataBoundary();
  request.ContentType = "multipart/form-data; boundary=" + boundary;
  Stream requestStream = request.GetRequestStream();
  postData.WriteMultipartFormData(requestStream, boundary);
  if (fileToUpload != null)
  {
    fileToUpload.WriteMultipartFormData(requestStream, boundary, fileMimeType, fileFormKey);
  }
  byte[] endBytes = System.Text.Encoding.UTF8.GetBytes("--" + boundary + "--");
  requestStream.Write(endBytes, 0, endBytes.Length);
  requestStream.Close();
  using (WebResponse response = request.GetResponse())
  using (StreamReader reader = new StreamReader(response.GetResponseStream()))
  {
    return reader.ReadToEnd();
  };
}

Obviously you’ll need to tailor the usage to your own needs, but this at least shows it in action.

I’m no expert on this by any means, but this is what got me through the magic of file upload via POST to ImageShack, so hopefully it’ll help someone else out there, too.

I have to admit - I’m a Windows Live Writer convert. I tried earlier versions and wasn’t impressed, but I’m all over it now.

I’m also an ImageShack user. I love their free image hosting service for its ability to save me bandwidth on image hosting. It makes a surprising difference. (I even use YFrog on Twitter.)

The only real problem I ran into was that Windows Live Writer wants to upload every image to your blog for hosting. I don’t want that - I want my images on ImageShack. That means leaving Windows Live Writer to upload the image from some other uploader tool, getting the URL to the image, and manually inserting it. Sort of a pain in the workflow, if you know what I mean.

As an experiment to see how different applications enable extensibility through plugins, and to ease a problem I was seeing, I wrote a Windows Live Writer plugin that uploads images to ImageShack directly. No more having to leave Windows Live Writer.

Simply drop the plugin DLL in the Windows Live Writer plugins folder, paste your ImageShack registration key into the plugin options box, and upload/insert images from the “Insert” menu in WLW:

It was surprisingly easy to write, which was cool.

It’s free and open source.Go pick it up on Google Code.

Sometimes you have a snippet of code in Visual Studio that you’d like to share over Twitter. You used to have to post the code snippet somewhere (or take a screen shot), then manually tweet the link to that snippet.

Not anymore!

CR_CodeTweet is a plugin for DXCore that adds a context menu and hotkey that allows you to select code in Visual Studio and automatically post it to CodePaste.NET, then tweet a link to that posted snippet - all without leaving Visual Studio.

Select your code and use a simple context menu…

…fill in your tweet info and your selected snippet hits the interwebs!

Interested? It’s free! Go get it!

Last year about this time I posted a database maintenance page that I created for cleaning up Subtext database things. Since then I’ve upgraded to Subtext 2.1.0.5 and, while the old page still works, my blog has become more popular so the referral cleanup is more difficult. There are so many records coming in that even with a long timeout set, there’s too much and the delete operation times out. That leaves a huge amount of junk in the transaction log and is just problems.

I updated the page so you can select which referrers you want to delete using a checkbox list and the number of referrals deleted at any given time is 1000 so you don’t have to worry about the timeout issue. Again, this page will let you:

• Clear the error log. Yes, you can do this from the error log page, too, but it’s nice to have all of this in a central location.
• See how many referrals you have in your database vs. how many of those are from search engines. The page lists out what qualifies as a search engine or spam referral so you’ll know what this means. It’s basically just a list of expressions that the page tries to match the URL against - nothing fancy. This new version only shows you the matches from the selected search engines, though.
• Remove search engine referrals from the referral log. Qualifying spam referrals are also removed.
• Reindex the referrals table and shrink the database. Do that after you clear out the garbage referrals.
• See some size statistics on your database.

Download the zip file, then drop the enclosed ASPX page in your Subtext “Admin” folder. It’s an administration page so you do have to be logged in as an admin to use it. It doesn’t add any navigation links to the admin site, so you do need to manually enter the URL to the page to get to it.

[Download SubtextDatabaseMaintenance2105.zip]