GeekSpeak comments edit

I have some files (like my local Subversion repository, some documents, etc.) that I need to sync between computers and I was recommended Dropbox as the way to get that done. I signed up, installed it, and it works brilliantly.

That said, my primary complaint is that it only synchronizes files inside a special “My Dropbox” folder that it creates. Anything you want to synchronize has to live in there. Thing is, while I don’t mind changing the location of some things, like my documents, I really would rather not change the location of other things, like my local Subversion repository. I like it in “C:\LocalSVN” rather than “C:\Documents and Settings\tillig\My Documents\My Dropbox\LocalSVN” or whatever.

Turns out you can use the magic of symbolic links to fix that right up. If you create a symbolic link (junction point) inside “My Dropbox” to content that actually lives outside “My Dropbox” then the content gets synchronized just fine but can live wherever you want.

If you are in Windows XP, you’ll need to go get a free copy of Junction and put it somewhere in your path like your C:\Windows\System32 folder. In Windows Vista or Windows 7, you’ll use the built-in “mklink” command.

  1. Get Dropbox set up and synchronizing on your computer without the external content.
  2. Open a command prompt as an administrator on your machine.
  3. Change to the “My Dropbox” folder that you set up. In Vista or Windows 7 it’ll be like: cd "\Users\yourusername\Documents\My Dropbox" In Windows XP it’ll be like: cd "\Documents and Settings\yourusername\My Documents\My Dropbox"
  4. Create a directory link to the folder containing the external content. In Vista or Windows 7 it’ll be like: mklink /d "External Content" "C:\Path\To\External Content" In Windows XP it’ll be like: junction "External Content" "C:\Path\To\External Content"

That’s it. Dropbox will see the symbolic directory link as a new folder with content it needs to synchronize and it’ll get done.

Note that you can do things the other way around, too - move the content into the “My Dropbox” folder and then create the symbolic link from the original location into the moved content… but this way it means you don’t have to do the moving to begin with. Admittedly, I kinda wish I had figured this out before I moved everything, but now I know.

media comments edit

Creative Vado
HDI got a Creative Vado HD camera for Christmas from Jenn and have been messing around a bit with it. I like the recording quality, but I found in Windows Vista you have to install an H.264 codec in order to get the videos to play in Windows Media Player. (In Windows 7 they provide an H.264 codec for Windows Media Player so you don’t have to install anything else.)

While playback is fine and dandy, I’m not really interested in editing the videos I take with the camera in the simplistic editor they provide. I suppose if I just zapped something really quick to jam up on YouTube that’d be great, but I find I end up with several clips where I need to trim the start/end on them, maybe crossfade them into each other - nothing really major, but more than the little app offers. I have Sony Vegas to do my video editing and it works great.

Except on H.264.

See, Windows does not come with a codec that Sony Vegas can use to edit these files. You can get the audio, but not the video. Really dumb, and dumber still that Sony didn’t pre-package one just in case, particularly since H.264 is so popular.

The answer: x264vfw - a free, open-source Video for Windows H.264 codec.

Once you install x264vfw - three clicks, tops - magically everything works the way you want it to. Playback in Windows Media Player, editing in Sony Vegas… fantastic.

Note that I have seen other solutions for this posted elsewhere encouraging you to install the whole K-Lite Mega Codec Pack and everything that comes with it. That’s overkill. You only need a VFW (Video for Windows) H.264 codec, and K-Lite uses x264vfw internally anyway. I’m a fan of only installing the things you absolutely have to, and just x264vfw will take care of it.

UPDATE 1/4/2010: This appears to have broken in Sony Vegas 10. I had it working in Vegas 9 and for various reasons upgraded to Vegas 10 and now it doesn’t work. It appears that the recommendation in the Sony Vegas knowledge base is to use Windows Live Movie Maker to convert the video to a different format (since Windows can read/play it but Vegas can’t). For that purpose, I created a new custom setting in Windows Live Movie Maker with the following values:

  • Width: 1280
  • Height: 720
  • Bit rate: 8500
  • Frame rate: 30
  • Audio format: 192kbps, 48kHz, stereo

The width, height, and frame rate match the Vado HD video. The other values are just slightly higher quality than the Vado so you won’t lose too much quality in the conversion. Doing a few tests, I don’t really notice a difference in the source and converted materials.

After conversion, Sony Vegas easily reads the files and you can edit them as expected.

Personally, I think it’s pretty lame that Windows Live Movie Maker can read these files but a higher-end program like Vegas can’t. You would think Vegas would take advantage of the same set of codecs WLMM does… but apparently not.

personal comments edit

I just finished upgrading the last of our computers at home to Windows 7 and now that I have Windows Live Writer installed I figured I’d do my yearly retrospective - see what’s gone on this year and recap.

In January I found that they were ending Google Notebook so I had to move all of my notes over to PBworks. That was sort of painful, but I’m really enjoying PBworks now and I have a ton of stuff in there. I also released version 2.2.1.0 of CR_Documentor, a bug fix release.

In February I showed you how to upgrade your Windows Home Server capacity with an eSATA port replicator and upgraded my main TV to a 52” Samsung LN52A750 (still great, and still recommended).

March was pretty eventful. I upgraded my blog to Subtest 2.1.0.5. I reflected a bit on why it’s a good idea to keep a cool head in a tough situation. I ran into some User Account Control issues in Windows Server 2008 and wrote about some power toys that will help you out with that as well as providing my own Visual Studio related elevation toy. I went to MIX09 and blogged all three days (1, 2, 3). My MIX trip then spun off some travel luggage recommendations.

At the beginning of April I found that most of my network configuration problems were due to Verizon updating my FiOS router automatically, requiring me to restore my router to factory defaults more than once since then. I found that an HDMI switchbox can help solve issues where your TV loses the HDMI signal when your computer goes to sleep. Jenn and I took a three-day vacation to Vegas, which is always a good time. I released version 2.3.0.1 of CR_Documentor and put out a solicitation for input (with not much response, to be honest). I wrote about some challenges I’ve faced while trying to write multi-tenant ASP.NET apps. I also wrote about some challenges I faced with my Blackberry due to the Facebook application - if you’re having trouble synchronizing, check your default services.

May found me offering some tips on proper use for bullet lists and finishing off the ripping of all of my DVDs. (Did you know an average DVD is 6.7GB?) I wrote a script to automatically set the album artist on your iTunes tracks. I reviewed ASP.NET 3.5 Enterprise Application Development with Visual Studio 2008. I explained how to get iTunes music playing in Windows Media Center, though Windows 7 fixes a lot of the issues there. Plus I showed you how to use the MSBuild engine in your programs and take advantage of the file finding functionality in there.

June started out where I showed you how to use Typemock Isolator to skip generic constructors. I got an HD camcorder and struggled with the file formats. I reflected a bit on blogs that rip off content and publish it as their own. I talked about defragmenting your Windows Home Server drives, then I went to see the B-52s in concert. June ended on a huge down note, though, as my Grandma Jeanne passed away at 86.

July started out with a nifty trick - I showed how to change Windows Service runtime behavior using Typemock Isolator. I also showed you how to get the Windows OS version from inside MSBuild. I talked about the SQL Server installer constantly requiring a reboot (which always seems to be the case for me) and how to fix it. I updated my custom NAnt tasks to .NET 2.0. On a personal note, my birthday was awesome and I went to see Tears for Fears in concert. The biggest thing in July, though, was Jenn and I running our own fireworks shoot in Sandy. Scary and exhilarating.

August started out brilliantly with one of our famous 24 marathons. I discovered the coolness that is Asset UPnP on Windows Home Server and showed you how to back Windows Home Server up to MozyHome. I showed you how to write Firefox extensions using Visual Studio and I even released one of my own - Firefox NTLMAuth, a plugin to help you with Windows pass-through authentication in Firefox.

Jenn and I started September with a trip to Victoria, BC, Canada - good times. I found a little gotcha when using the Windows Vista DVD burner and it tells you there aren’t any files to burn. I updated my iTunes metadata copying program for the latest iTunes. I reviewed Professional ASP.NET MVC 1.0. Jenn and I went to see The Killers in concert (awesome!). I posted a couple of ASP.NET AJAX tips - using ASP.NET AJAX String.format() in jQuery validation and parsing currency values. Finally, I created a quick DXCore plugin that helps other plugin writers explore contexts.

In October I started out with another jQuery/ASP.NET AJAX tip - converting relative to absolute paths. I did a one-year retrospective with my Windows Home Server. I discovered the hugest gotcha with the “COMPLUS_Version” environment variable and the .NET runtime

In November I had to manually uninstall a Windows Home Server add-in and showed how to upgrade PerfectDisk for Windows Home Server. I ran into a weird edge case with XML serialization while debugging a Visual Studio add-in. I did some work on my Media Center and figured out the Windows 7 supported media formats, how to fix that one-pixel line in Windows Media Center playback, and talked a bit about metadata and artist names on music files. I showed how to create icons for your context menu items in DXCore and how to put your log4net.config outside your app.config/web.config file. November ended, as it always does, with my least favorite holiday ever.

In December I updated my Subtext database maintenance page for Subtext 2.1.0.5, but I did two other programming things that I was more proud of: I released a DXCore plugin, CR_CodeTweet, that lets you tweet code snippets from inside Visual Studio; and I released a Windows Live Writer plugin that lets you upload images to ImageShack. As part of that ImageShack upload plugin, I figured out how to post multipart/form-data using .NET WebRequest.

Overall, the year was decent, but not great. Like everyone else, we’ve had our challenges with the economy. There have been a couple of fairly difficult personal issues to face as well. On the bright side, Jenn and I are both healthy, and we’ve got a nice place to live, we both have jobs, and we’re otherwise doing well, so I can’t say we’re in a bad spot. I’ll be glad to see 2009 past and I look forward to seeing what 2010 holds.

net, gists comments edit

While making my ImageShack plugin for Windows Live Writer I had to figure out how to make a web request that posts data to an endpoint in “multipart/form-data” format. That’s a lot different than URL-encoding everything into a querystring format and sticking that in the body of the POST request. If you’re uploading one or more files, you need to format things differently.

I looked all over for some built-in function that does this, but it doesn’t seem to exist in the .NET base class library. Surprising, but also not surprising. I found a question on StackOverflow that talked about it and an article with some tiny code snippets in it, but that didn’t tell me what was really going on in the request and the code I found was either incomplete, confusing, or both.

So, in the spirit of “Not Invented Here,” I decided to write my own. :)

First was figuring out what the format looked like in the request. I found an article that showed just that. Not a very thorough article, but it got me started. Let me boil it down in terms that [at least I] understand.

  • Generate a “boundary.” A boundary is a unique string that serves as a delimiter between each of the form values you’ll be sending in your request. Usually these boundaries look something like
    ---------------------------7d01ecf406a6
    with a bunch of dashes and a unique value.
  • Set the request content type to multipart/form-data; boundary= and your boundary, like:
    multipart/form-data; boundary=---------------------------7d01ecf406a6
  • Any time you write a standard form value to the request stream, you’ll write:
    • Two dashes.
    • Your boundary.
    • One CRLF (\r\n).
    • A content-disposition header that tells the name of the form field you’ll be inserting. That looks like:
      Content-Disposition: form-data; name="yourformfieldname"
    • Two CRLFs.
    • The value of the form field - not URL encoded.
    • One CRLF.
  • Any time you write a fileto the request stream (for upload), you’ll write:
    • Two dashes.
    • Your boundary.
    • One CRLF (\r\n).
    • A content-disposition header that tells the name of the form field corresponding to the file and the name of the file. That looks like:
      Content-Disposition: form-data; name="yourformfieldname"; filename="somefile.jpg" 
    • One CRLF.
    • A content-type header that says what the MIME type of the file is. That looks like:
      Content-Type: image/jpg
    • Two CRLFs.
    • The entire contents of the file, byte for byte. It’s OK to include binary content here. Don’t base-64 encode it or anything, just stream it on in.
    • One CRLF.
  • At the end of your request, after writing all of your fields and files to the request, you’ll write:
    • Two dashes.
    • Your boundary.
    • Two more dashes.

That’s how it works. The problem is that it’s not built-in to the HttpWebRequest class, so you have some manual work to do. Lucky for you, I’ve got some snippets that do a lot of heavy lifting.

First, here’s a method you can use to create the boundary:

/// <summary>
/// Creates a multipart/form-data boundary.
/// </summary>
/// <returns>
/// A dynamically generated form boundary for use in posting multipart/form-data requests.
/// </returns>
private static string CreateFormDataBoundary()
{
  return "---------------------------" + DateTime.Now.Ticks.ToString("x");
}

Next, here’s an extension method I wrote for generic dictionaries with string names and string values, sort of like you’d have if you were generating a querystring or a regular post request. This method takes the dictionary and writes the name/value pairs out to a supplied stream in the proper format.

using System;
using System.Collections.Generic;
using System.IO;
using System.Web;

namespace MultipartFormData
{
  /// <summary>
  /// Extension methods for generic dictionaries.
  /// </summary>
  public static class DictionaryExtensions
  {
    /// <summary>
    /// Template for a multipart/form-data item.
    /// </summary>
    public const string FormDataTemplate = "--{0}\r\nContent-Disposition: form-data; name=\"{1}\"\r\n\r\n{2}\r\n";

    /// <summary>
    /// Writes a dictionary to a stream as a multipart/form-data set.
    /// </summary>
    /// <param name="dictionary">The dictionary of form values to write to the stream.</param>
    /// <param name="stream">The stream to which the form data should be written.</param>
    /// <param name="mimeBoundary">The MIME multipart form boundary string.</param>
    /// <exception cref="System.ArgumentNullException">
    /// Thrown if <paramref name="stream" /> or <paramref name="mimeBoundary" /> is <see langword="null" />.
    /// </exception>
    /// <exception cref="System.ArgumentException">
    /// Thrown if <paramref name="mimeBoundary" /> is empty.
    /// </exception>
    /// <remarks>
    /// If <paramref name="dictionary" /> is <see langword="null" /> or empty,
    /// nothing wil be written to the stream.
    /// </remarks>
    public static void WriteMultipartFormData(
      this Dictionary<string, string> dictionary,
      Stream stream,
      string mimeBoundary)
    {
      if (dictionary == null || dictionary.Count == 0)
      {
        return;
      }
      if (stream == null)
      {
        throw new ArgumentNullException("stream");
      }
      if (mimeBoundary == null)
      {
        throw new ArgumentNullException("mimeBoundary");
      }
      if (mimeBoundary.Length == 0)
      {
        throw new ArgumentException("MIME boundary may not be empty.", "mimeBoundary");
      }
      foreach (string key in dictionary.Keys)
      {
        string item = String.Format(FormDataTemplate, mimeBoundary, key, dictionary[key]);
        byte[] itemBytes = System.Text.Encoding.UTF8.GetBytes(item);
        stream.Write(itemBytes, 0, itemBytes.Length);
      }
    }
  }
}

That takes care of regular form fields, but what about files? Here’s an extension method for FileInfo objects that will write the file out to a supplied stream, also in the proper format:

using System;
using System.IO;
using System.Text;

namespace MultipartFormData
{
  /// <summary>
  /// Extension methods for <see cref="System.IO.FileInfo"/>.
  /// </summary>
  public static class FileInfoExtensions
  {
    /// <summary>
    /// Template for a file item in multipart/form-data format.
    /// </summary>
    public const string HeaderTemplate = "--{0}\r\nContent-Disposition: form-data; name=\"{1}\"; filename=\"{2}\"\r\nContent-Type: {3}\r\n\r\n";

    /// <summary>
    /// Writes a file to a stream in multipart/form-data format.
    /// </summary>
    /// <param name="file">The file that should be written.</param>
    /// <param name="stream">The stream to which the file should be written.</param>
    /// <param name="mimeBoundary">The MIME multipart form boundary string.</param>
    /// <param name="mimeType">The MIME type of the file.</param>
    /// <param name="formKey">The name of the form parameter corresponding to the file upload.</param>
    /// <exception cref="System.ArgumentNullException">
    /// Thrown if any parameter is <see langword="null" />.
    /// </exception>
    /// <exception cref="System.ArgumentException">
    /// Thrown if <paramref name="mimeBoundary" />, <paramref name="mimeType" />,
    /// or <paramref name="formKey" /> is empty.
    /// </exception>
    /// <exception cref="System.IO.FileNotFoundException">
    /// Thrown if <paramref name="file" /> does not exist.
    /// </exception>
    public static void WriteMultipartFormData(
      this FileInfo file,
      Stream stream,
      string mimeBoundary,
      string mimeType,
      string formKey)
    {
      if (file == null)
      {
        throw new ArgumentNullException("file");
      }
      if (!file.Exists)
      {
        throw new FileNotFoundException("Unable to find file to write to stream.", file.FullName);
      }
      if (stream == null)
      {
        throw new ArgumentNullException("stream");
      }
      if (mimeBoundary == null)
      {
        throw new ArgumentNullException("mimeBoundary");
      }
      if (mimeBoundary.Length == 0)
      {
        throw new ArgumentException("MIME boundary may not be empty.", "mimeBoundary");
      }
      if (mimeType == null)
      {
        throw new ArgumentNullException("mimeType");
      }
      if (mimeType.Length == 0)
      {
        throw new ArgumentException("MIME type may not be empty.", "mimeType");
      }
      if (formKey == null)
      {
        throw new ArgumentNullException("formKey");
      }
      if (formKey.Length == 0)
      {
        throw new ArgumentException("Form key may not be empty.", "formKey");
      }
      string header = String.Format(HeaderTemplate, mimeBoundary, formKey, file.Name, mimeType);
      byte[] headerbytes = Encoding.UTF8.GetBytes(header);
      stream.Write(headerbytes, 0, headerbytes.Length);
      using (FileStream fileStream = new FileStream(file.FullName, FileMode.Open, FileAccess.Read))
      {
        byte[] buffer = new byte[1024];
        int bytesRead = 0;
        while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
        {
          stream.Write(buffer, 0, bytesRead);
        }
        fileStream.Close();
      }
      byte[] newlineBytes = Encoding.UTF8.GetBytes("\r\n");
      stream.Write(newlineBytes, 0, newlineBytes.Length);
    }
  }
}

Finally, you need to put all of that together when you make your web request. That means setting the content type on your request, writing the parameters to the request stream using the extension methods, and getting the response back. An example of what this might look like is here:

public string ExecutePostRequest(
  Uri url,
  Dictionary<string, string> postData,
  FileInfo fileToUpload,
  string fileMimeType,
  string fileFormKey
){
  HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url.AbsoluteUri);
  request.Method = "POST";
  request.KeepAlive = true;
  string boundary = CreateFormDataBoundary();
  request.ContentType = "multipart/form-data; boundary=" + boundary;
  Stream requestStream = request.GetRequestStream();
  postData.WriteMultipartFormData(requestStream, boundary);
  if (fileToUpload != null)
  {
    fileToUpload.WriteMultipartFormData(requestStream, boundary, fileMimeType, fileFormKey);
  }
  byte[] endBytes = System.Text.Encoding.UTF8.GetBytes("--" + boundary + "--");
  requestStream.Write(endBytes, 0, endBytes.Length);
  requestStream.Close();
  using (WebResponse response = request.GetResponse())
  using (StreamReader reader = new StreamReader(response.GetResponseStream()))
  {
    return reader.ReadToEnd();
  };
}

Obviously you’ll need to tailor the usage to your own needs, but this at least shows it in action.

I’m no expert on this by any means, but this is what got me through the magic of file upload via POST to ImageShack, so hopefully it’ll help someone else out there, too.

downloads, net, blog comments edit

I have to admit - I’m a Windows Live Writer convert. I tried earlier versions and wasn’t impressed, but I’m all over it now.

I’m also an ImageShack user. I love their free image hosting service for its ability to save me bandwidth on image hosting. It makes a surprising difference. (I even use YFrog on Twitter.)

The only real problem I ran into was that Windows Live Writer wants to upload every image to your blog for hosting. I don’t want that - I want my images on ImageShack. That means leaving Windows Live Writer to upload the image from some other uploader tool, getting the URL to the image, and manually inserting it. Sort of a pain in the workflow, if you know what I mean.

As an experiment to see how different applications enable extensibility through plugins, and to ease a problem I was seeing, I wrote a Windows Live Writer plugin that uploads images to ImageShack directly. No more having to leave Windows Live Writer.

Simply drop the plugin DLL in the Windows Live Writer plugins folder, paste your ImageShack registration key into the plugin options box, and upload/insert images from the “Insert” menu in WLW:

Insert ImageShack Upload menu
option

It was surprisingly easy to write, which was cool.

It’s free and open source.Go pick it up on Google Code.