Sadly, duty called and I didn't get to make it to Mix '08 this year. To make up for it, I think I'll take a couple days this week to watch some recorded sessions via teh intarweb and then drive to the Indian casino in Everett.

Here's what's in my queue:

http://sessions.visitmix.com/?selectedSearch=T26 -- Justin Smith on the NetFx 3.5 WCF Web Programming Model
http://sessions.visitmix.com/?selectedSearch=T19 – Pablo on talking to Windows Live Services via AtomPub
http://sessions.visitmix.com/?selectedSearch=BT02 – Mike Flasko’s Astoria overview
http://sessions.visitmix.com/?selectedSearch=T22 – Scott Hanselman’s MVC overview
http://sessions.visitmix.com/?selectedSearch=T28 – John Lam on the DLR
http://sessions.visitmix.com/?selectedSearch=T07 – another Astoria talk, this one from Pablo
http://sessions.visitmix.com/?selectedSearch=T13 – our very own EugeneOs on web services and Silverlight

But of course the big one for me is Creating a RESTful API with Windows Communication Foundation, presented by Haider Sabri and some friends from the MySpace developer API team. These guys have been building the public-facing MySpace REST API as a set of WCF services and were kind enough to do a talk at Mix about their experience. They even built a cool demo app -- http://www.restchess.com using WCF and shared the source for everyone to play with. There's a sweet (and highly reusable) OAuth channel in there along with lots of other goodies...thanks guys!

I've gotten a lot of requests (internally and externally) for requests for resources on how to use the new WCF Web Programming Model features in .NET 3.5.

MSDN has a lot of great content on this stuff, but it's kind of sprinkled around in various places due to the way the MSDN table of contents is laid out. I figured it would be nice to have links to all of the important topics in one place, so here's the "mini-TOC" for the Web Programming Model content:

Conceptual Overviews:

• Web Programming Model
• AJAX and JSON
• WCF Syndication (Atom and RSS)
• Partial Trust

Class Library Reference (not exhaustive):

• System.ServiceModel.Web Namespace
• System.ServiceModel.Syndication Namespace
• System.Runtime.Serialization.Json Namespace
• SyndicationFeed Class
• SyndicationItem Class
• WebOperationContext Class
• WebServiceHost Class
• WebGetAttribute Class
• WebInvokeAttribute Class
• WebHttpBehavior Class
• WebScriptEnablingBehavior Class
• DataContractJsonSerializer Class

Configuration Schema:

• enableWebScript
• webHttp 
• webHttpBinding 

Samples:

• Web Programming Model
  • Basic Web Programming Model
  • Advanced Web Programming
  • UriTemplate
  • UriTemplate Table
  • UriTemplate Table Dispatcher
  • WebContentTypeMapper
  • HTML Form Handler
  • Push-Style Streaming
• AJAX and JSON
  • Basic AJAX Service
  • AJAX Service Without Configuration
  • AJAX Service Using Complex Types
  • AJAX Service with JSON and XML
  • JSON Serialization
  • Weakly-typed JSON Serialization (AJAX)
• Syndication
  • Intro to Syndication
  • Loosely-Typed Extensions
  • Strongly Typed Extensions
  • Streaming Feeds
  • Feed Formatter (JSON)
• Partial Trust
• Partial Trust client and server
• Everything and (a picture of) the Kitchen Sink (Thanks, Justin!)
• http://samples.netfx3.com/PictureServices/

Lots of folks have asked me how to add various XMLism like XML Namespace prefixes to the serialized output of a SyndicationFeed. This is actually easy to do once you know the trick but it's admittedly not the most obvious thing in the world.

The thing to remember is that the System.Xml stack treats prefix declarations as special kinds of attributes. If you want to emit an XML namespace declaration of xmlns:contoso=http://schemas.contoso.com you need to emit attribute whose name is the prefix you're declaring (e.g. "contoso"), whose value is the namespace URI that corresponds to the prefix (e.g. 'http://schemas.contoso.com'). This attribute needs to be in the special XML namespace declaration namespace (!) of http://www.w3.org/2000/xmlns/

Since SyndicationFeed supports additional XML attributes through the AttributeExtensions property, you can do this on SyndicationFeed as follows (using the C# 3.0 collection initializer syntax, natch):

SyndicationFeed feed = new SyndicationFeed()
{

   AttributeExtensions =

  {
     { new XmlQualifiedName("contoso", "http://www.w3.org/2000/xmlns/"), "http://schemas.constoso.com" }
  }
};

If you like syntactic sugar, here are a few extension methods that add a helper function to all of the OM constructs that support attribute extensibility:

using System;
using System.Xml;
using System.ServiceModel.Syndication;

namespace Samples
{
    public static class SyndicationExtensions
    {
        public static void DeclareNamespace(this SyndicationFeed feed, string prefix, string nsUri)
        {
            feed.AttributeExtensions.Add(new XmlQualifiedName( prefix, "http://www.w3.org/2000/xmlns/" ), nsUri);
        }

        public static void DeclareNamespace(this SyndicationItem item, string prefix, string nsUri)
        {
            item.AttributeExtensions.Add(new XmlQualifiedName(prefix, "http://www.w3.org/2000/xmlns/"), nsUri);
        }

        public static void DeclareNamespace(this SyndicationCategory category, string prefix, string nsUri)
        {
            category.AttributeExtensions.Add(new XmlQualifiedName(prefix, "http://www.w3.org/2000/xmlns/"), nsUri);
        }

        public static void DeclareNamespace(this SyndicationLink link, string prefix, string nsUri)
        {
            link.AttributeExtensions.Add(new XmlQualifiedName(prefix, "http://www.w3.org/2000/xmlns/"), nsUri);
        }

        public static void DeclareNamespace(this SyndicationPerson person, string prefix, string nsUri)
        {
            person.AttributeExtensions.Add(new XmlQualifiedName(prefix, "http://www.w3.org/2000/xmlns/"), nsUri);
        }
    }
}

On a side note, I'm becoming a huge fan of extension methods. They are not perfect, but I expect that their existence will have a pretty substantial impact on the way the internals of the framework get factored in future versions...

One of the things that the DataContractSerializer does is create a mapping between CLR type names and XML qnames.

Lots of people know that you can control this mapping on a per-type basis via the [DataContract] attribute, e.g.

namespace Contoso.Serialization
{
  [DataContract( Namespace="http://schemas.contoso.com/2008" )]
  public class Person
  {

     [DataMember]
     public string Name { get; set; }
  }
}

Having to explicitly specify this on every type gets a little laborious, especially if you just want to say "put all types in this CLR namespace into this XML namespace".

Few people know that there's a simple way to do this via the assembly-level ContractNamespaceAttribute:

[assembly: ContractNamespace( "http://schemas.contoso.com/2008", ClrNamespace="Contoso.Serialization" )]

If you have that, then you can just say [DataContract] (sans Namespace="...") and it's just as if you had said [DataContract( Namespace="http://schemas.contoso.com/2008" )] for all [DataContract] types in the Contoso.Serialization namespace.

That comes in handy from time to time and can save some keystrokes.

This has been around since .NET 3.0, but I didn't know about it until recently. Figured not many other people did either :)

I thought this post from Ganesh Prasad was both cheeky and refreshing:

Though I like REST and consider it a very elegant model for SOA, it's a little tiresome to hear day in and day out that it's so much more elegant than the SOAP-based Web Services model. In fact, I'm getting so tired of this shrill posturing that I'm going to stick it to the RESTafarians right now, in their own style. Watch.

Paying the RESTafarians Back in Their Own Coin

I think I would get along well with Ganesh.

If you want to use the [WebGet]/[WebInvoke] programming model in WCF, you need an endpoint with the right binding and the right endpoint behavior.

The binding is the out-of-the-box WebHttpBinding. The endpoint behavior can be either WebHttpBehavior or WebScriptEnablingBehavior.

What's the difference between the two?

The WebHttpBehavior is a general-purpose behavior that supports UriTemplate dispatch and POX/JSON/byte stream formats on the wire. Use it for general purpose HTTP/REST/Web-Style services.

The WebScriptEnablingBehavior is a "profile" of the WebHttpBehavior functionality designed specifically for interop with ASP.NET AJAX clients. It adds in some AJAX-isms like the ability to automatically generate ASP.NET AJAX client proxies.

I get asked by many, many folks if there's a way to cleanse the .svc file extension from WCF service URI's running in IIS.

Of course there is, and Jon Flanders shows you how.

Thanks for writing that up, Jon!

Performance is something we take pretty dang seriously on WCF. I had the privilege of wearing the Performance PM hat during the last few milestones of WCF v1, so I got an up close and personal introduction to all the work that goes in to squeezing every last bit of performance possible from a bigtime server stack. I wish I could say I was responsible for the fantastic performance we have in the product today, but that honor rightly belongs to every feature developer on the team -- great perf is something that's baked in from the ground up, and it takes a commitment from every single person involved in the code to make a stack perform to its fullest. Of course, having the SWAT team of BobDimp, ErikC and their roving band of brianiacs didn't hurt either :)

The reason I bring this up is because we release the .NET StockTrader application today, which is an end-to-end real-world app that stacks up WCF 3.0's performance against Websphere 6.1. I'm pretty stoked about the summary results:

• .NET 3.0 hosted on IIS with an Http binding and XML encoding offers 124% better throughput than the fastest WebSphere/EJB Web Service implementation tested; and 46% better throughput than the JDBC (no entity beans) WebSphere implementation tested.
• .NET 3.0 self-hosted over Http/XML offers 225% better throughput than the fastest WebSphere/EJB Web Service implementation tested; and 113% better throughput than the JDBC WebSphere implementation tested.
• .NET 3.0 with binary encoding over a TCP binding offers 488% better throughput than the fastest WebSphere EJB Web service implementation tested; and 284% better throughput than the JDBC WebSphere implementation tested.

Gotta love going fast.

Upgraded to DasBlog 1.9.7 last night -- if you're reading this, things seem to be going ok. Still some issues to work out with Free Text Box and the editing UI, but I'll get to those later. Must read Harry Potter now.

Day Three of TechEd Orlando is pretty much in the books -- newsflash: when it rains down here, it really rains.

I've spent most of my time here hanging out in the SOA + Web Services area of the Technical Learning Center. For those of you at Tech Ed, it's the blue one. I'll be there tomorrow as well, so if you have a question about WCF or just want to talk about the web you should come on by.

Tomorrow, my good friends Ed Pinto and Kenny Wolf will be doing a 400-level talk on WCF Architecture and Extensibility. This is the talk to go to if you're interested in exploring WCF from the inside out. It's going to be a great talk; I'll probably pop out of the TLC for an hour or two and go hear the PintoWolf do their thing.

And if you're still here on Friday afternoon, I'm doing a talk on WCF and the Web at 1pm. I'll be talking about UriTemplates, WebGet/WebInvoke, JSON, RSS, and all the other webby goodness I've been blogging about recently. Should be a good time -- hope to see you there.

Scott's got some good comments on the Ruby aesthetic.

I find some of the Ruby idioms to be really interesting. Specifically, the attempt to bring programming language closer to the natural language is an admirable goal. That said, I wonder how well they translate and whether this style is ultimately a help or a hindrance to Ruby coders who don't natively speak English (at the very least, it puts a different slant on the globalization problem).

And I know Matz is Japanese, but it seems like a lot of this comes in the framework/standard libary. I think Ruby has landed in an interesting place -- it has a nice mix of core language features (particularly block closures and extensible types) that enable some nice patterns, but the real trick is in the framework and how it makes use of them to enable the idioms that Scott finds so attractive.

C# 3.0 is getting to the point where you can do some (admittedly not all) of the same things -- extension methods and lambda expressions can be pushed pretty far. I'm interested to see what people will do with them and if some of those aesthetics end up crossing over. What will be really interesting is to see how the overall design of the BCL evolves now that we have these core features at the language layer and a few key features (LINQ, WPF, WF) that tend to really light up with libraries that embrace functional rather than inheritance-based extensibility and composition.

As we were working through the design of the UriTemplate API, we had three broad classes of scenarios in mind. One class involves creating URI's that conform to a common pattern, which our implementation supports via the BindByName and BindByPosition methods. You use these methods when you need to create the URI for a new request or build a URI that someone else will later dereference via an embedded link. I blogged some sample code that shows how this works in a previous post.

A second, more interesting class of scenarios involves determining whether a specific URI matches a given pattern. If the URI matches the pattern, we can treat the URI as structured data and parse it according to the pattern which describes its structure. This functionality is implemented by the Match and MatchSingle API's on System.UriTemplate.

Addressing the second class of scenarios (which involve a single URI and a single pattern at a time) opens the door to the third class of scenarios, which involves matching a candidate URI against a set of templates and taking some arbitrary action based on which subset of the templates matched the candidate URI. This is implemented in the System.UriTemplateTable class which forms the basis of our URI dispatch engine, the discussion of which will probably get punted to a later post.

Many aspects of the final design were influenced by the second and third classes -- as much if not more so than the simple case of Bind().

A bit about template syntax

When you write code like the following:

UriTemplate t = new UriTemplate( "literal/{var}?k1={v1}" );

you are defining a pattern expression over the path and query components of the URI. The path component is divided into segments, where each segment in the template can be one of three tokens:

• A literal segment. Just what you'd think -- the corresponding segment in the URI must match this literal string. The match is done case-insensitively against the URL-escaped template string (canonicalization and escaping for template strings follow the logic of System.Uri).
• A named variable segment -- the name of the segment is surrounded with curly braces. Variables can match whole segments only, and the individual names of the variables within the template must be unique. During Match(), we use the variable name to create a key/value pair where the key is the name of the variable and the value is the unescaped value of the corresponding segment in the URI being matched.
• Path wildcard segments. We allow a special token (written as *) to appear at the end of the template as a stand in for 'the rest of the path'

The query component is written as an unordered sequence of name/value pairs. The left-hand side (the key) must be a literal value and cannot be templatized. The right-hand side can be a literal value or a named variable in {curlyBraces}.

Bind and Match are inverses

One basic operating assumption you can make about UriTemplate is that the Match() operation is the logical inverse of Bind(). If you create a URI from a template and a lexical environment using BindXXX(), you can assume that calling Match on the same template with the URI you just created will result in a successful match and create the same lexical environment you started with:

Uri baseAddress = new Uri( "http://localhost:81" );
string artist = "Led Zeppelin";
string album = "Four";
 
UriTemplate template = 
 new UriTemplate("music/{artist}/{album}?format={format}" );

Uri boundUri = template.BindByPosition( baseAddress, artist, album, "rss" );
 
//boundUri:
// http://localhost:81/music/Led%20Zeppelin/Four?format=rss

UriTemplateMatch match = template.Match( baseAddress, boundUri );

Debug.Assert( match != null );
Debug.Assert( match.BoundVariables["artist"] == artist );
Debug.Assert( match.BoundVariables["album"] == album );
Debug.Assert( match.BoundVariables["format"] == "rss" );

Although it's technically a weak invariant[1], it's a handy rule of thumb for reasoning about the intent of the design.

UriTemplateMatch

The return value of a successful match is an instance of System.UriTemplateMatch (a failed match will return null). Inside of UriTemplateMatch are a number of useful constructs:

    public class UriTemplateMatch
    {
        public UriTemplateMatch (){}
   
        //The starting point for the template match
        public Uri BaseUri{ get; }

        //The full candidate URI (as it came in off the wire -> untouched)
        public Uri RequestUri{ get; } 

        //the set of path segments, starting after EndpointUri -> decoded
        public Colletion<string> RelativePathSegments { get; }

        //a name/value view of the entire query string -> VALUES are decoded
        public NameValueCollection QueryParameters { get; }

        //a name/value view of the template variables -> decoded
        public NameValueCollection BoundVariables { get; }

        //the template that was matched -> whatever you passed into the ctor of UriTemplate
        public UriTemplate MatchedTemplate { get; }

        //the unmatched part of the path
        //(for wildcard templates, e.g. {foo}/bar/*) -> untouched
        public Collection<string> WildcardSegments{ get; }
    }

The member most relevant to this discussion is BoundVariables, which contains a name/value view of values obtained by extruding the candidate URI through the template and associating each variable in the template with its corresponding URI segment value. WildcardSegments and RelativePathSegments are quite nice because they give you just the parts of the URI that are relevant to your app (saves you from having to do unnatural acts -- like grepping through the URI to find a file extension, which is basically what you have to do on the platform today...).

I think that's probably enough rambling for one post -- I'll come back to Match() when I talk about UriTemplateTable in the next post.

--------

[1] The only time this invariant breaks is if the values you pass to BindXXX() contain slashes. Escaped or unescaped, slashes are considered evil by System.Uri for security reasons and are never escaped on the wire. As such, a call to Bind() with values containing slashes will result in an output URI with more segments that the template it came from, which will cause the subsequent Match() to fail. This gives rise to my URI canonicalization haiku:

oy, freaking slashes
real or escaped, they all suck.
I blame Roy Fielding...

Just kidding, Roy. :)

You know you want one. I mean, who wouldn't want this wrapped around a coffee mug:

Now available on Cafe Press. Buy 6 for all your friends!

I talked at a high level about the new System.UriTemplate API in my post last week on the Zen of the Web Programming Model but there's definitely more to drill into.

As a start, here's some more extensive sample code that shows the basics of Bind() and Match():

using System;
using System.Collections.Specialized;

//To run this sample locally, create a new Console Application Project and
//add a reference to Microsoft.ServiceModel.Web.dll
public class BasicUriTemplate
{
    public static void Main()
    {
        Uri prefix = new Uri("http://localhost/");

        //A UriTemplate is a "URI with holes". It describes a set of URI's that
        //are structurally similar. This UriTemplate might be used for organizing
        //weather reports:
        UriTemplate template = new UriTemplate("weather/{state}/{city}");

        //You can convert a UriTemplate into a Uri by filling
        //the holes in the template with parameters.

        //BindByPosition moves left-to-right across the template
        Uri positionalUri = template.BindByPosition(prefix, "Washington", "Redmond");

        Console.WriteLine("Calling BindyByPosition...");
        Console.WriteLine(positionalUri);
        Console.WriteLine();

        //BindByName takes a NameValueCollection of parameters. 
        //Each parameter gets substituted into the UriTemplate "hole"
        //that has the same name as the parameter.
        NameValueCollection parameters = new NameValueCollection();
        parameters.Add("state", "Washington");
        parameters.Add("city", "Redmond");

        Uri namedUri = template.BindByName(prefix, parameters);

        Console.WriteLine("Calling BindyByName...");
        Console.WriteLine(positionalUri);
        Console.WriteLine();


        //The inverse operation of Bind is Match(), which extrudes a URI
        //through the template to produce a set of name/value pairs.
        Uri fullUri = new Uri("http://localhost/weather/Washington/Redmond");
        UriTemplateMatch results = template.Match(prefix, fullUri);

        Console.WriteLine(String.Format("Matching {0} to {1}", template.ToString(), fullUri.ToString()));

        if (results != null)
        {
            foreach (string variableName in results.BoundVariables.Keys)
            {
                Console.WriteLine(String.Format("   {0}: {1}", variableName, results.BoundVariables[variableName]));
            }
        }

        Console.ReadLine();
    }
}

Of the two, Match() is my personal favorite. Here's the output from this program:

Calling BindyByPosition...
http://localhost/weather/Washington/Redmond

Calling BindyByName...
http://localhost/weather/Washington/Redmond

Matching weather/{state}/{city} to http://localhost/weather/Washington/Redmond
   state: washington
   city: redmond

More on the template syntax next time when we talk about Match().

The importance of Content-Type

One of the amazing things about the web is that it's a vast interconnected graph of heterogeneous data that can be traversed in a generic way.

Half of this story is enabled through a uniform interface for moving the bits around -- URI's and HTTP methods. The other half is about uniform formats for entity bodies so systems can process data once it gets to them. One without the other is like a yin without a yang.

One of the nice things about the web is that the base mechanism for moving bits around (URI's and HTTP methods) doesn't much care about the format those bits conform to. Rather, there's a general notion of metadata (MIME type) and a well-known location for finding that metadata (the HTTP Content-Type header). And that turns out to be enough to at least bootstrap the whole system.

If you look at the MIME types in use today, some of them are pretty concrete:

• text/html - the standard format for projecting data about how to render user experience
• application/xhtml - another vocabulary for talking about UX
• image/* - a bunch of concrete formats for getting pixels to appear on the screen 

While there are many concrete formats, there are many abstract meta-formats:

• text/xml and application/xml - Good old POX. If you get one of these content types, you know that the abstract data model is fixed but the concrete token space (the set of element qnames) is unspecified.
• application/json - similar to XML, except the data model is atom/record/sequence instead of element/attribute
• application/atom+xml -- constrains POX to list structures and guarantees some well-known metadata about items in the list.
• application/soap+xml -- constrains POX by adding a notion of "headers" and "body"

There's also the grand daddy of them all - application/octet-stream, which says you're allowed to intepret the following stream of bytes as just that -- a stream of bytes.

MIME types (and support for pluggable handlers for specific MIME types) are what enables the web to deal with heterogeneous data without constraining the set of possible data types that might be exchanged.

Formats and the Web Programming Model

WCF interacts with data formats at a couple of different layers in the stack -- both at the encoder level and serializer level (I wrote a long time ago about the difference between the two). The Web Programming Model has new features at both levels to deal with all sorts of different data formats.

At the binding layer, the WebHttpBinding can read and write three different kinds of data:

• XML
• JSON (requires Orcas -- not in the CTP)
• Opaque binary streams

Practically speaking, this means we can handle any type of data you throw at us. Worst case, you end up programming against System.IO.Stream. The PhotoFeeds sample in the BizTalk Services SDK has a nice example of this at work.

At the programming model layer, one of the big new features is the managed object model for Atom + RSS -- this makes it easy to produce and consume syndicated data from managed code. You can use the object model stand-alone without WCF, or compose it with [WebGet]/[WebInvoke] for networking.

There's lots of other more "behind the scenes" features that I'll cover in due time.