If you want to use the [WebGet]/[WebInvoke] programming model in WCF, you need an endpoint with the right binding and the right endpoint behavior.

The binding is the out-of-the-box WebHttpBinding. The endpoint behavior can be either WebHttpBehavior or WebScriptEnablingBehavior.

What's the difference between the two?

The WebHttpBehavior is a general-purpose behavior that supports UriTemplate dispatch and POX/JSON/byte stream formats on the wire. Use it for general purpose HTTP/REST/Web-Style services.

The WebScriptEnablingBehavior is a "profile" of the WebHttpBehavior functionality designed specifically for interop with ASP.NET AJAX clients. It adds in some AJAX-isms like the ability to automatically generate ASP.NET AJAX client proxies.

I get asked by many, many folks if there's a way to cleanse the .svc file extension from WCF service URI's running in IIS.

Of course there is, and Jon Flanders shows you how.

Thanks for writing that up, Jon!

Performance is something we take pretty dang seriously on WCF. I had the privilege of wearing the Performance PM hat during the last few milestones of WCF v1, so I got an up close and personal introduction to all the work that goes in to squeezing every last bit of performance possible from a bigtime server stack. I wish I could say I was responsible for the fantastic performance we have in the product today, but that honor rightly belongs to every feature developer on the team -- great perf is something that's baked in from the ground up, and it takes a commitment from every single person involved in the code to make a stack perform to its fullest. Of course, having the SWAT team of BobDimp, ErikC and their roving band of brianiacs didn't hurt either :)

The reason I bring this up is because we release the .NET StockTrader application today, which is an end-to-end real-world app that stacks up WCF 3.0's performance against Websphere 6.1. I'm pretty stoked about the summary results:

• .NET 3.0 hosted on IIS with an Http binding and XML encoding offers 124% better throughput than the fastest WebSphere/EJB Web Service implementation tested; and 46% better throughput than the JDBC (no entity beans) WebSphere implementation tested.
• .NET 3.0 self-hosted over Http/XML offers 225% better throughput than the fastest WebSphere/EJB Web Service implementation tested; and 113% better throughput than the JDBC WebSphere implementation tested.
• .NET 3.0 with binary encoding over a TCP binding offers 488% better throughput than the fastest WebSphere EJB Web service implementation tested; and 284% better throughput than the JDBC WebSphere implementation tested.

Gotta love going fast.

Upgraded to DasBlog 1.9.7 last night -- if you're reading this, things seem to be going ok. Still some issues to work out with Free Text Box and the editing UI, but I'll get to those later. Must read Harry Potter now.

Day Three of TechEd Orlando is pretty much in the books -- newsflash: when it rains down here, it really rains.

I've spent most of my time here hanging out in the SOA + Web Services area of the Technical Learning Center. For those of you at Tech Ed, it's the blue one. I'll be there tomorrow as well, so if you have a question about WCF or just want to talk about the web you should come on by.

Tomorrow, my good friends Ed Pinto and Kenny Wolf will be doing a 400-level talk on WCF Architecture and Extensibility. This is the talk to go to if you're interested in exploring WCF from the inside out. It's going to be a great talk; I'll probably pop out of the TLC for an hour or two and go hear the PintoWolf do their thing.

And if you're still here on Friday afternoon, I'm doing a talk on WCF and the Web at 1pm. I'll be talking about UriTemplates, WebGet/WebInvoke, JSON, RSS, and all the other webby goodness I've been blogging about recently. Should be a good time -- hope to see you there.

Scott's got some good comments on the Ruby aesthetic.

I find some of the Ruby idioms to be really interesting. Specifically, the attempt to bring programming language closer to the natural language is an admirable goal. That said, I wonder how well they translate and whether this style is ultimately a help or a hindrance to Ruby coders who don't natively speak English (at the very least, it puts a different slant on the globalization problem).

And I know Matz is Japanese, but it seems like a lot of this comes in the framework/standard libary. I think Ruby has landed in an interesting place -- it has a nice mix of core language features (particularly block closures and extensible types) that enable some nice patterns, but the real trick is in the framework and how it makes use of them to enable the idioms that Scott finds so attractive.

C# 3.0 is getting to the point where you can do some (admittedly not all) of the same things -- extension methods and lambda expressions can be pushed pretty far. I'm interested to see what people will do with them and if some of those aesthetics end up crossing over. What will be really interesting is to see how the overall design of the BCL evolves now that we have these core features at the language layer and a few key features (LINQ, WPF, WF) that tend to really light up with libraries that embrace functional rather than inheritance-based extensibility and composition.

As we were working through the design of the UriTemplate API, we had three broad classes of scenarios in mind. One class involves creating URI's that conform to a common pattern, which our implementation supports via the BindByName and BindByPosition methods. You use these methods when you need to create the URI for a new request or build a URI that someone else will later dereference via an embedded link. I blogged some sample code that shows how this works in a previous post.

A second, more interesting class of scenarios involves determining whether a specific URI matches a given pattern. If the URI matches the pattern, we can treat the URI as structured data and parse it according to the pattern which describes its structure. This functionality is implemented by the Match and MatchSingle API's on System.UriTemplate.

Addressing the second class of scenarios (which involve a single URI and a single pattern at a time) opens the door to the third class of scenarios, which involves matching a candidate URI against a set of templates and taking some arbitrary action based on which subset of the templates matched the candidate URI. This is implemented in the System.UriTemplateTable class which forms the basis of our URI dispatch engine, the discussion of which will probably get punted to a later post.

Many aspects of the final design were influenced by the second and third classes -- as much if not more so than the simple case of Bind().

A bit about template syntax

When you write code like the following:

UriTemplate t = new UriTemplate( "literal/{var}?k1={v1}" );

you are defining a pattern expression over the path and query components of the URI. The path component is divided into segments, where each segment in the template can be one of three tokens:

• A literal segment. Just what you'd think -- the corresponding segment in the URI must match this literal string. The match is done case-insensitively against the URL-escaped template string (canonicalization and escaping for template strings follow the logic of System.Uri).
• A named variable segment -- the name of the segment is surrounded with curly braces. Variables can match whole segments only, and the individual names of the variables within the template must be unique. During Match(), we use the variable name to create a key/value pair where the key is the name of the variable and the value is the unescaped value of the corresponding segment in the URI being matched.
• Path wildcard segments. We allow a special token (written as *) to appear at the end of the template as a stand in for 'the rest of the path'

The query component is written as an unordered sequence of name/value pairs. The left-hand side (the key) must be a literal value and cannot be templatized. The right-hand side can be a literal value or a named variable in {curlyBraces}.

Bind and Match are inverses

One basic operating assumption you can make about UriTemplate is that the Match() operation is the logical inverse of Bind(). If you create a URI from a template and a lexical environment using BindXXX(), you can assume that calling Match on the same template with the URI you just created will result in a successful match and create the same lexical environment you started with:

Uri baseAddress = new Uri( "http://localhost:81" );
string artist = "Led Zeppelin";
string album = "Four";
 
UriTemplate template = 
 new UriTemplate("music/{artist}/{album}?format={format}" );

Uri boundUri = template.BindByPosition( baseAddress, artist, album, "rss" );
 
//boundUri:
// http://localhost:81/music/Led%20Zeppelin/Four?format=rss

UriTemplateMatch match = template.Match( baseAddress, boundUri );

Debug.Assert( match != null );
Debug.Assert( match.BoundVariables["artist"] == artist );
Debug.Assert( match.BoundVariables["album"] == album );
Debug.Assert( match.BoundVariables["format"] == "rss" );

Although it's technically a weak invariant[1], it's a handy rule of thumb for reasoning about the intent of the design.

UriTemplateMatch

The return value of a successful match is an instance of System.UriTemplateMatch (a failed match will return null). Inside of UriTemplateMatch are a number of useful constructs:

    public class UriTemplateMatch
    {
        public UriTemplateMatch (){}
   
        //The starting point for the template match
        public Uri BaseUri{ get; }

        //The full candidate URI (as it came in off the wire -> untouched)
        public Uri RequestUri{ get; } 

        //the set of path segments, starting after EndpointUri -> decoded
        public Colletion<string> RelativePathSegments { get; }

        //a name/value view of the entire query string -> VALUES are decoded
        public NameValueCollection QueryParameters { get; }

        //a name/value view of the template variables -> decoded
        public NameValueCollection BoundVariables { get; }

        //the template that was matched -> whatever you passed into the ctor of UriTemplate
        public UriTemplate MatchedTemplate { get; }

        //the unmatched part of the path
        //(for wildcard templates, e.g. {foo}/bar/*) -> untouched
        public Collection<string> WildcardSegments{ get; }
    }

The member most relevant to this discussion is BoundVariables, which contains a name/value view of values obtained by extruding the candidate URI through the template and associating each variable in the template with its corresponding URI segment value. WildcardSegments and RelativePathSegments are quite nice because they give you just the parts of the URI that are relevant to your app (saves you from having to do unnatural acts -- like grepping through the URI to find a file extension, which is basically what you have to do on the platform today...).

I think that's probably enough rambling for one post -- I'll come back to Match() when I talk about UriTemplateTable in the next post.

--------

[1] The only time this invariant breaks is if the values you pass to BindXXX() contain slashes. Escaped or unescaped, slashes are considered evil by System.Uri for security reasons and are never escaped on the wire. As such, a call to Bind() with values containing slashes will result in an output URI with more segments that the template it came from, which will cause the subsequent Match() to fail. This gives rise to my URI canonicalization haiku:

oy, freaking slashes
real or escaped, they all suck.
I blame Roy Fielding...

Just kidding, Roy. :)

You know you want one. I mean, who wouldn't want this wrapped around a coffee mug:

Now available on Cafe Press. Buy 6 for all your friends!

I talked at a high level about the new System.UriTemplate API in my post last week on the Zen of the Web Programming Model but there's definitely more to drill into.

As a start, here's some more extensive sample code that shows the basics of Bind() and Match():

using System;
using System.Collections.Specialized;

//To run this sample locally, create a new Console Application Project and
//add a reference to Microsoft.ServiceModel.Web.dll
public class BasicUriTemplate
{
    public static void Main()
    {
        Uri prefix = new Uri("http://localhost/");

        //A UriTemplate is a "URI with holes". It describes a set of URI's that
        //are structurally similar. This UriTemplate might be used for organizing
        //weather reports:
        UriTemplate template = new UriTemplate("weather/{state}/{city}");

        //You can convert a UriTemplate into a Uri by filling
        //the holes in the template with parameters.

        //BindByPosition moves left-to-right across the template
        Uri positionalUri = template.BindByPosition(prefix, "Washington", "Redmond");

        Console.WriteLine("Calling BindyByPosition...");
        Console.WriteLine(positionalUri);
        Console.WriteLine();

        //BindByName takes a NameValueCollection of parameters. 
        //Each parameter gets substituted into the UriTemplate "hole"
        //that has the same name as the parameter.
        NameValueCollection parameters = new NameValueCollection();
        parameters.Add("state", "Washington");
        parameters.Add("city", "Redmond");

        Uri namedUri = template.BindByName(prefix, parameters);

        Console.WriteLine("Calling BindyByName...");
        Console.WriteLine(positionalUri);
        Console.WriteLine();


        //The inverse operation of Bind is Match(), which extrudes a URI
        //through the template to produce a set of name/value pairs.
        Uri fullUri = new Uri("http://localhost/weather/Washington/Redmond");
        UriTemplateMatch results = template.Match(prefix, fullUri);

        Console.WriteLine(String.Format("Matching {0} to {1}", template.ToString(), fullUri.ToString()));

        if (results != null)
        {
            foreach (string variableName in results.BoundVariables.Keys)
            {
                Console.WriteLine(String.Format("   {0}: {1}", variableName, results.BoundVariables[variableName]));
            }
        }

        Console.ReadLine();
    }
}

Of the two, Match() is my personal favorite. Here's the output from this program:

Calling BindyByPosition...
http://localhost/weather/Washington/Redmond

Calling BindyByName...
http://localhost/weather/Washington/Redmond

Matching weather/{state}/{city} to http://localhost/weather/Washington/Redmond
   state: washington
   city: redmond

More on the template syntax next time when we talk about Match().

The importance of Content-Type

One of the amazing things about the web is that it's a vast interconnected graph of heterogeneous data that can be traversed in a generic way.

Half of this story is enabled through a uniform interface for moving the bits around -- URI's and HTTP methods. The other half is about uniform formats for entity bodies so systems can process data once it gets to them. One without the other is like a yin without a yang.

One of the nice things about the web is that the base mechanism for moving bits around (URI's and HTTP methods) doesn't much care about the format those bits conform to. Rather, there's a general notion of metadata (MIME type) and a well-known location for finding that metadata (the HTTP Content-Type header). And that turns out to be enough to at least bootstrap the whole system.

If you look at the MIME types in use today, some of them are pretty concrete:

• text/html - the standard format for projecting data about how to render user experience
• application/xhtml - another vocabulary for talking about UX
• image/* - a bunch of concrete formats for getting pixels to appear on the screen 

While there are many concrete formats, there are many abstract meta-formats:

• text/xml and application/xml - Good old POX. If you get one of these content types, you know that the abstract data model is fixed but the concrete token space (the set of element qnames) is unspecified.
• application/json - similar to XML, except the data model is atom/record/sequence instead of element/attribute
• application/atom+xml -- constrains POX to list structures and guarantees some well-known metadata about items in the list.
• application/soap+xml -- constrains POX by adding a notion of "headers" and "body"

There's also the grand daddy of them all - application/octet-stream, which says you're allowed to intepret the following stream of bytes as just that -- a stream of bytes.

MIME types (and support for pluggable handlers for specific MIME types) are what enables the web to deal with heterogeneous data without constraining the set of possible data types that might be exchanged.

Formats and the Web Programming Model

WCF interacts with data formats at a couple of different layers in the stack -- both at the encoder level and serializer level (I wrote a long time ago about the difference between the two). The Web Programming Model has new features at both levels to deal with all sorts of different data formats.

At the binding layer, the WebHttpBinding can read and write three different kinds of data:

• XML
• JSON (requires Orcas -- not in the CTP)
• Opaque binary streams

Practically speaking, this means we can handle any type of data you throw at us. Worst case, you end up programming against System.IO.Stream. The PhotoFeeds sample in the BizTalk Services SDK has a nice example of this at work.

At the programming model layer, one of the big new features is the managed object model for Atom + RSS -- this makes it easy to produce and consume syndicated data from managed code. You can use the object model stand-alone without WCF, or compose it with [WebGet]/[WebInvoke] for networking.

There's lots of other more "behind the scenes" features that I'll cover in due time.

Part 1

I find that I often get a chuckle out of presenting seemingly obvious information in non-obvious ways. That basic juxtaposition and fundamental element of irony goes a long way towards explaining the inner workings of my somewhat obtuse sense of humor (well, that and fart jokes). Anyway, I think that's what I had rolling around in my head when I creating the following slide about the relative popularity of the various HTTP methods:

The point being, of course, that GET and POST are by far the most common methods on the Web today (Side note -- I wonder how different this graph would be had early versions of HTML forms done PUT and DELETE bindings instead of just GET and POST). Which brings us to the second element of the Web programming model -- how we think about verbs and actions on the Web.

"View It" + "Do It"

when it comes to verbs,
everyone knows what GET means
all else is obscure

On the Web, 80% of the world is GET. The rest is relative chaos.

GET is special because, by and large, everybody knows what it means. It has a distinct notion of "retrieval" and it allows clients to make certain assumptions about the safety and idempotency of the request. Safety means that if I issue a GET request to you, it won't inadvertently cause some random side effect to happen as a result (or if it does, the sever will never blame the client for causing those side effects). Idempotency means that the side effects of executing the request multiple times will be the same as the side effects induced by executing the request once -- for the case of GET, because it's safe, it will be zero side effects in either case. Practically, this means I can usually pound on the "Reload" button in my browser without fear. Why am I not fearful? Because GET has bounded semantics.

It's funny to observe how the Web punishes those who violate the bounded semantics of GET. I remember a couple of years back the havoc caused by the Google Web Accelerator. This was a seemingly cool idea -- a little local web proxy/crawler that would prefetch sites you often visit so they load faster. It accomplished this by looking at the sites in your history and following walking the link graph on every page. The funny part came about two days after the thing launched -- lots of people started waking up to find that someone had come along during the night and made all their wiki content go away. In reality, the GWA had simply come along and issued an HTTP GET request to all the DELETE PAGE URI's in the wiki. D'oh. This wasn't the GWA's fault -- rather, it was the fault of web sites that were stepping outside the bounded semantics of GET. Things break when you don't play by the rules.

When you play by the rules, all sorts of interesting things are possible. One of the major reasons the web is scalable is because of distributed hierarchical caches. The reason these things work (why it doesn't matter if your GET request makes it all the way back to the server) is because bounded semantics of GET imply that the server wouldn't do anything interesting with the request once it gets it anyway. Intermediate nodes know they can safely return a cached response precisely because everybody has a common understanding of what GET means and what GET implies. Common understanding == power. 

Compare the bounded semantics of GET to the relatively unbounded semantics of everything else. PUT and DELETE are bounded in the sense that they are unsafe but still idempotent -- although that notion of idempotency gets really fuzzy in the face of multiple concurrent writers. As for POST (which makes up 90% of the remaining 20%), well -- let's just say that there's lots of things POST should do if you read the spec but what a given POST actually does is really anybody's guess.

Once you step out of the world of "view it" and into the "do its", it gets really murky really fast. If you want to talk about the fundamental semantics that drive today's real-world web, there really are just two -- GET and INVOKE, "view it" and "do it". INVOKE is an abstract semantic that can be refined in many different ways for many different purposes.

[WebGet] and [WebInvoke]

You've been able to express both "view it" and "do it" semantics with WCF since the beginning. All you had to do was write a service contract that looks like this:

[ServiceContract]
interface ICustomer
{
  //"View It"
  [OperationContract]
  Customer GetCustomer():
  
  //"Do It"
  [OperationContract]
  Customer UpdateCustomerName( string id, 
                               string newName );
}

Now if you actually went and exposed this contract over the BasicHttpBinding (which uses SOAP 1.1) and looked at a Fiddler trace of the underlying HTTP messages, you'd see that both of these things get sent out using HTTP POST. Why? At this point, it's mainly a SOAP thing. SOAP takes a shortcut and uses the unbounded semantics of POST to subsume the bounded semantics of GET. SOAP 1.2 does have the web method feature (thanks to folks like Mark) but it hasn't really taken off for various reasons. No, SOAP just happily uses POST for everything -- and in doing so, misses out on things like caching that could have potentially been done transparently had SOAP just used the same verb other people were using to mean the same thing.

It's not just a SOAP issue, though. If you take that contract and expose it over an endpoint using the WebHttpBinding + WebHttpBehavior (which WebServiceHost gets you for free, incidentally) you've gotten one step closer because WebHttpBinding doesn't use SOAP. It washes the SOAP off the message using MessageVersion.None. Try the above experiment in this configuration and your protocol traces will still show that GetCustomer is bound to the HTTP POST method. Why? Because the runtime can't read your method names. You have to provide some additional metadata to clue us in that you really want the GetCustomer method to be bound to the HTTP GET method. That additional method comes in the form of the [WebGet] attribute, which is new in the BizTalk Services SDK.

[WebGet] and [WebInvoke] let you control how individual operations get bound to chunks of the endpoint's URI space and the HTTP methods associated with those URI's. For example, adding [WebGet] and [WebInvoke] like so:

[ServiceContract]
interface ICustomer
{
  //"View It"
  [OperationContract]
  [WebGet]
  Customer GetCustomer():
  
  //"Do It"
  [OperationContract]
  [WebInvoke]
  Customer UpdateCustomerName( string id, 
                               string newName );
}

Lets me do things like:

• GET /GetCustomer
• POST /UpdateCustomerName

[WebInvoke] defaults to POST but you can use it for other "do it" verbs too:

[ServiceContract]
interface ICustomer
{
  //"View It“ -> HTTP GET
  [OperationContract]
  [WebGet( UriTemplate=“customers/{id}” )]
  Customer GetCustomer( string id ):
  
  //"Do It“ -> HTTP PUT
  [OperationContract]
  [WebInvoke( UriTemplate=“customers/{id}”, Method=“PUT” )]
  Customer UpdateCustomer( string id, Customer newCustomer );
}

The Advanced Web Programming sample shows this off in more detail. 

That about wraps it up for verbs. I've got one more "meta" post queued up to cover formats and entity bodies, and then its on to code :)

Ouch, it looks like Jef ran headlong into a bit of a brick wall using the Web programming model inside of ASP.NET. 

That one's my fault for not getting the time to write an ASP.NET sample before the SDK shipped. Had I done that, I could have saved Jef some pain and a long blog post.

So Jef, I apologize. I owe you a beer next time I see you. Let me try to make it up to you.

The basic gist of Jef's post is that "there's too much *@#%!^ config required to use your stuff!". I agree. That's why we added two types to the Web programming model for making this easier -- System.ServiceModel.Web.WebServiceHost and System.ServiceModel.Web.WebServiceHostFactory. Both of these guys ship in the BizTalk Services SDK.

I like to think of them as the 'do the right thing' service host for Web-style services. Absent any configuration, they will auto-configure a default Web endpoint at the service's base address. The WebServiceHostFactory knows about IIS metabase settings so it can properly configure the endpoint's binding for HTTPS. It will also add the WebHttpBehavior to the endpoint to enable web-style dispatching.

Net/net, all you have to do to use the Web programming model in ASP.NET is to create a web site project that references the Microsoft.ServiceModel.Web assembly. Once you've done that, you can write the service in one file (e.g. Service.svc):

<% @ServiceHost Factory="System.ServiceModel.Web.WebServiceHostFactory"
                Service="Service" 
                Language="C#" %>
                
using System;
using System.Collections.Generic;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.ServiceModel.Web;
using System.Text;

[ServiceContract]
public class Service
{

    //you can hit this at http://<app>/Service.svc/EchoWithGet?s=hello, world
    [OperationContract]
    [WebGet()]
    public string EchoWithGet(string s)
    {
        return "In EchoWithGet: You said " + s;
    }
}

Note the Factory attribute in the @ServiceHost directive. That's the key to the whole thing. Once you have that, you don't need any service or endpoint config. The only thing you need in web.config is the reference to Microsoft.ServiceModel.Web.

Hope that helps with keeping simple things simple.

Note: in the SDK build, we're a bit aggressive about requiring a trailing slash in some cases (particularly if you have UriTemplate="" or UriTemplate="*"). That will be much more usable by RTM, but for now just a heads up.

In our Mix talk, Don joked about how we were taking off the 'philosoper's robes' when it came to designing this feature set. If you're waiting for the "Four Tenets of REST Orientation" to come out of this work, I think you'll be waiting for quite a while. There are no tenets or orientations here -- just some practical stuff based on what people are doing on the web today. We've made some observations about what people are struggling with on the platform as it is now, identified their pain points, and hopefully given them some aspirin to make their coding lives simpler and more productive.

That said, I would get thrown out of CSD if we didn't have a few guiding technical principles backing up what we are doing. Also, I would hate to deprive you of the quasi-mystical experience you've come to expect from the Indigo team, so I'll posit that all of theory you need to know about our Web features is contained succinctly in the following paragraph (encoded in haiku for your pleasure (well, probably more mine than yours)):

with GET and INVOKE,
it's the URI, stupid.
yes, formats matter

The center of the haiku is about URI's, which also happen to be the center of the model (convenient, that) so that seems like a reasonable starting point.

It's the URI, Stupid

One thing is pretty clear: if you take URI's out of the web, the whole thing falls over.

URI's have some nice properties -- transcribability (the ability to be easily written down) being chief among them. The URI RFC goes into some depth on this feature and how it enables even the low end 'cocktail napkin' style of URI transmission between two humans. This has led to a couple of things -- number one, URI support is pretty ubiquitous throughout the application stack. URI's are supported directly in browsers and file system explorers, and URI-based mechanisms are the defacto app-level metaphor for referencing one piece of information from within another (I know I definitely use Word's Add Hyperlink feature a lot more than I use OLE these days). URI's have become so ubiquitous that folks are starting to optimize away the need for transmission-by-cocktail-napkin by designing systems around URI's that are easy for people reason about and remember directly. They leverage the hierarchical nature of URI paths to logical URI layouts where the hierarchy of the URI path reflects the organization structure of the underlying information space. 

Although the URI RFC and W3C Web Arch are fairly prescriptive about the intended usage of the URI, in the wild people end up using them in a lot of different ways. The usage spectrum runs the gamut from "Pure Opaque Identifier" at one end to "Eval-able structured expression" at the other. A URI like http://www.microsoft.com/downloads/details.aspx?FamilyID=5d9c6b2d-439c-4ec2-8e24-b7d9ff6a2ab2&DisplayLang=en is a good example of the URI-as-GUID school; although it may have some internal structure you're much better off if you just treat it as an opaque sequence of bytes and don't spend too much time trying to understand where it came from or what it means. Contrast that with a URI like http://flickr.com/search/?q=Bill+Gates&page=2 which, if you squint at it hard enough could almost be seen as

       (skip-to-page (search "Bill Gates") 2)

The similarity is much deeper than simple syntactic transformation -- there's a whole remote evaluation motif embedded here.

Given that I know http://flickr.com/search/?q=Bill+Gates&page=2 points to the second page, I'm pretty confidant I can come up with a URI that gets me page 3 of a search on 'Steve Ballmer' without having to read any docs or asking the server to construct it for me.

Of course, there are URI systems that fall in between the two extremes -- take http://www.microsoft.com/silverlight/. Can you treat that as an opaque identifier? Of course. But then again, I bet you could figure out what the URI for 'Windows' was without resorting to a search engine.

No matter where they fall on the GUID <-> sexpr spectrum, we've heard our customers tell us that they want their URI's to be more than just something they bolt onto the system at deployment time. Rather, they want to think about the URI structure of their system as being a primary thing and are finding that building server systems that handle a particular URI space in the way that they want is rather hard.

One of the challenges in building such systems is that URI space for a given service is practically unbounded. The set of all URI's a server might process might be data dependent (consider the case where you give every customer in your system a unique URI), which poses something of a notation problem. How do we write down an infinite set of URI's?

Common practice on the web today is to use some sort of templating syntax to punch holes into the URI at strategic places (much the way String.Format lets you punch holes into an arbitrary string at strategic places). Our URI template syntax looks like this:

              music/{artist}/{album}

The bits in {curlyBraces} are variables, which means that a URI Template is an efficient notation for an (unbounded) set of structurally equivalent URI's.

As part of the BizTalk Services SDK (and ultimately part of .NET 3.5) we're providing a new API for dealing with URI templates called (intuitively enough) System.UriTemplate. There are a couple of interesting things you can do with a UriTemplate once you have one:

• You can call Bind() with a set of parameters to produce a fully closed URI that matches the template.
• You can call Match() with a candidate URI, which extrudes the candidate URI through the template to give you back a lexical environment (a.k.a. a dictionary) with all data that poked through the holes in the template.

Bind() and Match() have this nice property of being inverses so that you can call Match( Bind( x ) ) and come back with the same environment you started with[1]. The code sample below give the basic gist of this API:

Uri baseAddress = new Uri( "http://localhost:81" );
string artist = "Led Zeppelin";
string album = "Four";
 
UriTemplate template = 
 new UriTemplate("music/{artist}/{album}?format={format}" );

Uri boundUri = template.BindByPosition( baseAddress, artist, album, "rss" );
 
//boundUri:
// http://localhost:81/music/Led%20Zeppelin/Four?format=rss

UriTemplateMatch match = template.Match( baseAddress, boundUri );

Debug.Assert( match.BoundVariables["artist"] == artist );
Debug.Assert( match.BoundVariables["album"] == album );
Debug.Assert( match.BoundVariables["format"] == "rss" );

There are many times (especially on the server, where you need a reasonable way of doing URI-centric dispatch) that you want to keep track of a set of UriTemplates in a data structure that has dictionary-like semantics. In our world, that's what System.UriTemplateTable is for. It lets you select the best match given a set of templates and a candidate URI. This is basically URI-dispatcher-in-a-box -- and it's not coupled to any particular networking stack (WCF included) so you can use it wherever you want.

I'll give UriTemplate and UriTemplateTable a much deeper treatment in the next couple posts. For now:

uri templates
are just URI's with holes.
behold, .Bind() and .Match()!

Tomorrow is all about verbs.

[1] That's not always true -- there are evil characters that you can put in the variables you pass to Bind() (e.g. slashes) that System.Uri will special-case the escaping for, which will cause this invariant to fail. But unless you're pathological, this will hold in all the cases you care about.

I've been enjoying reading some of the reactions to our Web Programming Model features -- particularly Jef Newsom's post Some Interesting History and David Ing's Simple Object Apology Protocol. Both make some great points and are well worth the read.

Jef writes:

So, it is interesting to me to watch guys like Clemens Vasters and Don Box (and Steve Maine, though I think he was less heavily involved in 'expectorating' the WS-splat and a lot more heavily involved in consuming and evangelizing prior to getting into the big house) starting to shed the WS-Splat baggage and just GET it. And I think that they are being pragmatic about it and trying to scratch the 80% itch (GET) really well, but they are adding in a decent set of support for the REST (ha, pun, ha ha, er, um...) of the story without being to religious about being RESTful. And when questions come up like [paraphrased] "Well, shouldn't you layer soap into that?" they look very relaxed and calm as they say [paraphrased] "Nah, you're not going to need it here." From the outside, it looks a little cathartic.

Cathartic? Maybe. But I don't think the fact that we're embracing the stuff that works on the web today in any way devalues the rich protocol infrastructure we build in WCF V1. Rather, the mea cupla embodied in these bits is two fold:

• Not every scenario requires infrastructure support for interoperable distributed transactions, or message-based security and reliability through intermediaries. And you know what? That's fine.
• Like all scenarios, complexity should be pay-for-play. Simple things should be simple and hard things should be possible.

I'll admit that we could have done a better job internalizing that last point in V1. We did a great job of making hard things possible (the answer to "how do I turn on X?" is generally "just configure another endpoint with a different binding") but that came at the expense of some really simple stuff like HTTP GET. Sorry about that, we're making that better as we speak.

But the fact that we're coming back to the basics in Orcas doesn't mean we're deleting all that rich protocol code we wrote in V1 from the code base. This isn't an either/or decision -- our "just configure another endpoint" story is still intact, and it's really nice to have that protocol richness in your back pocket because when you

David writes:

By having such a broad choice of what to use (REST, HTTP/POX, SOAP, Remoting etc. etc.) then for the people looking for the 'one simple answer' it just gets increasingly complex and confusing. The complexity tax of these news things are start to look like the federal deficit - always going the wrong way.

This gets back to one of the major reasons why we built WCF in the first place. When we started this thing, Microsoft had a metric shedload of distributed communications API's -- ASMX, MSMQ, COM+, Remoting -- and one of the primary value-adds of WCF is that it unified all these guys into a single framework so you didn't have to learn 4 different API's and figure out how to choose between them to get your job done.

This fundamental value proposition hasn't changed in Orcas.

Rather, the API unification of WCF has just gotten bigger, because now we're bringing a whole bunch of rich HTTP scenarios into the WCF tent. With the Web features in Orcas you can use the high-level WCF programming model in places where you would normally have to start at a much lower layer of the framework (System.Net.HttpListener or System.Web.IHttpHandler) and roll a whole bunch of boilerplate yourself.

Does using WCF mean you don't have to make decisions about things like 'when do I use durable messaging?' or 'should I stick an HTTP GET head on this thing?'. Of course not -- we're a framework, not an oracle. At the end of the day, you still have to make the right call for your app. What we as framework authors can offer you is the simple guarantee that the same core set of concepts, metaphors, and programming idioms you need to learn to be productive on WCF in once scenario will help you be productive on WCF generally. You don't have to learn six wildly different API's anymore. There's a simple answer to the question of 'how do I build distributed apps on the Microsoft platform' and that answer is WCF.

As such, I think the reach programming model work we're doing now actually increases the value of the rich protocol stack in V1 because it gets rid of the traditional scenario cliff between rich and reach. Take a framework like Rails, for instance - which is optimized around making a common set of scenarios very easy. Granted, the stuff Rails optimizes for hits the 80% case. But if you suddenly have to step into the other 20% for whatever reason, you're kind of screwed (as the Twitter guys are finding out). We want to protect folks who take a bet on WCF from falling off similar cliffs in the distributed systems space. We're taking a big step in that direction with the Orcas Web features, and you can expect us to continue down that road over time.

First, go download the SDK from http://labs.biztalk.net.

Now that you've downloaded it, you've got a whole bunch of new stuff to play with. Let me give you the guided tour of the Programmable Web stuff (the Connectivity and Identity stuff will have to wait for another day).

All the assemblies live in the \Assemblies folder underneath root installation directory. If you installed them in the default place, you'll find them in C:\Program Files\BizTalk Services SDK\Assemblies. This is where you can point your Visual Studio "Add Reference" dialog at if you want to build apps on top of this stuff. Inside this folder, you'll find four assemblies relating to the programmable web features. Here's what's in each of them:

Microsoft.ServiceModel.Web.Tools - this one contains a nifty little Visual Studio add-in that makes it easier to generate XML documents that have a specific shape. The idea behind this tool is simple -- lots of folks on the web are eschewing XSD in favor of simple web pages that say "here's what I expect the XML in this request to look like". For example, check out the FlickR API docs. Using the example instance document in that page, you could write your own parser on top of System.Xml.XmlReader and get a working solution, but what you might really like is some set of serializable types that could just parse all that stuff for you. Normally, you'd have to write a bunch of types yourself and mark them up with a bunch of XmlSerializer attributes, but that's kind of a pain. With this tool, you can just copy the example XML blob onto the clipboard, pop over to Visual Studio, and choose Edit -> Paste XML as Serializable Type. The tool will infer the document structure from the example and cough up the code for the XML Serializable type system automatically. Pretty cool, and a nice time saver. Note that you'll need to copy the add-in files to the right directory to make Visual Studio aware of the add-in -- there's handy batch file plus instructions for doing this in the Tooling sample (Samples\Web\Tooling). You may need to bounce VS after installing the tool in order to use it for the first time.

Microsoft.Workflow.Activities - this assembly contains two new Workflow Activities for integrating with simple web services. The WebGetActivity lets you do HTTP GET's on URI's, the WebInvokeActivity does everything else (POST, PUT, DELETE -- you can bring your own verb). These guys are great for bringing web resources into the declarative programs you write with Workflow.

Microsoft.ServiceModel.Web.Client - this is an experimental API we're looking at for improving the programming experience of consuming web data from imperative code. It's a spiritual cousin of the System.Net.WebClient API but has some more advanced features (a pipeline processing model, better redirect support, more access to HTTP-isms). The best place to look at this is the client side of the PhotoFeeds sample, which uses this API exclusively. This one is still quite early and we're really looking for community feedback here to tell us where we should go with this.

Microsoft.ServiceModel.Web - this is the main course. We took as many of the programmable web features we're building in Orcas for WCF and plunked them into this assembly so people didn't have to wait for Orcas Beta 2 to build apps on it and give feedback. The one thing that's in Orcas but not in these bits is JSON support (that has a hard dependency on some internal changes to System.Runtime.Serialization so we could decouple it). There's a lot in here so I'll break it down further:

• System.UriTemplate. You can think of this as the API manifestation of Joe Gregorio's URI Template idea. URI Templates are a great idea, so we built some OM around them and put it into the platform. Now you have an easy way of constructing a whole bunch of URI's that match a specific pattern (you can also go backwards - use a UriTemplate to destructure a candidate URI into a set of pattern variables, which is pretty powerful).
•  System.UriTemplateTable. Simple idea here -- give us a map of UriTemplate -> Some Data + a candidate URI and we'll do an efficient lookup in the table to tell you which template matches the candidate URI. Easier and faster than foreach'ing over a set of regex's. Like it's friend UriTemplate, System.UriTemplateTable is lightweight and stand-alone so you can use it anywhere (even outside of WCF).
• [WebGet] and [WebInvoke] attributes for service operations. These two attributes give the WCF dispatch plumbing the necessary metadata to hang some code (your service operation) at a particular URI Template + HTTP Method. I'll blog a lot more about how this all works in a future post, but [WebGet( UriTemplate = "customers/{customerId}" )] captures the basic zen of the model.
• WebHttpBinding -- this is our new standard binding for this stuff. It uses HTTP/MessageVersion.None + a new encoder that suppoerts XML, JSON (in Orcas) as well as raw byte streams. You hand us a Stream and a content-type, we'll do the right thing. But most of the time, you won't have to worry about configuring endpoints to use this binding because we have the...
• System.ServiceModel.WebServiceHost/Factory - this is a custom service host that eliminates the need to add or configure the WCF plumbing to do simple web-style services. Hosting these things in IIS is a one-liner with zero-config. This class is all about keeping simple things simple.
• System.ServiceModel.Syndication Namespace - this is the manage object model for producing and consuming RSS/Atom feeds. We have a format-netural OM for programmatically manipulating syndication data, plus a set of formatters that can project that data model into concrete formats (we ship Atom and RSS in the box, the whole thing's extensible so you can write your own if you want).

That should at least give you a sense of what's in this SDK (and remember, there's all the connectivity and federated identity stuff that we haven't even touched on yet). I'll be blogging about all of these in future posts, so stay tuned.

Phew, Mix is over. The conference technically ended yesterday but we had a follow-on event with a few customers to dive deeper into Web Programming Model stuff that Don and I covered in our Navigating the Programmable Web talk (the recording of which is now live on http://sessions.mix.com). Bottom line is that I'm just now getting some time to blog...

I'm really stoked to announce that all the bits that Don and I used in our demo are now available to the public as part of the newly refreshed BizTalk Services SDK, which can be found at http://labs.biztalk.net (webby folks, don't let the BizTalk name scare you -- remember, URI's are opaque ;) )

The feature set in this SDK is really quite amazing. Using these bits, you can do simple HTTP GET's on URI's. You can syndicate data using Atom + RSS. You can build RESTful services that use PUT, DELETE, and other features of HTTP. You can use the Relay service as a broker for doing NAT traversal securely. You can use the new Direct Connect feature of the RelayBinding to get better performance by establishing direct TCP connections with the remote host, even when both sides are sitting behind NAT's and firewalls. You can use the Identity service to establish trust between those two parties, so the whole thing is done securely. And finally (the killer in my book) is you can use the Service Bus capability to do simple pub/sub and eventing. Whew! We've always said that WCF is a big tent -- nothing proves that story more in my mind than the bits in this SDK.

There are a bunch of samples included in the SDK that give you the full tour of the features. I'll be blogging about the web programming model features here over the next several days. One bit of housekeeping about the samples, though -- if you're running the Web samples on vanilla 3.0 (no Orcas beta), than you'll need to slightly modify the samples in order to get them to work. It's pretty straightforward, though -- just replace this stuff:

Update 5/11/2007: We just refreshed the bits on http://labs.biztalk.net. The samples have been updated to work correctly on both .Net 3.0 and .Net 3.5 Beta 1.

using(ChannelFactory<IService> cf = new ChannelFactory<IService>() )
{

    cf.Endpoint.Behaviors.Add( new WebHttpBehavior() );

    //YOUR CODE HERE
}

with this stuff:

using( WebChannelFactory<IService> cf = new WebChannelFactory<IService>() )

{

       cf.Endpoint.Address = new EndpointAddress("http://localhost:8000");

       cf.Endpoint.Binding = new WebHttpBinding();

       cf.Endpoint.Behaviors.Add(new WebHttpBehavior());

       //YOUR CODE HERE
}

Sorry about that -- we'll get it fixed in the download ASAP.

In the mean time, have fun playing with the bits! The best way to get started is by looking at the samples:

Technology Samples

• Basic Web Programming – this sample demonstrates how to build a simple WCF service that responds to HTTP GET and POST without requiring clients to use the SOAP protocol.
• Advanced Web Programming – this sample demonstrates more advanced capabilities of the Web Programming model, including Uri Templates and support for additional HTTP methods such as PUT and DELETE.
• Basic Syndication – this sample demonstrates how to use the Syndication API to produce and consume data exposed as RSS and Atom feeds.
• Web Activities – this sample demonstrates how to interact with Web resources using Windows Workflow Foundation
• Paste as Serializable Type – this sample demonstrates how to use the Paste-As-Serializable-Type Visual Studio add-in to easily create CLR types from XML instance documents.

Extensibility Samples

· Push-Style Streaming – this sample explores various approaches to streaming API’s in relation to the Web Programming Model.

· Streaming Feeds – this sample explores how the Syndication API can be used to efficiently work with feeds containing large numbers of items, both on the client and the server.

Scenario Samples

· PhotoFeeds – this end-to-end scenario sample demonstrates many aspects of the Web Programming Model in the context of a simple web-enabled photo sharing system.

There are lots of samples for the connectivity and identity stuff too -- keep an eye on Dennis' blog for information on that stuff.

I'm really looking forward to community feedback on this stuff, so if you have comments feel free to post them on your blog (I'll be keeping an eye on the 'WCF Web Programming' technorati tag, so feel free to use that), or just leave a comment here.

Happy hacking!