Scott's got some good comments on the Ruby aesthetic.

I find some of the Ruby idioms to be really interesting. Specifically, the attempt to bring programming language closer to the natural language is an admirable goal. That said, I wonder how well they translate and whether this style is ultimately a help or a hindrance to Ruby coders who don't natively speak English (at the very least, it puts a different slant on the globalization problem).

And I know Matz is Japanese, but it seems like a lot of this comes in the framework/standard libary. I think Ruby has landed in an interesting place -- it has a nice mix of core language features (particularly block closures and extensible types) that enable some nice patterns, but the real trick is in the framework and how it makes use of them to enable the idioms that Scott finds so attractive.

C# 3.0 is getting to the point where you can do some (admittedly not all) of the same things -- extension methods and lambda expressions can be pushed pretty far. I'm interested to see what people will do with them and if some of those aesthetics end up crossing over. What will be really interesting is to see how the overall design of the BCL evolves now that we have these core features at the language layer and a few key features (LINQ, WPF, WF) that tend to really light up with libraries that embrace functional rather than inheritance-based extensibility and composition.

Phew, Mix is over. The conference technically ended yesterday but we had a follow-on event with a few customers to dive deeper into Web Programming Model stuff that Don and I covered in our Navigating the Programmable Web talk (the recording of which is now live on http://sessions.mix.com). Bottom line is that I'm just now getting some time to blog...

I'm really stoked to announce that all the bits that Don and I used in our demo are now available to the public as part of the newly refreshed BizTalk Services SDK, which can be found at http://labs.biztalk.net (webby folks, don't let the BizTalk name scare you -- remember, URI's are opaque ;) )

The feature set in this SDK is really quite amazing. Using these bits, you can do simple HTTP GET's on URI's. You can syndicate data using Atom + RSS. You can build RESTful services that use PUT, DELETE, and other features of HTTP. You can use the Relay service as a broker for doing NAT traversal securely. You can use the new Direct Connect feature of the RelayBinding to get better performance by establishing direct TCP connections with the remote host, even when both sides are sitting behind NAT's and firewalls. You can use the Identity service to establish trust between those two parties, so the whole thing is done securely. And finally (the killer in my book) is you can use the Service Bus capability to do simple pub/sub and eventing. Whew! We've always said that WCF is a big tent -- nothing proves that story more in my mind than the bits in this SDK.

There are a bunch of samples included in the SDK that give you the full tour of the features. I'll be blogging about the web programming model features here over the next several days. One bit of housekeeping about the samples, though -- if you're running the Web samples on vanilla 3.0 (no Orcas beta), than you'll need to slightly modify the samples in order to get them to work. It's pretty straightforward, though -- just replace this stuff:

Update 5/11/2007: We just refreshed the bits on http://labs.biztalk.net. The samples have been updated to work correctly on both .Net 3.0 and .Net 3.5 Beta 1.

using(ChannelFactory<IService> cf = new ChannelFactory<IService>() )
{

    cf.Endpoint.Behaviors.Add( new WebHttpBehavior() );

    //YOUR CODE HERE
}

with this stuff:

using( WebChannelFactory<IService> cf = new WebChannelFactory<IService>() )

{

       cf.Endpoint.Address = new EndpointAddress("http://localhost:8000");

       cf.Endpoint.Binding = new WebHttpBinding();

 

       cf.Endpoint.Behaviors.Add(new WebHttpBehavior());

       //YOUR CODE HERE
}

Sorry about that -- we'll get it fixed in the download ASAP.

In the mean time, have fun playing with the bits! The best way to get started is by looking at the samples:

Technology Samples

• Basic Web Programming – this sample demonstrates how to build a simple WCF service that responds to HTTP GET and POST without requiring clients to use the SOAP protocol.
• Advanced Web Programming – this sample demonstrates more advanced capabilities of the Web Programming model, including Uri Templates and support for additional HTTP methods such as PUT and DELETE.
• Basic Syndication – this sample demonstrates how to use the Syndication API to produce and consume data exposed as RSS and Atom feeds.
• Web Activities – this sample demonstrates how to interact with Web resources using Windows Workflow Foundation
• Paste as Serializable Type – this sample demonstrates how to use the Paste-As-Serializable-Type Visual Studio add-in to easily create CLR types from XML instance documents.

Extensibility Samples

· Push-Style Streaming – this sample explores various approaches to streaming API’s in relation to the Web Programming Model.

· Streaming Feeds – this sample explores how the Syndication API can be used to efficiently work with feeds containing large numbers of items, both on the client and the server.

Scenario Samples

· PhotoFeeds – this end-to-end scenario sample demonstrates many aspects of the Web Programming Model in the context of a simple web-enabled photo sharing system.

There are lots of samples for the connectivity and identity stuff too -- keep an eye on Dennis' blog for information on that stuff.

I'm really looking forward to community feedback on this stuff, so if you have comments feel free to post them on your blog (I'll be keeping an eye on the 'WCF Web Programming' technorati tag, so feel free to use that), or just leave a comment here.

Happy hacking!

I think the skill tree would go something like this:

Level 0: Line Numbers and GOTO
Level 1: Iteration and Procedural Abstraction
Level 2: Classes and Encapsulation
Level 3: Virtual functions and Inherit-And-Override
Level 5: Pointer Arithmetic and Bit Twiddling

Once you hit Level 5, you're pretty much done with the starter dungeons and can head off to seek your fortune across the countryside.

It's interesting to note that Pointers are now coming much later in the development cycle. These used to be a Level 2 thing; now that many introductory classes are taught in managed language I think it's something people encounter much later on...sometimes, sadly, not at all...

Anyway, once you hit level 5 you're free to pursue membership in one or more Guild. It's common to maintain some level of membership in more than one, but few people rise to high ranks in all of them. The three biggest guilds are as follows:

The Object Oriented Guild

• Design Patterns
• Separating interface from implementation
• Conditional elimination through polymorphism

The Highly Concurrent Guild

• Asynchronous paths and threading optimizations
• Multithreaded state management
• Lock-free concurrency

The Metaprogramming Guild

• Functional programming
• Closures and Higher-order procedures
• Code is Data, Data is Code

I make the observation that one of these Guilds is full up with Java guys; I know lots of folks around the Big House who are trying to make it easier for Joe User to get into the other two...

I came across Eliotte Rusty Harold's missive PUT is not UPDATE via rest-discuss.

Not gonna lie -- I think that reducing PUT to bitblt() is a pretty limiting view of HTTP.

The essential difference between PUT and POST is that POST allocates a new name while PUT doesn't [1]. Neither verb has anything to say about representation.

But I agree -- the SQL analogy of PUT <-> UPDATE is tired. PUT isn't UPDATE, it's

public virtual Representation ApplyState( Representation state );

For PUT, the request-uri is the "this" pointer. The object behind the "this" pointer can apply the incoming state in whatever way it wants -- including but not limited to a simple bitwise copy of the new state into itself.

Just the opinion of one dude with a blog, naturally.

[1] That's the purist view of POST, of course...

Congrats to my friends Kabir and Jigna -- their startup, Bluedot.us, is getting some positive buzz around the web. They did an interview with Scoble last week, which is up on PodTech. Check it out. 

If you haven't heard of Blue, they're a social bookmarking site similar in spirit (but better done, IMO -- and I'm not just saying that because my friends run the company) than del.icio.us. The Blue guys win on user experience, hands down...their site is much more something normal people could use, while del.icio.us is a little too geeky for the masses.

I love the two-way RSS integration -- you can point Blue at an RSS feed from your blog and have items in the feed show up as Dots, or you can get all your dots out of Blue as an RSS feed. That's killer.

Keep up the great work guys!

Breaking radio silence to make note of the fact that John Lam will soon be hanging out Building 42, since he's taken a PM job on the CLR team working on dynamic language support. That's freaking sweet -- RubyCLR here we come :)

Details are on his blog: Dynamic Languages, Microsoft, and me

Congrats to Jim Hugunin and the IronPython team for shipping IronPython 1.0 today! The bits are up on CodePlex.

A dynamic language implementation targeting the CLR built by Microsoft and released under a BSD license...what's the world coming to? :)

I just noticed that Eric Lawrence has released version 1.2 of his fantabulous Fiddler HTTP debugging proxy (get it now at http://www.fiddlertool.com).

If you're doing POX/HTTP work (with Indigo or pretty much anything else on Windows), Fiddler is definitely worth having in your toolbox.

One nice little addition in Fiddler 1.2 -- you can bump the font size up! This makes Fiddler much more suitable for using in demos and presentations, which is something I've been wanting to for a long time. Thanks Eric!

More fun with list comprehensions...I started investigating some of the language plugins for Dr. Scheme and found out that one of them (Swindle) support Haskell-like list comprehensions [1].

Here's a very simple list comprehension in Swindle:

>(list-of x (x <- '(1 2 3))
(1 2 3)

list-of is one of several collectors; there are others. For example, using product-of instead of list-of evaluates to 6 in the above example.

Projections on the accumulated data are handled thusly:

> (list-of (list x (* x x)) (x <- '(1 2 3)))
((1 1) (2 4) (3 9))

Similar to Haskell, Swindle supports range definitions in the generator:

> (list-of x (x <- 0 2 .. 10))
(0 2 4 6 8 10)

Using clauses like where  and until combined with predicates gets interesting:

> (define (less-than-5 x) (< x 5))
> (list-of x (x <- 0 1 .. while less-than-5))
(0 1 2 3 4)

You can even write the world's wierdest for loop:

> (loop-for (x <- 0 then (+ x 1) until (= x 10)) do (display x))
0123456789

Comprehensions in Swindle support arbitrary numbers of generators. For example, here's a simple Cartesian product:

>(list-of (list x y) (x <- '(a b c)) (y <- '(1 2 3)))
((a 1) (a 2) (a 3) (b 1) (b 2) (b 3) (c 1) (c 2) (c 3))

You can use the when clause to apply filtering:

> (list-of (list x y) (x <- '(1 2 3)) (y <- '(1 2 3)) when (<= y x))
((1 1) (2 1) (2 2) (3 1) (3 2) (3 3))

Swindle also supports the ability to define your own iterators as one-arg functions:

> (define (foo yield)
      (yield 'hello)
      (yield 'list)
      (yield 'comprehensions)
      (yield collect-final))

> (list-of x (x <- (function-iterator foo)))
(hello list comprehensions)

Not sure why I explicitly have to yield collect-final in that example, especially given this output:

> (list-of x (x <- (lambda (yield) (yield 'hello) (yield 'world))))
(hello world)

Anyway, pretty cool stuff. Sort of lessens my desire to learn Haskell now, though :)

[1] It looks like Swindle has pretty much everything and the kitchen sink. Sometimes two kitchen sinks, from the looks of things. Need to check this out more. 

When it comes to academic approaches to teaching Computer Science, there are three major schools of thought. You have the MIT cabal, which is Scheme-based, the CMU cabal which is all about ML and the Yale cabal, which is centered on Haskell. Northwestern is part of the MIT cabal, so most of my academic exposure to functional programing concepts came via Scheme/LISP.

Last week on a lark, I decided to see how the Haskell crowd lived. I checked out a couple books from the MS Library, downloaded a Haskell environment, and started puttering. I'm by no means a Haskell virtuoso now, but I'm getting a decent flavor of the language. At least enough to begin to see the various Haskell-isms that are popping up with increasing frequency in things like LINQ and other projects that are on my radar.

The guard syntax in Haskell is pretty nifty, especially when it comes to defining recursive functions. For example, here's factorial in Haskell:

  fact x | x == 0 = 1
         | otherwise = x * fact (x-1)

and in Scheme:

   (define (fact x)
      (if (eq? x 0 )
           1
           (* x (fact (- x 1)))))

I like the way Haskell lifts the explicit conditional into the function definition syntax.

I like the pattern matching capabilities of Haskell too:

pop [single] = single
pop [first:rest] = (first,rest)

That's a simple stack pop function defined for two cases (both of which take a single parameter, a list representing the stack). If there's only one element on the stack, we just return it. If there's more than one, return a tuple where the first element in the tuple is the one we popped off the stack and the second element in the tuple is the rest of the stack. The Haskell syntax is again more terse than the Scheme equivalent:

(define (pop stack)
     (if (eq? (cdr stack) '())
             (car stack)
          (list (car stack) (cdr stack))))

One place where Haskell wins over scheme is that Haskell has list comprehensions. List comprehensions are a way of specifying a list of items based on membership criteria, rather than populating the list items explicitly. For example, here's how you pull out the even members of some list s:

[x | x <- s, even x]

Which is basically the following LINQ-ism (I think):

from x in s
where even(x)
select x

Map in Haskell can be done in Haskell as:

map function list = [function x | x <- list]

And in quasi-LINQ as:

public static IEnumerable<TRet> Map<TArg, TRet>( Func<TArg, TRet> function, IEnumerable<TArg> list)
{
    return from x in list
           select function(x);
}

There's an example that clearly illustrates the syntactic overhead of static typic, I think. Anyway, I have much left to explore in this area.

At this point, I have two major beefs with Haskell. The first is that the syntax is almost too expressive -- there's a lot of "magic" going on under the hood sytactically, and until you internalize the magic it's hard to look at a piece of Haskell syntax and fully wrap your head around it. The second (and this may just be a function of my Haskell environment) is that the language doesn't seem to be very REPL-friendly, which gets in the way of experimentation. One of the things that I really like about Scheme is that you can just fire up a REPL and start hacking away -- that feature really lowers the barrier to entry for me.

Got an MSDN subscription? Beta 2 of Vista and Office 2007 was released today...

http://www.microsoft.com/windowsvista/ has more details.

Have fun...

In case you haven't gotten around to buying your own copy of The Reasoned Schemer yet, here's the source code for a nice little predicate logic system implemented in Scheme.

Thanks to LtU for the link.

 

Since it seems like continuations are The New Hotness, here's a nice little introduction to Scheme's call-with-current-continuation:

http://community.schemewiki.org/?call-with-current-continuation 

Nikhil posted his wish list for browsers and scripting.

One thing that I wish for in the CLR world is native support for the "objects as associative arrays" concept that's fundamental to many dynamically typed scripting languages. Being able to access an object's state via either foo.bar or foo["bar"] is tremendously powerful.

It irritates me that converting a structured CLR type into IDictionary<string, object>/IList<object> requires a deep object copy and is thus a linear operation with respect to the number of nodes in the object graph. It would be so nice to be able to convert freely between a statically typed representation and a structurally typed representation in sublinear time.

Viva la expando! (in spirit, but almost certainly not in practice)

Update: fixed typo

public void Foo<T>( T instance )  where T: has DataContractAttribute
{
   ...
}

If only...

I'm heading out tomorrow night for Mix '06.

If you want to talk about POX, REST, RSS, AJAX, Indigo, or whatever, I'll be spending a lot of my time hanging out in the Sandbox. I won't quite have Phil Haacks' eye-burning aloha shirt, but hopefully I'll be easy to find.

This should be fun. I haven't gone to a conference with the sole purpose of having conversations with customers and getting a feel for their issues in a really long time. I'm sure it will be educational.

See you there.

This is illustrative of a rather fundamental and all-too-common misunderstanding: REST-RPC.

Of course, the project description includes the obligatory quote about Amazon:

Amazon offer both a SOAP and a REST-compliant API. Tim O'Reilly expects as much as 85% of the usage to go via the REST interface; see his blog entry "REST vs. SOAP at Amazon".

I've already made my thoughts known on the alleged RESTfulness of the Amazon API.

REST != POX

Thanks to Stefan Tilkov for the link.

BTW, looks like I'll be heading to the Mix 06 conference at the end of March. That's shaping up to be really cool.

Paraphrasing Mark: Apps do not become magically interoperable simply by exposing a WSDL document. True interoperability is directly a function of the number of clients that can understand the application protocol encoded in that WSDL. (Insert obligatory reference to the REST principle of constrained interfaces).

Is there a RESTful corollary? Something like: Apps do not become magically interoperable simply by exposing representations of their internal resources in a RESTful manner using HTTP as the application protocol. True interoperability is directly a function of the number of clients that can understand the representation of a resource living behind a given URI. 

An example of this corollary: remember back in the wild days of HTML 1.0, when all the browser vendors were trying to out-innovate each other by coming up with a whole bunch of crazy tags? You could build a site that was theoretically interoperable (i.e. you implemented HTTP correctly), but at the end of the day that site still looked like crap in certain browsers.

This is an issue that is addressed partially by using the Content-Type: header appropriately. But even if we’re both speaking text/xml, if you send me 

<person>
   <name>Steve</name>
</person> 

when I’m expecting 

<entity type=”person”>
   <characteristic type=”name” value=”Steve” />
</entity> 

I’m still hosed. I suppose we could invent a whole bunch of app-specific content types, but that just seems to punt the problem over to a different part of the stack. 

In order to interoperate, two parties need to agree on both the verbs they use as well as how the nouns are represented (why else would we have RDF?). Most discussions of REST and the constrained interface principle seem to skip over that fact.

REST: it’s not just HTTP…

Contributing this handy script to the collective memory, because I always forget how to do this:

This little blob of VBScript converts from a Windows account name to the associated SID:

strComputer = "."
Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")

Set objAccount = objWMIService.Get _
    ("Win32_UserAccount.Name='user',Domain='domain'")
Wscript.Echo objAccount.SID

It works equally well for local accounts (pass your machine name for 'Domain') and system accounts (ask for Win32_SystemAccount.Name instead of Win32_UserAccount.Name).

Hopefully the number of times you will need to do this will asymptotically approach zero, but I'm blogging it just in case I ever need to use httpcfg.exe on XPSP2 :)

For more details, see http://www.microsoft.com/technet/scriptcenter/resources/qanda/dec04/hey1203.mspx

Brian Vargas: What Does "Web Paradigm" Mean, Anyway? (Part 1)

What about the other direction? Can we bring the desktop to the web? The answer is that we've been trying to from the very beginning! The truth is, the web paradigm is only good at a few specific things. Most notable among them are intentional directed arcs between different nodes on the network (a.k.a. links), commonality in languages, and ubiquity in protocol support. It is notoriously bad at everything else. Notice that web applications have always striven to behave more like desktop applications. Since the very beginning, any web application of any complexity yearned to present a stateful, responsive, user-driven application flow. Sessions, cookies, and Javascript were all created with this in mind. Witness the advent of Ajax as the latest effort in this campaign to make the web more like the desktop. It's the next logical step in a path that began with the <FORM> element all those years ago.

Interesting observations.

I recently had my eyes opened to what's possible with AJAX UI's from http://www.bindows.net (hat tip to Sells). Those guys have gone and replicated pretty much all of the Windows XP common control set as in AJAX form -- the result of which is that their demo app looks very much like an XP UI. Totally blew my mind. 

And then I look at the daily builds of Vista that I have running on the desktop in my office. Suddenly, having "XP in a browser" doesn't seem so cool anymore. Having the "old and busted" available in a browser doesn't seem so neat when you have "the new hotness" sitting right in front of you.

I'm all for up-leveling the browser experience, but I don't think it will ever catch up to the desktop at this rate. Of course, looking at the web 5 years ago, I would have been very surprised to see things like Bindows available today. Maybe the next five years will surprise me.

 

This is sort of yesterday’s news, but it looks like Google’s building a big honkin’ schematized storage system that you can stick all sorts of stuff into. Dare says it reminds him of Amazon’s Simple Queue Service. Personally, it strikes as being a little like WinFS-in-the-sky. Of course nobody on the outside really knows what the heck Google Base is…

From the screenshots I’ve seen floating around the Web, it seems like Google Base is trying to do three basic things:

1)      Provide a backend storage solution for structured data.

2)      Schematize that data and paint it with rich metadata

3)      Provide search capability on both data and metadata to leverage stored information in interesting ways.

All of this fits in well with Google’s apparent strategy of “own as much information as possible, surface it through broad-reaching apps and API’s, and monetize it through advertising”.

(2) is a pretty fundamental step to enabling others to build applications on top of this system. Unifying data models is one of the really hard parts of integration work, and there’s a lot of power in being able to look at all the information you know about in terms of a common entity model. Once you have your information universe (infoverse?) framed in terms of these common entities, it becomes much easier to build new apps on top of that data store because the integration costs have been drastically lowered. In essence, you’re spending your time figuring out how to make your application cooler and less time worrying about how to make your Customer look like my Customer.

One thing that’s pretty apparent is that solving (2) is really hard. Building generalized, canonical schemas for things like Customers, Contacts, Events, [insert data entity here] is a tremendously challenging knowledge representation perspective (it’s hard to find a “one size fits all” solution). Google appears to tackle this by letting users schematize their information themselves (see this screenshot – note the “define a new type” option). It will be interesting to see if they’re going to try to apply a tagging/folksonomy approach to this – I define my version of a Recipe, you can use it if you want. Maybe they’re looking to “the community” to grow up a set of canonical schemas in sort of an ad hoc manner.  I suspect that if they go down this route they’ll have to solve one of the big problems with tagging today – proliferation of synonyms, where everyone uses different terms to define the same thing.

So where’s the money in this for Google? First off it gives people yet another reason for people to go look at a Google-owned site. More eyeballs == more ads == more money. I’m also wondering if they’re not trying to make something of a standards play – leverage the reach of Google Base and turn it into the de facto data model that every application speaks (why invent your own schema when everybody else already uses Google’s?). I don’t know how they’d monetize that leverage directly, though, since it doesn’t cost anything to use the format...

We'll see where this goes.

Looks like Aaron’s caught the WSE transport bug. Not too long ago, he brought the SMTP transport I hacked out in the WSE 2.0 days up to date for the 3.0 release. Now he’s written an HTTP transport that uses HTTP.SYS so you can host it outside of IIS. Cool!

Erik Meijer's guest blogging over at Lambda the Ultimate, talking about linguistic pedigree of the new VB.NET / Linq features. Fun read for PL geeks.

VB9 is making me (almost ;) ) want to be a VB programmer.

The coolest VB9 feature in my mind? Dynamic interfaces, hands down.

Dynamic interfaces are a way of declaring your expectations to the compiler without actually asserting demands about the concrete type of something. The compiler uses the dynamic interface to give you statement completion and design time IntelliCrack, but everything is late bound at compile time so you're really building a dynamically typed system. In his post, Erik refers to this as "strong duck typing" which seems appropriate.

The dynamic interfaces story gets better when you realize that they've taught the binding infrastructure about XElement's, effectively turning interface member accesses into name lookups under the hood. That maps very nicely to how I think about structural typing and XML.

I'd love to see them take this a step futher and introduce the dynamic interface concept directly into the CLR, so it wasn't just a VB thing. I didn't get to see any of the dynamic-languages-and-the-CLR talks at PDC, so maybe I have a pleasant surprise coming. I hope so.

JSON (JavaScript Object Notation) is a data exchange format that’s becoming increasingly popular among the AJAX crowd. Several AJAX frameworks (e.g. Ajax.NET and Microsoft’s ‘Atlas’ project) are embracing JSON on the wire because it’s very simple to produce and consume from a JavaScript-based browser environment.

JSON and objects

Linguistically, JSON is nothing more than JavaScript’s standard object initialization syntax. JScript has long has the ability to initialize objects using a lightweight syntax similar to the following:

     var person = { name: “Steve”, age: 27,  jobTitle: “Program Manager”  };

When the JScript interpreter evaluates this statement, the result is that the person variable holds an object with data slots labeled name, age, and jobTitle. That is, the statement person.name evaluates to “Steve” and person.jobTitle evaluates to “Program Manager”. When you realize that JScript has the ability to dynamically evaluate arbitrary strings via the eval() method, it doesn’t take a great intellectual leap to see how the object initialization syntax can be applied to data exchange. All I have to do is send you a string containing valid initialization syntax by some mechansim (e.g. XmlHttpRequest). You then call eval() on that string and then you’ve got a nice object structure that you can party on. JSON is attractive precisely because the client-side deserialization routines are already embedded in the core JScript platform (you unfortunately have to write the client-side serialization piece yourself, but that’s almost trivially easy to do using JScript prototypes and a few lines of code). That is, you can send JSON to a script-enabled browser and be reasonably sure they’ll be able to evaluate it — something not necessarily true with XML today.

From a data model perspective, the JSON data model is atoms, pairs, sequences, arrays. LISP-heads are shaking their heads going ‘all you need are atoms and pairs’, but the JSON data model maps very well into both Indigo’s [DataContract] system as well as most object-oriented type systems (the nullability of value types may cause issues in some places, however). JSON’s atoms are strings, numbers, booleans, and the ‘null’ token. Objects are expresses as sequences of name : value pairs, with arrays being a special type of sequence containing an ordered list of unnamed values. Beyond that, there’s not much else that you need to know to build a JSON parser (all the details can be found at JSON.org).

JSON and nominal typing

One thing that I really like about JSON is that strict JSON has absolutely no notion of nominal type. JSON documents consist of anonymous complex types that remain forever unnamed. For example, consider the following JSON document:

{
  name: “Steve”,
  address: {
       street: “2323 Foobar Lane”,
       city: “Seattle”,
       state: “WA”,
       zip: 98911
  }
}

Notice how the value of the address field is a complex structure but there’s no idea of “type name” anywhere in the intialization string. When this document is evaluated, the address slot will be bound to an instance of some anonymous complex type. You can drill into this object and ask it for its data (it knows about stree, city, state, and zip — you can ask it for other fields but it will return ‘null’. Asking for missing members is intentionally not an exception — see below on open content models). Compare that to the following code, which is legal JScript but not technically legal JSON:

{
  name: “Steve”,
  address: new StreetAddress( “2323 Foobar Lane”, “Seattle”, “WA”, 98911 }
}

This code requires a shared concept of a nominal ‘StreetAddress” type and requires that both the serializing party and the deserializing party know about it. As long as the client side has a preexisting notion of the Address type the eval() trick still works, but this rubs me the wrong way. I think that you want to keep both sides as independent as possible, and avoid coupling both sides to a shared nominal type system if at all possible. It’s my opinion that data should arrive on the wire gooey and typeless; it’s up to the party consuming the data to inflict whatever local notion of type it needs to. Coupling happens at the structural level (“I need a string called name and an int called zip ”) not at the nominal type level (“I need an instance of a Quux”). Shared nominal typing just makes everything more difficult, and when you’re dealing with a dynamically typed language like JScript that doesn’t really require nominal typing to be successful I think it makes sense to avoid it.

 JSON and XML

One of the nice things about JSON is that there’s a trivial isomorphism between a JSON document and an equivalent XML Infoset. For example, the structure above can be represented in XML as follows:

<object>
   <name>Steve</name>
   <address>
         <street>2323 Foobar Lane</street>
         <city>Seattle</city>
         <state>WA</state>
         <zip>98911</zip>
   </address>
</object>

One side observation is that the JSON representation is much nicer, as the data-to-markup ratio is much higher. Anyway, there are a few things related to array serialization that you need to shake out but in general the process of creating a JSON-equivalent Infoset from an arbitrary JSON document is straightforward and mechanistic.

Because this isomorphism exists, I look at JSON as an alternative serialization of element-normal XML sans namespaces. This is a very useful comparison to make in my mind, because there’s been a lot of work already done to figure out exactly how far you can push this type of XML data. The lack of a JSON equivalent for XML attributes is less troubling to me than the lack of namespace support – but all the arguments both for and against XML namespaces can also be applied to JSON. This comparison is a helpful framework for judging the technical capabilities of JSON within an overall architecture.

JSON-equivalent Infosets and XSD schemas

I currently know of no technologies for describing the expected structure of JSON instance documents in a machine-readable way. As such, the “metadata problem” is something that is definitely on the front burner when it comes to JSON. However, because JSON instance documents are really just a special class of XML instance documents in disguse, it’s probably easier to focus on describing the JSON-equivalent Infoset using exsiting XML technologies and describe the actual JSON itself indirectly. That is, I could write a description of the JSON-equivalent Infoset using something like an XSD schema or a RelaxNG grammar and let you figure out what the corresponding curly-brace representation would look like by applying the mechansitic conversion from XML to JSON.

Of the two description options (XSD and RelaxNG), RelaxNG is unquestionably the better option when it comes to describing JSON documents. This is because interior forest nodes in a JSON documents are implicitly unbounded (they support open content) and unordered — elements at the same level of the heirarchy are not constrained to appear in any particular order. That is, a JSON consumer doesn’t care if you send city, state, zip or zip, city, otherStuff, state so long as the expected elements are all present eventually. You could potentially describe JSON Infosets using XSD’s xs:all construct, but that would require a closed content model as xs:all does not compose well with xs:any due to the inherent UPA violations. RelaxNG does not have these sorts of restrictions and is quite capable of validating JSON-type grammars. As such, it’s technically a much better option for describing JSON-equivalent Infosets.

Conclusion

I can definitely see why JSON appeals to web developers. It’s a simple, lightweight data format that’s extremely easy to produce and consume from the environments in which web developers work. The protocol overhead of JSON is significantly smaller than the corresponding XML, and any JScript environment that supports eval() can consume JSON data and render it in the local programming model as objects. JSON is essentially equivalent to element-normal XML minus namespaces, and that isomorphism is both a blessing and a curse. It’s helpful because existing XML technologies can be brought to bear on JSON without a large degree of pain, but it also means that JSON will be unequipped to solve the large-scale integration problems that the XML community addresses via XML namespaces. I can see JSON being successful for point-to-point integration and data exchange between a brower client and a server backend, but I don’t see it rivaling XML as a data exchange format for large-scale data integration anytime soon. AJAX and JSON are at the point where XML, SOAP, and XML-RPC were about 5 years ago; they just haven’t realized that yet.

 

Last time around, I was excited about PDC because I didn’t know what was coming and I didn’t know what to expect. This time around, now that I’m inside the belly of the beast, I’m excited precisely because I do know what’s coming. That’s all I want to say, because I don’t want to spoil the surprise J

I’ll be at the conference all week, so if you want to talk about hosting WCF services or just generally geek out, stop by the Communications track lounge and say hello.

It’s going to be a great week. I’m looking forward to being able to blog about some of the stuff that I’ve had to keep under my hat for a long time now, so expect to see things pick up around here too once PDC gets under our collective belts.

Until then…see you in L.A.!

John Cavnar-Johnson wrote a piece called “Mort gets the message” over on Pluralsight, where he talks about how Indigo looks at structured data.

He’s absolutely right when he says “We need an easy way to send and receive structured (but not necessarily typed) messages”.

I think the reason people like Tim and Steve advocate the bare-XML approach is because that’s the only programming model that lets you work directly with the data in a purely structural manner. When you think about your data as “an element named Foo, whose children are Bar and Baz…”, you don’t think about Foo being an “instance of” some nominal type. Taking a nominal type system out of the picture reduces complexity.

Nominal type systems are good for classifying sets of behaviors, but not very good at describing structured data. The reason why is that there are simply more combinatorial possibilities in the world of data. To give each possible structure a name (which is what you’re doing with a nominal type), you either need a really large number of types (e.g. Foo, FooWithJustBar, FooWithBazAndOtherStuffButNotBar, FooWithBarAndBazOrExactlySixQuuxes, etc) or a really complex type description language (e.g. XSD). Neither are particularly attractive.

Dynamic languages like LISP, Python, JavaScript, or Ruby solve this problem by getting rid of nominal typing everywhere. Doing dynamic data processing in these languages is much better than in a strongly-typed language, but you lose the benefits that strong typing brings to behavioral objects. For example, Ruby-with-method-Intellisense would be pretty neat.

Statically-typed languages have their own way of dealing with structurally typed data. This comes in the form of a strongly-typed wrapper for the loosely-typed data, such as XmlDocument. Using these classes, you can manipulate data at a structural level but the experience is quite painful. XmlDocument allows you to cast off the chains of nominal typing, but saddles you with burden of the XML API’s. In the end, you haven’t really gained much in terms of complexity savings.

Ultimately, I think we want to go where Erik Meijer and Peter Drayton want to take us — to a world that’s statically typed when possible, but dynamically typed when needed. Imagine a dynamically typed message body passed around to statically type message processors. There are a lot of possibilities there.

More on this after PDC.

 

 

 

By the way, David Wang (the man behind a lot of the cool automation that I blogged about here) has a blog. Check out his post on Ins and Outs of Virtual Server 2005 Administration and IIS.

Freaking cool stuff.

I’ve been spending a bunch of time lately dogfooding some internal installation automation that our test team will use to get fresh bits of Indigo running on fresh bits of Longhorn and IIS7. I don’t even want to think about the number Virtual PC images I’ve created in the past couple of weeks.

The automation that the IIS team has developed is really pretty sweet. With one command, I can run a script that does all of the following:

• Checks to see if Virtual Server is installed and installs it if necessary
• Creates a new virtual machine and empty .vhd
• Boots the VM into WinPE (the Windows preinstallation environment used by OEM’s)
• Formats the .vhd
• Runs an internal tool to grab the lastest Longhorn bits out of the IIS build lab and install them on the VM
• Install IIS7

After that’s done, control passes to the Indigo automation:

• Connect to the Indigo drop server and grab the latest retail build
• Install Indigo and the MSMQ QFE’s
• Run a tool to register Indigo with IIS7
• Grabs the necessary components of our build verification tests and runs a  subset of the Indigo BVT’s

How does it do all this? JavaScript. Lots and lots of JavaScript :)

It’s pretty amazing what you can do with command-line batch files and JScript automation. If this is state-of-the-art, then I’m really interested to see what happens when Monad ships and the command shell becomes an order of magnitude more powerful.

 

I haven’t been keeping a close eye on Ralf Westphal’s series on Software Cells, but maybe I should be. This picture is really quite interesting, especially if you take its recursive nature into account.

I like the fact that Ralf hasn’t tried to depict what each circle is supposed to represent. That makes the diagram “zoom” well.

Zoom all the way in and this diagram could represent a call tree, showing a single entry point that calls smaller functions which themselves call even smaller functions. The arrows represent function calls.

Zoom out a little bit and imagine the outer circle as an object library. If you diagram out the interactions between the various parts of Enterprise Library, you’d get a diagram that looked like this.

Zoom out a little bit more and imagine the outer circle as the IT structure of a single enterprise, with the exterior circle reprsenting a business function and the interior circles being the various line-of-business applications that support it. The arrows de