Category: technology

CF8: AHAH!

In my last few posts, I looked at the mechanisms CF8 provides to get data into the browser. Just getting data is not enough, of course – you still need to plug that data into elements in the browser’s HTML DOM tree so that is is made visible to users, allowing them to view and interact with it. This calls for JavaScript skills, and possibly the use of toolkits like jQuery or Prototype to ease the task. Such applications can be extremely powerful and interactive, but do require a lot of custom JavaScript code.

Your average Web 1.0 application performs tasks like this on the server – hence the plethora of templating languages, designed to allow programmers to plug data into HTML markup. ColdFusion was the first, of course, and as we like to modestly claim, still the best! 😉

It seems inevitable that someone would come up with the idea to combine these two approaches, asynchronously fetching markup from the server, and plugging that markup into the HTML DOM tree in the browser. It’s a pretty neat idea – let the server process the template for a given fragment of HTML, which allows programmers to code in a familiar server-side templating language, use AJAX to fetch that markup fragment, and push it into the browser’s HTML DOM tree. Given the need for a catchy Web 2.0 acronym, the technique has come to be called AHAH, short for Asynchronous HTML And HTTP, pronounced “a-ha!” or perhaps “ah! ah!” – I’ll leave the choice to you, dear reader.

CF8 provides support for AHAH in a number of different ways – with the ColdFusion.navigate() JavaScript API, the various AJAX “container” tags (CFDIV, CFLAYOUTAREA, CFWINDOW and CFPOD), and the AjaxLink() and AjaxOnLoad() built-in functions. The “container” tags are so called since they act as containers for HTML markup, allowing the contained markup to swapped out for new markup fragments fetched from the server. In my next few posts, I’ll talk about the functionality these tags and functions offer, and how you can use them in your CF8 AJAX applications.

So far, I’ve discussed templating on the server, and using custom JavaScript to plug data into the HTML DOM tree on the browser. There is also a third approach – templating on the browser. To date, the only framework that I know of which accomplishes this is Spry. So here’s another way to build your AJAX applications – use Spry JSON datasets in conjunction with CF8 JSON web services to pull data into the browser. But more on that later…

Beautiful Code

Tim Bray, whose work I admire deeply, is being published along with many other great programmers in Beautiful Code. Follow the link to the O’Reilly site to get a sample chapter, “Finding Things”, which happens to be Tim Bray’s, on search, from small to big.

I like the idea of this book, being as it is more about the art and craft of computer programming, rather than the science.

When I was at Aztec Software, we used to have short half-day courses that were run on a volunteer basis. Amongst these courses was one called, if memory serves right, “Industrial Strength Programming”. The first task students were given was to compile the Apache web server – I love courses like this, since they teach by real-world example. In order to get the compile working, a student was forced to go through, and understand, at least a little bit, the Apache makefile, and in the process, encounter some of the decisions the Apache web server team made while designing their build process. We learn by doing, and we learn best when we see how someone else has solved similar problems, understand what judgment calls they had to make, and so gain greater skill in the art and craft of programming.

Wouldn’t it be great if there were college-level computer courses that dissected, let’s say, Apache Tomcat, or PostgreSQL, and so taught students how theoretical computer science concepts could be applied in real projects, and what trade-offs had to be made? Or (nudge, nudge) if someone were to take apart ColdSpring, MG, Fusebox or BlogCFC, explain how they’re structured internally, what design decisions were taken, and why?

We do this every day, in a smaller way, through our blogs and technical articles, but I do believe that a larger, structured course in the art and craft of programming could prove invaluable. This idea first came up a few years ago, in conversation with a very good friend of mine, and one of the most creative technical minds I know, Ramesh Srinivasraghavan, now Head of Research and Development at Aztec Software.

Open source provides invaluable material to learn from, in the form of source code. But that is only half the story – the knowledge and opinions that guide the design of successful software projects are more valuable by far.

Beautiful Code is definitely going on my reading list – as soon as I have some time to read! At the moment, I’m in the throes of preparations for our move to Berkeley, and, of course, tying neat little ribbons on Scorpio code in preparation for the final release.

For reasons beyond my ken, it’s cheaper on Amazon than direct from O’Reilly. Read it!

CF8: Of Star Wars, Layer Cake and CFAJAXPROXY

Now that I’ve covered the server-side JSON support we have in place, it’s time to talk about something a little different – the client-side tools that CF8 provides to leverage the stuff we’ve put in on the server.CF Developer Encased in Carbonite

If you’re a CF developer, you’d have to be encased in carbonite not to have heard that CF8 adds a slew of new tags and functions to help you build AJAX applications. The first of these to be built was the CFAJAXPROXY tag.

Once automatic JSON<->CFML serialization and deserialization was built into CF’s remote CFC invocation layer, the next logical step was to provide a mechanism that would allow invocation of CFC functions remotely from a web browser in the simplest manner possible. The easy way out would have been to just write a JavaScript function that works like CFINVOKE, perhaps something like this:

ColdFusion.invoke(<Path to CFC>, <Function>, <Arguments>)

We did consider it, but it just didn’t feel natural enough… Continue reading “CF8: Of Star Wars, Layer Cake and CFAJAXPROXY”

CF8: Evolving JSON Support, Part 2

In my last post, I looked at the new JSON functions available in ColdFusion 8, and showed how they could be used to consume JSON web services. Today I’ll look at how CF8 lets you expose your applications as JSON web services.

Even with just the CF8 JSON functions, it is easy to create JSON web services – consume arguments from the caller (in the URL or FORM scopes) in JSON format, deserializeJSON() those to CFML types, run your logic on the CFML objects, then serializeJSON() your return type to get the data back to the caller in JSON format. Easy to do, from a CFC or a CFM.

But with our credo of making hard things easy (and easier!) that just wasn’t good enough.

Lost In Translation?

I’ve always liked the notion that the transformation from and to data formats for communication should be independent of application logic. Taking that a step further, I like to think that application designers should not need to worry about data formats (well, not too much, anyway!) – the client of a web service should be able to decide the data format in which it wishes to communicate, depending on its capabilities. Perhaps one client likes to talk SOAP web services, another prefers XML, and, perhaps, is willing to play with WDDX, and yet another chooses to chatter away in JSON. Once an application designer decides that a particular bit of logic should be exposed as a web service, why should it matter what format the data is exposed as?

RESTful Web Services

Before I get into what we’ve done with CF8 in this space, a little background first – did you know you could invoke CFC functions over HTTP? You should – to my mind, it’s one of the coolest things about CF, which I’ve written about before. Here’s how it works:

http://myhost/my/application/callme.cfc?method=echo&input1=yodel&input2=hey

Simple – form a URL to the location of your CFC, and pass it the function to invoke in the method request parameter (URL or FORM scope). Other request parameters are provided to the CFC as arguments. Of course, you have to be sure that the CFC function you’re invoking is marked access=remote, or CF will not allow you to call it remotely, as a web service. By default, the return format is WDDX – CF will automatically serialize the return value from your CFC to WDDX and return that to the caller. Arguments can be provided to the CFC in complex nested CFML types using the special request parameter argumentCollection, which can take a WDDX-formatted string, which must represent a CFML struct, and is deserialized to form the arguments scope for the CFC function to be invoked.

In the currency of the dynamic web, CF makes it simple to expose CFCs as REST web services. If you are planning to design web services into you applications, and don’t know what REST is, you should – REST, or REpresentational State Transfer, outlines a way of thinking about web services architecture using the mechanisms built into HTTP, rather than reinventing the wheel a la SOAP web services. For more see Roy Fielding’s Ph.D. dissertation (Roy Fielding originated the term, relevant chapter here), and for an easier, more fun, introduction, see Duncan Cragg’s REST dialogues. Personally, I’m a huge fan of REST web services – while SOAP has it’s place, I think it really is needlessly bloated and overly complex for the vast majority of use cases it is applied to.

Talking Tongues

Getting back on topic, CF8 introduces the new returnFormat attribute for HTTP invocations of CFC functions, which allows a caller to specify the format of data they’d like to get back. returnFormat supports the following options:

  • wddx – the default, return data as WDDX
  • json – serialize the return value from the function to JSON and return it
  • plain – do nothing with the return value from the function, return it as-is. Note that the plain return format works only for simple types or XML documents. If you try to return a complex type (struct, array, query) with returnFormat=plain, CF will throw an error, since complex types cannot be returned as-is; they must be serialized to a string representation (WDDX or JSON) to transmit over HTTP. Also note that if you’re returning a XML document, you should remember to set the HTTP response content type appropriately – text/xml for most cases, or as appropriate for the particular XML dialect that you’re returning.

Taking my example above, a user who wants a JSON response would modify the URL like so:

http://myhost/my/application/callme.cfc?method=echo&input1=yodel&input2=hey&returnFormat=json

To flesh things out, CF8 can also accept JSON in the argumentCollection request parameter. For instance:

http://myhost/my/application/callme.cfc?method=echo&argumentCollection={"input1":"yodel","input2":"hey"}&returnFormat=json

For completeness, returnFormat is also available as a CFFUNCTION attribute – if you would rather your remote functions had a default other than WDDX you can set it on your functions, and save callers from have to specify it in their requests.

So what’s it good for?

For starters, you can now invoke CFCs directly from AJAX applications – there any any number of AJAX frameworks, including Spry, jQuery and Prototype, which provide support for consuming data from JSON web services. Oh, and lest we forget, CF8 offers up the cfajaxproxy tag to aid in this effort – but more on that later.

Or, if you buy into the “web as a platform” meme (I know I do!), you could choose to expose public JSON web services from your applications, as do del.icio.us, and many others. I particularly like the fact that services such as del.icio.us, built on the backs of their users’ tag gardening activity, open up the data for others to access and do with as they will. Then there’s the it-makes-commercial-sense approach, with companies such as Amazon leading the way – Amazon Web Services provide an astonishing array of functionality (XML rather than JSON, but that’s besides the point), providing pay-as-you-go services, such as S3, and the eCommerce Service, which allows developers to build their own storefronts on top of Amazon’s catalogues.

Think you could be the next Amazon? CF8 makes the technology end of things easier – by a lot – and it certainly doesn’t hurt that CF8 ups the ante on performance to let you operate at Amazon-scale that much more easily.

CF8: Evolving JSON Support

When we first decided to include AJAX functionality in CF8, well, that was just about all that we had decided! Which was fun, because it gave me a broad mandate to go out and come up with some crazy ideas, and scary, because I didn’t have a clue what to do about it.

So we started small, and built native support for JSON into the server, with a fast Java JSON parser of our own, which provided the basis for the new JSON functions in CF8:

serializeJSON(<CFML object>, <serializeQueryByColumns>)

deserializeJSON(<JSON string>, <strictTypes>)

isJSON(<JSON string>)

I’m not going to get into the specifics of these functions here – Ben Nadel’s done a great job covering them on his blog. Also, a note, these function signatures are not final – we’re tweaking them a bit (just a bit, applications you write now will not be broken!) for the final release. But more on that later.

JSON has been gaining currency as an alternative to XML for data interchange for a variety of reasons: it’s lighter, easier to read and write, maps well to basic constructs in many languages (such as arrays and structs), and, perhaps most importantly for the dynamic web, plays well with yer average browser’s language of choice, JavaScript.

With just these basic functions, it becomes easy to consume JSON-powered web services, mash them up as you will, and spit them back out again. For instance, here’s a tasty morsel of a CFC I wrote to get data from del.icio.us JSON web services:

<cfcomponent output="false">

	<!---
	Gets del.icio.us tags for a user.
	The web service returns a JSON object, mapping tag names to
	numbers of posts with that tag, which deserializes to a CFML Struct.
	More details of the web service available at:
	http://del.icio.us/help/json/tags
	--->
	<cffunction name="getTags" access="remote" output="false">
		<cfargument name="userid">
		<cfset tagsURL = "http://del.icio.us/feeds/json/tags/#userid#?raw">
		<cfhttp url="#tagsURL#">
		<cfreturn deserializeJSON(cfhttp.fileContent)>
	</cffunction>

	<!---
	Gets posts for a user, optionally for a specified set of tags.
	The web service returns a JSON array of objects, one object per
	post. Each post object has the keys:
	u - URL
	d - description
	t - tags
	This deserializes to a CFML Array of Structs, with one Struct per post.
	More details of the web service available at:
	http://del.icio.us/help/json/posts
	--->
	<cffunction name="getPosts" access="remote" output="false">
		<cfargument name="userid">
		<cfargument name="tags">
		<cfset var tagList = replace(tags, ",", "+")>
		<cfset tagsURL = "http://del.icio.us/feeds/json/#userid#/#tagList#?raw&count=30">
		<cfhttp url="#tagsURL#">
		<cfreturn deserializeJSON(cfhttp.fileContent)>
	</cffunction>

</cfcomponent>

CFHTTP to the service URL, deserializeJSON on the response, and, all of a sudden, you have native CFML types to extract data from. Doesn’t get much easier than that! I’ll show off an AJAX application that I wrote with this CFC in a little while.

Given CF’s ability to consume data from a variety of sources (databases, LDAP servers, FTP, MS Exchange, HTTP, etc.), along with native support for common data formats (XML, and now JSON), and the large variety of web services out there, you’re now limited only by your own imagination in the mashup applications you can build. Go forth and multiply!

New Beginnings

So it’s that time of year again – all the cool kids are out at CFUnited. I would have loved to have been there, but it is not to be.

Why? Well, with much reluctance, I am leaving Adobe, for entirely personal reasons. If I could, I would have stayed on here for many years to come, and seen ColdFusion through many more releases. The team is, without a doubt, the finest I’ve ever worked with, it’s been quite an experience putting ColdFusion 8 out, working on everything from language design to server scalability, and it has been a pleasure getting to know all the great folks in the ColdFusion community, with whom I will continue to be involved. As Damon likes to say, you can join the ColdFusion team, but you can never leave… Sinister and megalomaniacal, that’s our Damon! 😉

Anasuya and I will be moving to California, where she’ll get down to the arduous task of writing her D.Phil. thesis, and I’ll enroll in the Master’s at the UC Berkeley School of Information, a.k.a. the iSchool. In recent years, I’ve found myself fascinated by the impact that the technologies we build have upon society. The iSchool’s faculty and roster of courses, which meld computer science and the social sciences, should help shape these interests, and give me space to think about what I want to do next. I’m really excited to be studying at the iSchool – this will be a whole new set of challenges and possibilities.

It’s been a lovely 7 years in Bangalore – I met Anasuya and got married here, studied classical guitar at the Bangalore School of Music, jammed in various bands with weird names, circumnavigated the city on a bicycle, did the Koshy’s Sunday brunch (many times), drank too much beer at Peco’s, and so much more. Sadly, I’ve also watched the city decline in this time. Just as the software and services boom drew engineers to Bangalore, so it also drew land sharks, and their ilk. Couple that with an unstable political environment, and you have a recipe for disaster. I can only hope that the citizen’s groups working to better the city will find success in their efforts.

This blog will continue to be, of course, and while I’ll still be writing about technology, I’ll also be talking about the social impact of computing (intelligently, I hope!). And there will be lots more CF8 in the days to come – I’ve been rather insanely busy helping mop things up for the final release build, but I should have more time free to blog CF8 features from now onwards, especially the AJAX stuff, of which I am particularly proud.

New ColdFusion blogs

A couple of new ColdFusion team blogs for you to watch – as though there isn’t enough Scorpio buzz out there already!

Rakshith, a great guy who worked with me on the AJAX stuff, has got his blog up – he’ll be writing up new Scorpio features that he worked on, as will I, in the days to come.

Ahmad Patan, a member of our QA team, has started the CFPDF blog, to talk about new PDF manipulation and PDF form processing features in Scorpio. The QA guys pair up with development team members right from design time, so they end up knowing the features inside out, upside down, sideways, backward, and any other way you’d like to write your CFML code.