jQuery Ajax Transports

1 July 2011
15:30

A few weeks ago I published a jQuery plugin called jquery.iframe-transport.js, which I had written after reading up on the Ajax rewrite introduced with jQuery 1.5. The Ajax rewrite was big news for jQuery, as it enables things like non-XMLHttpRequest transports to be integrated seamlessly, and also because it introduced Deferred/Promise-style programming.

I've long wanted to get rid of the rather heavy-handed jquery.form.js plugin, which I've been using for some time to support traditional HTML file uploads in Ajax-based applications. But the jQuery Ajax subsystem is pretty hard to digest if you're trying to extend it in a non-trivial manner. The documentation and the examples are quite limited at this time, and I had to carefully read through and debug the relevant jQuery code to get my plugin to work.

In this article, I'll explain what this particular plugin does, and why it does that in the way it does. I'm sure there's room for improvement here, and would welcome any feedback guiding me in the right direction.

File Uploads in an Ajax Context

First, some background.

Regular, non-multipart form submissions are easy to convert into Ajax requests: you collect the values that are to be submitted from the various form elements in the DOM, put those values into an array, and use that array as the body of the XMLHttpRequest via the data parameter. File uploads, however, can not be handled that way because the Javascript code (traditionally) only has access to the filename, but not the contents of the selected file (the File API changes that, of course).

So, beyond Flash and the File API, the only way you have to upload a file without triggering a full page reload is to create a hidden iframe and submitting the form with that iframe as the target. Submitting the form that way retains the state of the outer page, which is what we want. The file is uploaded to the server, and the response of the server is in turn loaded back into the iframe.

This is where it can get tricky, because browsers apparently insist on interpreting the content of iframes as an HTML document. If you really wanted to respond with a JSON payload, your response is likely to get garbled by automatic HTML encoding of reserved characters and such. But there's a workaround for that, too: The server side code puts the actual payload in a textarea element inside an HTML document. That way the client-side Javascript code is able to reliably extract the actual payload and work with it.

So what I wanted was a way to make $.ajax() use this technique behind the scenes whenever the form to be submitted included file fields. It should look like a regular Ajax invocation to client code, and do the hidden iframe creation, form submission, and response extraction automatically.

Data Types, Prefilters, Converters, Transports, Oh My!

The jQuery Ajax functionality includes a couple of extension points for adding custom behavior: Prefilters, Converters, and Transports. For each of these you basically register a function that is invoked before, after, or instead of the actual XMLHttpRequest exchange, respectively.

I'll go into more detail about those three hooks shortly, but first lets talk about Data Types, as this is a central aspect of how the Ajax subsystem in jQuery works. jQuery itself defines a few different data types such as html, script, or json. Using the dataType option of Ajax requests, you can specify what kind of response you expect back from the server, thereby telling jQuery how it should process the response before handing it back to your code. But you don't really need to specify the data type in most cases, because it can also be determined by looking at the Content-Type header of the HTTP response. (Rather confusingly, the Ajax options in jQuery also allow you to set a contentType option, but this time you're specifying the content type of the request payload).

In those cases where the specified dataType does not match the server-specified content type of the response, jQuery tries to convert the data into the format you expect. This is done by the converters, a number of which are already included, such as the conversion from plain text to JSON. You can register custom data types, and you can also register converters to convert from/to that data type.

Prefilters on the other hand are invoked before the request is sent. They can inspect the provided Ajax options and make modifications as needed. The documentation includes a couple examples for what this might be useful for, but as far as I can tell they won't help with implementing an iframe transport.

Finally, transports handle the actual mechanism of sending the request and handling the response. They can be registered just for specific data types, or for all data types. When they match the requested data type, they get a chance to inspect the Ajax options of the request, and eventually return an object with methods to send (and abort) the request.

Implementing an Iframe Transport

It's clear that all this extensibility should allow us to make a transport that routes $.ajax() requests through a form submission in a hidden iframe. What was not at all obvious to me (and still isn't), is what a good implementation of such a transport would look like, in particular with respect to data types.

So for now I ended up with registering the transport for any data type:

$.ajaxTransport("+*", function(options, origOptions, jqXHR) {
  // Check the options for file upload fields, and return an object
  // implementing the iframe transport if any are specified.
  // Otherwise, pass on to the default implementation.
});

To even get a chance at getting invoked, I needed to prioritize the transport registration by prefixing the wildcard with a plus sign, which is an (as far as I can tell) undocumented trick I learned by reading the jQuery code.

Although this works, the registration for "+*" is a strong indicator that this code is not ideal.

The other interesting aspect of the code is how it interprets the response. Here's the corresponding code in the plugin (somewhat simplified for better readability):

iframe.bind("load", function() {
  var doc = this.contentDocument.documentElement,
      textarea = doc.getElementsByTagName("textarea")[0],
      type = textarea ? textarea.getAttribute("data-type") : null;
  completeCallback(200, "OK", {
    text: type ? textarea.value : doc ? doc.innerHTML : null
  }, "Content-Type: " + (type || "text/html"));
});

This code looks for the first textarea in the response, and checks whether it has a data-type attribute. If it does, the response jQuery will see has the content type specified in the data-type attribute, and the text content given as the value of the textarea element. Things like parsing this text content as JSON or XML is then done by jQuery itself via its converters.

Using Data Types for Transport Selection?

A better implementation might be based on an iframe data type. The transport would simply register for that data type. Then there'd be converters for transforming the text/html response into the actual data type by extracting the data from the embedded textarea.

But how do you tell jQuery to use the data type iframe for the transport, and then also have it automatically convert the response to the actual payload format which has a different data type? It turns out that jQuery now lets you specify multiple data types using the dataType option. The documentation gives the following examples:

multiple, space-separated values: As of jQuery 1.5, jQuery can convert a dataType from what it received in the Content-Type header to what you require. For example, if you want a text response to be treated as XML, use "text xml" for the dataType. You can also make a JSONP request, have it received as text, and interpreted by jQuery as XML: "jsonp text xml." Similarly, a shorthand string such as "jsonp xml" will first attempt to convert from jsonp to xml, and, failing that, convert from jsonp to text, and then from text to xml.

So we could in theory trigger the transport and the required conversions by setting the dataType option to, say, "iframe json". Since our server-side upload handler will return a text/html response, there would first be a conversion from html to the iframe data type. Then jQuery would try to convert that to JSON, which would require us to define additional converters from the iframe type to the other types.

But remember how I said above that you usually don't even need to specify the data type at all, as it will be determined from the MIME type of the response? In my projects, I use this to automatically support different response types on the same request, such as HTML that should be rendered, and JSON for Ajax redirects or other actions. For example, assume you have a simple form in your application; when the form is submitted over Ajax, if there are validation errors, the server will send back the form HTML including information about the errors. If it is successful, the server instead sends back a redirection command in a JSON payload, which gets interpreted by the client-side script. This whole approach could be the topic for an entire post… The gist is that I can not specify an expected data type, as I'm dealing with different types of responses that the client-side code can't predict when it's issuing requests.

So in the end I don't see how I could avoid the "+*" registration trick. Assuming I am not missing anything, this appears to be a flaw in the design. A transport mechanism and the type of the payload are not necessarily correlated in any way. That happens to work for JSONP (which is basically a transport mechanism bound to JSON), but for something like an iframe transport, there really is no relationship between the transport and the data type.

Maybe the jQuery Ajax options should simply have a transport option that would be more or less independent of the data type?

Conclusion

Despite this design issue, the iframe transport plugin works pretty well. If you need to support HTML file uploads in an Ajax-based application, you should probably have a look. It's a lot more lightweight than the other options out there, and it integrates nicely into the jQuery Ajax subsystem so you don't need to change all your client-side code around just to support a file upload or two somewhere.

Of course, it explicitly doesn't help you with the File API or other fancy things such as Flash-based uploads. Those could still use the iframe transport as a fallback, I guess.

If you'd like to give it a spin, check out the node.js based demo application.

Reactions

  1. Julian Aubourg says:

    5 July 2011
    01:13

    Nice article Christopher.

    I think you're a bit hasty in dismissing prefilters in your architecture because they're exactly what you need to solve your automatic dataType problem.

    As I understand it, the problem is that you'd need to specify a dataType to select your iframe transport, yet want to be able to use automatic dataType selection. You suggest that, maybe, a "tranport" option is missing. It's easy enough to implement it using a prefilter:

    
    $.ajaxPrefilter( "*", function( options ) {
        if ( options.transport ) {
            // Redirect to the temporary transport dataType
            return options.transport;
        }
    });
    
    $.ajaxTransport( "iframe", function( options ) {
        // Remove the temporary transport dataType
        options.dataTypes.shift();
        // Your transport code goes here
    });
    
    // How to use the iframe transport in a request
    $.ajax( url, {
        transport: "iframe"
    });
    

    Returning a dataType in a prefilter will "redirect" to this dataType (so if you had dataType set to "*", it would get transformed into "TRANSPORT *"). options.dataTypes is the array containing the sequence of dataTypes used by the conversion logic in ajax (in our example [ "TRANSPORT", "*" ]). Remove the temporary transport dataType, and you're good to go.

    Of course, this will also work if and when you specify a dataType ;)

    You could also be a little less generic simplify the prefilter by having a more specific custom option:

    
    $.ajaxPrefilter( "*", function( options ) {
        if ( options.iframe ) {
            // Redirect to the temporary iframe dataType
            return "iframe";
        }
    });
    

    Hope this helps.

  2. Christopher Lenz says:

    5 July 2011
    08:08

    Thanks for the comment, Julian. Very helpful indeed!

    The big thing I was missing was that the transport can simply move its data type out of the way before handing control back to jQuery.

    I'll play with your suggestions and update the plugin if it all works out as planned.

  3. Kevin Boudloche says:

    29 August 2011
    20:35

    Thanks for the code, I've made a couple changes to it that I think are improvements.

    Basically, instead of using the form that the file input is in, I created a new hidden form and appended the file input into it. This allows you to use file inputs that may not be in the same form or in a form at all. Then, in the cleanUp method, you simply append the input elements back in their original positions. I forked your repository, however i can't clone it from here due to a proxy; I'll update my fork tonight.

    On a side note, i really want to come up with a better way of handling the data so that it can accept a string such as the one returned from $(form).serialize(), as well as a basic object such as { "foo":"bar" }. I'm sure it can't be too hard.

  4. Kevin Boudloche says:

    31 August 2011
    03:57

    I committed some changes, however I haven't had time to do any testing on it. I'll have more time to test tomorrow afternoon. https://github.com/tentonaxe/jquery-iframe-transport

    I like Julian's idea with the transport: "iframe" rather than iframe: true since it can take care of other custom transports built in the same way.

    I have a question for you though. What is the textarea for? I don't see a reason for it.

  5. Christopher Lenz says:

    31 August 2011
    10:35

    On a side note, i really want to come up with a better way of handling the data so that it can accept a string such as the one returned from $(form).serialize(), as well as a basic object such as { "foo":"bar" }. I'm sure it can't be too hard.

    Actually I think it is pretty hard; check out the $.deparam() implementation in the BBQ plugin, for example.

    It would be way better if there was a way for a prefilter to stop jQuery from serializing the parameters in the first place. Unfortunately, by the time the prefilters are called, the data has already been processed.

    (Supporting an object instead of the array for the parameters should be doable though – actually I thought that was already supported.)

    I committed some changes, however I haven't had time to do any testing on it. I'll have more time to test tomorrow afternoon. https://github.com/tentonaxe/jquery-iframe-transport

    Cool, I'll check out your changes as soon as I get a chance. In general I agree that not using the original form is probably better.

    I like Julian's idea with the transport: "iframe" rather than iframe: true since it can take care of other custom transports built in the same way.

    Not sure, I don't want to try to establish a general standard here without the blessing of the jQuery team. So I'd prefer keeping the iframe: true method for now.

    I have a question for you though. What is the textarea for? I don't see a reason for it.

    As I wrote in the original post, when the response data type is not HTML but rather JSON (for example), characters that are special in HTML (such as & or <) get mangled by the browser, making it impossible to extract the actual payload. The simplest way around that is to stuff the payload into a <textarea>.

  6. Kevin Boudloche says:

    31 August 2011
    23:03

    If processData converts this object

    {"foo":"bar","bar":"foo"}

    to this string:

    "foo=bar&bar=foo"

    it is easy to take that string and turn it back into an object.

    var obj = {}, str = options.data, temp,temp2; temp = str.split("&"); for (var i = 0; i < temp.length; i++) { temp2 = temp[i].split("="); obj[temp2[0]] = temp2[1]; }

  7. Joshua Wilson says:

    6 February 2012
    05:46

    This is great stuff. Thanks for putting this together.

    I do have a problem though. If I implement this using your example in the demo it works, but if I follow the example on the code site, http://cmlenz.github.com/jquery-iframe-transport/ , it doesn't work. The difference that I found is the code site triggers on a 'Submit' button, while the demo does not have a 'Submit' button in the html relying on 'form change'. I would like to use the submit feature if possible. What am I doing wrong?

    I would like to be able to send multiple files each with a different data element associated with the file to be uploaded, if that is possible. Any ideas?

    Thanks, Joshua

  8. Madhusudhan Srinivasa says:

    17 October 2012
    19:21

    I would really love to see if cross domain uploads are possible.