Ivan Zuzak – blog

Pmrpc discovery and publish-subscribe support + systematization of cross-context browser communication systems

Posted in web development by Ivan Zuzak on June 15, 2010

This blog has moved to a new domain. You can view this post here.

The pmrpc cross-context browser communication library has grown from the last time I blogged about it. Last time I wrote about adding support for RPC-style communication for WebWorkers. Today I’ll first introduce two new features – dynamic discovery of remote procedures and a publish-subscribe communication model. Also, I’ll write about our systematization of existing systems for cross-context browser communication.

Discovery and publish subscribe

Up until now, pmrpc supported only remote procedure call as the communication model for inter-window and WebWorker communication. This model of communication is based on a client-server interaction where the client both a) has an object reference of the destination window/webworker context and b) knows the name of the remote procedure it wants to call. Here’s an example of a pmrpc RPC call:

// server [window object A] exposes a procedure in it's window
pmrpc.register( {
  publicProcedureName : "HelloPMRPC",
  procedure : function(printParam) { alert(printParam); }
} );
// client [window object B] calls the exposed procedure remotely, from its own window
pmrpc.call( {
  destination : window.frames["myIframe"],
  publicProcedureName : "HelloPMRPC",
  params : ["Hello World!"]
} );

This works well for cases when the client knows how to obtain a reference of the server context object. However, it turns out that in many real-world cases the client does not know which window/iframe/worker implements the procedure it wants to call. Mashups and widget portals are the best example. Imagine that both the client and the server are widgets on iGoogle. Although you may be in control of both the client and server iframe contents, in most cases you will not know the names of iframe elements which contain them and which are located on the top container page. And you need the values of the name attributes so that you can obtain a reference to the iframe object which you could then use to call postMessage.

This referencing problem was the motivation for implementing both the discovery and publish-subscribe features. Both features enable clients to call a remote procedure without a-priori knowing which destination object implements the  procedure. First, the publish-subscribe feature enables clients to perform a remote call without specifying the destination context object reference. If a publish-subscribe call is made, the pmrpc infrastructure will make a RPC call for the specified procedure to all window, iframe and WebWorker object it can find (thus simulating a “publish” event on a channel named by the remote procedure name). The API for using the publish-subscribe feature is simple; it reuses the pmrpc.call method and instead of passing the reference of the destination context object, a “publish” string parameter is passed instead:

// server [window object A] exposes a procedure in it's window
pmrpc.register( {
  publicProcedureName : "HelloPMRPC",
  procedure : function(printParam) { alert(printParam); }
} );
// client [window object B] calls the exposed procedure remotely, from its own window
pmrpc.call( {
  destination : "publish",
  publicProcedureName : "HelloPMRPC",
  params : ["Hello World!"]
} );

Under the hood, the publish-subscribe feature uses the discovery feature to discover all window, iframe and WebWorker context objects and retrieves their references. Also, the publish-subscribe feature may be used with other pmrpc features, like access control and retries.

The discovery feature enables dynamic discovery of procedures registered at remote contexts and their filtering using various optional criteria. Discovered procedures may be filtered by the destination window/iframe/webworker context object, destination context origin (specified as a regular expression) and destination procedure name (also specified as a regular expression). For each discovered procedure that wasn’t filtered out, the procedure name, origin of the destination context, the access control rules specified for the procedure and the destination context object. The feature is implemented as a new pmrpc methodpmrpc.discover. The method is asynchronous and accepts a parameter object which specifies the filters and a callback method which will be called when all registered procedures are discovered. Here’s an example:

// server [window object A] exposes a procedure in it's window
pmrpc.register( {
  publicProcedureName : "HelloPMRPC",
  procedure : function(printParam) { alert(printParam); }
} );
// client [window object B] discovers and calls the exposed procedure remotely, from its own window
var discoCallback = function(discoveredProcedures) {
  if (discoveredProcedures.length > 0) {
    console.log(discoveredProcedures[0].publicProcedureName);
    console.log(discoveredProcedures[0].destinationOrigin);
    console.log(discoveredProcedures[0].procedureACL);
    pmrpc.call( {
      destination : discoveredProcedures[0].destination,
      publicProcedureName : "HelloPMRPC",
      params : ["Hello World!"]
    } );
  }
};
pmrpc.discover( {
  destination : [window.frames["someiFrameName"], window.frames["myIframe"]],
  origin : ".*goodOrigin.*",
  publicProcedureName : ".*Hello.*",
  callback : discoCallback
} );

The implementation of the discover method is based on recursively visiting every iframe in the window.frames object starting from window.top:

  1. The method fetches a reference to the top window object (window.top), adds the reference to the result array and repeats the procedure for each nested iframe (by iterating through the window.frames object). This way, all iframes in the current window are visited. This process is depicted by the figure below.
  2. The method adds all local web worker objects to the result array.
  3. On each context object in the result array, a special pmrpc infrastructure method is invoked to obtain a list of registered procedures.
  4. The list of all obtained procedures is filtered by the specified criteria.
  5. The method invokes the callback, passing the list of discovered procedures as a parameter.

It’s not an ideal way of discovering remote contexts, but it’s better than nothing and works with the latest versions of all major browsers.

Systematization of cross-context browser communication systems

It’s evident that the WWW is becoming ever more componentized as almost every web site is a mashup, contains widgets or uses WebWorkers. As a result, there has been a increase in development of systems that enable communication between these different browser contexts – windows, iframes and WebWorkers. Pmrpc is only one example of such system and lots of other systems exist. However, as the number of these systems increases, the understanding of each systems’ capabilities and their comparison and evaluation becomes a more difficult task since no effort is being made to systematize this ecosystem.

This is why Marko and I are trying to create a systematization of existing cross-context communication systems. Our work on this systematization is in its early stages but is openly available to anyone at a pmrpc wiki page. At the moment we are trying to do the following:

  1. Make a list of all mechanisms for cross-context communication. We are including both in-browser mechanisms like the postMessage API, cookies and URL fragment, and 3rd party libraries like easyXDM and pmrpc.
  2. Define dimensions for classifying the gathered systems. Each dimension defines a system characteristic whereas values along each dimension define a set of possible design choices.  Currently, we have defined about a dozen dimension and are still thinking some through. Here are some examples: Type of system (browser built-in, pure client-side framework, server-mediated initialization framework, server-mediated communication framework), Transport mechanism (cookies, URL fragment, postMessage, …), Programming model (message-oriented, shared memory, RPC, publish-subscribe).
  3. Evaluate the listed systems according to the defined dimensions.

I’d love to hear what you think of the new pmrpc features and the systematization (see more examples at our API docs page)! Do you know of any cross-context communication systems that are not listed? Do you think some other system characteristic should be included in the classification? @izuzak

RPC for Web Workers and distributed computing within the browser

Posted in web development by Ivan Zuzak on December 21, 2009

This blog has moved to a new domain. You can view this post here.

In a previous post I wrote about pmrpc – a project of mine for developing a JavaScript library for cross-domain inter-window RPC-style communication in HTML5 browsers. RPC communication is based on two technologies: HTML5 cross-domain messaging as the transport mechanism and JSON-RPC as the RPC protocol and message format, while the library supports other advanced features (access control, asynchronous calls, retries…). At the end of the post I outlined a few things Marko and I wanted to implement next in pmrpc, one of which was support for Web Workers. From the post:

Web Workers are another specification being developed in parallel with HTML5. The specification “defines an API for running scripts in the background independently of any user interface scripts”. What’s interesting is that the postMessage API is used to communicate with web workers and therefor pmrpc could be extended to include support for web workers.

Well, the API used to send messages from a web page to a worker and vice versa is not exactly the same as the cross-document messaging API, but is very similar. So, the idea for supporting Web Workers was leave the pmrpc API for registering, calling and unregistering procedure unchanged, and just extend it to also use the Web Worker API for message transport (in addition to the cross-doc messaging). And about two weeks ago we released a new version of pmrpc with Web Worker support. Here’s an example of using pmrpc with Web Workers:

// [worker object A - testingWorker.js]

// load pmrpc library
importScripts('pmrpc.js');

// expose a procedure
pmrpc.register( {
publicProcedureName : "HelloPMRPC",
procedure : function(printParam) {
              alert(printParam); } } );

// [window object B]

// load pmrpc library

// create worker
var testingWorker =
  new Worker("testingWorker.js");

// calls the exposed procedure
pmrpc.call( {
destination : testingWorker,
publicProcedureName : "HelloPMRPC",
params : ["Hello World!"] } );

A few notes on worker support. Pmrpc currently supports only normal (“dedicated”) Web Workers. Shared Workers (workers shared between multiple web pages) and nested workers (workers within workers) are currently not supported since, as far as I know, Shared and nested Workers are not implemented yet in any browser. Also, since Workers must be loaded from the same origin as the page that created them – the access control feature in pmrpc make no sense and are therefore not supported for Web Workers.

Next on our list is enabling discovery of registered procedures scattered over multiple browser workers, iframes and windows. Currently, pmrpc requires that you tell it where the procedure you are calling is, but sometimes you don’t know where it is, or don’t care – e.g. you just want to call the “DrawMap” procedure loaded from domain “http://www.good.com” if it exists. Discovery should enable discovery of methods based just on their names. The problems in implementing this feature (knowing which windows, iframes and workers are currently running and syncing the list of registered procedures with each of them) made me think about where systems running in browsers are heading – towards a form of distributed & parallel computing, only within a single browser. Distributed and concurrent execution, discovery, access control, synchronization, remote invocation, fault tolerance and naming are all concepts well know in distributed computing, and will possibly be re-invented in the context of web browsers to support this new shift. I’m not sure how far pmrpc will go in this direction, but I hope it will be a good example for other developers and research projects.

Do you think that web browsers need more powerful mechanisms supporting this new form of distributed computing? @izuzak

Inter-window browser communication and how to make it better

Posted in web development by Ivan Zuzak on October 10, 2009

This blog has moved to a new domain. You can view this post here.

As promised in my post on named arguments in JavaScript, this post will be about pmrpc – a project I’ve been working on with Marko for the last few months.

But first, a bit of background. From mid 2007. when I was interning at Google up until today and my dissertation-related research, an issue that’s consistently been popping up in anything I worked on is inter-window communication. Inter-window communication (IWC) is the process of transporting data from one window object within a browser to another window object inside the same browser. What’s relevant here is that window objects are both full-size windows and iframes – window objects that can be embedded within other window objects (think of an iGoogle page and all the gadgets in it).

The reason why IWC is an issue is the security model of communication within browsers. Same Origin Policy (SOP), a security policy introduced to reduce security risks, restricts access of processes executing within browsers. In essence, SOP prevents a document or script loaded from one domain from getting or setting properties of a document from another domain. For example, a page loaded from http://www.google.com can not directly access data in an iframe loaded from http://www.yahoo.com.

Although SOP turns Web applications into a set of isolated units, the need for communication between those islands is growing rapidly. With the advancement web technologies, web applications were slowly becoming componentized and many current web applications are based on mashing-up content from different sources – map gadgets, finance widgets, blog commenting toolboxes, games, chat widgets and many, many others. Since widgets, content encapsulated within an iframe, are the most popular method of including cross-domain content on a web page, inter-widget communication is most often implemented as inter-window communication.

A lot of groups and projects are researching models and developing systems that address the problem of communication between widgets and between windows in general. So here’s a short overview of this broad space of R&D. Various widget specifications are trying to formally define and standardize what a widget actually is. Browser security in general and various issues concerning intra-browser communication and mashup security are being heavily researched. Low-level IWC mechanisms are being proposed and implemented, sometimes even as browser hacks, and high-level communication models are being researched and developed. Even patents have been filed for methods of inter-window and inter-widget communication.

Among all of these efforts, one is catching on on a wider scale – the HTML5 cross-document messaging specification aka the postMessage API. The postMessage API has been implemented in all popular web browsers and enables secure message-oriented communication between any window objects – scripts in any window may send messages to any other window and receive messages from scripts in other windows. Security is established by two access-control mechanisms: 1) the message sender may specify the domain from which the destination window must be loaded in order for the message to be delivered, and 2) the message receiver may inspect the domain from which a message was sent and drop unwanted messages.

The problem with postMessage is that it’s a very low-level and general communication mechanism which requires a lot of boiler-plate code every time it is used and even more if one wants to implement advanced communication or access-control models. Have a look at the example on using postMessage in the HTML5 spec to see what I’m talking about. However, it’s exactly this generality and low-levelness that makes postMessage an excellent foundation to build upon! Some of the projects I mentioned above have been working on advanced communication libraries based on postMessage, but none that I know of are open-source and applicable in any context but rather tied to a specific application environment. Furthermore, advanced access control, fault tolerance and many other aspects of inter-window communication have been under-explored. These drawbacks were the main motivators for developing pmrpc.

Pmrpc (postMessage remote procedure call) is a JavaScript library for advanced inter-window cross-domain communication. The library utilizes the postMessage API as an underlying communication mechanism and extends it to a remote procedure call (RPC) model using JSON-RPC. JSON-RPC is a transport-independent protocol that uses the JSON data format for formatting messages. The pmrpc library provides a simple API for exposing and calling procedures from windows or iFrames on different domains.

Here’s a simple example for calling a function in another window:

1. First, a procedure is registered for remote calls in the window or iFrame that contains the procedure:

// [window object A] exposes a procedure in it's window
pmrpc.register( {
  publicProcedureName : "HelloPMRPC",
  procedure : function(printParam) { alert(printParam); } } );

2. Second, the procedure is called from a remote window or iFrame by specifying the window object which contains the remote procedure, name of the procedure and parameters:

// [window object B] calls the exposed procedure remotely, from its own window
pmrpc.call( {
  destination : window.frames["myIframe"],
  publicProcedureName : "HelloPMRPC",
  params : ["Hello World!"] } );

Furthermore, pmrpc also provides several features for advanced communication: callbacks for successful and unsuccessful calls, access control lists on both the client and server side, fault tolerance using user-defined retries and timeouts for each call, registering asynchronous procedures. The features and their usage are explained in detail on the pmrpc API docs page and a short presentation, so I'll just give an example of an advanced pmrpc call: 1. First, a synchronous procedure named "HelloPMRPC" accessible from windows and iframes loaded from any page on "http://www.myFriendlyDomain1.com" or exactly from the "http://www.myFriendlyDomain2.com" page is registered:

// [window object A] exposes a procedure in it's window
var publicProcedureName = "HelloPMRPC";
var procedure = function (alertText) { alert(alertText); return "Hello!"; };
var accessControlList = {
  whitelist : ["http://www.myFriendlyDomain1.com*", "http://www.myFriendlyDomain2.com"],
  blacklist : []
};

pmrpc.register( {
  "publicProcedureName" : publicProcedureName,
  "procedure" : procedure,
  "acl" : accessControlList
} );

2. Second, the procedure "HelloPMRPC" is called with a single positional parameter "Hello world from pmrpc client!". In case of success, the return value will be alerted while in case of errors the error description will be alerted. The call will be retried up to 15 times, waiting 1500 milliseconds between retries. Windows or iFrames loaded from any domain, except from http://evil.com, may process the call:

// [window object B] calls the exposed procedure remotely, from its own window
var parameters = {
  destination : window.frames["targetIframeName"],
  publicProcedureName : "HelloPMRPC",
  params : ["Hello world from pmrpc client!"],
  onError : function(statusObj) {alert(statusObj.description);},
  retries : 15,
  timeout : 1500,
  destinationDomain :  {  whitelist : ["*"],  blacklist : ["http://evil.com*"] }
};

pmrpc.call(parameters);

Pmrpc is an open-source project developed by Marko Ivankovic and myself and is available under the Apache v2.0 license. The library works in all web browsers that implement the postMessage API (Firefox 3, Google Chrome, Internet Explorer 8). For more information please visit the project homepage on Google Code and the API docs page for a full description of the API, feature list and usage examples. I'll end this post with a few ideas on making pmrpc even better: 1. discovery - currently, in order to call a remote procedure one needs to have a reference to a window that exposes that procedure. Sometimes obtaining a reference to a window is not very practical and easy (or possible) and a way to find windows by various criteria is needed. For example, discover all windows with a specific name or location or discover windows that expose a procedure with a specific name. 2. web workers - web workers are another specification being developed in parallel with HTML5. The specification "defines an API for running scripts in the background independently of any user interface scripts". What's interesting is that the postMessage API is used to communicate with web workers and therefor pmrpc could be extended to include support for web workers. 3. publish-subscribe mechanism - there are lots of communication models other than remote procedure call. One of the most popular models is publish-subscribe in which publishers publish events on channels and subscribers subscribe for events on these channels. Messages are delivered from publishers to subscribers without having a direct reference to each other. I'd appreciate any feedback on pmrpc and our ideas for future work. What are your ideas for the future of inter-window communication? @izuzak