This post adapts my talk from JSConf EU 2013.
The web platform has, historically, been somewhat of a kludge. It’s grown, organically, into something with no real sense of cohesion. Most of its APIs have been poorly designed, by C++ developers, via a binding layer meant originally for CORBA.
The problem is in the process. Generally, new things have been introduced into our web platform via a months or years of mailing-list standardization, writing something in prose and IDL, driven by scenario-solving—without much concern for actual utility, much less usability. Implementers expose some fundamental capability in terms of a high-level API or declarative form that burrows down directly to the C++ layer, giving you limited customizability. After all this time, it eventually ends up in your hands, and you end up telling the standards bodies that it’s a huge mess, or that it solves half of your problems half of the time.
Despite all this, we’ve somehow done OK. Actually, a bit more than OK, given that the web is the most successful platform ever. How did we manage this?
Well, we wrap up APIs with horrible usability into ones that are quite pleasant, like jQuery. We “prolyfill,” creating libraries like Sizzle to implement CSS selector matching, or libraries like Angular to implement custom elements, in the hope that eventually native support will appear. We transpile from languages like CoffeeScript or SASS to add new features to our authoring languages. And one case, promises, we even built an interoperable standard from the ground up.
We need our platform to be better, and so we make it better, by ourselves.
The Extensible Web Manifesto
There’s a better way, where we work together toward the future.
What these standards bodies have realized is that the web platform is our language, but like all languages, it must evolve.
This evolution of our shared language takes place in two acts:
- extending our basic vocabulary;
- starting to incorporate “slang”.
Extending our Vocabulary
Extending our vocabulary means two things:
Explaining the features of the platform that are already there. Wouldn’t it be weird if we had compound words like “scifi,” but didn’t have the words “science” or “fiction”? If some standards body, perhaps the French Making Up Words Consortium, just handed us the word “sandpaper,” but we had no way in our language to talk about “sand” or “paper” individually? The web is like that today, and we’ll go over a few examples.
Giving you new low-level features that you can use. If you wanted to invent the word “scifi,” somebody had better have come up with the words for “science” and “fiction”! Similarly, there’s lots of things we just don’t have “words” for on the web, yet. That’s where native apps are hurting us.
So with this in mind, let’s look at some examples.
The most fundamental unexplained gap in the platform is simply: how do those damn elements even work?
Somehow, you feed a string containing some angle brackets into the browser, and they get turned into these JS objects with terrific APIs, which we call “the DOM.” How did that happen?
Custom elements explain this process, saying that you register a mapping of tag names to element prototypes with the browser, and that’s what the HTML parser is actually using under the hood. This is great! This is the democratization of HTML!
And better yet, this means no more crazy widget libraries with their own crazy semantics. No more jQuery UI with its
.option thing (sometimes a setter, sometimes a getter, sometimes a method call); no more Dojo digits; no more
Bootstrap craziness; no more WinJS with its funky
winControl property. Just tags, that turn into elements, which
behave like you’d expect: they have properties, getters, setters, methods, and all that.
The Shadow DOM
But what about the existing tags? Half of the reason these widget libraries exist is so that you can create your own
<select> element, because the existing one isn’t styleable or customizable.
In general, think of all the “magic” tags that exist today, like
<input type="date">, or
<video>, or even good old
<li>, whose bullet seems to come out of nowhere. In all cases, there’s some extra
“stuff” the browser is creating, and allowing users to interact with, and sometimes even allowing you to style via
ridiculous vendor-prefixed pseudo-elements like
::-moz-placeholder. But where does this extra stuff live?
The answer is: in the shadow DOM. And what’s great about the shadow DOM, is that once we actually have a realistic basis for these hidden parts of the DOM, in reality instead of in C++ magic-land, you’ll be able to actually start hooking into them instead of rebuilding an entire element just to customize its behavior and styling. That day is almost here.
The web audio API is a good example of both facets of the “new vocabulary” theme. You can do fundamentally new things with web audio, like positional audio or audio synthesis or so many other cool possibilities.
But remember the
<audio> tag, from way back in 2009? It’s kind of the quintessential instance of “here’s some C++
magic thrown over the wall to you web developers; have fun!” Well, from an extensible web perspective, the
should be explained in terms of web audio.
There are of course many other APIs which exist solely to expose a new low-level hardware or platform feature to the web platform. One of the older examples on the hardware side is the geolocation API. On the software side, good examples include the notifications API and fullscreen API. But more and more are popping up as we attempt to close all the gaps preventing full parity with native apps; one particular driver of this is the work on Firefox OS and the related device APIs.
ES6 and ES7
Finally, I want to call out ECMAScript 6 (which is nearing finalization) and ECMAScript 7 (for which efforts are just starting to ramp up). Extending the web’s programming language is adding new vocabulary at its most literal level, and the TC39 committee driving the evolution of ECMAScript does not disappoint in their efforts here.
In ES6 we’ll be getting subclassable built-in objects, so that you can finally extend
Date or the new
Set types, in a way that actually works. We’ll also be getting proxies, which allow an object almost-complete
control over the meta-object protocol underlying all interactions with it. And for ES7, the proposal for
Object.observe is starting to firm up. Plus there is talk of adding weak references to the language, now that some of
their trickier aspects have been worked out.
The second half of the extensible web philosophy is that we need to tighten the feedback loop between developers and standards bodies.
In short, we need to incorporate this kind of invented “slang” back into our shared language. Let’s take a look at some of the examples of this so far.
<template> element is a generalization of the
<script type="text/x-template"> trick. By rolling it into the browser, additional benefits can be realized,
allowing the template tree to be treated as an inert version of a real DOM tree, and for the element parsing and
serialization rules to specifically call out templating use cases.
<dialog> element obviates all of the annoying
dialog or “lightbox” libraries we keep having to ship, each with their own strange semantics. Instead, it’s a simple
tag, with some imperative APIs, some declarative features, and a nice
::backdrop pseudo-element. Sweet!
CSS is slowly but surely starting to roll in innovations from SASS and elsewhere.
CSS hierarchies, still under development, brings SASS’s nested selectors to
the browser. CSS variables uses a clever trick to get something with the same
benefits as the variables in SASS and others, but fitting in well with CSS’s existing semantics. And
CSS cascade introduces the
unset keyword which reduces all those complicated
CSS reset stylesheets rules to virtually nothing.
Pointer events finally unify mouse and touch events into a single abstraction. In one stroke, this obviates many libraries built to work around this strange dichotomy introduced by mobile Safari, and around other strangeness relating to trying to use mouse events on a touch device. They will be a welcome addition to the web platform.
When a pattern is adopted by jQuery, Dojo, Angular, Ember, WinJS, and YUI, as well as many other popular dedicated libraries, it’s time to put it into the platform. Promises are on track for ES6, and are being added to browsers now.
The extensible web is an ongoing project, and several efforts are being headed up to expose even more capabilities to developers, or roll even more common patterns into the platform. Here’s a brief taste of those I’m watching closely.
postMessage, object URLs,
poignantly emphasized in a brilliant presentation by Max Ogden, we’re clearly missing a
unifying abstraction here, and that abstraction is streams.
Node.js has led the way with their battle-tested implementations, but they’ve also learned some lessons we should be
sure to heed in order to design a good browser stream API. In the end, the goal is to be able to take various sources of
binary data (HTTP requests, camera data, payloads stored in IndexedDB, the output of a web audio graph, …) and pipe them
into various sinks (
<audio> tags; other windows, frames, or workers; filesystem or remote HTTP
endpoints; or completely custom consumption code). It’s going to be really cool, but we have some work to do before we
get something as well-designed as promises were.
The basic act of doing an HTTP request has so much complexity on the web platform: cross-domain protection; redirect following; deserialization from bytes; cookie jars; caches… We want to provide the basic building block, and then the ability to layer and compose each of these features on top of it.
There’s active investigation going on into how to expose compression primitives to the web. This is clearly something where native bindings will be more performant, and although there are impressive polyfills, native APIs, preferably with asynchronous off-main-thread compression, will enable new scenarios. This work is in its early stages, so if you want to get involved, reach out.
class Elements extends Array
My personal favorite new feature is the upcoming
Elements collection. It’s a proper array subclass, using the
aforementioned ES6 subclassable builtin support, to give you something where you can finally use
filter, and all your favorite methods.
As part of this effort we added two methods,
queryAll, to both
Element.prototype and to
Elements.prototype. They act as better versions of
querySelectorAll, in that they treat relative
"> div" the way you would expect instead of throwing an error. The versions on
as composite operations over all elements in the collection, just like in jQuery.
This is the beginning of a new, friendlier DOM, and I’m pretty excited about it.
What do we need? What is preventing you from building the web apps of your dreams? You tell us! The extensible web is waiting for your participation!
The best thing you can do to get involved in the extensible web is prolyfill. There’s only so much standardization bandwidth to go around, so if you can create a de-facto standard like jQuery, or an open specification with wide implementer suppport like Promises/A+, the world is waiting.
For example, if you wanted to figure out what a zlib API for the browser should look like, the best thing you can do is:
- Learn what the constraints and use cases are. (And not just your use cases, but everyone’s!)
- Design an API and library to prolyfill this gap.
- Evangelize its use among developers, so that everyone recognizes it as the clear solution that browsers should just ship and be done with it.
More generally, if you want to be involved in helping the web succeed by guiding us toward better standards, then let’s talk. It’s an area I’ve been diving into over the last year, stemming from my Promises/A+ work but expanding into many other things. Finding the right approach and content is delicate, as these people are jaded by newbies coming out of the woodwork to demand feature X. But if you approach in good faith and avoid a prideful demeanor, they’re often happy to listen. I’ve had a few success stories in this area already, and by this time next year I want to have a lot more.
In fact, I gave a talk on this subject at LXJS, titled “How to Win Friends and Influence Standards Bodies”. I’ll probably be adapting it into blog post form soon.
To close, I’d like to give a short message of hope. It’s easy to think about all these cool things that are coming, and then get depressed about having to support IE8 or Android 2.3 at your job. But that’s the price we pay for an open, interoperable web. We can’t just march to the tune of a single vendor, upgrading in lockstep. Instead we work through this collaborative, cooperative process, to build our shared language. In the end, the future is longer than the past, and I look forward not only to living in that future, but to helping shape it, together with you all.