Web application development is different (and better)
On both front and back end, the Web challenges conventional wisdom.
The Web is different, and I can see why programmers might have little tolerance for the paths it chose, but this time the programmers are wrong. It’s not that the Web is perfect – it certainly has glitches. It’s not that success means something is better. Many terrible things have found broad audiences, and there are infinite levels to the Worse is Better conversations. And of course, the Web doesn’t solve every programming need. Many problems just don’t fit, and that’s fine.
So why is the Web better?
The Web made it possible to address the project scalability problem, making some key choices that allowed both human and technical distribution of responsibilities. Those choices are compromises, but the balance they achieved lets developers at any level contribute to the Web.
The Web had great timing. It had a few years at the beginning to settle and develop a story about what it was. Next came a few years of insane rampaging growth in which developers large and small could make mistakes, energy to fix those mistakes, and then a quieter period for those fixes to set in. Even in the quiet period, the Web maintained mammoth scale, and web projects continued growing in scale and scope.
It’s not just that the Web is a great way to build distributed applications. It’s not just that it showed up at a time when a lot of us wanted to do that. It’s that the front end, the back end, and the connections between them are built with a unique set of features and constraints that make it much easier to build great (and maintainable!) applications.
In the Beginning, there was HTML. HTML, to be precise, with hypertext links serving as GOTO-like glue. Content, structure, formatting, behavior, and links were all stuck together in text documents, creating Ted Nelson’s worst technical nightmare.
As the web (and web sites) got bigger, search and replace hit their limits. The base layer, HTML, still contains a mix of content, structure, and hyperlink behavior, but other separations of concerns made it much easier to maintain and distribute web content. Eventually that spawned the web application context we have today.
HyperText Markup Language (HTML) remains approachable to a large number of people. Not everyone, but many more people than “programmers” or “designers”. With basic knowledge of how markup works and a few key tags, people can create content that others can use in many different contexts, from basic pages to complex applications. I still do a double-take when I see “HTML code,” but that usage is correct – HTML is not a programming language per se, but certainly encodes content and structure.
Sketching out an application in HTML is often easier than it is on paper or in Photoshop. Text is easy to store and manipulate (though encoding headaches exist). Google relies on this, to take just one key search engine example. The markup structure also defines a tree. The Document Object Model (DOM), or one of the many wrappers that seek to improve the DOM experience, give developers access to that tree.
Cascading Style Sheets (CSS) spares people writing HTML the headache of figuring out what exactly it should look like. Despite continuing concerns about the challenges of layout within a web browser, and efforts to add more logic to what is becoming a complex world, CSS hit a sweet spot. Its declarative approach made it easy for humans to create it and browsers to apply it efficiently. The ‘cascade’, though sometimes mysterious in its details, usually only sees use in situations complex enough to require that level of solution. Most amazing to me, CSS has also helped integrate hardware acceleration for animation and transformation in ways that would be much more difficult in a procedural environment.
CSS plays a key role in web architecture, though, beyond helping make things beautiful. CSS makes it much much easier to manage libraries of HTML documents, including application components, reducing redundancy and errors. The simple fact that style information applies to multiple documents has many benefits:
- From a programming perspective, the “separation of concerns” between HTML and CSS fits neatly with the “Don’t Repeat Yourself” (DRY) approach to programming. Style sheets may have repetition internally, but compared to every document repeating presentation information, it’s an easy win.
- From a network performance perspective, separate reused stylesheets makes caching work more efficiently.
- From a design perspective, it makes it much easier to have designers manage design with much simpler guidelines for people creating content.
Yes, having presentation information in separate documents is sometimes inconvenient. (Using the style attribute doesn’t win points for elegance.) That inconvenience, though, disappears rapidly as soon a site or application includes more than one document!
CSS had two cultural problems, one from the design side and one from the programming side. While HTML documents were (mostly) naturally responsive, reflowing to fit whatever size browser container they needed, the additional control CSS gave designers encouraged them to create beautiful but brittle layouts. New features for layout and media queries are making Responsive Web Design workable once again. On the programming side, well, CSS isn’t programming, as this excessively plausible satire of that attitude makes clear.
The Web keeps doing more with these tools, of course.
What does that mean? It means that:
- Developers can wrap APIs to tune their interfaces.
- Developers can fill gaps in browsers that are missing APIs or other functionality.
- Developers can extend the HTML vocabulary, using Web Components. You want a calendar, or a tab bar, or a tooltip? Just add one with brick or create one with Polymer. Or roll your own. HTML’s existing vocabulary becomes a foundation wall rather than a boundary wall.
The browser itself, not just the markup language, is now extensible, and these tools work across browser environments. Some of those extensions may be as simple as using new elements and providing CSS styles for them. Programmers hold the keys to more substantial extensions, adding behavior, but in this model they can share their extensions with non-programmers who just include them in their markup. You don’t have to be a programmer to use the new features, and the familiar toolset still works. (Complex polyfills can create questions around accessibility, but WAI-ARIA provides a foundation for many cases and the W3C is working on maintaining accessibility in the harder cases.)
Single-Page Apps, the most prominent of those approaches, is a great match for the expectations of developers coming from other desktop or mobile application models, or even Flash. The Web can accommodate this model (on both the front and back ends), and browser performance has improved to the point where it can compete in other ways with native applications. However, it discards much of what made Web culture scalable. Programmers may be kings and queens of their applications, but there aren’t enough kings and queens to go around.
Almost at the Beginning, in 1994, there was the URL. A uniform resource locator or web address, the URL not only connects the front and back ends of web development, but makes hyperlinking easy. Despite a variety of attempts (from the CueCat to QR codes) to replace text-based URLs with computer-readable graphics, the URL remains both popular and the foundation for its attempted successors.
Fragment identifiers, the optional part on the right end of a URL that starts with #, let you go further, reaching to a specific point in a document. Unfortunately, they quickly become complicated once you leave behind the basic expectations of either the HTML vocabulary or multi-page browsing models. Re-creating fragment identifiers in a single-page app was not my favorite part of the project a few years ago. Not everyone is as excited as I am about fragment identifiers of course (even proposing syntax to extend their pointing abilities – more here).
On the back end, the way that URLs, at least the parts to the right of the domain name, actually work has changed over the years. They used to map to files on the server, and still sometimes do, but many web frameworks now centralize control through routing approaches. PHP has been a good standard-bearer for distributing application logic across a large set of files, while Rails made routing a central component of its development story. Despite the different models, back-end developers have largely accepted the notion that URLs can work as pointers to different aspects of the same application.
Stop for a second, though, and think about your expectations of other application contexts. When I open an application on the desktop or a mobile device, my options are pretty much that it opens where I was last time or it opens on a new screen. It’s not so easy to have an application open up the way I want it, especially if I have multiple “the way I want it” options. With the Web, this is generally trivial – URLs let you link to specific points in an application, and halfway-smart development models let you log in as needed and get on with your work. Maybe there’s room to extend this capability further, but URLs definitely create capabilities for web apps that other apps have barely begun to consider.
(URLs have, in a sense, been punished for their success. Tim Berners-Lee grew less excited about them as he shifted focus toward visions of the Semantic Web. Instead of a mostly clear story about URLs, the W3C now spends much of its time working around the theology of the URI. If you have a dark sense of humor, a fondness for tech conversation, and lots of time on your hands, you could waste a year on this. Just seek out httpRange-14, the closed but never actually ending tear in the URI space-time continuum, or its predecessor conversation.)
In the Beginning, there was HTTP. All it offered clients was the ability to GET content from a URL, but over time the number of methods – verbs – expanded. GET and POST remain dominant, but PUT and DELETE have also become central for developers managing information in web applications.
HTTP’s original charm was its simplicity. It didn’t actually do much, and its ubiquity and apparent harmlessness combined to make firewall vendors largely willing to let it pass through. Because interest in consuming Web content grew faster than bandwidth, HTTP’s maintainers placed high priority on caching and proxies. After a few iterations that led to a protocol that was still relatively simple, but with a strong interest in performance. (As is all too often the case, many of the advantages of that simplicity are fading with the need to add security, but the architectural legacy remains.)
Statelessness was another feature of HTTP, forcing developers to consider how they would maintain conversations across multiple requests. Most of us settled for cookies – I now repent having written a book on them – that made the challenge less visible to users and developers, but also, because they offered persistence, opened the door to endless user tracking strategies.
Around the turn of the millennium, “Web Services” lurched forward, using HTTP POST requests for transportation and as an easy way to cut through firewalls. Like Single-Page Apps, they used web infrastructure in pursuit of an architecture that didn’t resemble web architecture or take much advantage of its features. Instead, it took much of its inspiration from CORBA, and enterprise architectures of the time.
At the time, I worried mostly that Web Services were destroying XML, turning it toward programming tasks for which it wasn’t well-suited. Fortunately HTTP at least was looking out for itself, and struck back in the unusual form of a doctoral dissertation chapter that found traction.
REST recognized the strengths of HTTP, in particular its providing a uniform (and minimal) interface to resources. HTTP’s constraints were in fact its virtues, though the religious-sounding nature of that claim has led to pretty much infinite debate about what is and is not truly RESTful. Early discussions of CREATE-READ-UPDATE-DELETE (CRUD) as parallels to the HTTP verbs led to of applications and frameworks (notably Rails) that used those methods blindly.
The most remarkable part of REST’s growth, to me, is that proponents of Web Services insisted that to reach both the public world and the internal enterprise world, Web Services had to base themselves on the expectations of enterprise developers. To a substantial extent, REST has demonstrated that enterprise developers can lean a lot from the architecture and practices of a larger public Web world.
More recently, it’s not just REST, but hypermedia interfaces pushing against traditional expectations of how you structure a program. Despite some programmers’ complaints that documents and hypertext as the basis of web programming is just an unfortunate quirk, I can’t help but see document-based approaches to distributed development as a feature, not a bug.
HTTP does have some key limitations – it’s a protocol that expects clients and servers and intermediaries to be transferring relatively large chunks of content in bursts. It doesn’t do peer-to-peer, handle tiny content efficiently, or do a great job of dealing with always-open connections. Those cases teach different lessons and need different technologies, notably WebRTC for the peer-to-peer and WebSockets for the small and always-on.
However, like its Web peers, HTTP has triumphed in a wide range of situations, not just despite its limitations, but because of them. HTTP even has a descendant, Constrained Application Protocol, which supports the RESTful HTTP verbs in a lighter-weight context!
In Worse is Better, Richard Gabriel argued that “Unix and C are the ultimate computer viruses.” They were. Today, the Web has proven even more viral.
Structured document exchange was not a field anyone forecast would change the way we write and share programs, but it has turned out to be a remarkably good fit. It doesn’t solve every problem – please don’t try to rewrite Photoshop or video-processing software to run in a browser – but it hits an 80/20 point and then some, with signs of going further.
Why has this worked so well? Distributed application development has always been difficult, and the Web has been distributed almost from the start. Some of it, I think, has to do with the peculiar stresses of the early Web: the need to scale projects with limited resources, and the need to scale content delivery with limited bandwidth. Those best practices grew out of challenging situations we have largely forgotten. The other key piece, though, is the URL, which provided glue and a basic set of expectations.
I was delighted to see the Extensible Web Manifesto last year, which seemed to me to pursue a vision of building on this existing web infrastructure to create new possibilities. “Small mutations” sound good, but are they small? Lately I’m getting a stronger sense that others read it as an opportunity to tear things down and rebuild them in programmers’ own image, chucking or at the very least destabilizing the very layers that make the Web scalable and accessible.
Let’s extend the Web and help it do more – but let’s do that by valuing the many strengths it already brings.