Enterprise Rails

What to Expect in This Book

If you are new to Rails, the first book to read is Agile Web Development with Rails by Dave Thomas et al. (Pragmatic Bookshelf). The book you have in your hands, by contrast, is not a how-to guide for writing your first Rails application. This should be the second book you read.

This book deals in larger concepts, the formulas for how pieces fit together. It is not a compendium of the Rails API or a reference of the Ruby programming language. Books on these topics exist, and they are good to keep on your bookshelf, but they contain descriptions of tools rather than a formula for putting those tools together to get your job done.

This book gives you the tools to develop applications for the enterprise world for websites with global scale. Scaling comes in two forms. The first is the scaling we traditionally think of in terms of handling thousands, hundreds of thousands, or even millions of users, typified by Twitter and other large scale websites like Google, Amazon, or eBay. The second type of scaling is a more practical, human-focused concern. As your business needs change or expand, and as the types of developers in your organization and their quantities increase, will each developer still have the ability to contribute to the product in a meaningful way? Can new features be added easily and in parallel without conflict, or is the application difficult to modify by multiple developers at once? Will you be able to harness the hard work of the past in building the future, or will each new bold direction require a rewrite of the entire application?

Scaling of both varieties is facilitated through careful design of your application’s software architecture, rather than through the choice of language or platform. It’s a common misconception that scaling problems will be solved by the materialization of a faster Ruby interpreter, or by learning a magical set of Ruby incantations that aren’t described in the beginner books. Certainly, there are good and bad ways to describe any algorithm, but these are problems solved by those with comprehensive computer science training, not by the speed of the interpreter. Choosing a bad implementation for an algorithm will have similarly poor results in Ruby as in Java or Perl or otherwise.

The purpose of the Ruby interpreter and the Rails framework is to give you a tableau on which to develop your masterpiece. That’s where this book comes in. This book is about the principles involved in architecting serious web applications. The principles are universal, regardless of which technology you are using in the application tier. Of course, as you may have guessed, Ruby and Ruby on Rails will be used to describe all of these principles.

As it happens, Ruby is a terrific language, with many advanced features not found in today’s compiled languages. Not only is Ruby feature-rich, but it is also succinct to the point of marvel. What often takes dozens of lines of code in Java can often be written in just a few lines in Ruby. Rails, too, is a best-of-breed platform for developing web applications with little overhead. The commonly touted benchmark is that Ruby on Rails development proceeds at a clip of 10 times the rate of Java development. Big names like yellowpages.com have invested a lot of time and effort (and money) into rebuilding their entire sites on Ruby on Rails for the long-term benefits they will reap down the road in having a simpler and more efficient (by metric of code volume) framework.

However, there is a problem with this benchmark, and with the ethos of many in the Rails community as it exists today. Because so much effort has been put toward showing how simple it is to develop with Rails, and how much more productive you can be than with Java or other alternatives, little effort has been put toward showing Rails developers how to build applications that can truly stand up to the challenges that their Java cousins have had to prove themselves worthy against.

This doesn’t mean that a Rails application cannot stand up to the challenges imposed by constant traffic and large scale. Of course Rails can. However, there has been a dearth of public examples, and by the same token, there has not been much public discourse within the Rails community about how to design Rails applications to scale to the same levels that have been achieved by Java applications.

The secret is that the principles are the same. They were the same even before Java was de facto. The difference is that in Rails, with Ruby, the principles are so much easier to achieve once you know what you are doing. Because Ruby is so succinct, describing how to achieve the goals of good software architecture for the web is almost invisible when written in Ruby code.

Whether or not the revered “Gang of Four” Designs Patterns by Erich Gamma et al. (Addison-Wesley) needed to be explicitly retooled and retold for a Ruby audience has received its share of debate; achieving patterns is trivial in Ruby even though they required intricate structing of code in Java and C+. Indeed, the singleton pattern is achieved in Ruby by saying include Singleton. The factory pattern is so simply reproduced it barely warrants a name.

Most of the effort of architecting in Ruby is not found in tens of thousands of lines of application code. Instead, it’s in how you use Ruby and Rails to tie together all the other parts of your application stack: the database, your servers and clients, other services, and users of your application.

In this book, we begin by putting Rails in the correct context. The purpose of the original Rails book, Agile Web Development with Rails by Dave Thomas et al. (Pragmatic Bookshelf), was to sell Rails to the world, so the viewpoint is somewhat myopic. From its perspective, Rails may as well be the only element in the stack. The database is obligatory, so magical migrations are created to hand wave it away and ensure you never need to learn a scrap of DDL; anything else is pejoratively labeled as “legacy” and ignored. In the real world, databases and legacy systems tend to outlive everything else, hence the seemingly inescapable term “legacy” itself, so it’s worth paying them their due.

Topics

This book introduces “architecture” for enterprise web applications, from the ground up. What are the topics of web architecture, and why aren’t they found in most books on Rails? In truth, the success or failure of a web application has only partially to do with what is classically called the application layer. This may come as a surprise to those who have cut their development teeth on Rails, because the Rails view is that the application layer is all there is. It turns out that it’s the edges around the application that can make the biggest difference: databases, caches, and in a service-oriented architecture (SOA), the constellation of back-end services and front-end websites that make up the entire application.

First, below your application is the database. A schema stays with you for a very long time, so how you structure your data determines whether you can guarantee the integrity of your data, and whether your queries will be fast or slow. How those queries are written makes a big difference too, meaning you need to understand SQL even if you are using an object-relational-mapper like ActiveRecord. To write an application that is fast, you need to know into which queries a set of ActiveRecord statements will translate so that you can issue your queries in an intelligent way. “It works” usually is not enough for an enterprise application; as a developer you need to know how it works, how it should work, and why each way is as it is.

If you plan to avoid the hassles of optimizing a database schema and writing optimized queries by caching query results or rendered pages, be prepared for difficult times ahead if you want speed and consistency at the same time. It’s easy to make a cache that returns old, stale, invalid data. Correctly implementing a cache that is up-to-date in real-time is no simple task. What goes in the cache should be chosen carefully, as well as in what format. And the most difficult challenge still remains: when and how to invalidate or rebuild elements in the cache. Many people naively treat caching as a trivial problem, but depending on a cache that is out of sync with reality can be far worse than a slow site. Relying on stale data can lead you to make incorrect decisions, sell products you don’t have, double-book a flight, or not sell products you do have because the cache doesn’t know about them.

Once you have mastered these areas, suddenly the problems are raised an order of magnitude. It’s the rare website that is powered by a single monolithic application with a single database. To scale, not only to handle ever more users, but also to handle application and organizational complexity, SOA is almost always a necessary architectural evolution. In SOA, many applications are responsible for different slices of the overall problem. How do you choose how to split up a monolithic application, and further, how to glue the pieces back together to give a site’s visitors a unified experience?

In this book, we’ll cover these topics in detail. Of course, the application layer itself is extremely important, so we’ll start there, with the proper way to think about and structure your application. We’ll see how and when to separate code into logical elements, called modules. Then we’ll extract code into plugins to be shared by multiple applications. In many books, these topics are treated as advanced topics; in this book, they come first so you will actually have an opportunity to use them before you get entrenched in a design.

After looking within, we’ll look downward to the database layer. We’ll see how to build a solid foundation for our application with proper data modeling. First, we’ll learn about referential integrity and database constraints, culminating in writing trigger-based stored procedures to ensure complex relationships are satisfied. Then, we’ll discuss rigorous levels of database normalization, including third normal form and domain key/normal form, which will help us ensure our data’s integrity.

We’ll introduce the concept of domain tables, and how these special tables can be incorporated naturally in Rails. We’ll see how to base Rails models on database views. After that, we’ll get our first taste of caching by materializing a database view, increasing database performance by orders of magnitude.

Next, we’ll look to the sides as we explore service-oriented architecture. An oft-misunderstood concept, we’ll spend a good deal of time concentrating on theory. Then we’ll build multiple RPC-based back-end services to be consumed by a thin front-end client. We’ll build a REST web service, too, but we’ll see how to build any type of REST service, not just the subset supported by ActiveResource.

Finally, we’ll revisit caching, treating it like the sleeping monster it really can be, giving you the tools to ensure with certainty that the caches you create can be depended upon to be accurate.

Throughout the book, a heavy emphasis will be placed on testing, both unit tests within an application and integration tests when we connect multiple applications together. In fact, we never create a single view or traditional controller, instead, we exercise the model classes we write with our tests.

How This Book Is Organized

In Chapter 1, we start out by taking a tour of an ideal enterprise systems layout, highlighting all of the elements that aren’t Rails, as well as noting the various places where Rails can fit in.

The rest of this book can be divided principally into three major sections. The first deals with the Rails framework itself. The next few chapters examine Rails itself. While Rails gives you a Model-View-Controller (MVC) framework to start your projects, there is much to be desired in terms of structuring and maintaining large applications. Chapters 2 and 3 fill these gaps. First, Chapter 2 dives into plugins, and how they can be used to improve application clarity, while also encouraging you to write reusable code. Chapter 3 introduces modules: what they are, and how and when to use them.

The next major section of the book, Chapters 4 through 12, deals with the database. The database layer has not really been given its due in the Rails community, and Chapter 4 begins by providing an overview of why it is such a critical part of your application. In Chapter 5, we start building a data model for the example application we’ll work with throughout this book: a sales website for movie tickets.

Although the schema we’ll design would be sufficient for most Rails books, we’ll see quickly that it was a naive design. In Chapter 6, we refactor the schema to be in third normal form (3NF). In Chapter 7, we’ll pick out a special type of table called a domain table, and we’ll incorporate these tables naturally into a Rails application. In Chapter 8, we’ll expose some more problems with our schema, then tighten it one step further by moving from third normal form to domain key normal form. In Chapter 9, we’ll get even more advanced with our introduction of stored procedures and triggers. We’ll use them to enforce relationships that built-in database constraints cannot handle, giving you the power to completely lock down your database schema.

Chapters 10 through 12 introduce some new database-related features to Rails developers. In Chapter 10, you’ll see how to base an ActiveRecord model on a database view. This is useful for automatic filtering, or for filtering on data that’s not easily available at the application layer. In Chapter 11, we’ll show how to build support for multiple table inheritance in Rails. Rails supports single table inheritance, but it is not always the right tool for the job. In Chapter 12, we’ll get our first taste of caching when we materialize the view we created in Chapter 10, giving our view a huge boost in performance.

Chapters 13 through 18 deal with service-oriented architectures and different techniques for connecting systems together. In Chapter 13, we start by defining SOA, as well as going over the scenarios when it is the right choice to solve a problem. Chapter 14 covers considerations that go into designing a service-oriented architecture, including guidelines for designing an API. In Chapter 15, we build our first SOA service. In Chapter 16, we build upon our accomplishments in Chapter 15, connecting two back-end services together and testing them with the same interface a thin front-end client would use. Chapter 17 is a critical exploration of REST, helpful for placing it in its proper context for web services, but not necessarily for service-oriented architectures. In Chapter 18, we build a RESTful web-service.

Chapter 19 is a culmination of many of the previous chapters. There, we implement a fully correct service-layer cache to enhance application performance. We also go over other places where caching can be a dependable way to improve performance.

Who Is This Book For?

The purpose of this book is to provide the background you need to build your bridge: a large-scale, enterprise website. It is assumed that you’ve already read the manuals of the tools you need to get the job done, e.g., books on the syntax of Ruby and of Rails. This book fills in the background that transforms you from a layman who has tools into an expert ready to make the most of those tools.

This book is geared toward three general types, which we’ll call the Student, the Glass Ceiling, and the Travelers.

For the Student, this book is full of the theory behind engineering large-scale enterprise web applications, so if you are embarking upon a class on software engineering for the Web, this book is for you. This may even be your textbook, and if so, Mazel Tov! You will learn a lot here.

Our Glass Ceiling audience are those who have read half a dozen manuals about their tools and toolkits, but still find they don’t have the background to jump to the next level of web application design. These are the so-called newbies. The tools are beginning to make sense, but how to use them together effectively may still seem murky. If you are in this group, this book is for you, too. Soon all the pieces will be dovetailing nicely.

The final group that may benefit from this book are the Travelers. Travelers may have a lot of experience with other frameworks, but they are wondering how to make it all go smoothly with Ruby on Rails. For this audience, this book is a great refresher in all the basic theory behind solid web application design, followed by a hearty dose of how to integrate all that theory into Rails.