Chapter 1. Programming on the Web

You probably have been using the Web now for many years to read news, shop, gather information, and communicate with your friends. You start your web browser (Internet Explorer, Firefox, Safari, Opera, Chrome, or another) and navigate around the Web. You may even have a MySpace page or a blog somewhere and have written a bit of HyperText Markup Language (HTML) to customize the look of your page. Some web pages are just flat content, where you see the exact same thing every time you visit that page, and other pages have highly dynamic content and display something very different based on your actions or what you type.

In this book, you will learn how to write applications that generate those dynamic web pages and how to run your applications on the Google App Engine infrastructure.

A perfect example of an interactive and dynamic page is Google Search (Figure 1-1).

Google Search

Figure 1-1. Google Search

When you come to Google Search, you can type anything into the search box and click the Google Search button. Figure 1-2 shows search results for your request.

Google Search results

Figure 1-2. Google Search results

Google Search is a “web application”—it is software that runs on the Web. The Google Search application takes as its input many requests per second from web browsers around the world. When the Google Search application receives the request, it springs into action looking for web pages in its large datastore that match the search terms, sorts those pages based on relevance, and sends an HTML page back to your browser, which shows you the results of your search.

The Google Search engine is quite complex and makes use of a vast amount of data around the Web to make its decisions about what pages to show you and in what order to present your pages. The web applications that you write will generally be much simpler—but all the concepts will be the same. Your web applications will take incoming requests from browsers and your software will make decisions, use data, update data, and present a response to the user.

Because you will be writing a program, rather than just writing documents, the response that you give to the user can be as dynamic and as unique or customized for each user as you like. When a program is building your web pages, the sky is the limit.

The Request/Response Cycle

For you to be able to write your web applications, you must first know a few basic things about how the Web works. We must dig a little deeper into what happens when you click on a page and are shown a new page. You need to see how your web application is part of the cycle of requesting and displaying web pages. We call this the HyperText Transport Protocol (HTTP) request/response cycle.

The request/response cycle is pretty easy to understand. It starts when the user clicks on a web page or takes some other action and ends when the new page is displayed to the user. The cycle involves making a request and getting a response across the Internet by connecting to software and data stored in data centers connected to the Internet (Figure 1-3).

Connecting across the Internet

Figure 1-3. Connecting across the Internet

Although you probably have a general notion as to what is going on when you are using your web browser to surf the Web, before you can really develop web applications, you need to understand the process in more detail (Figure 1-4).

The request/response cycle

Figure 1-4. The request/response cycle

Your browser looks at the Uniform Resource Locator, or URL (i.e., that you clicked on. It then opens an Internet connection to the server in the URL ( and requests the /search document. It also sends any data that you have typed into form fields along with the request for the /search document.

When your browser makes the connection, requests a document, and sends any input data that you have typed, it is called an “HTTP request” because your browser is requesting a new document to display.

The HTTP request information is routed across the Internet to the appropriate server. The server is usually one of hundreds or thousands of servers located in one of Google’s many data centers around the world. When the server receives the request, it looks at the document that is being requested (/search), which user the request is coming from (in case you have previously logged in with this browser), and the data from any input fields. The server application then pulls data from a database or other source of data, produces a new HTML document, and sends that document back to your web browser. When your web browser receives the new HTML document, it looks at the HTML markup and CSS (Cascading Style Sheets), formats the new page, and shows it to you.

Although there are many steps that take a few paragraphs to describe, the whole process from clicking on one page to the display of the next page usually happens in less than a second. Of course, if your network connection is down or very slow or the server is running slowly, the process happens in “slow motion,” so the fact that there are several steps is a little more obvious as the progress bar crawls across your screen while your browser is waiting for the request to be sent and the response to be retrieved over your slow Internet connection.

If the page references images or other files in the HTML, your browser also makes separate requests for those documents. Many browsers will show a status bar as each of these requests/response cycles are processed by showing a running count on the browser’s status bar as the files are retrieved:

Retrieving ""-Completed 5 of 8 items.

This message simply means that to display the page you requested, your browser needs to retrieve eight documents instead of just one. And although it is making progress, so far it has received only five of the documents. A document can be an HTML page, CSS layout, image file, media file, or any number of different types of documents.

In a sense, the HTTP request/response cycle determines the overall layout of the book: to build a web application, you need to have a general understanding of all aspects of the request/response cycle and what is happening at both ends (browser and server) of the request/response cycle (Figure 1-5).

The technologies of the Web

Figure 1-5. The technologies of the Web

You need to learn about how the browser operates using HTML and CSS so that you know how to properly format web pages for display in the browser. You also need to learn about how to add interactivity to web pages using JavaScript and AJAX (Asynchronous JavaScript and XML).

You need to understand the mechanics of how the browser makes its requests using the HTTP protocol—in particular, the different types of requests (GET or POST) and how to handle incoming data entered by the user on forms or files to be uploaded as part of the request.

Inside of the server, you need to learn the Python programming language, the Google Datastore facility, how to generate Dynamic HTML easily using templates, and how to use the Google memory cache to make sure that your applications continue to be fast when being used by many users at the same time.

The browser technologies and HTTP topics are generic and apply to programming in any web application environment such as Ruby on Rails, PHP, Java Servlets, Django, Web2Py, or any of the literally hundreds of other web application frameworks. Learning these topics will be of use in any web programming environment.

Most of this book focuses on the unique aspects of programming in the Google App Engine framework. We cover Python, templates, the Datastore, and the memcache to give you a solid introduction to the App Engine environment and the Google Cloud.

Get Using Google App Engine now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.