Embedded Android

Preface

Android’s growth is phenomenal. In a very short time span, it has succeeded in becoming one of the top mobile platforms in the market. Clearly, the unique combination of open source licensing, aggressive go-to-market, and trendy interface is bearing fruit for Google’s Android team. Needless to say, the massive user uptake generated by Android has not gone unnoticed by handset manufacturers, mobile network operators, silicon manufacturers, and app developers. Products, apps, and devices “for,” “compatible with,” or “based on” Android seem to be coming out ever so fast.

Beyond its mobile success, however, Android is also attracting the attention of yet another, unintended crowd: embedded systems developers. While a large number of embedded devices have little to no human interface, a substantial number of devices that would traditionally be considered “embedded” do have user interfaces. For a goodly number of modern machines, in addition to pure technical functionality, developers creating user-facing devices must also contend with human-computer interaction (HCI) factors. Therefore, designers must either present users with an experience they are already familiar with or risk alienating users by requiring them to learn a lesser-known or entirely new user experience. Before Android, the user interface choices available to the developers of such devices were fairly limited and limiting.

Clearly, embedded developers prefer to offer users an interface they are already familiar with. Although that interface might have been window-based in the past—and hence a lot of embedded devices were based on classic window-centric, desktop-like, or desktop-based interfaces—Apple’s iOS and Google’s Android have forever democratized the use of touch-based, iPhone-like graphical interfaces. This shift in user paradigms and expectations, combined with Android’s open source licensing, have created a groundswell of interest about Android within the embedded world.

Unlike Android app developers, however, developers wanting to do any sort of platform work in Android, including porting or adapting Android to an embedded device, rapidly run into quite a significant problem: the almost total lack of documentation on how to do that. So, while Google provides app developers with a considerable amount of online documentation, and while there are a number of books on the topic, such as O’Reilly’s Learning Android, embedded developers have to contend with the minimalistic set of documents provided by Google at http://source.android.com. In sum, embedded developers seriously entertaining the use of Android in their systems were essentially reduced to starting with Android’s source code.

The purpose of this book is to remedy that situation and to enable you to embed Android in any device. You will, therefore, learn about Android’s architecture, how to navigate its source code, how to modify its various components, and how to create your own version for your particular device. In addition, you will learn how Android integrates into the Linux kernel and understand the commonalities and differences it has with its Linux roots. For instance, we will discuss how Android leverages Linux’s driver model to create its very own hardware layer and how to take “legacy” Linux components such as glibc and BusyBox and package them as part of Android. Along the way, you will learn day-to-day tips and tricks, such as how to use Android’s repo tool and how to integrate with or modify Android’s build system.

Learning How to Embed Android

I’ve been involved with open source software since the mid-’90s. I was fortunate enough to join in before it became recognized as the powerful software movement that it is today and, therefore, witness its rise firsthand in the early 2000s. I’ve also made my share of open source contributions and, yes, participated in a couple of, shall we say, colorful flame wars here and there. Among other things, I also wrote the first edition of O’Reilly’s Building Embedded Linux Systems.

So when Android—which I knew was Linux-based—started becoming popular, I knew enough about Linux’s history and embedded Linux to know that it was worth investigating. Then, I was naively thinking: “I know Linux fairly well and Android is based on Linux; how hard could it be?” That is, until I actually started to seriously look into and, most importantly, inside Android. That’s when I realized that Android was very foreign. Little of what I knew about Linux and the packages it’s commonly used with in embedded systems applied to Android. Not only that, but the abstractions built in Android were even weirder still.

So began a very long (and ongoing) quest to figure things out. How does Android work? How is it different from regular Linux? How can I customize it? How can I use it in an embedded system? How do I build it? How does its app development API translate into what I know about Linux’s user-space? etc. And the more I dug into Android, the more alien it felt and the more questions I had.

The first thing I did was to actually go to http://developer.android.com and http://source.android.com and print out everything I could get my hands on, save for the actual developer API reference. I ended up with a stack of about 8 to 10 inches of paper. I read through most of it, underlined a lot of the key passages I found, added plenty of notes in the margins, and created a whole list of questions I couldn’t find answers for. In parallel, I started exploring the sources made available by Google through the Android Open Source Project (AOSP). In all honesty, it took me about 6 to 12 months before I actually started feeling confident enough to navigate within the AOSP.

The book you presently hold is a result of the work I’ve done on Android since starting to explore it—including the various projects I’ve been involved in, such as helping different development teams customizing Android for use in their embedded designs. And I’ve learned enough about Android to say this: By no means is this book exhaustive. There are a lot of things about Android and its internals that this book doesn’t and can’t cover. This book should, nevertheless, allow you to jump-start your efforts in molding Android to fit your needs.

Audience for This Book

This book is primarily geared toward developers who intend to create embedded systems based on Android or who would like to take Android and customize it for specific uses. It’s assumed you know about embedded systems development and have at least a good handle on how Linux works and how to interact with its command line.

I don’t assume you have any knowledge of Java, and you can get away without knowing Java for quite a few of the tasks required to customize Android. However, as your work within Android progresses, you’ll find it necessary to start becoming familiar with Java to a certain degree. Indeed, many of Android’s key parts are written in Java, and you’ll therefore need to learn the language in order to properly integrate most additions to specific parts of the stack.

This book isn’t, however, about either app development or Java programming in any way. If these are the topics you are interested in, I recommend you look elsewhere. There are quite a few books on each of these topics already available. This book isn’t about embedded systems, either, and there are books on that topic, too. Finally, this book isn’t about embedded Linux, which also has its own books. Still, being familiar with Linux’s use in embedded systems is something of a plus when it comes to Android. Indeed, though Android is a departure from all things traditionally known as “embedded Linux,” many of the techniques typically used for creating embedded Linux systems can guide and help in the creation of embedded Android systems.

This book will also be helpful to you if you’re interested in understanding Android’s internals. Indeed, customizing Android for use in embedded systems requires knowing at least some basics about its internals. So while the discussion isn’t geared toward a thorough exploration of Android’s sources, the explanations do show how to interact with the various parts of the Android stack at a fairly intimate level.

Organization of the Material

Like many other titles, this book gradually builds in complexity as it goes, with the early chapters serving as background material for later chapters. If you’re a manager and just want to grab the essentials, or if you’re wondering which set of chapters you have to read through before you can start skipping chapters and read material selectively, I recommend you at least read through the first three chapters. That doesn’t mean that the rest isn’t relevant, but the content is much more modular after that.

Chapter 1, Introduction, covers the general things you should know about Android’s use in embedded systems, such as where it comes from, how its development model and licensing differ from conventional open source projects, and the type of hardware required to run Android.

Chapter 2, Internals Primer, digs into Android’s internals and exposes you to the main abstractions it comprises. We start by introducing the app development model that app developers are accustomed to. Then we dig into the Android-specific kernel modifications, how hardware support is added in Android, the Android native user-space, Dalvik, the system server, and the overall system startup.

Chapter 3, AOSP Jump-Start, explains how to get the Android sources from Google, how to compile them into a functional emulator image, and how to run that image and shell into it. Using the emulator is an easy way to explore Android’s underpinnings without requiring actual hardware.

Chapter 4, The Build System, provides a detailed explanation of Android’s build system. Indeed, unlike most open source projects out there, Android’s build system is nonrecursive. This chapter explains the architecture of Android’s build system, how it’s typically used within the AOSP, and how to add your own modifications to the AOSP.

Chapter 5, Hardware Primer, introduces you to the types of hardware for which Android is designed. This includes covering the System-on-Chips (SoCs) typically used with Android, the memory layout of typical Android systems, the typical development setup to use with Android, and a couple of evaluation boards you can easily use for prototyping embedded Android systems.

Chapter 6, Native User-Space, covers the root filesystem layout, the adb tool, Android’s command line, and its custom init.

Chapter 7, Android Framework, discusses how the Android Framework is kick-started, the utilities and commands used to interact with it, and the support daemons required for it to operate properly.

Appendix A, explains how to get a legacy stack of “embedded Linux” software to coexist with Android’s user-space.

Appendix B, shows you how to extend the Android stack to add support for new hardware. This includes showing you how to add a new system service and how to extend Android’s Hardware Abstraction Layer (HAL).

Appendix C, provides you with pointers to help you customize what’s included by default in AOSP-generated images.

Appendix D, contains a commented set of the default init.rc files used in version 2.3/Gingerbread and version 4.2/Jelly Bean.

Appendix E, lists a number of resources you may find useful, such as websites, mailing lists, books, and events.

Software Versions

If you hadn’t already guessed it when you picked up this book, the versions we cover here are likely way behind the current Android version. And that is likely to be the case forever forward. In fact, I don’t ever expect any version of this book to be able to apply to the latest release of Android. The reason is very simple: Android releases occur every six months. It took almost two years to write this book and, from past experience, it takes anywhere from six months to a year, if not more, to update an existing title to the latest version of the software it covers.

So either you stop reading right now and return this book right away, or you read on for a cogent explanation on how to best use this book despite its almost guaranteed obsolescence.

Despite its very rapid release cycle, Android’s internal architecture and the procedures for building it have remained almost unchanged since its introduction about five years ago. So while this book was first written with 2.3/Gingerbread in mind, it’s been relatively straightforward to update it to also cover 4.2/Jelly Bean with references included to other versions, including 4.0/Ice-Cream Sandwich and 4.1/Jelly Bean where relevant. Hence, while new versions add new features, and many of the software components we discuss here will be enriched with every new version, the underlying procedures and mechanisms are likely to remain applicable for quite some time still.

Therefore, while you can be assured that I am committed to continuing to monitor Android’s development and updating this title as often as I humanly can, you should still be able to benefit from the explanations contained in this book for quite a few more versions than the ones covered.

Note

Some actually expect 2.3/Gingerbread to be around for a very long time given that its hardware requirements are much more modest than later versions. At the AnDevCon IV conference in December 2012, for instance, the keynote speaker from Facebook explained that it expected to have to support its app on devices running 2.3/Gingerbread for a very long time, given that that version runs on cheaper hardware than more recent versions.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This icon signifies a tip, suggestion, or general note.

Caution

This icon indicates a warning or caution.

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Embedded Android by Karim Yaghmour (O’Reilly). Copyright 2013 Karim Yaghmour, 978-1-449-30829-2.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

Safari® Books Online

Note

Safari Books Online is an on-demand digital library that lets you easily search over 7,500 technology and creative reference books and videos to find the answers you need quickly.

With a subscription, you can read any page and watch any video from our library online. Read books on your cell phone and mobile devices. Access new titles before they are available for print, and get exclusive access to manuscripts in development and post feedback for the authors. Copy and paste code samples, organize your favorites, download chapters, bookmark key sections, create notes, print out pages, and benefit from tons of other time-saving features.

O’Reilly Media has uploaded this book to the Safari Books Online service. To have full digital access to this book and others on similar topics from O’Reilly and other publishers, sign up for free at http://my.safaribooksonline.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North

Sebastopol, CA 95472

800-998-9938 (in the United States or Canada)

707-829-0515 (international or local)

707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://oreil.ly/embedded-android.

To comment or ask technical questions about this book, send email to:

[e-mail]

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

This is my second book ever and my first in 10 years. I’m somewhat skeptical about self-diagnosis, especially when I’m doing it myself—as I’m doing right here, but I clearly seem to have a tendency to be naively drawn to exploring uncharted territory. When I set out to write my first book, Building Embedded Linux Systems, in 2001, there wasn’t any book describing in full what embedded Linux was about. It took me two years to write down what was in fact half the material I originally thought would take me one year to write. In the same way, there was practically no information about embedded Android when I set out to write the present book in 2011. Somewhat coincidentally, it’s taken me two years to finish the manuscript you’re presently holding in your hands (or, these days, looking at on your screen, tablet, phone, or whichever device hadn’t yet been conceived as I’m writing these lines.)

Overall, I’ve found that writing books feels like attrition warfare. Maybe that’s because of the topics I choose, or maybe it’s just my own quirks. Still, akin to attrition warfare, writing books on ambitious topics isn’t something that can be done alone. Indeed, when I set out writing this book, I knew but a fraction of what you’ll find in these pages. While you can bet that I’ve done a tremendous amount of research, I should also highlight that what you have here is the result of a very large number of interactions I’ve had with many talented developers, each of whom taught me a little bit more than what I knew then. Therefore, if you’ve ever asked me a question at a conference or during a class, or if you’ve ever explained to me what you’re doing with Android or what problems you’re encountering with it or, better yet, have sent me in the right direction when I was lost with Android, know that part of you is somewhere in here.

It also takes a special breed of publisher to make this type of book possible. As with my first book, everyone at O’Reilly has simply been exceptional. I would like to first thank Mike Hendrickson for believing in this project and in my ability to deliver it. It’s also been a tremendous privilege to once more have the chance to work with Andy Oram as an editor. He’s again done a fantastic job at vetting the text you’re reading and, oftentimes, pointing out technical issues. In addition to Andy, I’d also like to thank Rachel Roumeliotis and Maria Stallone for gently reminding me to continue pushing this book forward.

Another aspect of writing this type of book is that utmost caution has to be exercised in order to ensure technical accuracy. It was therefore crucial for me to have a strong technical review team. As such, I would like to start by thanking Magnus Bäck, Mark Gross, and Amit Pundir for agreeing very early on in this project to review the book and for having provided generous feedback over the long period when it was written. This initial group was joined along the way by many other talented individuals. Hardware guru David Anders provided key feedback on the hardware chapter. Robert PJ Day did a great job of making sure it made sense for those who’ve never been exposed to Android. Benjamin Zores ironed out several aspects of the stack’s internals. Finally, some readers of the book’s early versions, such as Andrew Van Uitert and Maxime Ripard, gracefully shared with me some issues they found along the way.

I would like to most especially thank Linaro’s Android team and Bernhard Rosenkränzer specifically for almost single-handedly pointing out the vast majority of discrepancies between the earlier version of this book, which was very 2.3/Gingerbread-centric, and 4.2/Jelly Bean. If you’re happy to hold a book that covers two major Android versions, one of which is the latest one at the time of this writing, thank Bernhard. Not only did he force my hand in updating the book, but his input was by far the most extensive—and often the most detailed. I would therefore like to warmly thank Zach Pfeffer for offering his team’s help and making it possible for Bernhard to contribute, along with Vishal Bhoj, Fahad Kunnathadi, and YongQin Liu.

As I said earlier, people I’ve met along the way at conferences have been instrumental in this writing. I would therefore like to single out two organizations that have gone out of their way to make it possible for me to participate in their conferences. First, I’d like to thank the BZ Media team, who’ve been organizing the AnDevCon conferences since early 2011 and who trusted me early on to talk about Android’s internals and have continued inviting me since. Special thanks to Alan Zeichick, Ted Bahr, Stacy Burris, and Katie Serignese. I’d also like to thank the Linux Foundation for giving me the chance to keynote, speak, and participate in a number of events they’ve been organizing over the years, including the Android Builders Summit, the Embedded Linux Conference, and the Embedded Linux Conference Europe. Special thanks to Mike Woster, Amanda McPherson, Angela Brown, Craig Ross, Maresa Fowler, Rudolf Streif, Dominic Duval, Ibrahim Haddad, and Jerry Cooperstein.

A special thanks also to the team at RevolutionLinux, especially Benoit des Ligneris, Bruno Lambert, and Patrick Turcotte, for agreeing to be my guinea pigs early on. Your trust has borne fruit.

Finally, a very special thanks to Google’s Android team for having created one of the best brain-teasers I’ve run into in a while. I say this sincerely: Exploring this operating system has been one of the funnest things I’ve done in some time. Kudos to the entire team for creating an amazing piece of software and making it available so generously under such a permissive license. And while I understand this is an unconventional open-source project where transparency isn’t (for good reason) on the agenda, I’d like to thank those Android developers who’ve helped (or in some cases at least tried) in various ways. Thanks to Brian Swetland for filling in the blanks every so often on LWN and to Chet Haase.

These acknowledgments would be incomplete without closing with those who are closest to my heart. Thank you Sonia, Anaïs, Thomas, and Vincent for your loving patience throughout. Les mains invisibles qui ont écrit les espaces entre les lignes sont les leurs et je leur en suis profondémment reconnaissant.^[1]

---

^[1] The invisible hands that wrote the spaces between the lines are theirs, and for this I am profoundly grateful to them.