Chapter 4. The Build System
The goal of the previous chapter was to get you up and running as quickly as possible with custom AOSP development. There’s nothing precluding you from closing this book at this point and starting to dig in and modify your AOSP tree to fit your needs. All you need to do to test your modifications is to rebuild the AOSP, start the emulator again, and, if need be, shell back into it using ADB. If you want to maximize your efforts, however, you’ll likely want some insight into Android’s build system.
Despite its modularity, Android’s build system is fairly complex and doesn’t resemble any of the mainstream build systems out there; none that are used for most open source projects, at least. Specifically, it uses make in a fairly unconventional way and doesn’t provide any sort of menuconfig-based configuration (or equivalent for that matter). Android very much has its own build paradigm that takes some time to get used to. So grab yourself a good coffee or two—things are about to get serious.
Warning
Like the rest of the AOSP, the build system is a moving target. So while the following information should remain valid for a long time, you should be on the lookout for changes in the AOSP version you’re using.
Comparison with Other Build Systems
Before I start explaining how Android’s build system works, allow me to begin by emphasizing how it differs from what you might already know. First and foremost, unlike most make-based build systems, the Android build system doesn’t rely on recursive makefiles. Unlike the Linux kernel, for instance, there isn’t a top-level makefile that will recursively invoke subdirectories’ makefiles. Instead, there is a script that explores all directories and subdirectories until it finds an Android.mk file, whereupon it stops and doesn’t explore the subdirectories underneath that file’s location—unless the Android.mk found instructs the build system otherwise. Note that Android doesn’t rely on makefiles called Makefile. Instead, it’s the Android.mk files that specify how the local “module” is built.
Warning
Android build “modules” have nothing to do with kernel “modules.” Within the context of Android’s build system, a “module” is any component of the AOSP that needs to be built. This might be a binary, an app package, a library, etc., and it might have to be built for the target or the host, but it’s still a “module” with regards to the build system.
Another Android specificity is the way the build system is configured. While most of us are used to systems based on kernel-style menuconfig or GNU autotools (i.e., autoconf, automake, etc.), Android relies on a set of variables that are either set dynamically as part of the shell’s environment by way of envsetup.sh and lunch or are defined statically ahead of time in a buildspec.mk file. Also—always seeming to be a surprise to newcomers—the level of configurability made possible by Android’s build system is fairly limited. So while you can specify the properties of the target for which you want the AOSP to be built and, to a certain extent, which apps should be included by default in the resulting AOSP, there is no way for you to enable or disable most features, as is possible à la menuconfig. You can’t, for instance, decide that you don’t want power management support or that you don’t want the Location Service to start by default.
Also, the build system doesn’t generate object files or any sort of intermediate output within the same location as the source files. You won’t find the .o files alongside their .c source files within the source tree, for instance. In fact, none of the existing AOSP directories are used in any of the output. Instead, the build system creates an out/ directory where it stores everything it generates. Hence, a make clean is very much the same thing as an rm -rf out/. In other words, removing the out/ directory wipes out anything that was built.
The last thing to say about the build system before we start
exploring it in more detail is that it’s heavily tied to GNU
make. And, more to the point, version 3.81; even the newer 3.82
won’t work with many AOSP versions without patching. The build system in fact heavily relies on many GNU
make-specific features such as the define, include, and ifndef directives.
Architecture
As illustrated in Figure 4-1, the entry point to making sense of the build system is the main.mk file found in the build/core/ directory, which is invoked through the top-level makefile, as we saw earlier. The build/core/ directory actually contains the bulk of the build system, and we’ll cover key files from there. Again, remember that Android’s build system pulls everything into a single makefile; it isn’t recursive. Hence, each .mk file you see eventually becomes part of a single huge makefile that contains the rules for building all the pieces in the system.
Configuration
One of the first things the build system does is pull in the build configuration through the inclusion of config.mk. The build can be configured either by the use of the envsetup.sh and lunch commands or by providing a buildspec.mk file at the top-level directory. In either case, some of the following variables need to be set.
TARGET_PRODUCTAndroid flavor to be built. Each recipe can, for instance, include a different set of apps or locales or build different parts of the tree. Have a look at the various single product .mk files included by the AndroidProducts.mk files in build/target/product/, device/samsung/crespo/, and device/htc/passion/ for examples in 2.3/Gingerbread. In case of 4.2/Jelly Bean, look at device/asus/grouper/ and device/samsung/amgnuro/ instead of Crespo and Passion. Values include the following:
genericThe “vanilla” kind, the most basic build of the AOSP parts you can have.
fullThe “all dressed” kind, with most apps and the major locales enabled.
full_crespoSame as
fullbut for Crespo (Samsung Nexus S).full_grouperSame as
fullbut for Grouper (Asus Nexus 7).simAndroid simulator (see The Simulator: A Piece of Android’s History). Even though this is available in 2.3/Gingerbread, this target has since been removed and isn’t in 4.2/Jelly Bean.
sdk
TARGET_BUILD_VARIANTSelects which modules to install. Each module is supposed to have a
LOCAL_MODULE_TAGSvariable set in its Android.mk to at least one of the following:[18]user,debug,eng,tests,optional, orsamples. By selecting the variant, you will tell the build system which module subsets should be included—the only exception to this is packages (i.e., modules that generate .apk files) for which these rules don’t apply. Specifically:engIncludes all modules tagged as
user,debug, oreng.userdebugIncludes both modules tagged as
useranddebug.userIncludes only modules tagged as
user.
TARGET_BUILD_TYPEDictates whether or not special build flags are used or
DEBUGvariables are defined in the code. The possible values here are eitherreleaseordebug. Most notably, the frameworks/base/Android.mk file chooses between either frameworks/base/core/config/debug or frameworks/base/core/config/ndebug, depending on whether or not this variable is set todebug. The former causes theConfigBuildFlags.DEBUGJava constant to be set totrue, whereas the latter causes it to be set tofalse. Some code in parts of the system services, for instance, is conditional onDEBUG. Typically,TARGET_BUILD_TYPEis set torelease.TARGET_TOOLS_PREFIXBy default, the build system will use one of the cross-development toolchains shipped with it underneath the prebuilt/ directory — prebuilts/ as of 4.2/Jelly Bean. However, if you’d like it to use another toolchain, you can set this value to point to its location.
OUT_DIRBy default, the build system will put all build output into the out/ directory. You can use this variable to provide an alternate output directory.
BUILD_ENV_SEQUENCE_NUMBERIf you use the template build/buildspec.mk.default to create your own buildspec.mk file, this value will be properly set. However, if you create a buildspec.mk with an older AOSP release and try to use it in a future AOSP release that contains important changes to its build system and, hence, a different value, this variable will act as a safety net. It will cause the build system to inform you that your buildspec.mk file doesn’t match your build system.
In addition to selecting which parts of the AOSP to build and
which options to build them with, the build system also needs to know
about the target it’s building for. This is provided through a BoardConfig.mk file, which will specify
things such as the command line to be provided to the kernel, the base
address at which the kernel should be loaded, or the instruction set
version most appropriate for the board’s CPU (TARGET_ARCH_VARIANT). Have a look at build/target/board/ for a set of per-target
directories that each contain a BoardConfig.mk file. Also have a look at the
various device/*/TARGET_DEVICE/BoardConfig.mk files included in the AOSP.
The latter are much richer than the former because they contain a lot
more hardware-specific information. The device name (i.e.,
TARGET_DEVICE) is derived from the PRODUCT_DEVICE specified in the product
.mk file provided for the TARGET_PRODUCT set in the configuration. In
2.3/Gingerbread, for example, device/samsung/crespo/AndroidProducts.mk
includes device/samsung/crespo/full_crespo.mk, which
sets PRODUCT_DEVICE to crespo. Hence, the build system looks for a
BoardConfig.mk in device/*/crespo/, and there happens to be one
at that location. The same goes on in 4.2/Jelly Bean for the PRODUCT_DEVICE set in device/asus/grouper/full_grouper.mk to
grouper, thereby pointing the build
system to device/*/grouper/BoardConfig.mk.
The final piece of the puzzle with regard to configuration is the
CPU-specific options used to build Android. For ARM, those are contained
in build/core/combo/arch/arm/armv*.mk, with
TARGET_ARCH_VARIANT determining the
actual file to use. Each file lists CPU-specific cross-compiler and
cross-linker flags used for building C/C++ files. They also contain a
number of ARCH_ARM_HAVE_* variables
that enable others parts of the AOSP to build code conditionally based
on whether a given ARM feature is found in the target’s CPU.
envsetup.sh
Now that you understand the kinds of configuration input the build system needs, we can discuss the role of envsetup.sh in more detail. As its name implies, envsetup.sh actually is for setting up a build environment for Android. It does only part of the job, though. Mainly, it defines a series of shell commands that are useful to any sort of AOSP work:
$cd ~/android/aosp-2.3.x$. build/envsetup.sh$helpInvoke ". build/envsetup.sh" from your shell to add the following functions to your environment: - croot: Changes directory to the top of the tree. - m: Makes from the top of the tree. - mm: Builds all of the modules in the current directory. - mmm: Builds all of the modules in the supplied directories. - cgrep: Greps on all local C/C++ files. - jgrep: Greps on all local Java files. - resgrep: Greps on all local res/*.xml files. - godir: Go to the directory containing a file. Look at the source to view more functions. The complete list is: add_lunch_combo cgrep check_product check_variant choosecombo chooseproduct choo setype choosevariant cproj croot findmakefile gdbclient get_abs_build_var getbug reports get_build_var getprebuilt gettop godir help isviewserverstarted jgrep lu nch m mm mmm pgrep pid printconfig print_lunch_menu resgrep runhat runtest set_j ava_home setpaths set_sequence_number set_stuff_for_environment settitle smokete st startviewserver stopviewserver systemstack tapas tracedmdump
In 4.2/Jelly Bean, hmm has replaced help, and the command set made available to you has been expanded:
$cd ~/android/aosp-4.2$. build/envsetup.sh$hmmInvoke ". build/envsetup.sh" from your shell to add the following functions to y our environment: - lunch: lunch <product_name>-<build_variant> - tapas: tapas [<App1> <App2> ...] [arm|x86|mips] [eng|userdebug|user] - croot: Changes directory to the top of the tree. - m: Makes from the top of the tree. - mm: Builds all of the modules in the current directory. - mmm: Builds all of the modules in the supplied directories. - cgrep: Greps on all local C/C++ files. - jgrep: Greps on all local Java files. - resgrep: Greps on all local res/*.xml files. - godir: Go to the directory containing a file. Look at the source to view more functions. The complete list is: addcompletions add_lunch_combo cgrep check_product check_variant choosecombo cho oseproduct choosetype choosevariant cproj croot findmakefile gdbclient get_abs_b uild_var getbugreports get_build_var getlastscreenshot getprebuilt getscreenshot path getsdcardpath gettargetarch gettop godir hmm isviewserverstarted jgrep key_ back key_home key_menu lunch _lunch m mm mmm pid printconfig print_lunch_menu re sgrep runhat runtest set_java_home setpaths set_sequence_number set_stuff_for_en vironment settitle smoketest startviewserver stopviewserver systemstack tapas tr acedmdump
You’ll likely find the croot and godir commands quite useful for traversing the
tree. Some parts of it are quite deep, given the use of Java and its
requirement that packages be stored in directory trees bearing the same
hierarchy as each subpart of the corresponding fully qualified package
name. For instance, a file part of the com.foo.bar package must be stored under the
com/foo/bar/ directory. Hence, it’s
not rare to find yourself 7 to 10 directories underneath the AOSP’s
top-level directory, and it rapidly becomes tedious to type something
like cd ../../../ ... to return to an
upper part of the tree.
m and mm are also quite useful since they allow you to, respectively, build from the top level regardless of where you are or just build the modules found in the current directory. For example, if you made a modification to the Launcher and are in packages/apps/Launcher2, you can rebuild just that module by typing mm instead of cd’ing back to the top level and typing make. Note that mm doesn’t rebuild the entire tree and, therefore, won’t regenerate AOSP images even if a dependent module has changed. m will do that, though. Still, mm can be useful to test whether your local changes break the build or not until you’re ready to regenerate the full AOSP.
Although the online help doesn’t mention lunch, it is one of the commands defined by envsetup.sh. When you run lunch without any parameters, it shows you a list of potential choices. This is the list from 2.3/Gingerbread:
$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo4g-userdebug
5. full_crespo-userdebug
Which would you like? [generic-eng]This is the list from 4.2/Jelly Bean:
$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. full-eng
2. full_x86-eng
3. vbox_x86-eng
4. full_mips-eng
5. full_grouper-userdebug
6. full_tilapia-userdebug
7. mini_armv7a_neon-userdebug
8. mini_armv7a-userdebug
9. mini_mips-userdebug
10. mini_x86-userdebug
11. full_mako-userdebug
12. full_maguro-userdebug
13. full_manta-userdebug
14. full_toroplus-userdebug
15. full_toro-userdebug
16. full_panda-userdebug
Which would you like? [full-eng]These choices are not static. Most depend on what’s in the
AOSP at the time envsetup.sh runs.
They’re in fact individually added using the add_lunch_combo() function that the script
defines. In 2.3/Gingerbread, for instance, envsetup.sh adds generic-eng and simulator by default:
# add the default one here
add_lunch_combo generic-eng
# if we're on linux, add the simulator. There is a special case
# in lunch to deal with the simulator
if [ "$(uname)" = "Linux" ] ; then
add_lunch_combo simulator
fiIn 4.2/Jelly Bean, simulator is
no longer a valid target and envsetup.sh does this instead:
# add the default one here add_lunch_combo full-eng add_lunch_combo full_x86-eng add_lunch_combo vbox_x86-eng add_lunch_combo full_mips-eng
envsetup.sh also includes all the vendor-supplied scripts it can find. Here’s how it’s done in 2.3/Gingerbread:
# Execute the contents of any vendorsetup.sh files we can find.
for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/build/vendorsetup.sh device/*
/*/vendorsetup.sh 2> /dev/null`
do
echo "including $f"
. $f
done
unset fHere’s how it’s done in 4.2/Jelly Bean:
# Execute the contents of any vendorsetup.sh files we can find.
for f in `/bin/ls vendor/*/vendorsetup.sh vendor/*/*/vendorsetup.sh device/*/*/v
endorsetup.sh 2> /dev/null`
do
echo "including $f"
. $f
done
unset fIn 2.3/Gingerbread the device/samsung/crespo/vendorsetup.sh file, for instance, does this:
add_lunch_combo full_crespo-userdebug
Similarly, in 4.2/Jelly Bean the device/asus/grouper/vendorsetup.sh file does this:
add_lunch_combo full_grouper-userdebug
So that’s how you end up with the menu we saw earlier.
Note that the menu asks you to choose a combo.
Essentially, this is a combination of a TARGET_PRODUCT and TARGET_BUILD_VARIANT, with the exception of
the simulator in 2.3/Gingerbread. The
menu provides the default combinations, but the others remain valid and
can be passed to lunch as parameters
on the command line. In 2.3/Gingerbread, for instance, you can do
something like this:
$lunch generic-user============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=generic TARGET_BUILD_VARIANT=user TARGET_SIMULATOR=false TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================ $lunch full_crespo-eng============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=full_crespo TARGET_BUILD_VARIANT=eng TARGET_SIMULATOR=false TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================
Once lunch has finished running
for a generic-eng combo, it will set
up environment variables described in Table 4-1 in your current shell to provide the
build system with the required configuration information.
| Variable | Value |
PATH | $ANDROID_JAVA_TOOLCHAIN:$PATH:$ANDROID_BUILD_PATHS |
ANDROID_EABI_TOOLCHAIN | |
ANDROID_TOOLCHAIN | $ANDROID_EABI_TOOLCHAIN |
ANDROID_QTOOLS | |
ANDROID_BUILD_PATHS | |
ANDROID_BUILD_TOP | |
ANDROID_JAVA_TOOLCHAIN | $JAVA_HOME/bin |
ANDROID_PRODUCT_OUT | |
OUT | ANDROID_PRODUCT_OUT |
BUILD_ENV_SEQUENCE_NUMBER | 10 |
OPROFILE_EVENTS_DIR | |
TARGET_BUILD_TYPE | release |
TARGET_PRODUCT | generic |
TARGET_BUILD_VARIANT | eng |
TARGET_BUILD_APPS | empty |
TARGET_SIMULATOR | false |
PROMPT_COMMAND | \"\033]0;[${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}]
${USER}@${HOSTNAME}: ${PWD}\007\" |
JAVA_HOME | /usr/lib/jvm/java-6-sun |
Of course, if you get tired of always typing build/envsetup.sh and lunch, all you need to do is copy the build/buildspec.mk.default into the top-level directory, rename it to buildspec.mk, and edit it to match the configuration that would have otherwise been set by running those commands. The file already contains all the variables you need to provide; it’s just a matter of uncommenting the corresponding lines and setting the values appropriately. Once you’ve done that, all you have to do is go to the AOSP’s directory and invoke make directly. You can skip envsetup.sh and lunch.
Function Definitions
Because the build system is fairly large—there are more than 40
.mk files in build/core/ alone—there are benefits in being
able to reuse as much code as possible. This is why the build system
defines a large number of functions in the definitions.mk file. That file
is actually the largest one in the build system at about 60KB, with
about 140 functions on about 1,800 lines of makefile code in
2.3/Gingerbread. It’s still the largest file in the build system in
4.2/Jelly Bean at about 73KB, 170 functions, and about 2,100 lines of
makefile code. Functions offer a variety of operations, including file
lookup (e.g., all-makefiles-under and
all-c-files-under), transformation
(e.g., transform-c-to-o and transform-java-to-classes.jar), copying (e.g.,
copy-file-to-target), and utility
(e.g., my-dir.)
Not only are these functions used throughout the rest of the build system’s components, acting as its core library, but they’re sometimes also directly used in modules’ Android.mk files. Here’s an example snippet from the Calculator app’s Android.mk:
LOCAL_SRC_FILES := $(call all-java-files-under, src)
Although thoroughly describing definitions.mk is outside the scope of this book, it should be fairly easy for you to explore it on your own. If nothing else, most of the functions in it are preceded with a comment explaining what they do. Here’s an example from 2.3/Gingerbread:
###########################################################
## Find all of the java files under the named directories.
## Meant to be used like:
## SRC_FILES := $(call all-java-files-under,src tests)
###########################################################
define all-java-files-under
$(patsubst ./%,%, \
$(shell cd $(LOCAL_PATH) ; \
find $(1) -name "*.java" -and -not -name ".*") \
)
endefMain Make Recipes
At this point you might be wondering where any of the goodies are actually generated. How are the various images such as RAM disk generated or how is the SDK put together, for example? Well, I hope you won’t hold a grudge, but I’ve been keeping the best for last. So without further ado, have a look at the Makefile in build/core/ (not the top-level one). The file starts with an innocuous-looking comment:
# Put some miscellaneous rules here
But don’t be fooled. This is where some of the best meat is. Here’s the snippet that takes care of generating the RAM disk, for example, in 2.3/Gingerbread:
# ----------------------------------------------------------------- # the ramdisk INTERNAL_RAMDISK_FILES := $(filter $(TARGET_ROOT_OUT)/%, \ $(ALL_PREBUILT) \ $(ALL_COPIED_HEADERS) \ $(ALL_GENERATED_SOURCES) \ $(ALL_DEFAULT_INSTALLED_MODULES)) BUILT_RAMDISK_TARGET := $(PRODUCT_OUT)/ramdisk.img # We just build this directly to the install location. INSTALLED_RAMDISK_TARGET := $(BUILT_RAMDISK_TARGET) $(INSTALLED_RAMDISK_TARGET): $(MKBOOTFS) $(INTERNAL_RAMDISK_FILES) | $(MINIGZIP) $(call pretty,"Target ram disk: $@") $(hide) $(MKBOOTFS) $(TARGET_ROOT_OUT) | $(MINIGZIP) > $@
And here’s the snippet that creates the certs packages for checking over-the-air (OTA) updates in the same AOSP version:
# ----------------------------------------------------------------- # Build a keystore with the authorized keys in it, used to verify the # authenticity of downloaded OTA packages. # # This rule adds to ALL_DEFAULT_INSTALLED_MODULES, so it needs to come # before the rules that use that variable to build the image. ALL_DEFAULT_INSTALLED_MODULES += $(TARGET_OUT_ETC)/security/otacerts.zip $(TARGET_OUT_ETC)/security/otacerts.zip: KEY_CERT_PAIR := $(DEFAULT_KEY_CERT_PAIR) $(TARGET_OUT_ETC)/security/otacerts.zip: $(addsuffix .x509.pem, $(DEFAULT_KEY_CERT_PAIR)) $(hide) rm -f $@ $(hide) mkdir -p $(dir $@) $(hide) zip -qj $@ $< .PHONY: otacerts otacerts: $(TARGET_OUT_ETC)/security/otacerts.zip
Obviously there’s a lot more than I can fit here, but have a look at Makefile for information on how any of the following are created:
Properties (including the target’s /default.prop and /system/build.prop).
RAM disk.
Boot image (combining the RAM disk and a kernel image).
NOTICE files: These are files required by the AOSP’s use of the Apache Software License (ASL). Have a look at the ASL for more information about NOTICE files.
OTA keystore.
Recovery image.
System image (the target’s /system directory).
Data partition image (the target’s /data directory).
OTA update package.
SDK.
Nevertheless, some things aren’t in this file:
- Kernel images
Don’t look for any rule to build these. There is no kernel part of the official AOSP releases—some of the third-party projects listed in Appendix E, however, actually do package kernel sources directly into the AOSPs they distribute. Instead, you need to find an Androidized kernel for your target, build it separately from the AOSP, and feed it to the AOSP. You can find a few examples of this in the devices in the device/ directory. In 2.3/Gingerbread, for example, device/samsung/crespo/ includes a kernel image (file called kernel) and a loadable module for the Crespo’s WiFi (bcm4329.ko file). Both of these are built outside the AOSP and copied in binary form into the tree for inclusion with the rest of the build.
- NDK
While the code to build the NDK is in the AOSP, it’s entirely separate from the AOSP’s build system in build/. Instead, the NDK’s build system is in ndk/build/. We’ll discuss how to build the NDK shortly.
- CTS
The rules for building the CTS are in build/core/tasks/cts.mk.
Cleaning
As I mentioned earlier, a make
clean is very much the equivalent of wiping out the out/ directory. The clean target itself is defined in main.mk. There are, however, other cleanup
targets. Most notably, installclean,
which is defined in cleanbuild.mk,
is automatically invoked whenever you change TARGET_PRODUCT, TARGET_BUILD_VARIANT or PRODUCT_LOCALES. For instance, if I had first
built 2.3/Gingerbread for the generic-eng combo and then used lunch to switch the combo to
full-eng, the next time I started
make, some of the build output would
be automatically pruned using installclean:
$ make -j16
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
...
============================================
*** Build configuration changed: "generic-eng-{mdpi,nodpi}" -> "full-eng-{en_US,
en_GB,fr_FR,it_IT,de_DE,es_ES,mdpi,nodpi}"
*** Forcing "make installclean"...
*** rm -rf out/target/product/generic/data/* out/target/product/generic/data-qem
u/* out/target/product/generic/userdata-qemu.img out/host/linux-x86/obj/NOTICE_F
ILES out/host/linux-x86/sdk out/target/product/generic/*.img out/target/product/
generic/*.txt out/target/product/generic/*.xlb out/target/product/generic/*.zip
out/target/product/generic/data out/target/product/generic/obj/APPS out/target/p
roduct/generic/obj/NOTICE_FILES out/target/product/generic/obj/PACKAGING out/tar
get/product/generic/recovery out/target/product/generic/root out/target/product/
generic/system out/target/product/generic/dex_bootjars out/target/product/generi
c/obj/JAVA_LIBRARIES
*** Done with the cleaning, now starting the real build.In contrast to clean, installclean doesn’t wipe out the entirety of
out/. Instead, it only nukes the
parts that need rebuilding given the combo configuration change. There’s
also a clobber target which is
essentially the same thing as a clean.
Module Build Templates
What I just described is the build system’s architecture and the
mechanics of its core components. Having read that, you should have a
much better idea of how Android is built from a top-down perspective.
Very little of that, however, permeates down to the level of AOSP
modules’ Android.mk files. The
system has in fact been architected so that module build recipes are
pretty much independent from the build system’s internals. Instead,
build templates are provided so that module authors can get their
modules built appropriately. Each template is tailored for a specific
type of module, and module authors can use a set of documented
variables, all prefixed by LOCAL_, to
modulate the templates’ behavior and output. Of course, the templates
and underlying support files (mainly base_rules.mk) closely interact with the rest
of the build system to deal properly with each module’s build output.
But that’s invisible to the module’s author.
The templates are themselves found in the same location as the
rest of the build system in build/core/. Android.mk gets access to them through the
include directive. Here’s an
example:
include $(BUILD_PACKAGE)
As you can see, Android.mk files don’t actually include the .mk templates by name. Instead, they include a variable that is set to the corresponding .mk file. Table 4-2 provides the full list of available module templates.
| Variable | Template | What It Builds | Most Notable Use |
BUILD_EXECUTABLE | executable.mk | Target binaries | Native commands and daemons |
BUILD_HOST_EXECUTABLE | host_executable.mk | Host binaries | Development tools |
BUILD_RAW_EXECUTABLE | raw_executable.mk | Target binaries that run on bare metal | Code in the bootloader/ directory |
BUILD_JAVA_LIBRARY | java_library.mk | Target Java libaries | Apache Harmony and Android Framework |
BUILD_STATIC_JAVA_LIBRARY | static_java_library.mk | Target static Java libraries | N/A, few modules use this |
BUILD_HOST_JAVA_LIBRARY | host_java_library.mk | Host Java libraries | Development tools |
BUILD_SHARED_LIBRARY | shared_library.mk | Target shared libraries | A vast number of modules, including many in external/ and frameworks/base/ |
BUILD_STATIC_LIBRARY | static_library.mk | Target static libraries | A vast number of modules, including many in external/ |
BUILD_HOST_SHARED_LIBRARY | host_shared_library.mk | Host shared libraries | Development tools |
BUILD_HOST_STATIC_LIBRARY | host_static_library.mk | Host static libraries | Development tools |
BUILD_RAW_STATIC_LIBRARY | raw_static_library.mk | Target static libraries that run on bare metal | Code in bootloader/ |
BUILD_PREBUILT | prebuilt.mk | Copies prebuilt target files | Configuration files and binaries |
BUILD_HOST_PREBUILT | host_prebuilt.mk | Copies prebuilt host files | Tools in prebuilt/ and configuration files |
BUILD_MULTI_PREBUILT | multi_prebuilt.mk | Copies prebuilt modules of multiple but known types, like Java libraries or executables | Rarely used |
BUILD_PACKAGE | package.mk | Built-in AOSP apps (i.e., anything that ends up being an .apk) | All apps in the AOSP |
BUILD_KEY_CHAR_MAP | key_char_map.mk | Device character maps | All device character maps in AOSP |
These build templates allow Android.mk files to be usually fairly lightweight:
LOCAL_PATH := $(call my-dir)include $(CLEAR_VARS)
LOCAL_
VARIABLE_1:=value_1LOCAL_
VARIABLE_2:=value_2... include $(BUILD_MODULE_TYPE)
Note
Note that CLEAR_VARS, which
is provided by clear_vars.mk,[19] is very important. Recall that the build system includes
all Android.mk into what amounts
to a single huge makefile. Including CLEAR_VARS ensures that the LOCAL_* values set for modules preceding
yours are zeroed out by the time your Android.mk is included. Also, a single
Android.mk can describe multiple
modules one after the other. Hence, CLEAR_VARS ensures that previous module
recipes don’t pollute subsequent ones.
Here’s the Service Manager’s Android.mk in 2.3/Gingerbread, for instance (frameworks/base/cmds/servicemanager/):[20]
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_SHARED_LIBRARIES := liblog
LOCAL_SRC_FILES := service_manager.c binder.c
LOCAL_MODULE := servicemanager
ifeq ($(BOARD_USE_LVMX),true)
LOCAL_CFLAGS += -DLVMX
endif
include $(BUILD_EXECUTABLE)And here’s the one[21] from 2.3/Gingerbread’s Desk Clock app (packages/app/DeskClock/):
LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := $(call all-java-files-under, src) LOCAL_PACKAGE_NAME := DeskClock LOCAL_OVERRIDES_PACKAGES := AlarmClock LOCAL_SDK_VERSION := current include $(BUILD_PACKAGE) include $(call all-makefiles-under,$(LOCAL_PATH))
As you can see, essentially the same structure is used in both modules, even though they provide very different input and result in very different output. Notice also the last line from the Desk Clock’s Android.mk, which basically includes all subdirectories’ Android.mk files. As I said earlier, the build system looks for the first makefile in a hierarchy and doesn’t look in any subdirectories underneath the directory where one was found, hence the need to manually invoke those. Obviously, the code here just goes out and looks for all makefiles underneath. However, some parts of the AOSP either explicitly list subdirectories or conditionally select them based on configuration.
The documentation at http://source.android.com used to
provide an exhaustive list of all the LOCAL_* variables with their meaning and use.
Unfortunately, at the time of this writing, this list is no longer
available. The build/core/build-system.html file, however,
contains an earlier version of that list, and you should refer to that
one until up-to-date lists become available again. Here are some of the
most frequently encountered LOCAL_*
variables:
LOCAL_PATHThe path of the current module’s sources, typically provided by invoking
$(call my-dir).LOCAL_MODULEThe name to attribute to this module’s build output. The actual filename or output and its location will depend on the build template you include. If this is set to
foo, for example, and you build an executable, then the final executable will be a command called foo and it will be put in the target’s /system/bin/. IfLOCAL_MODULEis set tolibfooand you includeBUILD_SHARED_LIBRARYinstead ofBUILD_EXECUTABLE, the build system will generate libfoo.so and put it in /system/lib/.Note that the name you provide here must be unique for the particular module class (i.e., build template type) you are building. There can’t be two libfoo.so libraries, for instance. It’s expected that the module name will have to be globally unique (i.e., across all module classes) at some point in the future.
LOCAL_SRC_FILESThe source files used to build the module. You may provide those by using one of the build system’s defined functions, as the Desk Clock uses
all-java-files-under, or you may list the files explicitly, as the Service Manager does.LOCAL_PACKAGE_NAMEUnlike all other modules, apps use this variable instead of
LOCAL_MODULEto provide their names, as you can witness by comparing the two Android.mk files shown earlier.LOCAL_SHARED_LIBRARIESUse this to list all the libraries your module depends on. As mentioned earlier, the Service Manager’s dependency on liblog is specified using this variable.
LOCAL_MODULE_TAGSAs I mentioned earlier, this allows you to control under which
TARGET_BUILD_VARIANTthis module is built. Usually, this should just be set tooptional.LOCAL_MODULE_PATHUse this to override the default install location for the type of module you’re building.
A good way to find out about more LOCAL_* variables is to look at existing
Android.mk files in the AOSP. Also,
clear_vars.mk contains the full
list of variables that are cleared. So while it doesn’t give you the
meaning of each, it certainly lists them all.
Also, in addition to the cleaning targets that affect the AOSP globally, each module can define its own cleaning rules by providing a CleanSpec.mk, much like modules provide Android.mk files. Unlike the latter, though, the former aren’t required. By default, the build system has cleaning rules for each type of module. But you can specify your own rules in a CleanSpec.mk in case your module’s build does something the build system doesn’t generate by default and, therefore, wouldn’t typically know how to clean up.
Output
Now that we’ve looked at how the build system works and how module build templates are used by modules, let’s look at the output it creates in out/. At a fairly high level, the build output operates in three stages and in two modes, one for the host and one for the target:
Intermediates are generated using the module sources. These intermediates’ format and location depend on the module’s sources. They may be .o files for C/C++ code, for example, or .jar files for Java-based code.
Intermediates are used by the build system to create actual binaries and packages: taking .o files, for example, and linking them into an actual binary.
The binaries and packages are assembled together into the final output requested of the build system. Binaries, for instance, are copied into directories containing the root and /system filesystems, and images of those filesystems are generated for use on the actual device.
out/ is mainly separated into
two directories, reflecting its operating modes: host/ and target/. In each directory, you will find a
couple of obj/ directories that
contain the various intermediates generated during the build. Most of
these are stored in subdirectories named like the one that the BUILD_* macros presented earlier and serve a
specific complementary purpose during the build system’s
operation:
EXECUTABLES/
JAVA_LIBRARIES/
SHARED_LIBRARIES/
STATIC_LIBRARIES/
APPS/
DATA/
ETC/
KEYCHARS/
PACKAGING/
NOTICE_FILES/
include/
lib/
The directory you’ll likely be most interested in is out/target/product/PRODUCT_DEVICE/.
That’s where the output images will be located for the PRODUCT_DEVICE defined in the corresponding
product configuration’s .mk. Table 4-3 explains the content of that
directory.
| Entry | Description |
| android-info.txt | Contains the code name for the board for which this product is configured |
| clean_steps.mk | Contains a list of steps that must be executed to clean
the tree, as provided in CleanSpec.mk files by calling the
add-clean-step
function |
| data/ | The target’s /data directory |
| installed-files.txt | A list of all the files installed in data/ and system/ directories |
| obj/ | The target product’s intermediaries |
| previous_build_config.mk | The last build target; will be used on the next make to check if the config has
changed, thereby forcing an installclean |
| ramdisk.img | The RAM disk image generated based on the content of the root/ directory |
| root/ | The content of the target’s root filesystem |
| symbols/ | Unstripped versions of the binaries put in the root filesystem and /system directory |
| system/ | The target’s /system directory |
| system.img | The /system image, based on the content of the system/ directory |
| userdata.img | The /data image, based on the content of the data/ directory |
Have a look back at Chapter 2 for a refresher on the root filesystem, /system, and /data. Essentially, though, when the kernel boots, it will mount the RAM disk image and execute the /init found inside. That binary, in turn, will run the /init.rc script that will mount both the /system and /data images at their respective locations. We’ll come back to the root filesystem layout and the system’s operation at boot time in Chapter 6.
Build Recipes
With the build system’s architecture and functioning in mind, let’s take a look at some of the most common, and some slightly uncommon, build recipes. We’ll only lightly touch on using the results of each recipe, but you should have enough information to get started.
The Default droid Build
Earlier, we went through a number of plain make commands but never really explained the default target. When you run plain make, it’s as if you had typed:[22]
$ make droiddroid is in fact the default
target as defined in main.mk. You
don’t usually need to specify this target manually. I’m providing it
here for completeness, so you know it exists.
Seeing the Build Commands
When you build the AOSP, you’ll notice that it doesn’t
actually show you the commands it’s running. Instead, it prints out only
a summary of each step it’s at. If you want to see everything it does,
like the gcc command lines for
example, add the showcommands target
to the command line:
$ make showcommands
...
host Java: apicheck (out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/c
lasses)
for f in ; do if [ ! -f $f ]; then echo Missing file $f; exit 1; fi; unzip -qo $
f -d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes; (cd ou
t/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes && rm -rf META-I
NF); done
javac -J-Xmx512M -target 1.5 -Xmaxerrs 9999999 -encoding ascii -g -extdirs ""
-d out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes \@out/host
/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list-uniq || ( rm
-rf out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/classes ; exit 41
)
rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list
rm -f out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/java-source-list
-uniq
jar -cfm out/host/common/obj/JAVA_LIBRARIES/apicheck_intermediates/javalib.jar b
uild/tools/apicheck/src/MANIFEST.mf -C out/host/common/obj/JAVA_LIBRARIES/apich
eck_intermediates/classes .
Header: out/host/linux-x86/obj/include/libexpat/expat.h
cp -f external/expat/lib/expat.h out/host/linux-x86/obj/include/libexpat/expat.h
Header: out/host/linux-x86/obj/include/libexpat/expat_external.h
cp -f external/expat/lib/expat_external.h out/host/linux-x86/obj/include/libexpa
t/expat_external.h
Header: out/target/product/generic/obj/include/libexpat/expat.h
cp -f external/expat/lib/expat.h out/target/product/generic/obj/include/libexpat
/expat.h
...Illustrating what I explained in the previous section, this is the same as:
$ make droid showcommandsAs you’ll rapidly notice when using this, it generates a lot of output and is therefore hard to follow. You may, however, want to save the standard output and standard error into files if you’d like to analyze the actual commands used to build the AOSP:
$ make showcommands > aosp-build-stdout 2> aosp-build-stderrYou can also do something like this to merge all output into a single file:
$ make showcommands 2>&1 | tell build.logSome also report that they prefer using the nohup command instead:
$ nohup make showcommandsBuilding the SDK for Linux and Mac OS
The official Android SDK is available at http://developer.android.com. You can, however, build your own SDK using the AOSP if, for instance, you extended the core APIs to expose new functionality and would like to distribute the result to developers so they can benefit from your new APIs. To do so, you’ll need to select a special combo:
$. build/envsetup.sh$lunch sdk-eng$make sdk
Once this is done, the SDK will be in out/host/linux-x86/sdk/ when built on Linux and in out/host/darwin-x86/sdk/ when built on a Mac. There will be two copies, one a ZIP file, much like the one distributed at http://developer.android.com, and one uncompressed and ready to use.
Assuming you had already configured Eclipse for Android development using the instructions at http://developer.android.com, you’ll need to carry out two additional steps to use your newly built SDK. First, you’ll need to tell Eclipse the location of the new SDK. To do so, go to Window→Preferences→Android, enter the path to the new SDK in the SDK Location box, and click OK. Also, for reasons that aren’t entirely clear to me at the time of this writing, you also need to go to Window→Android SDK Manager, deselect all the items that might be selected except the first two under Tools, and then click “Install 2 packages...” Once that is done, you’ll be able to create new projects using the new SDK and access any new APIs you expose in it. If you don’t do that second step, you’ll be able to create new Android projects, but none of them will resolve Java libraries properly and will, therefore, never build.
Building the SDK for Windows
The instructions for building the SDK for Windows are slightly different from Linux and Mac OS:
$. build/envsetup.sh$lunch sdk-eng$make win_sdk
The resulting output will be in out/host/windows/sdk/.
Building the CTS
If you want to build the CTS, you don’t need to use envsetup.sh or lunch. You can go right ahead and type:
$ make cts
...
Generating test description for package android.sax
Generating test description for package android.performance
Generating test description for package android.graphics
Generating test description for package android.database
Generating test description for package android.text
Generating test description for package android.webkit
Generating test description for package android.gesture
Generating test plan CTS
Generating test plan Android
Generating test plan Java
Generating test plan VM
Generating test plan Signature
Generating test plan RefApp
Generating test plan Performance
Generating test plan AppSecurity
Package CTS: out/host/linux-x86/cts/android-cts.zip
Install: out/host/linux-x86/bin/adbThe cts command includes its own online help. Here’s the corresponding sample output from 2.3/Gingerbread:
$cd out/host/linux-x86/bin/$./ctsListening for transport dt_socket at address: 1337 Android CTS version 2.3_r3 $ cts_host >helpUsage: command options Available commands and options: Host: help: show this message exit: exit cts command line Plan: ls --plan: list available plans ls --plan plan_name: list contents of the plan with specified name add --plan plan_name: add a new plan with specified name add --derivedplan plan_name -s/--session session_id -r/--result result_type: derive a plan from the given session rm --plan plan_name/all: remove a plan or all plans from repository start --plan test_plan_name: run a test plan start --plan test_plan_name -d/--device device_ID: run a test plan using the specified device start --plan test_plan_name -t/--test test_name: run a specific test ... $cts_host > ls --planList of plans (8 in total): Signature RefApp VM Performance AppSecurity Android Java CTS
Once you have a target up and running, such as the emulator, you can launch the test suite and it will use adb to run tests on the target:
$ ./cts start --plan CTS
Listening for transport dt_socket at address: 1337
Android CTS version 2.3_r3
Device(emulator-5554) connected
cts_host > start test plan CTS
CTS_INFO >>> Checking API...
CTS_INFO >>> This might take several minutes, please be patient...
...Building the NDK
As I had mentioned earlier, the NDK has its own separate build system, with its own setup and help system, which you can invoke like this:
$cd ndk/build/tools$export ANDROID_NDK_ROOT=$aosp-root/ndk./make-release --helpUsage: make-release.sh [options] Valid options (defaults are in brackets): --help Print this help. --verbose Enable verbose mode. --release=nameSpecify release name [20110921] --prefix=nameSpecify package prefix [android-ndk] --development=pathPath to development/ndk directory [/home/karim/ opersys-dev/android/aosp-2.3.4/development/ndk] --out-dir=pathPath to output directory [/tmp/ndk-release] --force Force build (do not ask initial question) [no] --incremental Enable incremental packaging (debug only). [no] --darwin-ssh=hostnameSpecify Darwin hostname to ssh to for the build. --systems=listList of host systems to build for [linux-x86] --toolchain-src-dir=pathUse toolchain sources frompath
When you are ready to build the NDK, you can invoke make-release as follows, and witness its rather emphatic warning:
$./make-releaseIMPORTANT WARNING !! This script is used to generate an NDK release package from scratch for the following host platforms: linux-x86 This process is EXTREMELY LONG and may take SEVERAL HOURS on a dual-core machine. If you plan to do that often, please read docs/DEVELOPMENT.TXT that provides instructions on how to do that more easily. Are you sure you want to do that [y/N]yDownloading toolchain sources... ...
Updating the API
The build systems has safeguards in case you modify the AOSP’s core API. If you do, the build will fail by default with a warning such as this:
******************************
You have tried to change the API from what has been previously approved.
To make these errors go away, you have two choices:
1) You can add "@hide" javadoc comments to the methods, etc. listed in the
errors above.
2) You can update current.xml by executing the following command:
make update-api
To submit the revised current.xml to the main Android repository,
you will need approval.
******************************
make: *** [out/target/common/obj/PACKAGING/checkapi-current-timestamp] Error 38
make: *** Waiting for unfinished jobs....As the error message suggests, to get the build to continue, you’ll need to do something like this:
$ make update-api
...
Install: out/host/linux-x86/framework/apicheck.jar
Install: out/host/linux-x86/framework/clearsilver.jar
Install: out/host/linux-x86/framework/droiddoc.jar
Install: out/host/linux-x86/lib/libneo_util.so
Install: out/host/linux-x86/lib/libneo_cs.so
Install: out/host/linux-x86/lib/libneo_cgi.so
Install: out/host/linux-x86/lib/libclearsilver-jni.so
Copying: out/target/common/obj/JAVA_LIBRARIES/core_intermediates/emma_out/lib/cl
asses-jarjar.jar
Install: out/host/linux-x86/framework/dx.jar
Install: out/host/linux-x86/bin/dx
Install: out/host/linux-x86/bin/aapt
Copying: out/target/common/obj/JAVA_LIBRARIES/bouncycastle_intermediates/emma_ou
t/lib/classes-jarjar.jar
Copying: out/target/common/obj/JAVA_LIBRARIES/ext_intermediates/emma_out/lib/cla
sses-jarjar.jar
Install: out/host/linux-x86/bin/aidl
Copying: out/target/common/obj/JAVA_LIBRARIES/core-junit_intermediates/emma_out/
lib/classes-jarjar.jar
Copying: out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/emma_out/l
ib/classes-jarjar.jar
Copying current.xmlThe next time you start make, you won’t get any more errors regarding API changes. Obviously at this point you’re no longer compatible with the official APIs and are therefore unlikely to be able to get certified as an “Android” device by Google.
Building a Single Module
Up to now, we’ve looked at building the entire tree. You can also build individual modules. Here’s how you can ask the build system to build the Launcher2 module (i.e., the Home screen):
$ make Launcher2You can also clean modules individually:
$ make clean-Launcher2If you’d like to force the build system to regenerate the system
image to include your updated module, you can add the snod target to the command line:
$ make Launcher2 snod
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=2.3.4
TARGET_PRODUCT=generic
...
target Package: Launcher2 (out/target/product/generic/obj/APPS/Launcher2_interme
diates/package.apk)
'out/target/common/obj/APPS/Launcher2_intermediates//classes.dex' as 'classes.d
ex'...
Install: out/target/product/generic/system/app/Launcher2.apk
Install: out/host/linux-x86/bin/mkyaffs2image
make snod: ignoring dependencies
Target system fs image: out/target/product/generic/system.imgBuilding Out of Tree
If you’d ever like to build code against the AOSP and its Bionic library but don’t want to incorporate that into the AOSP, you can use a makefile such as the following to get the job done:[23]
# Paths and settings
TARGET_PRODUCT = generic
ANDROID_ROOT = /home/karim/android/aosp-2.3.x
BIONIC_LIBC = $(ANDROID_ROOT)/bionic/libc
PRODUCT_OUT = $(ANDROID_ROOT)/out/target/product/$(TARGET_PRODUCT)
CROSS_COMPILE = \
$(ANDROID_ROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-
# Tool names
AS = $(CROSS_COMPILE)as
AR = $(CROSS_COMPILE)ar
CC = $(CROSS_COMPILE)gcc
CPP = $(CC) -E
LD = $(CROSS_COMPILE)ld
NM = $(CROSS_COMPILE)nm
OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
RANLIB = $(CROSS_COMPILE)ranlib
READELF = $(CROSS_COMPILE)readelf
SIZE = $(CROSS_COMPILE)size
STRINGS = $(CROSS_COMPILE)strings
STRIP = $(CROSS_COMPILE)strip
export AS AR CC CPP LD NM OBJCOPY OBJDUMP RANLIB READELF \
SIZE STRINGS STRIP
# Build settings
CFLAGS = -O2 -Wall -fno-short-enums
HEADER_OPS = -I$(BIONIC_LIBC)/arch-arm/include \
-I$(BIONIC_LIBC)/kernel/common \
-I$(BIONIC_LIBC)/kernel/arch-arm
LDFLAGS = -nostdlib -Wl,-dynamic-linker,/system/bin/linker \
$(PRODUCT_OUT)/obj/lib/crtbegin_dynamic.o \
$(PRODUCT_OUT)/obj/lib/crtend_android.o \
-L$(PRODUCT_OUT)/obj/lib -lc -ldl
# Installation variables
EXEC_NAME = example-app
INSTALL = install
INSTALL_DIR = $(PRODUCT_OUT)/system/bin
# Files needed for the build
OBJS = example-app.o
# Make rules
all: example-app
.c.o:
$(CC) $(CFLAGS) $(HEADER_OPS) -c $<
example-app: ${OBJS}
$(CC) -o $(EXEC_NAME) ${OBJS} $(LDFLAGS)
install: example-app
test -d $(INSTALL_DIR) || $(INSTALL) -d -m 755 $(INSTALL_DIR)
$(INSTALL) -m 755 $(EXEC_NAME) $(INSTALL_DIR)
clean:
rm -f *.o $(EXEC_NAME) core
distclean:
rm -f *~
rm -f *.o $(EXEC_NAME) coreIn this case, you don’t need to care about either envsetup.sh or lunch. You can just go ahead and type the magic incantation:
$ makeObviously this won’t add your binary to any of the images
generated by the AOSP. Even the install target here will be of value only if
you’re mounting the target’s filesystem off NFS, and that’s valuable
only during debugging, which is what this makefile is assumed to be
useful for. To an extent, it could also be argued that using such a
makefile is actually counterproductive, since it’s far more complicated
than the equivalent Android.mk that
would result if this code were added as a module part of the
AOSP.
Still, this kind of hack can have its uses. Under certain circumstances, for instance, it might make sense to modify the conventional build system used by a rather large codebase to build that project against the AOSP yet outside of it; the alternative being to copy the project into the AOSP and create Android.mk files to reproduce the mechanics of its original conventional build system, which might turn out to be a substantial endeavor in and of itself.
Building Recursively, In-Tree
You can, if you really want to, hack yourself a makefile to build within the AOSP a component that is based on recursive makefiles instead of trying to reproduce the same functionality using Android.mk files, as was suggested in the last section. Several of the AOSP forks mentioned in Appendix E, for instance, include the kernel sources at the top level of the AOSP and modify the AOSP’s main makefile to invoke the kernel’s existing build system.
Here’s another example where an Android.mk was created by Linaro’s Bernhard Rosenkränzer in order to build ffmpeg—which relies on a GNU autotools-like script—using its original build files:
include $(CLEAR_VARS) FFMPEG_TCDIR := $(realpath $(shell dirname $(TARGET_TOOLS_PREFIX))) FFMPEG_TCPREFIX := $(shell basename $(TARGET_TOOLS_PREFIX)) # FIXME remove -fno-strict-aliasing once the aliasing violations are fixed FFMPEG_COMPILER_FLAGS = $(subst -I ,-I../../,$(subst -include \ system/core/include/arch/linux-arm/AndroidConfig.h,,$(subst -include \ build/core/combo/include/arch/linux-arm/AndroidConfig.h,, \ $(TARGET_GLOBAL_CFLAGS)))) -fno-strict-aliasing -Wno-error=address \ -Wno-error=format-security ifneq ($(strip $(SHOW_COMMANDS)),) FF_VERBOSE="V=1" endif .PHONY: ffmpeg droidcore: ffmpeg systemtarball: ffmpeg REALTOP=$(realpath $(TOP)) ffmpeg: x264 $(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates/libvpx.a mkdir -p $(PRODUCT_OUT)/obj/ffmpeg cd $(PRODUCT_OUT)/obj/ffmpeg && \ export PATH=$(FFMPEG_TCDIR):$(PATH) && \ $(REALTOP)/external/ffmpeg/configure \ --arch=arm \ --target-os=linux \ --prefix=/system \ --bindir=/system/bin \ --libdir=/system/lib \ --enable-shared \ --enable-gpl \ --disable-avdevice \ --enable-runtime-cpudetect \ --disable-libvpx \ --enable-libx264 \ --enable-cross-compile \ --cross-prefix=$(FFMPEG_TCPREFIX) \ --extra-ldflags="-nostdlib -Wl,-dynamic-linker, \ /system/bin/linker,-z,muldefs$(shell if test $(PRODUCT_SDK_VERSION) -lt 16; \ then echo -n ',-T$(REALTOP)/$(BUILD_SYSTEM)/armelf.x'; fi),-z,nocopyreloc, \ --no-undefined -L$(REALTOP)/$(TARGET_OUT_STATIC_LIBRARIES) \ -L$(REALTOP)/$(PRODUCT_OUT)/system/lib \ -L$(REALTOP)/$(PRODUCT_OUT)/obj/STATIC_LIBRARIES/libvpx_intermediates -ldl -lc" \ --extra-cflags="$(FFMPEG_COMPILER_FLAGS) \ -I$(REALTOP)/bionic/libc/include -I$(REALTOP)/bionic/libc/kernel/common \ -I$(REALTOP)/bionic/libc/kernel/arch-arm \ -I$(REALTOP)/bionic/libc/arch-arm/include -I$(REALTOP)/bionic/libm/include \ -I$(REALTOP)/external/libvpx -I$(REALTOP)/external/x264" \ --extra-libs="-lgcc" && \ $(MAKE) \ TARGET_CRTBEGIN_DYNAMIC_O=$(REALTOP)/$(TARGET_CRTBEGIN_DYNAMIC_O) \ TARGET_CRTEND_O=$(REALTOP)/$(TARGET_CRTEND_O) $(FF_VERBOSE) && \ $(MAKE) install DESTDIR=$(REALTOP)/$(PRODUCT_OUT)
Basic AOSP Hacks
You most likely bought this book with one thing in mind: to hack the AOSP to fit your needs. Over the next few pages, we’ll start looking into some of the most obvious hacks you’ll likely want to try. Of course we’re only setting the stage here with the parts that pertain to the build system, which is where you’ll likely want to start anyway.
Note
While the following explanations are based on 2.3/Gingerbread, they’ll work just the same on 4.2/Jelly Bean, and likely many versions after that one, too. The fact is, these mechanisms have been constant for quite some time. Still, where relevant, changes in 4.2/Jelly Bean are highlighted.
Adding a Device
Adding a custom device is most likely one of the topmost items (if not the topmost) on your list of reasons for reading this book. I’m about to show you how to do just that, so you’ll likely want to bookmark this section. Of course I’m actually only showing you the build aspects of the work. There are a lot more steps involved in porting Android to new hardware. Still, adding the new device to the build system will definitely be one of the first things you do. Fortunately, doing that is relatively straightforward.
For the purposes of the current exercise, assume you work for a company called ACME and that you’re tasked with delivering its latest gizmo: the CoyotePad, intended to be the best platform for playing all bird games. Let’s get started by creating an entry for our new device in device/:
$cd ~/android/aosp-2.3.x$. build/envsetup.sh$mkdir -p device/acme/coyotepad$cd device/acme/coyotepad
The first thing we’ll need in here is an AndroidProducts.mk file to describe the various AOSP products that could be built for the CoyotePad:
PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/full_coyotepad.mkWhile we could describe several products (see build/target/product/AndroidProducts.mk for an example), the typical case is to specify just one, as in this case, and it’s described in full_coyotepad.mk:
$(call inherit-product, $(SRC_TARGET_DIR)/product/languages_full.mk) # If you're using 4.2/Jelly Bean, use full_base.mk instead of full.mk $(call inherit-product, $(SRC_TARGET_DIR)/product/full.mk) DEVICE_PACKAGE_OVERLAYS := PRODUCT_PACKAGES += PRODUCT_COPY_FILES += PRODUCT_NAME := full_coyotepad PRODUCT_DEVICE := coyotepad PRODUCT_MODEL := Full Android on CoyotePad, meep-meep
It’s worth taking a closer look at this makefile. First, we’re
using the inherit-product function to
tell the build system to pull in other product descriptions as the basis
of ours. This allows us to build on other people’s work and not have to
specify from scratch every bit and piece of the AOSP that we’d like to
include. languages_full.mk will
pull in a vast number of locales, and full.mk will make sure we get the same set of
modules as if we had built using the full-eng combo.
With regard to the other variables:
DEVICE_PACKAGE_OVERLAYSAllows us to specify a directory that will form the basis of an overlay that will be applied onto the AOSP’s sources, thereby allowing us to substitute default package resources with device-specific resources. You’ll find this useful if you’d like to set custom layouts or colors for Launcher2 or other apps, for instance. We’ll look at how to use this in the next section.
PRODUCT_PACKAGESAllows us to specify packages we’d like to have this product include in addition to those specified in the products we’re already inheriting from. If you have custom apps, binaries, or libraries located within device/acme/coyotepad/, for instance, you’ll want to add them here so that they are included in the final images generated. Notice the use of the
+=sign. It allows us to append to the existing values in the variable instead of substituting its content.PRODUCT_COPY_FILESAllows us to list specific files we’d like to see copied to the target’s filesystem and the location where they need to be copied. Each source/destination pair is colon-separated, and pairs are space-separated among themselves. This is useful for configuration files and prebuilt binaries such as firmware images or kernel modules.
PRODUCT_NAMEThe
TARGET_PRODUCT, which you can set either by selecting a lunch combo or passing it as part of the combo parameter to lunch, as in:$
lunch full_coyotepad-engPRODUCT_DEVICEThe name of the actual finished product shipped to the customer.
TARGET_DEVICEderives from this variable.PRODUCT_DEVICEhas to match an entry in device/acme/, since that’s where the build looks for the corresponding BoardConfig.mk. In this case, the variable is the same as the name of the directory we’re already in.PRODUCT_MODELThe name of this product as provided in the “Model number” in the “About the phone” section of the settings. This variable actually gets stored as the
ro.product.modelglobal property accessible on the device.
Version 4.2/Jelly Bean also includes a PRODUCT_BRAND that is typically set to
Android. The value of this variable is then available as the ro.product.brand global property. The latter
is used by some parts of the stack that take action based on the
device’s vendor.
Now that we’ve described the product, we must also provide some information regarding the board the device is using through a BoardConfig.mk file:
TARGET_NO_KERNEL := true TARGET_NO_BOOTLOADER := true TARGET_CPU_ABI := armeabi BOARD_USES_GENERIC_AUDIO := true USE_CAMERA_STUB := true
This is a very skinny BoardConfig.mk and ensures that we actually build successfully. For a real-life version of that file, have a look at device/samsung/crespo/BoardConfigCommon.mk in 2.3/Gingerbread, and also at device/asus/grouper/BoardConfigCommon.mk in 4.2/Jelly Bean.
You’ll also need to provide a conventional Android.mk in order to build all the modules that you might have included in this device’s directory:
LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) ifneq ($(filter coyotepad,$(TARGET_DEVICE)),) include $(call all-makefiles-under,$(LOCAL_PATH)) endif
It’s in fact the preferred modus operandi to put all
device-specific apps, binaries, and libraries within the device’s
directory instead of globally within the rest of the AOSP. If you do add
modules here, don’t forget to also add them to PRODUCT_PACKAGES as I explained earlier. If
you just put them here and provide them valid Android.mk files, they’ll build, but they
won’t be in the final images.
If you have several products sharing the same set of packages, you may want to create a device/acme/common/ directory containing the shared packages. You can see an example of this in 4.2/Jelly Bean’s device/generic/ directory. In that same version, you can also check how device/samsung/maguro/device.mk inherits from device/samsung/tuna/device.mk for an example of how one device can be based on another device.
Lastly, let’s close the loop by making the device we just added visible to envsetup.sh and lunch. To do so, you’ll need to add a vendorsetup.sh in your device’s directory:
add_lunch_combo full_coyotepad-eng
You also need to make sure that it’s executable if it’s to be operational:
$ chmod 755 vendorsetup.shWe can now go back to the AOSP’s root and take our brand-new ACME CoyotePad for a runchase:
$croot$. build/envsetup.sh$lunchYou're building on Linux Lunch menu... pick a combo: 1. generic-eng 2. simulator 3. full_coyotepad-eng 4. full_passion-userdebug 5. full_crespo4g-userdebug 6. full_crespo-userdebug Which would you like? [generic-eng]3============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=2.3.4 TARGET_PRODUCT=full_coyotepad TARGET_BUILD_VARIANT=eng TARGET_SIMULATOR=false TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm HOST_ARCH=x86 HOST_OS=linux HOST_BUILD_TYPE=release BUILD_ID=GINGERBREAD ============================================ $make -j16
As you can see, the AOSP now recognizes our new device and prints the information correspondingly. When the build is done, we’ll also have the same type of output provided in any other AOSP build, except that it will be a product-specific directory:
$ ls -al out/target/product/coyotepad/
total 89356
drwxr-xr-x 7 karim karim 4096 2011-09-21 19:20 .
drwxr-xr-x 4 karim karim 4096 2011-09-21 19:08 ..
-rw-r--r-- 1 karim karim 7 2011-09-21 19:10 android-info.txt
-rw-r--r-- 1 karim karim 4021 2011-09-21 19:41 clean_steps.mk
drwxr-xr-x 3 karim karim 4096 2011-09-21 19:11 data
-rw-r--r-- 1 karim karim 20366 2011-09-21 19:20 installed-files.txt
drwxr-xr-x 14 karim karim 4096 2011-09-21 19:20 obj
-rw-r--r-- 1 karim karim 327 2011-09-21 19:41 previous_build_config.mk
-rw-r--r-- 1 karim karim 2649750 2011-09-21 19:43 ramdisk.img
drwxr-xr-x 11 karim karim 4096 2011-09-21 19:43 root
drwxr-xr-x 5 karim karim 4096 2011-09-21 19:19 symbols
drwxr-xr-x 12 karim karim 4096 2011-09-21 19:19 system
-rw------- 1 karim karim 87280512 2011-09-21 19:20 system.img
-rw------- 1 karim karim 1505856 2011-09-21 19:14 userdata.imgAlso, have a look at the build.prop file in system/. It contains various global properties that will be available at runtime on the target and that relate to our configuration and build:
# begin build properties # autogenerated by buildinfo.sh ro.build.id=GINGERBREAD ro.build.display.id=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.1908 49 test-keys ro.build.version.incremental=eng.karim.20110921.190849 ro.build.version.sdk=10 ro.build.version.codename=REL ro.build.version.release=2.3.4 ro.build.date=Wed Sep 21 19:10:04 EDT 2011 ro.build.date.utc=1316646604 ro.build.type=eng ro.build.user=karim ro.build.host=w520 ro.build.tags=test-keys ro.product.model=Full Android on CoyotePad, meep-meep ro.product.brand=generic ro.product.name=full_coyotepad ro.product.device=coyotepad ro.product.board= ro.product.cpu.abi=armeabi ro.product.manufacturer=unknown ro.product.locale.language=en ro.product.locale.region=US ro.wifi.channels= ro.board.platform= # ro.build.product is obsolete; use ro.product.device ro.build.product=coyotepad # Do not try to parse ro.build.description or .fingerprint ro.build.description=full_coyotepad-eng 2.3.4 GINGERBREAD eng.karim.20110921.190 849 test-keys ro.build.fingerprint=generic/full_coyotepad/coyotepad:2.3.4/GINGERBREAD/eng.kari m.20110921.190849:eng/test-keys # end build properties ...
Warning
You may want to carefully vet the default properties before
using the build on a real device. Some developers have encountered
some severe issues due to default values. In both 2.3/Gingerbread and
4.2/Jelly Bean, for instance, ro.com.android.dataroaming is set to
true in some builds. Hence, if
you’re doing development on a device connected to a live cell network,
changing the value to false might
save you some money.
As you can imagine, there’s a lot more to be done here to make sure the AOSP runs on our hardware. But the preceding steps give us the starting point. However, by isolating the board-specific changes in a single directory, this configuration will simplify adding support for the CoyotePad to the next version of the AOSP that gets released. Indeed, it’ll just be a matter of copying the corresponding directory to the new AOSP’s device/ directory and adjusting the code therein to use the new APIs.
Adding an App
Adding an app to your board is relatively straightforward. As a starter, try creating a HelloWorld! app with Eclipse and the default SDK; all new Android projects in Eclipse are a HelloWorld! by default. Then copy that app from the Eclipse workspace to its destination:
$ cp -a ~/workspace/HelloWorld ~/android/aosp-2.3.x/device/acme/coyotepad/You’ll then have to create an Android.mk file in aosp-root/device/acme/coyotepad/HelloWorld/
to build that app:
LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := $(call all-java-files-under, src) LOCAL_PACKAGE_NAME := HelloWorld include $(BUILD_PACKAGE)
Given that we’re tagging this module as optional, it won’t be included by default in
the AOSP build. To include it, you’ll need to add it to the PRODUCT_PACKAGES listed in the CoyotePad’s
full_coyotepad.mk.
If, instead of adding your app for your board only, you would like to add a default app globally to all products generated by the AOSP alongside the existing stock apps, you’ll need to put it in packages/apps/ instead of your board’s directory. You’ll also need to modify one of the built-in .mk files, such as aosp-root/build/target/product/core.mk, to have your app built by default. This is not recommended, though, as it’s not very portable since it will require you to make this modification to every new AOSP release. As I stated earlier, it’s best to keep your custom modifications in device/acme/coyotepad/ in as much as possible.
Adding an App Overlay
Sometimes you don’t actually want to add an app but would rather modify existing ones included by default in the AOSP. That’s what app overlays are for. Overlays are a mechanism included in the AOSP to allow device manufacturers to change the resources provided (such as for apps), without actually modifying the original resources included in the AOSP. To use this capability, you must create an overlay tree and tell the build system about it. The easiest location for an overlay is within a device-specific directory such as the one we created in the previous section:
$cd device/acme/coyotepad/$mkdir overlay
To tell the build system to take this overlay into account, we need to modify our full_coyotepad.mk such that:
DEVICE_PACKAGE_OVERLAYS := device/acme/coyotepad/overlay
At this point, though, our overlay isn’t doing much. Let’s say we want to modify some of Launcher2’s default strings. We could then do something like this:
$mkdir -p overlay/packages/apps/Launcher2/res/values$cp>aosp-root/packages/apps/Launcher2/res/values/strings.xml \overlay/packages/apps/Launcher2/res/values/
You can then trim your local strings.xml to override only those strings that you need. Most importantly, your device will have a Launcher2 that has your custom strings, but the default Launcher2 will still have its original strings. So if someone relies on the same AOSP sources you’re using to build for another product, they’ll still get the original strings. You can, of course, replace most resources like this, including images and XML files. So long as you put the files in the same hierarchy as they are found in the AOSP but within device/acme/coyotepad/overlay/, they’ll be taken into account by the build system.
Warning
Overlays can be used only for resources. You can’t overlay source code. If you want to customize parts of Android’s internals, for instance, you’ll still have to make those modifications in situ. There’s no way, currently at least, to isolate those changes to your board.
Adding a Native Tool or Daemon
Like the example above of adding an app for your board, you can add your custom native tools and daemons as subdirectories of device/acme/coyotepad/. Obviously, you’ll need to provide an Android.mk in the directory containing the code to build that module:
LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := hello-world LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := hello-world.cpp LOCAL_SHARED_LIBRARIES := liblog include $(BUILD_EXECUTABLE)
As in the app’s case, you’ll also need to make sure hello-world is part of the CoyotePad’s
PRODUCT_PACKAGES.
If you intend to add your binary globally for all product builds instead of just locally to your board, you need to know that there are a number of locations in the tree where native tools and daemons are located. Here are the most important ones:
- system/core/ and system/
Custom Android binaries that are meant to be used outside the Android Framework or are standalone pieces.
- frameworks/base/cmds/
Binaries that are tightly coupled to the Android Framework. This is where the Service Manager and installd are found, for example.
- external/
Binaries that are generated by an external project that is imported into the AOSP. strace, for instance, is here.
Having identified from the list above where the code generating your binary should go, you’ll also need to add it as part of one of the global .mk files such as aosp-root/build/target/product/core.mk. As I said above, however, such global additions are not recommended since they can’t be transferred as easily to newer AOSP versions.
Adding a Native Library
Like apps and binaries, you can also add native libraries for your board. Assuming, as above, that the sources to build the library are in a subdirectory of device/acme/coyotepad/, you’ll need an Android.mk to build your library:
LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libmylib LOCAL_MODULE_TAGS := optional LOCAL_PRELINK_MODULE := false LOCAL_SRC_FILES := $(call all-c-files-under,.) include $(BUILD_SHARED_LIBRARY)
Note
Note that LOCAL_PRELINK_MODULE has been removed and is
no longer necessary as of 4.0/Ice-Cream Sandwich.
To use this library, you must add it to the libraries listed by the Android.mk file of whichever binaries depend on it:
LOCAL_SHARED_LIBRARIES := libmylib
You’ll also likely need to add relevant headers to an include/ directory located in about the same location as you put your library, so that the code that needs to link against your library can find those headers, such as device/acme/coyotepad/include/.
Should you want to make your library apply globally to all AOSP builds, not just your device, you’ll need a little bit more information regarding the various locations where libraries are typically found in the tree. First, you should know that, unlike binaries, a lot of libraries are used within a single module but nowhere else. Hence, these libraries will typically be placed within that module’s code and not in the usual locations where libraries used systemwide are found. The latter are typically in the following locations:
- system/core/
Libraries used by many parts of the system, including some outside the Android Framework. This is where liblog is, for instance.
- frameworks/base/libs/
Libraries intimately tied to the framework. This is where libbinder is.
- frameworks/native/libs/
In 4.2/Jelly Bean, many libraries that were in frameworks/base/libs/ in 2.3/Gingerbread have been moved out and into frameworks/native/libs/.
- external/
Libraries generated by external projects imported into the AOSP. OpenSSL’s libssl is here.
Similarly, instead of using a CoyotePad-specific include directory, you’d use a global directory such as system/core/include/ or frameworks/base/include/ or, in 4.2/Jelly Bean, frameworks/base/include/. Again, as stated earlier, you should carefully review whether such global additions are truly required, as they’ll represent additional work when you try to port for your device to the next version of Android.
[18] If you do not provide a value, defaults will be used.
For instance, all apps are set to optional by default. Also, some
modules are part of GRANDFATHERED_USER_MODULES in
user_tags.mk. No LOCAL_MODULE_TAGS need be specified
for those; they’re always included.
[19] This file contains a set list of variables starting with the
string LOCAL_. If a variable
isn’t specifically listed in this file, it won’t be taken into
account by CLEAR_VARS.
[20] This version is cleaned up a little (removed commented code, for instance) and slightly reformatted.
[21] Also slightly modified to remove white space and comments.
[22] This assumes you had already run envsetup.sh and lunch.
Get Embedded Android now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
