Mastering Embedded Linux Programming - Second Edition

Mastering Embedded Linux Programming - Second Edition

by Chris Simmonds
Released June 2017
Publisher(s): Packt Publishing
ISBN: 9781787283282

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Master the techniques needed to build great, efficient embedded devices on Linux

About This Book

  • Discover how to build and configure reliable embedded Linux devices
  • This book has been updated to include Linux 4.9 and Yocto Project 2.2 (Morty)
  • This comprehensive guide covers the remote update of devices in the field and power management

Who This Book Is For

If you are an engineer who wishes to understand and use Linux in embedded devices, this book is for you. It is also for Linux developers and system programmers who are familiar with embedded systems and want to learn and program the best in class devices. It is appropriate for students studying embedded techniques, for developers implementing embedded Linux devices, and engineers supporting existing Linux devices.

What You Will Learn

  • Evaluate the Board Support Packages offered by most manufacturers of a system on chip or embedded module
  • Use Buildroot and the Yocto Project to create embedded Linux systems quickly and efficiently
  • Update IoT devices in the field without compromising security
  • Reduce the power budget of devices to make batteries last longer
  • Interact with the hardware without having to write kernel device drivers
  • Debug devices remotely using GDB, and see how to measure the performance of the systems using powerful tools such as perk, ftrace, and valgrind
  • Find out how to configure Linux as a real-time operating system

In Detail

Embedded Linux runs many of the devices we use every day, from smart TVs to WiFi routers, test equipment to industrial controllers - all of them have Linux at their heart. Linux is a core technology in the implementation of the inter-connected world of the Internet of Things.

The comprehensive guide shows you the technologies and techniques required to build Linux into embedded systems. You will begin by learning about the fundamental elements that underpin all embedded Linux projects: the toolchain, the bootloader, the kernel, and the root filesystem. You’ll see how to create each of these elements from scratch, and how to automate the process using Buildroot and the Yocto Project.

Moving on, you’ll find out how to implement an effective storage strategy for flash memory chips, and how to install updates to the device remotely once it is deployed. You’ll also get to know the key aspects of writing code for embedded Linux, such as how to access hardware from applications, the implications of writing multi-threaded code, and techniques to manage memory in an efficient way. The final chapters show you how to debug your code, both in applications and in the Linux kernel, and how to profile the system so that you can look out for performance bottlenecks.

By the end of the book, you will have a complete overview of the steps required to create a successful embedded Linux system.

Style and approach

This book is an easy-to-follow and pragmatic guide with in-depth analysis of the implementation of embedded devices. It follows the life cycle of a project from inception through to completion, at each stage giving both the theory that underlies the topic and practical step-by-step walkthroughs of an example implementation.

Table of contents

  1. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
  2. Starting Out
    1. Selecting the right operating system
    2. The players
    3. Project life cycle
      1. The four elements of embedded Linux
    4. Open source
      1. Licenses
    5. Hardware for embedded Linux
    6. Hardware used in this book
      1. The BeagleBone Black
      2. QEMU
    7. Software used in this book
    8. Summary
  3. Learning About Toolchains
    1. Introducing toolchains
      1. Types of toolchains
      2. CPU architectures
      3. Choosing the C library
    2. Finding a toolchain
    3. Building a toolchain using crosstool-NG
      1. Installing crosstool-NG
      2. Building a toolchain for BeagleBone Black
      3. Building a toolchain for QEMU
    4. Anatomy of a toolchain
      1. Finding out about your cross compiler
      2. The sysroot, library, and header files
      3. Other tools in the toolchain
      4. Looking at the components of the C library
    5. Linking with libraries – static and dynamic linking
      1. Static libraries
      2. Shared libraries
        1. Understanding shared library version numbers
    6. The art of cross compiling
      1. Simple makefiles
      2. Autotools
        1. An example: SQLite
      3. Package configuration
      4. Problems with cross compiling
    7. Summary
  4. All About Bootloaders
    1. What does a bootloader do?
    2. The boot sequence
      1. Phase 1 – ROM code
      2. Phase 2 – secondary program loader
      3. Phase 3 – TPL
    3. Booting with UEFI firmware
    4. Moving from bootloader to kernel
    5. Introducing device trees
      1. Device tree basics
      2. The reg property
      3. Labels and interrupts
      4. Device tree include files
      5. Compiling a device tree
    6. Choosing a bootloader
    7. U-Boot
      1. Building U-Boot
      2. Installing U-Boot
      3. Using U-Boot
        1. Environment variables
        2. Boot image format
        3. Loading images
      4. Booting Linux
        1. Automating the boot with U-Boot scripts
      5. Porting U-Boot to a new board
        1. Board-specific files
        2. Configuring header files
      6. Building and testing
      7. Falcon mode
    8. Barebox
      1. Getting barebox
      2. Building barebox
      3. Using barebox
    9. Summary
  5. Configuring and Building the Kernel
    1. What does the kernel do?
    2. Choosing a kernel
      1. Kernel development cycle
      2. Stable and long term support releases
      3. Vendor support
      4. Licensing
    3. Building the kernel
      1. Getting the source
      2. Understanding kernel configuration – Kconfig
      3. Using LOCALVERSION to identify your kernel
      4. Kernel modules
    4. Compiling – Kbuild
      1. Finding out which kernel target to build
      2. Build artifacts
      3. Compiling device trees
      4. Compiling modules
      5. Cleaning kernel sources
      6. Building a kernel for the BeagleBone Black
      7. Building a kernel for QEMU
    5. Booting the kernel
      1. Booting the BeagleBone Black
      2. Booting QEMU
      3. Kernel panic
      4. Early user space
      5. Kernel messages
      6. Kernel command line
    6. Porting Linux to a new board
      1. A new device tree
      2. Setting the board compatible property
    7. Additional reading
    8. Summary
  6. Building a Root Filesystem
    1. What should be in the root filesystem?
      1. The directory layout
      2. The staging directory
      3. POSIX file access permissions
      4. File ownership permissions in the staging directory
      5. Programs for the root filesystem
        1. The init program
        2. Shell
        3. Utilities
        4. BusyBox to the rescue!
        5. Building BusyBox
        6. ToyBox – an alternative to BusyBox
      6. Libraries for the root filesystem
        1. Reducing the size by stripping
      7. Device nodes
      8. The proc and sysfs filesystems
        1. Mounting filesystems
      9. Kernel modules
    2. Transferring the root filesystem to the target
    3. Creating a boot initramfs
      1. Standalone initramfs
      2. Booting the initramfs
      3. Booting with QEMU
      4. Booting the BeagleBone Black
        1. Mounting proc
      5. Building an initramfs into the kernel image
      6. Building an initramfs using a device table
      7. The old initrd format
    4. The init program
      1. Starting a daemon process
    5. Configuring user accounts
      1. Adding user accounts to the root filesystem
    6. A better way of managing device nodes
      1. An example using devtmpfs
      2. An example using mdev
      3. Are static device nodes so bad after all?
    7. Configuring the network
      1. Network components for glibc
    8. Creating filesystem images with device tables
      1. Booting the BeagleBone Black
    9. Mounting the root filesystem using NFS
      1. Testing with QEMU
      2. Testing with the BeagleBone Black
      3. Problems with file permissions
    10. Using TFTP to load the kernel
    11. Additional reading
    12. Summary
  7. Selecting a Build System
    1. Build systems
    2. Package formats and package managers
    3. Buildroot
      1. Background
      2. Stable releases and long-term support
      3. Installing
      4. Configuring
      5. Running
      6. Creating a custom BSP
        1. U-Boot
        2. Linux
        3. Build
      7. Adding your own code
        1. Overlays
        2. Adding a package
      8. License compliance
    4. The Yocto Project
      1. Background
      2. Stable releases and supports
      3. Installing the Yocto Project
      4. Configuring
      5. Building
      6. Running the QEMU target
      7. Layers
        1. BitBake and recipes
      8. Customizing images via local.conf
      9. Writing an image recipe
      10. Creating an SDK
      11. The license audit
    5. Further reading
    6. Summary
  8. Creating a Storage Strategy
    1. Storage options
      1. NOR flash
      2. NAND flash
      3. Managed flash
        1. MultiMediaCard and Secure Digital cards
        2. eMMC
        3. Other types of managed flash
    2. Accessing flash memory from the bootloader
      1. U-Boot and NOR flash
      2. U-Boot and NAND flash
      3. U-Boot and MMC, SD, and eMMC
    3. Accessing flash memory from Linux
      1. Memory technology devices
        1. MTD partitions
        2. MTD device drivers
        3. The MTD character device, mtd
        4. The MTD block device, mtdblock
        5. Logging kernel oops to MTD
        6. Simulating NAND memory
      2. The MMC block driver
    4. Filesystems for flash memory
      1. Flash translation layers
    5. Filesystems for NOR and NAND flash memory
      1. JFFS2
        1. Summary nodes
        2. Clean markers
        3. Creating a JFFS2 filesystem
      2. YAFFS2
        1. Creating a YAFFS2 filesystem
      3. UBI and UBIFS
        1. UBI
        2. UBIFS
    6. Filesystems for managed flash
      1. Flashbench
      2. Discard and TRIM
      3. Ext4
      4. F2FS
      5. FAT16/32
    7. Read-only compressed filesystems
      1. squashfs
    8. Temporary filesystems
    9. Making the root filesystem read-only
    10. Filesystem choices
    11. Further reading
    12. Summary
  9. Updating Software in the Field
    1. What to update?
      1. Bootloader
      2. Kernel
      3. Root filesystem
      4. System applications
      5. Device-specific data
      6. Components that need to be updated
    2. The basics of software update
      1. Making updates robust
      2. Making updates fail-safe
      3. Making updates secure
    3. Types of update mechanism
      1. Symmetric image update
      2. Asymmetric image update
      3. Atomic file updates
    4. OTA updates
    5. Using Mender for local updates
      1. Building the Mender client
      2. Installing an update
    6. Using Mender for OTA updates
    7. Summary
  10. Interfacing with Device Drivers
    1. The role of device drivers
    2. Character devices
    3. Block devices
    4. Network devices
    5. Finding out about drivers at runtime
      1. Getting information from sysfs
        1. The devices: /sys/devices
        2. The drivers: /sys/class
        3. The block drivers: /sys/block
    6. Finding the right device driver
    7. Device drivers in user space
      1. GPIO
        1. Handling interrupts from GPIO
      2. LEDs
      3. I2C
      4. Serial Peripheral Interface (SPI)
    8. Writing a kernel device driver
      1. Designing a character driver interface
      2. The anatomy of a device driver
      3. Compiling kernel modules
      4. Loading kernel modules
    9. Discovering the hardware configuration
      1. Device trees
      2. The platform data
      3. Linking hardware with device drivers
    10. Additional reading
    11. Summary
  11. Starting Up – The init Program
    1. After the kernel has booted
    2. Introducing the init programs
    3. BusyBox init
      1. Buildroot init scripts
    4. System V init
      1. inittab
      2. The init.d scripts
      3. Adding a new daemon
      4. Starting and stopping services
    5. systemd
      1. Building systemd with the Yocto Project and Buildroot
      2. Introducing targets, services, and units
        1. Units
        2. Services
        3. Targets
      3. How systemd boots the system
      4. Adding your own service
      5. Adding a watchdog
      6. Implications for embedded Linux
    6. Further reading
    7. Summary
  12. Managing Power
    1. Measuring power usage
    2. Scaling the clock frequency
      1. The CPUFreq driver
      2. Using CPUFreq
    3. Selecting the best idle state
      1. The CPUIdle driver
      2. Tickless operation
    4. Powering down peripherals
    5. Putting the system to sleep
      1. Power states
      2. Wakeup events
      3. Timed wakeups from the real-time clock
    6. Further reading
    7. Summary
  13. Learning About Processes and Threads
    1. Process or thread?
    2. Processes
      1. Creating a new process
      2. Terminating a process
      3. Running a different program
      4. Daemons
      5. Inter-process communication
        1. Message-based IPC
          1. Unix (or local) sockets
          2. FIFOs and named pipes
          3. POSIX message queues
        2. Summary of message-based IPC
        3. Shared memory-based IPC
          1. POSIX shared memory
    3. Threads
      1. Creating a new thread
      2. Terminating a thread
      3. Compiling a program with threads
      4. Inter-thread communication
      5. Mutual exclusion
      6. Changing conditions
      7. Partitioning the problem
    4. Scheduling
      1. Fairness versus determinism
      2. Time-shared policies
        1. Niceness
      3. Real-time policies
      4. Choosing a policy
      5. Choosing a real-time priority
    5. Further reading
    6. Summary
  14. Managing Memory
    1. Virtual memory basics
    2. Kernel space memory layout
      1. How much memory does the kernel use?
    3. User space memory layout
    4. The process memory map
    5. Swapping
      1. Swapping to compressed memory (zram)
    6. Mapping memory with mmap
      1. Using mmap to allocate private memory
      2. Using mmap to share memory
      3. Using mmap to access device memory
    7. How much memory does my application use?
    8. Per-process memory usage
      1. Using top and ps
      2. Using smem
      3. Other tools to consider
    9. Identifying memory leaks
      1. mtrace
      2. Valgrind
    10. Running out of memory
    11. Further reading
    12. Summary
  15. Debugging with GDB
    1. The GNU debugger
    2. Preparing to debug
    3. Debugging applications
      1. Remote debugging using gdbserver
      2. Setting up the Yocto Project for remote debugging
      3. Setting up Buildroot for remote debugging
      4. Starting to debug
        1. Connecting GDB and gdbserver
        2. Setting the sysroot
        3. GDB command files
        4. Overview of GDB commands
          1. Breakpoints
          2. Running and stepping
          3. Getting information
        5. Running to a breakpoint
      5. Native debugging
        1. The Yocto Project
        2. Buildroot
    4. Just-in-time debugging
    5. Debugging forks and threads
    6. Core files
      1. Using GDB to look at core files
    7. GDB user interfaces
      1. Terminal user interface
      2. Data display debugger
      3. Eclipse
    8. Debugging kernel code
      1. Debugging kernel code with kgdb
      2. A sample debug session
      3. Debugging early code
      4. Debugging modules
      5. Debugging kernel code with kdb
      6. Looking at an Oops
      7. Preserving the Oops
    9. Further reading
    10. Summary
  16. Profiling and Tracing
    1. The observer effect
      1. Symbol tables and compile flags
    2. Beginning to profile
    3. Profiling with top
    4. Poor man's profiler
    5. Introducing perf
      1. Configuring the kernel for perf
      2. Building perf with the Yocto Project
      3. Building perf with Buildroot
      4. Profiling with perf
      5. Call graphs
      6. perf annotate
    6. Other profilers – OProfile and gprof
    7. Tracing events
    8. Introducing Ftrace
      1. Preparing to use Ftrace
      2. Using Ftrace
      3. Dynamic Ftrace and trace filters
      4. Trace events
    9. Using LTTng
      1. LTTng and the Yocto Project
      2. LTTng and Buildroot
      3. Using LTTng for kernel tracing
    10. Using Valgrind
      1. Callgrind
      2. Helgrind
    11. Using strace
    12. Summary
  17. Real-Time Programming
    1. What is real time?
    2. Identifying sources of non-determinism
    3. Understanding scheduling latency
    4. Kernel preemption
    5. The real-time Linux kernel (PREEMPT_RT)
      1. Threaded interrupt handlers
    6. Preemptible kernel locks
      1. Getting the PREEMPT_RT patches
      2. The Yocto Project and PREEMPT_RT
    7. High-resolution timers
    8. Avoiding page faults
    9. Interrupt shielding
    10. Measuring scheduling latencies
      1. cyclictest
      2. Using Ftrace
      3. Combining cyclictest and Ftrace
    11. Further reading
    12. Summary

Product information

  • Title: Mastering Embedded Linux Programming - Second Edition
  • Author(s): Chris Simmonds
  • Release date: June 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781787283282