Hands-On System Programming with Go

Hands-On System Programming with Go

by Alex Guerrieri
Released July 2019
Publisher(s): Packt Publishing
ISBN: 9781789804072

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore the fundamentals of systems programming starting from kernel API and filesystem to network programming and process communications

Key Features

  • Learn how to write Unix and Linux system code in Golang v1.12
  • Perform inter-process communication using pipes, message queues, shared memory, and semaphores
  • Explore modern Go features such as goroutines and channels that facilitate systems programming

Book Description

System software and applications were largely created using low-level languages such as C or C++. Go is a modern language that combines simplicity, concurrency, and performance, making it a good alternative for building system applications for Linux and macOS.

This Go book introduces Unix and systems programming to help you understand the components the OS has to offer, ranging from the kernel API to the filesystem. You'll then familiarize yourself with Go and its specifications. You'll also learn how to optimize input and output operations with files and streams of data, which are useful tools in building pseudo-terminal applications. You'll gain insights into how processes communicate with each other, and learn about processes and daemon control using signals, pipes, and exit codes. This book will also enable you to understand how to use network communication using various protocols, including TCP and HTTP. As you advance, you'll focus on Go's best feature - concurrency, which will help you handle communication with channels and goroutines, other concurrency tools to synchronize shared resources, and the context package to write elegant applications.

By the end of this book, you will have learned how to build concurrent system applications using Go

What you will learn

  • Explore concepts of system programming using Go and concurrency
  • Gain insights into Golang's internals, memory models and allocation
  • Familiarize yourself with the filesystem and IO streams in general
  • Handle and control processes and daemons' lifetime via signals and pipes
  • Communicate with other applications effectively using a network
  • Use various encoding formats to serialize complex data structures
  • Become well-versed in concurrency with channels, goroutines, and sync
  • Use concurrency patterns to build robust and performant system applications

Who this book is for

If you are a developer who wants to learn system programming with Go, this book is for you. Although no knowledge of Unix and Linux system programming is necessary, intermediate knowledge of Go will help you understand the concepts covered in the book

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On System Programming with Go
  3. Dedication
  4. About Packt
    1. Why subscribe?
  5. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Code in Action
        1. Playground examples
      4. Conventions used
    4. Get in touch
      1. Reviews
  7. Section 1: An Introduction to System Programming and Go
  8. An Introduction to System Programming
    1. Technical requirements
    2. Beginning with system programming
      1. Software for software
      2. Languages and system evolution
      3. System programming and software engineering
    3. Application programming interfaces
      1. Types of APIs
        1. Operating systems
        2. Libraries and frameworks
        3. Remote APIs
        4. Web APIs
    4. Understanding the protection ring
      1. Architectural differences
      2. Kernel space and user space
    5. Diving into system calls
      1. Services provided
        1. Process control
        2. File management
        3. Device management
        4. Information maintenance
        5. Communication
      2. The difference between operating systems
    6. Understanding the POSIX standard
      1. POSIX standards and features
        1. POSIX.1 – core services
        2. POSIX.1b and POSIX.1c – real-time and thread extensions
        3. POSIX.2 – shell and utilities
      2. OS adherence
        1. Linux and macOS
        2. Windows
    7. Summary
    8. Questions
  9. Unix OS Components
    1. Technical requirements
    2. Memory management
      1. Techniques of management
      2. Virtual memory
    3. Understanding files and filesystems
      1. Operating systems and filesystems
        1. Linux
        2. macOS
        3. Windows
      2. Files and hard and soft links
    4. Unix filesystem
      1. Root and inodes
      2. Directory structure
      3. Navigation and interaction
      4. Mounting and unmounting
    5. Processes
      1. Process properties
      2. Process life cycle
        1. Foreground and background
        2. Killing a job
    6. Users, groups, and permissions
      1. Users and groups
      2. Owner, group, and others
        1. Read, write, and execute
        2. Changing permission
    7. Process communications
      1. Exit codes
      2. Signals
      3. Pipes
      4. Sockets
    8. Summary
    9. Questions
  10. An Overview of Go
    1. Technical requirements
    2. Language features
      1. History of Go
      2. Strengths and weaknesses
    3. Namespace
      1. Imports and exporting symbols
    4. Type system
      1. Basic types
      2. Composite types
      3. Custom-defined types
    5. Variables and functions
      1. Handling variables
        1. Declaration
        2. Operations
        3. Casting
        4. Scope
      2. Constants
      3. Functions and methods
        1. Values and pointers
    6. Understanding flow control
      1. Condition
      2. Looping
    7. Exploring built-in functions
      1. Defer, panic, and recover
    8. Concurrency model
      1. Understanding channels and goroutines
    9. Understanding memory management
      1. Stack and heap
      2. The history of GC in Go
    10. Building and compiling programs
      1. Install
      2. Build
      3. Run
    11. Summary
    12. Questions
  11. Section 2: Advanced File I/O Operations
  12. Working with the Filesystem
    1. Technical requirements
    2. Handling paths
      1. Working directory
        1. Getting and setting the working directory
      2. Path manipulation
    3. Reading from files
      1. Reader interface
        1. The file structure
      2. Using buffers
        1. Peeking content
      3. Closer and seeker
    4. Writing to file
      1. Writer interface
      2. Buffers and format
      3. Efficient writing
      4. File modes
    5. Other operations
      1. Create
      2. Truncate
      3. Delete
      4. Move
      5. Copy
      6. Stats
      7. Changing properties
    6. Third-party packages
      1. Virtual filesystems
      2. Filesystem events
    7. Summary
    8. Questions
  13. Handling Streams
    1. Technical requirements
    2. Streams
      1. Input and readers
        1. The bytes reader
        2. The strings reader
      2. Defining a reader
      3. Output and writers
        1. The bytes writer
        2. The string writer
      4. Defining a writer
    3. Built-in utilities
      1. Copying from one stream to another
      2. Connected readers and writers
      3. Extending readers
      4. Writers and decorators
    4. Summary
    5. Questions
  14. Building Pseudo-Terminals
    1. Technical requirements
    2. Understanding pseudo-terminals
      1. Beginning with teletypes
      2. Pseudo teletypes
    3. Creating a basic PTY
      1. Input management
      2. Selector
      3. Command execution
      4. Some refactor
    4. Improving the PTY
      1. Multiline input
      2. Providing color support to the pseudo-terminal
      3. Suggesting commands
      4. Extensible commands
    5. Commands with status
      1. Volatile status
      2. Persistent status
        1. Upgrading the Stack command
    6. Summary
    7. Questions
  15. Section 3: Understanding Process Communication
  16. Handling Processes and Daemons
    1. Technical requirements
    2. Understanding processes
      1. Current process
        1. Standard input
        2. User and group ID
        3. Working directory
      2. Child processes
        1. Accessing child properties
        2. Standard input
    3. Beginning with daemons
      1. Operating system support
      2. Daemons in action
      3. Services
    4. Creating a service
      1. Third-party packages
    5. Summary
    6. Questions
  17. Exit Codes, Signals, and Pipes
    1. Technical requirements
    2. Using exit codes
      1. Sending exit codes
        1. Exit codes in bash
        2. The exit value bit size
        3. Exit and deferred functions
        4. Panics and exit codes
        5. Exit codes and goroutines
      2. Reading child process exit codes
    3. Handling signals
      1. Handling incoming signals
        1. The signal package
        2. Graceful shutdowns
        3. Exit cleanup and resource release
        4. Configuration reload
      2. Sending signals to other processes
    4. Connecting streams
      1. Pipes
        1. Anonymous pipes
        2. Standard input and output pipes
    5. Summary
    6. Questions
  18. Network Programming
    1. Technical requirements
    2. Communicating via networks
      1. OSI model
        1. Layer 1 – Physical layer
        2. Layer 2 – Data link layer
        3. Layer 3 – Network layer
        4. Layer 4 – Transport layer
        5. Layer 5 – Session layer
        6. Layer 6 – Presentation layer
        7. Layer 7 – Application layer
      2. TCP/IP – Internet protocol suite
        1. Layer 1 – Link layer
        2. Layer 2 – Internet layer
        3. Layer 3 – Transport layer
        4. Layer 4 – Application layer
    3. Understanding socket programming
      1. Network package
      2. TCP connections
      3. UDP connections
        1. Encoding and checksum
    4. Web servers in Go
      1. Web server
        1. HTTP protocol
        2. HTTP/2 and Go
      2. Using the standard package
        1. Making a HTTP request
        2. Creating a simple server
        3. Serving filesystem
        4. Navigating through routes and methods
        5. Multipart request and files
        6. HTTPS
      3. Third-party packages
        1. gorilla/mux
        2. gin-gonic/gin
      4. Other functionalities
        1. HTTP/2 Pusher
        2. WebSockets protocol
    5. Beginning with the template engine
      1. Syntax and basic usage
      2. Creating, parsing, and executing templates
      3. Conditions and loops
      4. Template functions
    6. RPC servers
      1. Defining a service
      2. Creating the server
      3. Creating the client
    7. Summary
    8. Questions
  19. Data Encoding Using Go
    1. Technical requirements
    2. Understanding text-based encoding
      1. CSV
        1. Decoding values
        2. Encoding values
        3. Custom options
      2. JSON
        1. Field tags
        2. Decoder
        3. Encoder
        4. Marshaler and unmarshaler
        5. Interfaces
        6. Generating structs
        7. JSON schemas
      3. XML
        1. Structure
        2. Document Type Definition
        3. Decoding and encoding
        4. Field tags
        5. Marshaler and unmarshaler
        6. Generating structs
      4. YAML
        1. Structure
        2. Decoding and encoding
    3. Learning about binary encoding
      1. BSON
        1. Encoding
        2. Decoding
      2. gob
        1. Interfaces
        2. Encoding
        3. Decoding
        4. Interfaces
      3. Proto
        1. Structure
        2. Code generation
        3. Encoding
        4. Decoding
        5. gRPC protocol
    4. Summary
    5. Questions
  20. Section 4: Deep Dive into Concurrency
  21. Dealing with Channels and Goroutines
    1. Technical requirements
    2. Understanding goroutines
      1. Comparing threads and goroutines
        1. Threads
        2. Goroutines
      2. New goroutine
      3. Multiple goroutines
      4. Argument evaluation
      5. Synchronization
    3. Exploring channels
      1. Properties and operations
        1. Capacity and size
        2. Blocking operations
        3. Closing channels
        4. One-way channels
      2. Waiting receiver
      3. Special values
        1. nil channels
        2. Closed channels
      4. Managing multiple operations
        1. Default clause
      5. Timers and tickers
        1. Timers
        2. AfterFunc
        3. Tickers
    4. Combining channels and goroutines
      1. Rate limiter
      2. Workers
      3. Pool of workers
      4. Semaphores
    5. Summary
    6. Questions
  22. Synchronization with sync and atomic
    1. Technical requirements
    2. Synchronization primitives
      1. Concurrent access and lockers
        1. Mutex
        2. RWMutex
        3. Write starvation
        4. Locking gotchas
      2. Synchronizing goroutines
      3. Singleton in Go
        1. Once and Reset
      4. Resource recycling
        1. Slices recycling issues
      5. Conditions
      6. Synchronized maps
      7. Semaphores
    3. Atomic operations
      1. Integer operations
        1. clicker
        2. Thread-safe floats
        3. Thread-safe Boolean
        4. Pointer operations
      2. Value
        1. Under the hood
    4. Summary
    5. Questions
  23. Coordination Using Context
    1. Technical requirements
    2. Understanding context
      1. The interface
      2. Default contexts
        1. Background
        2. TODO
      3. Cancellation, timeout, and deadline
        1. Cancellation
        2. Deadline
        3. Timeout
      4. Keys and values
    3. Context in the standard library
      1. HTTP requests
        1. Passing scoped values
        2. Request cancellation
      2. HTTP server
        1. Shutdown
        2. Passing values
      3. TCP dialing
        1. Cancelling a connection
      4. Database operations
      5. Experimental packages
    4. Context in your application
      1. Things to avoid
        1. Wrong types as keys
        2. Passing parameters
        3. Optional arguments
        4. Globals
      2. Building a service with Context
        1. Main interface and usage
        2. Exit and entry points
        3. Exclude list
        4. Handling directories
        5. Checking file names and contents
    5. Summary
    6. Questions
  24. Implementing Concurrency Patterns
    1. Technical requirements
    2. Beginning with generators
      1. Avoiding leaks
    3. Sequencing with pipelines
    4. Muxing and demuxing
      1. Fan-out
      2. Fan-in
    5. Producers and consumers
      1. Multiple producers (N * 1)
      2. Multiple consumers (1 * M)
      3. Multiple consumers and producers (N*M)
    6. Other patterns
      1. Error groups
      2. Leaky bucket
      3. Sequencing
    7. Summary
    8. Questions
  25. Section 5: A Guide to Using Reflection and CGO
  26. Using Reflection
    1. Technical requirements
    2. What's reflection?
      1. Type assertions
        1. Interface assertion
    3. Understanding basic mechanics
      1. Value and Type methods
        1. Kind
      2. Value to interface
      3. Manipulating values
        1. Changing values
        2. Creating new values
    4. Handling complex types
      1. Data structures
        1. Changing fields
        2. Using tags
      2. Maps and slices
        1. Maps
        2. Slices
      3. Functions
        1. Analyzing a function
        2. Invoking a function
      4. Channels
        1. Creating channels
        2. Sending, receiving, and closing
        3. Select statement
    5. Reflecting on reflection
      1. Performance cost
      2. Usage in the standard library
      3. Using reflection in a package
        1. Property files
          1. Using the package
    6. Summary
    7. Questions
  27. Using CGO
    1. Technical requirements
    2. Introduction to CGO
      1. Calling C code from Go
      2. Calling Go code from C
    3. The C and Go type systems
      1. Strings and byte slices
      2. Integers
      3. Float types
      4. Unsafe conversions
        1. Editing a byte slice directly
      5. Numbers
    4. Working with slices
    5. Working with structs
      1. Structures in Go
        1. Manual padding
      2. Structures in C
        1. Unpacked structures
        2. Packed structures
    6. CGO recommendations
      1. Compilation and speed
      2. Performance
      3. Dependency from C
    7. Summary
    8. Questions
  28. Assessments
    1. Chapter 1
    2. Chapter 2
    3. Chapter 3
    4. Chapter 4
    5. Chapter 5
    6. Chapter 6
    7. Chapter 7
    8. Chapter 8
    9. Chapter 9
    10. Chapter 10
    11. Chapter 11
    12. Chapter 12
    13. Chapter 13
    14. Chapter 14
    15. Chapter 15
    16. Chapter 16
  29. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On System Programming with Go
  • Author(s): Alex Guerrieri
  • Release date: July 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781789804072