book

Go for DevOps

by John Doak, David Justice

July 2022

Intermediate to advanced

634 pages

12h 13m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorsAbout the reviewers
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare your thoughtsDownload a Free PDF copy of this book
Technical requirementsUsing the Go PlaygroundUtilizing Go packagesDeclaring a packageImporting a packageUsing a packagePackage name conflictsPackages must be usedA Go Hello WorldUsing Go's variable typesGo's typesDeclaring variablesVariable scopes and shadowingFunction/statement variable must be usedLooping in GoC styleRemoving the init statementRemove the post statement too and you have a while loopCreating an infinite loopUsing conditionalsif statementselseLearning about functionsReturning multiple values and named resultsVariadic argumentsAnonymous functionsDefining public and privateUsing arrays and slicesArraysSlicesExtracting all valuesUnderstanding mapsDeclaring a mapAccessing valuesAdding new valuesExtracting all valuesUnderstanding Go pointersMemory addressesFunction arguments are copiesPointers to the rescueGetting to know about structsDeclaring a structDeclaring a custom typeCustom struct typesAdding methods to a typeChanging a field's valueChanging a field's value in a methodConstructorsComprehending Go interfacesDefining an interface typeImportant things about interfacesThe blank interface – Go's universal valueType assertionSummary
Handling errors in GoCreating an errorUsing an errorCreating named errorsCustom errorsWrapping errorsUtilizing Go constants Declaring a constantEnumeration via constantsPrinting enumeratorsUsing defer, panic, and recoverdeferpanicrecoverUtilizing goroutines for concurrencyStarting a goroutineSynchronizationWaitGroupsChannelsSending/receivingselect statementsChannels as an event signalMutexesRWMutexUnderstanding Go's Context typeUsing a Context to signal a timeoutHonoring a context when receivingContext in the standard libraryContext to pass valuesBest practicesUtilizing Go's testing frameworkCreating a basic test fileCreating a simple testTable Driven Tests (TDT)Creating fakes with interfacesThird-party testing packagesGenerics – the new kid on the blockType parametersUsing type constraintsWe could do better with constraintsCurrent built-in constraintsType constraints with methodsAdding type parameters to struct typesSpecifying the type when calling a generic functionGotchas to watch forWhen to use genericsSummary
Technical requirementsInstalling Go on your machinemacOS installation using the package installermacOS installation via HomebrewWindows installation using MSILinuxOther platformsA note on Go compiler version compatibilityBuilding code locallyCreating a module directory and go.mod fileUpdating a module when adding dependenciesAdding a hello worldRunning our first programSummary
All I/O in Go are filesI/O interfacesReading and writing to filesReading local filesWriting local filesReading remote filesStreaming file contentStdin/Stdout/Stderr are just filesReading data out of a streamWriting data into a streamOS-agnostic pathingWhat OS/platform am I running?Using filepathRelative and absolute pathingOS-agnostic filesystemsio.fs filesystemsembedWalking our filesystemThe io/fs futureSummary
Technical requirementsCSV filesBasic value separation using the strings packageUsing the encoding/csv packageUsing excelize when dealing with ExcelPopular encoding formatsThe Go field tagsJSONYAML encodingSummary
Technical requirementsAccessing SQL databasesConnecting to a Postgres databaseQuerying a Postgres databaseNull valuesWriting data to PostgresTransactionsPostgres-specific typesOther optionsStorage abstractionsCase study – data migration of an orchestration system – GoogleDeveloping REST services and clientsREST for RPCsDeveloping gRPC services and clientsProtocol buffersStating the prerequisitesGenerating your packagesWriting a gRPC clientWriting a gRPC serverCreating a server binaryCreating a client binaryCompany-standard RPC clients and serversSummary
Technical requirementsImplementing application I/OThe flag packageCustom flagsBasic flag error handlingShorthand flagsAccessing non-flag argumentsRetrieving input from STDINUsing Cobra for advanced CLI applicationsCode organizationThe optional Cobra generatorThe command packageHandling OS signalsCapturing an OS signalUsing Context to cancelSummary

Technical requirementsUsing os/exec to automate local changesDetermining the availability of essential toolsUsing SSH in Go to automate remote changesConnecting to another systemDesigning safe, concurrent change automationsComponents of a changeWriting a concurrent jobCase study – Network rolloutsWriting a system agentDesigning a system agentImplementing InstallImplementing SystemPerfSummary
Technical requirementsAn introduction to OpenTelemetryReference architecture for OpenTelemetryOpenTelemetry componentsLogging with contextOur first log statementStructured and leveled logs with ZapIngesting, transforming, and exporting logs using OpenTelemetryInstrumenting for distributed tracingThe life cycle of a distributed traceCorrelating traces and logsAdding log entries to spansInstrumenting for metricsThe life cycle of a metricClient/server metrics with OpenTelemetryAlerting on metrics abnormalitiesAdding and configuring AlertmanagerSummary
Technical requirementsUnderstanding the basics of GitHub ActionsExploring the components of a GitHub ActionHow to build and trigger your first GitHub ActionBuilding a continuous integration workflowIntroducing the tweeter command-line toolGoals of the tweeter continuous integration workflowContinuous integration workflow for tweeterBuilding a release workflowGitHub releasesRelease automation for tweeterCreating a custom GitHub Action using GoBasics of custom actionsGoals for the tweeter custom GitHub ActionCreating the tweeter actionPublishing a custom Go GitHub ActionThe basics of publishing actionsGoals for publishing the tweeter custom actionManaging action semantic versioningPublishing the tweeter action to the GitHub MarketplaceSummary
Technical requirementsEnvironment architectureUsing an Ops serviceBuilding a basic chatbotCreating event handlersCase Study – Regexes versus Lexer and ParserCreating our Slack applicationRunning the applicationsSummary
Technical requirementsBuilding an Amazon Machine ImageSetting up an AWS sourceDefining a build block and adding some provisionersExecuting a Packer buildValidating images with GossCreating a spec fileAdding a Packer provisionerCustomizing Packer with pluginsWriting your own pluginReleasing a pluginUsing our plugin in a buildDebugging a Packer pluginSummary
Technical requirementsAn introduction to IaCUnderstanding the basics of TerraformInitializing and applying infrastructure specs using TerraformUnderstanding the basics of Terraform providersDefining and provisioning cloud resourcesBuilding a pet store Terraform providerResources for building custom providersThe pet store providerPublishing custom providersSummary
Technical requirementsInteracting with the Kubernetes APICreating a KinD clusterUsing kubectl to interact with the APIDeploying a load-balanced HTTP application using GoIt all starts with mainCreating a ClientSetCreating a namespaceDeploying the application into the namespaceCreating the NGINX deploymentWaiting for ready replicas to match desired replicasCreating a Service to load-balanceCreating an ingress to expose our application on a local host portStreaming pod logs for the NGINX applicationExtending Kubernetes with custom resources and operatorsCustom Resource DefinitionsControllersStanding on the shoulders of giantsBuilding a pet store operatorInitializing the new operatorSummary
Technical requirementsWhat is the cloud?Learning the basics of the Azure APIsA background on cloud APIs and SDKsMicrosoft Azure identity, RBAC, and resource hierarchyCreating an Azure account and accessing the APIBuilding infrastructure using Azure Resource ManagerAzure SDK for GoSetting up your local environmentBuilding an Azure virtual machineUsing provisioned Azure infrastructureBuilding an Azure Storage accountSummary
Technical requirementsUsing overload prevention mechanismsCase study – AWS client requests overwhelm the networkUsing circuit breakersUsing backoff implementationsCombining circuit breakers with backoffUsing rate limiters to prevent runaway workflowsCase study – Google satellite disk eraseChannel-based rate limiterToken-bucket rate limiterBuilding workflows that are repeatable and never lostBuilding idempotent workflowsUsing three-way handshakes to prevent workflow lossUsing policies to restrict toolsDefining a gRPC workflow serviceCreating a policy engineWriting a policyCautions on policy enginesBuilding systems with an emergency stopUnderstanding emergency stopsBuilding an emergency-stop packageUsing the emergency-stop packageCase study – Google's network backbone emergency stopSummary
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a Free PDF copy of this book

Content preview from Go for DevOps

Using Common Data Formats

One of the key skills that a DevOps engineer requires is the ability to manipulate data across a variety of storage mediums.

In the last chapter, we interacted with the local filesystem to read and stream files. That is foundational for the skills we will be learning in this chapter.

This chapter will focus on how to manipulate common data formats that engineers commonly use. These formats are used to configure services, structure log data, and to export metrics, among the many other uses.

In this chapter, you will learn how to use comma-separated values (CSV) files to read and store data and encode/decode the popular JSON and YAML formats. You will discover how Go uses struct field tags to store metadata about fields. ...