Chapter 1. Foundations

Welcome to Learning GitHub Copilot!  I’m excited that you’re reading this book and hope you find it useful.  AI tools like ChatGPT have changed, and will continue to change, how we interact with software applications. GitHub Copilot and other similar tools have changed, and will continue to change, the way programmers create software applications. Through its ability to take context from existing code or natural language prompts, GitHub Copilot provides a richer code-generation capability than any we’ve seen before.

In this book, I’ll help you understand how to use the capabilities that GitHub Copilot provides - from code generation and completion to forming tests, mapping data, crafting complex SQL easily, and more.  I’ll look at examples of using it across multiple domains and programming languages. You’ll see how to leverage it to your advantage and craft prompts for it to get the best results. And you’ll understand how it does what it does, including why it sometimes doesn’t provide the best answers.

To begin though, some foundational knowledge is needed and that’s the purpose of this chapter. I’ll begin by explaining at a high level what GitHub Copilot is and how it works. Within that explanation, we’ll look at some of the key underlying technology that GitHub Copilot is built on, its overall flow, some usage considerations, how it differs from ChatGPT, and what you need to know about getting and installing it. So let’s get started.

Copilot catchall

There is a key item to be aware of as we are beginning this discussion.  The term copilot is a popular designation for AI applications that collect context and prompts, and return answers and suggestions.  In fact, Microsoft has an Office 365 Copilot that does this for Microsoft Office applications to summarize context from Word, Teams, Outlook, etc. and provide summarizations, suggested responses, and other useful interactions.

While it may be obvious, throughout this book, when I use the term Copilot, it will explicitly refer to GitHub Copilot. The only exceptions will be if I need to refer to another system that uses the same word. In those cases, I’ll explicitly identify them with their formal names such as Office 365 Copilot.

What is GitHub Copilot?

GitHub Copilot is a cloud-based generative AI tool. It’s well known that AI stands for Artificial Intelligence, which refers to computers doing tasks that previously only humans were thought to be able to accomplish because of required reasoning and skills. More recently, this has also taken on the aspect of interacting with humans in a natural and human-like way through chat interfaces.

Cloud-based refers to the pathway through which Copilot returns suggestions and generates answers. It refers to a cloud managed by GitHub. Generative expresses the AI’s ability to generate new results from the context it takes in. Copilot can offer up responses and suggestions for software development based on the context and prompts gathered from the user’s environment. It can be used for work in domains such as machine learning, AI, data science and analysis, and more. How well it does this depends on several factors that we’ll talk about in the next section.

How does Copilot work?

To understand how Copilot works, we need to understand some of the pieces that underpin its functionality and that of similar AI tools. This includes where it gets information to base responses on, and its overall flow.

The place that Copilot and most AI applications get their data from are models generated from large collections of  data, known as Large Language Models or LLMs. If you’re not familiar with that term, the next section provides a brief explanation.

Large Language Models

It’s difficult to talk about AI tooling today without talking about LLMs.  LLMs are AI models trained across huge amounts of existing data to predict the next words or other type of content (i.e. tokens) that fit given some input (a prompt).

What makes LLMs different from traditional computer models that can process formatted data, or respond to a math problem, is that large language models are taught to understand context as well as syntax and structure. This is done by algorithms that consider huge numbers of parameters to figure out what words or tokens make the most sense statistically to craft responses given the way the input is presented and not just the input itself. The model’s prediction capabilities are learned and tuned based on large collections of existing content. This training process is where the models can map how the various pieces of information relate in any given domain. Technically, given some sequence of tokens as a query, they can assess the input’s syntax and structure, infer context based off of the model’s training data, and predict what sequence of tokens would most likely come next. In simple terms, they can figure out what would sound right based off of all the data they’ve digested and continue the conversation.

To appreciate the difference from traditional processing, think of learning a language and then participating in a conversation with someone who speaks it as a first language.  You can learn the vocabulary and individual words and phrases. But in a conversation with someone who speaks this language well, you also need to be able to gather and understand the context of what the other person is saying in order to choose the right words or phrases to respond with.  And, you need to be able to frame your response in a context that will make sense to the other person. Language has syntax and structure. But the context of a conversation is how we make that syntax and structure make sense and convey meaning. LLMs are able to assess context from the inputs and supply relevant context in their outputs.

Turning back to Copilot, it is based on LLMs developed and managed by OpenAI, the same company behind ChatGPT. GitHub, in conjunction with OpenAI, developed Copilot over several years, experimenting with several flavors of LLM  before moving to the later models, known as GPT-3.5 and GPT 4, in use with the technology today.  Throughout their use of all of these models, though, the common characteristic of Copilot is the focus on creating software, and its conversational language is code.

 

Code and Generative AI

Programming languages by definition have a specific, required syntax and semantics.  And those are different for each language. If you are coding in Python versus Go, you will be using different tokens and structures to create the program, but the tokens and structure you use are well-defined and form a closed set. The model that Copilot uses is well suited for providing coding suggestions and related information that match syntax and structure.  But the real value-add is providing those in context - the context of what the coder is creating. The context that gets fed into Copilot can come from a number of different sources including:

  • From a set of code being written in an editor in a development environment
  • From interacting with the model via direct natural language queries - aka chat models
  • Through typical development activities in GitHub itself, such as pull requests (requires the Enterprise subscription plan)

I’ll be discussing these different interfaces more throughout the book. But regardless of the interface, the contextual information, directives, or questions that you enter into Copilot form the prompt, your side of the conversation that you expect the AI to respond to.  Processing your side of the conversation and deciding what to suggest based on the context provided and its training is the generative part of Copilot’s generative AI functionality.

To get to a suitable prompt that includes the best input to the AI, the prompt may undergo some evaluation and pre-processing. Ultimately, it is passed to the LLM to evaluate based on its training and return a response. The response can be thought of a prediction of what would satisfy the query. You can think of this process as being like describing a set of symptoms to a doctor over a Zoom call so they can try to reach a diagnosis based on their years of training.  Or describing to the car mechanic over the phone some issue that you are seeing with your vehicle, so they can suggest a fix. In these cases, the communication and context provided (the prompting process) and the training of the professional make all the difference. The interaction can result in one or more suggestions to address the issue (some of which may not apply), or indicate that the professional didn’t have enough context or understanding to really help. Generative AI behaves the same way.

When Copilot offers suggestions as you’re coding, your coding environment is the context it draws on to create a prompt for the AI model. Copilot can produce suggestions that mimic the coding style used in the files that are part of the project you’re working on. This can be both good and bad.  It can be good in the sense that Copilot suggestions often are similar to existing coding styles of the users.  It can be bad in that Copilot sometimes errs too much on making its suggestions almost similar to existing code and only using constructs used by the current code.  This can, in turn, lead to reinforcing limited and inefficient coding practices or having to wade through a larger set of suggestions to get to what the user really intends. And, of course, the quality and quantity (or not) of the examples that Copilot has to draw on from your context can influence its responses.

Ultimately, the generative AI employed with Copilot can be very useful, but comes with the unavoidable caveat that the coding suggestions will be biased by the context Copilot has to work with.  You can use this to your advantage to provide Copilot with examples of what sort of suggestions it should return.  But it can also be a disadvantage if the context that Copilot has to work with is limited. 

To best understand how Copilot works with context overall, let’s take a brief look at its high-level flow.

High-level Flow

To understand how Copilot works at a high level, we can trace through the basic workflow from the perspective of working in one of the supported integrated development environments (IDEs).  Currently, the supported IDEs for Copilot include Visual Studio, VS Code,, NeoVim, any of the JetBrains family of IDEs, and GitHub Codespaces.

About Codespaces

GitHub Codespaces are virtual machines (VMs) running in the cloud that provide a full-featured and customizable development environment for those working with GitHub. The interface is similar to VS Code and, in fact, they can be tailored with the same kind of extensions as used in VS Code in a browser or connected into a number of IDEs.  They are provided as an optional service via GitHub.

In the development environment a number of pieces of information are taken into account to form the context and ultimately the prompt for Copilot to work against.  These include:

  • Current file context: One of the key sources for Copilot to draw context from is the current file that a user is editing. 
  • Name of the currently active file: When named descriptively, this can provide a clue to Copilot about what the code is intended to do, for example TestConfig.go.
  • Content before and after the current cursor position: Copilot can draw context from the code and comments immediately before and after the current cursor position in the file. This can help it decide what should be filled in and/or understand gaps in the code.
  • Comments: As with human pair programmers or reviewers, Copilot can use comments to understand not only what code is doing, but also the intent of code that is not yet written. In fact, this is one of the primary way to provide context for Copilot – the more precise and detailed the comments are, the more likely the code that Copilot suggests will be relevant.
  • Other open files in the editor: Copilot uses the code being developed in any open files in the editor as context.  This is key to gathering not only information about the current task, but also augmenting information in the model. For example, one strategy for dealing with deprecated features that are stored in the LLM is to open a file in the editor with the replacement approach for the deprecated feature. From this example, Copilot can interpret preferred alternatives for coding whereas without it,  it might have relied on the deprecated approach that the LLM was trained on.

Copilot and closed files

Copilot currently is only able to draw context from open files in the editor, not closed files that are part of the project open in the editor. The ability to factor in code from closed files is being worked on, but current processing power and capabilities don’t allow for this.

When a user uses one of the interfaces that has had Copilot installed and activated, as they are entering code, Copilot gathers context from these sources.  That context is processed and ultimately sent to GitHub for processing into a prompt. The prompt is then passed through GitHub to the LLM and possible completions/answers are returned.

Once results are generated from the prompt to the LLM, GitHub then does some additional processing on the result (more on that later). After that, the suggestions/answers are returned back to the IDE where the user can evaluate them and choose to incorporate one (or not). The user can also ask Copilot to generate additional options - which may or may not be better/more complete than the original ones.

This process then continues and repeats as needed with the user/Copilot interaction. In this way, Copilot works as an assistant to help you in whatever use case you are working with, crafting boilerplate code, searching for a sophisticated algorithm, data or query generation, writing unit tests, learning a new programming language, etc.

You’ll see more of how the flow happens as we discuss how to use Copilot throughout the remaining chapters.  But before we get further along in our discussion, it’s worth understanding some of the usage considerations when using Copilot to help you produce software.

Usage Considerations

Now that we have some basic background on what Copilot is and how it works, let’s work through some considerations to keep in mind when using it. Specifically, in this section, we’ll look at the key areas of:

  • Timeliness
  • Relevance
  • Completeness
  • Accuracy
  • Privacy
  • Security

Since Copilot is our focus, we’ll present these in the context of using Copilot.  But these considerations are not unique to Copilot. They can apply generally to any current AI in use for some interactive task.

Timeliness

Timeliness here refers to the currency of suggestions from Copilot. While this may seem an odd consideration to start with, it is one that can potentially pervade all of the others. 

Since Copilot relies on models that were trained originally at a point in time, its data is based on what was current in the training data at that point.  That means, for example, that if the model it uses was trained two years ago, it does not necessarily know about changes since then.

In your everyday use of Copilot, this can translate into suggestions and answers being out of date.  This is arguably more important in the coding domain because results could contain deprecated code that, if you incorporate it, no longer works with your compiler or interpreter for example. Or, you could get a suggestion that references a version of a dependency that has a known vulnerability.  Or a response might include an outdated approach.

As another example, you might ask the chat interface in Copilot “Is X deprecated?” and get an answer that X is not deprecated, when in fact it is. Copilot is answering as of the point in time when it was trained.  Or, you might ask it “What is the current version of X?” and get a result from two years ago.  I’m sure you can see how this could cause issues.

To be fair, in most cases where you are interacting with the chat interface and you ask it “What is the current version of X?”, it will respond with an “as of the data of training” to make it clear and recommend that you consult the documentation for the latest information.

There are also additional technologies being explored to augment older training data. And there are techniques you can use to provide Copilot with more up-to-date context for deprecated items from within your environment to increase the likelihood that it will return more up-to-date responses. Those are discussed in Chapter Y. 

 

Relevance

The suggestions from Copilot are based on the GPT LLMs that were trained on a public code base – the code base from GitHub. With its industry-standard collaboration platform, GitHub arguably has the widest collection of repositories for open-source software.  This includes code written in the most commonly used programming languages today such as Python, Go, JavaScript, etc. 

Since the underlying LLM has been trained on this code base, it stands to reason that Copilot will be most effective when the user is working on the one of the more common programming languages or frameworks represented in GitHub.  The more a language or framework is represented in the GitHub repositories that the model was trained on, the more references it can learn from. Think of it like the subjects you spent more time studying in school or college - those are the ones you know the most about.

If you’re working in a less represented language or framework, the suggestions may not be as useful.  That doesn’t imply you can’t get relevant suggestions and answers, but you may find them less useful in answering your exact query.

Another key factor here that is beyond Copilot’s control is the amount of context it has to work with.  For example, if your code consists of a function name ParseData or you supply a query that is generic or ambiguous, such as Create a function to parse data, without additional context, the results returned from Copilot are unlikely to be a good fit.

Completeness

Another facet of using generative AI is that it can sometimes return incomplete or unusable suggestions.  While this rarely happens in the chat interface, it’s not uncommon for Copilot in the IDE integration to return a set of suggestions that are only partial solutions.  

The same caveat as discussed at the end of the previous section on relevancy applies here.  If your code or query is generic/ambiguous, Copilot may not have enough context to draw on to return a complete result.

However, there can be other instances where you have specific context provided to Copilot and it still may return only a comment or the first line of a function, for example. This is generally seen when working through the suggestion process in the IDE rather than in the chat interface. These may simply need additional prompting to complete or they may be indicative that there’s not enough information for the AI to form a complete response.

In the IDE integration, this can be a result of how Copilot returns large data. It may only return a line or two at a time and require you to accept multiple suggestions in sequence to get the overall result. Or, it can genuinely be that Copilot is not able to return a complete result and may just return a comment or an incomplete function, etc.  Sometimes, you can nudge Copilot by giving it a hint (typing a keyword for example) and that will be enough to make it return a more complete suggestion.

Copilot offers options to get more suggestions if the immediate one isn’t a good fit. I’ll delve into details on those approaches in Chapter 2.

Accuracy

Even when you get what appears to be a relevant, accurate result from Copilot (or multiple results), except in the most simplest of cases, you cannot guarantee that any response from Copilot is completely accurate. Copilot responses should always be reviewed carefully. 

This responsibility is similar to reviewing responses from any current AI application and the potential challenges with results with Generative AI in general. It’s not hard to find stories of AI results that made their way into official records with references that don’t exist. For example, you might find court briefs that were filed referencing previous cases that didn’t happen. The author personally knows of people who have tried to use AI to plan vacation itineraries where the AI returned a very promising agenda - only with hotels that didn’t actually exist.

Likewise, Copilot may return a response that is complete syntactically, but references constructs or variables that do not exist elsewhere in the code. This is a coding form of AI hallucination – where the AI appears to make up new objects and references that do not otherwise exist in your context.  This is rare, but not unheard of. To be fair, sometimes, this can be because Copilot needs to create suggestions around elements that don’t exist yet to reference. For example, if you ask Copilot how to open and write to a file in a programming language that you don’t know well, it may understandably insert a file name that has no meaning in your code. 

Regardless of the circumstances, it is always incumbent on the user to review Copilot’s responses for accuracy.  This should be no different than what you would do when reviewing a human’s contributions to your code to ensure that those are correct and do no harm.

Prompts and accuracy

Improving the results from Copilot can often be improved by developing a better prompt for the AI.  We’ll cover more about prompting for throughout the book.

Privacy

Another aspect of working with Copilot is data privacy.  As previously noted, a number of information sources (open files, current files, etc.) are read and information is collected and transferred through Copilot to factor into the prompt to the LLM.  That means the collected information is going across the web and being processed outside of your control. This may seem like a potential security risk, but Copilot includes options when signing up to specify whether or not you want to allow GitHub to include your context information as part of its data to help Copilot get better.  (See the bottom part of Figure 1-1.) If you do not, while the information will be gathered, it will not be retained.  It will be collected to construct the prompt and then discarded.

Figure 1-1. Options at signup for data privacy (individual plan)

Broadly, Copilot intersects with user data in three areas: user engagement data, prompts and suggestions. These can be roughly characterized as follows:

  • User engagement data - This is usage data about how you interact with Copilot (aka telemetry). This can include whether or not you accept or dismiss suggestions from Copilot, how you interact with the Chat UI, metrics such as latency, error messages, etc.
  • Prompts - The context information taken from your environment or a chat query that is passed back to GitHub as previously discussed.
  • Suggestions - The suggested code completions returned by Copilot and/or the responses to Chat queries.

Of these, by default, the user engagement data is tracked and could potentially include some anonymized data.  Mechanisms exist to encrypt data in transit and at rest. And controls are in place at GitHub to strictly control who can access any data on that side.

One other prevalent concern about Copilot is that, since it was trained on the GitHub public repositories, it could return suggestions that are exact copies of code from that codebase. The implication is that users could end up unwittingly violating the licensing terms and intellectual property rights of those repositories by having the duplicated code included in their work.  Copilot includes an option for individuals and administrators in charge of providing Copilot licenses to block exact public code matches if those are generated as part of the AI’s process. (See top part of Figure 1-1.) If that option is checked, then Copilot will filter out any exact matches from the public code base before returning suggestions to you.

Security

Last, but certainly not least, is the area of security in the results that Copilot returns. Security is an ever-present concern in any product or application used today and addressing that concern starts with secure coding practices.

I stated earlier that once Copilot gets potential results to return from the LLM, GitHub does some additonal processing on those.  This processing includes running algorithms to check for potential security vulnerabilities. This processing is not actually running a security scanning application against the result as that would take too long.  Instead, it is quickly looking for patterns that might indicate vulnerabilities and/or insecure coding practices. If those are identified, the proposed suggestion is rejected.

Even with these measures in place, there is no guarantee that something hasn’t slipped through.  And, as noted, this is not the same as running full scans with an application focused on finding vulnerabilities. Ultimately, the results returned from Copilot should still be subject to whatever other security checks you would use for code crafted by humans.

The clear and ever-present requirement when using generative AI is that you must always review and assess any suggestions it returns. You cannot assume that it completely and correctly interpreted the context. And, in all but the simplest of cases, you can not assume that the result has everything perfect.

To state this another way, while Copilot is often referred to as an AI pair programmer, it should not be considered to have the same understanding and level of familiarity with your code as an actual human pair programmer would. Rather, it is best to think of it as a skilled junior programmer – able to create some useful code based on what it can observe and has been told, but doesn’t fully possess the larger context.  And always requiring review to ensure the code produced is accurate, secure, and suitable to merge.

With respect to needing to review and assess timeliness, relevancy, completeness accuracy, privacy, and security, Copilot is really no different from other generative AIs, such as ChatGPT.  In fact, it may seem that we could just use ChatGPT to do code generation for us. But there are some key differentiations between the two applications.  The next section will give an overview of those.

Copilot vs ChatGPT

Looking at the various aspects of Copilot may lead you to wonder how it differs from ChatGPT given both use the same underlying models to produce responses. The short version is that Copilot is built and managed by GitHub and focuse only on the coding domain, whereas ChatGPT is targeted more broadly across any domain and managed/built by OpenAI.  The table below highlights differences in a number of more specific categories.

Table 1-1. Comparison of Copilot to ChatGPT
Category GitHub Copilot ChatGPT
Primary Function Understanding and generating code Understanding and generating natural language
Primary user Interface Code editors, chat Chat
Developer GitHub with OpenAI OpenAI
Primary use cases Writing and augmenting code and code documentation Conversational responses with text generation
Pricing Model Subscription-based Usage-based
Privacy May involve sharing data with GitHub for telemetry or sharing code in process May involve data being incorporated if controls not exercised
Training Data Code repos, documentation Diverse, broad text content

In general, you can think of Copilot as a very domain-specific implementation of generative AI. This is as opposed to the broader (domain-less) implementation and function of ChatGPT.  ChatGPT can certainly be used for generating code, but it lacks the integration with development environments, coding focus, and GitHub support and features that Copilot has. Additionally, without exercising the proper controls, there is a higher risk that, out of the box, your code could be incorporated into ChatGPT’s model through interaction with it.

Getting Copilot

Now that you understand some of the basics about Copilot and it’s intended use, how do you go about getting it?  What factors should you consider?  Copilot comes in three plans – Individual, Business, and Enterprise.  The table below explains some of the differences.

Table 1-2. Comparison of GitHub Copilot Plans
  Individual Business Enterprise
Target users Individual developers Organizations or enterprises Enterprises that use GitHub Enterprise Cloud
Features Coding suggestions, generation, and documenting plus interactive chat, CLI integration Individual features plus enterprise license/policy management and custom certificates Business features plus integration in GitHub such as pull request summaries and ability to train on internal codebases
Administration Individual Enterprise Enterprise
Cost $10/month or $100/year (free trial available for 30 days) $19/month/user $39/month/user

Latest info

The information in this table is accurate as of the time of this writing. Consult this page for the latest information.

Of course cost is a significant factor – if you are an individual user who only uses Copilot on a limited basis in your IDE and doesn’t need the extra features, the Individual plan is cheapest and makes sense. At a corporate or community level, if you need or want the extra ability to easily assign and manage licenses across multiple users as well as options such as whether or not to allow exact matches, the Business level subscription can provide that.  And if you need to manage multiple users and want the ability to train Copilot on your own codebases as well as take advantage of advanced features in GitHub such as generating pull request text, then the Enterprise plan is your only option.

Mixing plans

Unfortunately, as of this writing, GitHub does not allow mixing the various license types.

Regardless of the plan you choose, the steps are similar:

  • Sign up and register for license(s)
  • Establish a payment process
  • Install Copilot via an extension or whatever process you use to add functionality in your IDE

Copilot for free

Copilot does have a free option for certain verified maintainers of open source software on GitHub and students.  See the FAQ for more information.

Figure 1-2 shows the opening signup screen for an individual plan. Getting to this is as easy as clicking on your profile picture and selecting Try Copilot.

Figure 1-2. Individual signup screen to try Copilot

At an organization level, once the plan is setup, organization admins can add users and manage licenses as shown in Figure 1-3

Figure 1-3. Managing a Copilot Organization plan

Once you gain access to a Copilot subscription, you can install Copilot into whichever IDE you use.  For convenience, you can go to your Settings/Copilot screen and you’ll have links there to the documentation for the different IDEs as shown in Figure 1-4

Conclusion

I hope you’re starting to have a better conception of what GitHub Copilot is all about and how it potentially can be used.  I also hope you’re finding yourself intrigued to learn and understand more about it.

In this chapter, I’ve provided an overview of what Copilot is, how it works, some key considerations to keep in mind when using it, and how to go about getting ready to use it. Overall, this chapter paints a picture of how Copilot fits in with the current use and potential of AI to help you in a given domain. In this case, that domain is creating software, and more generally, coding.

There are a couple of key points to take away from this chapter. One is that AI can greatly simplify and support your efforts as a coder. A second is that context is king when it comes to helping Copilot provide you with the best results. And a third, and arguably the most important one, is that you still have the ultimate responsibility to review and assess any suggestions and answers from the AI for all of the usability considerations we covered.  Copilot is great at what it does, but it is only as good as the context we give it and the data it is trained on. And like human coders, those variables factor in to how good (or not) any result is.

In the next chapter, we’ll look at how we actually use and work with Copilot in the IDE and chat interfaces. That will help move you the next level of actually using the tool and complete the overall foundations you need for the rest of the book and more specific and advanced use cases for Copilot.

Get Learning GitHub Copilot now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.