Comparing GitHub Copilot and Codeium

Learn the advantages and limitations about these AI-powered, code completion tools.

In the evolving landscape of software development, AI-powered code completion tools have emerged as game-changers, helping developers write code more efficiently and accurately. Among the currently leading tools in this domain are GitHub Copilot and Codeium. While both tools provide similar functionality and features, they each have their distinct advantages and limitations. Having used and trained others on these tools, in this article, I’m going to delve into some of the similarities and differences.

Overview

At a high level, both products provide the same core functionality, code completion suggestions based on the contents and context of your current code, and an interactive chat feature for conversing with the Large Language Models (LLMs) around coding topics using natural language. Beyond that commonality, there are notable differences between the two in various categories.

Training models

GitHub Copilot, developed by GitHub in collaboration with OpenAI, leverages the power of OpenAI’s GPT-3.5/4 LLMs. These models have been trained on a large set of public code housed in GitHub.

Codeium leverages LLMs trained by the Codeium team primarily, which they claim has “slightly-better-than-GPT-3.5 performance.” At the Teams level in Codeium, there is also an option to select GPT-4 as an LLM to work against.

IDE integration

GitHub Copilot supports integrations in Visual Studio Code, Visual Studio, Vim, Neovim, the JetBrains suite of IDEs, GitHub Codespaces, and Azure Data Studio.

Codeium claims support for integrations in 40+ IDEs and web editors including the ones that are supported by Copilot. Notable beyond the typical IDEs are integrations for Jupyter, Colab, Databricks, Chrome, Eclipse, Sublime Text, and others.

I’ve used both Copilot and Codeium in the VS Code/Codespaces environments and both seem to work well there. Copilot has a tighter “feel” for integration which is not surprising given Microsoft/GitHub have more access to the underlying APIs. Codeium also proved to be challenging at times to get authenticated to in the virtual environment of GitHub Codespaces, but I was able to eventually get it working. Once authentication was successful, I found that it functioned well.

Language and framework support

Since Copilot leverages the ChatGPT models trained on the public repositories in GitHub, it can presumably give the best answers for the languages most widely present in those repositories. According to the documentation, Copilot “works especially well for Python, JavaScript, TypeScript, Ruby, Go, C# and C++.” And it “can also assist in query generation for databases.”

Codeium claims support for over 70 languages. An extensive list is in its FAQ.

Interfaces

A big part of using any tool for developing code is the user interface. Both GitHub Copilot and Codeium are UI-based with limited or no functionality available via command line and APIs currently. (Copilot has a couple of simple options it can do via the GitHub CLI utility, but they are not really meaningful yet.)

Both tools have code completion capabilities that present suggestions in grayed-out text until you accept the text. You can select the entire suggestion via the Tab key or choose to select only a “word” (Figure 4) or line (Figure 5) at a time.

When there is more than one suggestion that Copilot/Codeium thinks might make sense, they will both let you cycle through the suggestions with less than/greater than controls in the suggestion box (Figure 6).

Since these suggestions need to be returned inline in the IDE in near real-time, they may not always be a full set of code. Copilot also provides a feature to generate additional alternative suggestions using the Ctrl+Enter key sequence. This will bring up a separate pane in the IDE and list any alternative suggestions there (Figure 7).

If an alternate suggestion looks better than the current code, you can accept it using the provided link. One warning here is that the quality and completeness of these alternate suggestions can vary greatly. They may or may not be good alternatives and may not be fully formed replacements. In fact, sometimes, they are only comments.

Inline chat

Within the editor, both Copilot and Codeium offer inline chat options via a Ctrl/Cmd+I key sequence (Figures 8 and 9). Using this sequence, you can bring up a dialog box where you can type queries or prompts and then submit them. You can also abbreviate some functions, such as “/test“, “/doc“, etc. Depending on the length and type of response, the result may show up inline or it may be put into the separate query window.

Codeium also takes advantage of the VS Code Codelens functionality to offer other inline options. Under this set is a Refactor menu that is especially useful, allowing you to choose from many common tasks that you might need to do while writing code (Figure 10).

Once you choose a refactor option, Codeium presents the proposed changes in a typical “patch” format for you to review and accept or reject (Figure 11).

I found the options under this Refactor menu very convenient and helpful to use. There are also options to have Codeium explain code and generate doc strings (basically header comments). All of these capabilities are available in Copilot as well, but not through the Codelens inline interface. With Copilot, you would need to issue prompts to get these functions.

(As a side note, you can use comments inline in your code to prompt Copilot or Codeium to execute functions like explaining code. You can also ask questions in comments and get responses in comments generated through code completion. Copilot seemed to do a better job in responding to questions in comments.)

Dedicated chat interface

Both Copilot and Codeium provide separate dedicated chat interfaces with the ability to copy content from chat and put it into a file or terminal. Queries fed into the chat interfaces can range from simple shortcut commands (such as /simplify) to more extensive prompts with extensive additional context.

Both applications also have multiple ways of specifying the context for a query or prompt.

Codeium has a dedicated Context tab in the chat interface that can be used to set a context like “Respond back in Python” (Figure 14). You are also able to specify additional “pinned” contexts – for example, telling Codeium to only use certain files to answer questions. Codeium uses a “context engine” that is intended to do some additional pre-processing on the front end to gather a larger context sample and then rank and prioritize appropriately from that larger sample before sending to the LLM side. Codeium also supplies a search feature to allow devs to ask questions such as “How do we do X” in the current codebase.

As well, you can use the @ symbol to “mention” functions, classes, repos, or terminals in your prompt (Figure 15). 

Copilot includes simple ways to specify context such as use of a “#selection” tag to ensure the code currently selected is considered. At a broader level, Copilot has “agents” that allow you to specify your workspace, the terminal, or vscode as the context to use when responding to a prompt (Figure 16 and 17). Per the GitHub documentation, agents are “like specialized experts who can assist you with specific tasks.”

Currently, agents are available that know about your workspace, vscode, and the terminal. These can set context for queries through chat by mentioning them with the @ symbol in front. Some of these agents also provide a default context for using /commands. The @workspace agent is especially useful as it allows pulling in all of the files in your workspace to draw context from versus just what’s open in your editor.

Language and framework use cases

I’ve developed training and done proof of concepts using both products for JavaScript, Go, Python, and SQL generation and optimization. Both products seem to work well for these scenarios with a slight edge for Codeium in terms of the quality and number of suggestions it offered. 

I’ve also used both to do things like produce Kubernetes manifests (yaml files) for natural language prompts and both did equally well. The same held true for generating mapping data such as a mapping of states to area codes. (Neither product was able to provide a complete mapping on the first try due to limitations on the size of the amount of data returned.)

As another use case, I also had the products produce code in Python to then call Kubernetes APIs to manipulate the manifest they just generated and then convert that Python code to Go code. Both products did equally well with those use cases.

Pricing and plans

GitHub Copilot provides an initial 30-day trial that’s free for individual use. Post-trial, GitHub Copilot requires a per-month or per-year paid plan. The cheapest plan is the individual one at $10/month or $100/year. The price goes up from there to a Business model for organizations at $19/month/user. The main benefit of this plan is business licensing and administrative capabilities across users. 

There is also an option for an Enterprise-level plan ($39/month/user) that buys you more functionality as long as you are using GitHub Enterprise. In particular, it provides you with additional capabilities focused within your GitHub Enterprise instance, including limited ability to summarize pull requests, and having Copilot trained to answer questions on some of your documentation as long as it is stored in GitHub.

Read more details about the Copilot plans.

Codeium, on the other hand, offers a perpetually free option with sign up. The free option is comparable to Copilot’s individual option, but without any ongoing cost. Codeium also offer a Teams option for $12/month which provides similar administrative and licensing features for teams as the Copilot Business plan, as well as an option to use GPT-4. They also say this provides personalization on your code base and will eventually include doc searching capabilities.

An Enterprise version for Codeium is also available that includes a self-hosted option, priority support and fine-tuning, etc. As of the time of this writing, the price for that is noted as “Let’s talk”.

Read more details about the Codeium plans.

I’ve used both of the individual plans for Copilot and Codeium. Both worked well, and were comparable (within the qualifications mentioned in this article). In a previous corporate role and in my own business, I’ve had some experience with the Copilot Business model. Other than the usual learning curve and administrative headaches that can come with any enterprise software and rollouts, the experience was not bad, and the functionality seemed to work as intended. A drawback on the Copilot side is that (at least last time I checked) you could not mix Business and Enterprise licenses within an organization. Presumably this is the same for Codeium but there is no statement I could find about that.

Support

Copilot has the benefit of updates and communication channels from GitHub and the vast network of users it has. This may provide for broader insights and opportunities to have issues brought to light (and hopefully) addressed more publicly and quickly.

Codeium channels support discussions and issues through its Discord server which provides a less visible, but more close community setting for users and the product developers helping each other.

In my own experience, enterprise support/questions for Copilot support get attention fairly quickly, but individual questions/concerns can sometimes languish. Like any software Copilot can also suffer from minor, known issues that are problematic for an extended period of time, if they do not rise to the level of getting addressed.

For Codeium support, your issue may get a wider visibility and benefit from community input, but similar to Copilot, resolution may not occur unless it rises to an important enough level.

Privacy and security policies (as stated)

Both Codeium and Copilot obviously send context for completions, questions, etc. “over the wire”. But both include options to opt-out for telemetry based on code snippets. As previously mentioned, Codeium models were trained only on permissive licenses. Copilot includes an option to also opt-out / be warned about exact public code matches. Codeium is also SOC 2 compliant.

New features

As the world of AI is constantly and quickly evolving, adding new features (and thus additional value) is key to continued buy-in to both Copilot and Codeium. Both tools have multiple features in the beta/prototype stage.

Codeium’s new features and functionality seem to focus more on improving the core functionalities such as better suggestions, better context setting, etc.

Copilot, on the other hand, seems to be focused on providing larger extensibility and development environment features. For example, Copilot has recently announced GitHub Copilot Extensions which allows partner companies to provide targeted tools and environments as a packaged deal for developers to use. They are also adding dedicated virtual development environments with GitHub Copilot Workspaces.

Conclusion

At a core level, both GitHub Copilot and Codeium do a very good job for suggested code completions and development-oriented chat interfaces. In general, Copilot can provide a tighter integration while Codeium can provide a better UI integration experience. The quality of suggestions seems comparable with an edge to Codeium for generally providing a better set of alternative suggestions. Both products do a good job of handling the most prevalent languages and integration with popular IDEs, but Codeium supports a wider set of both.

Codeium provides a free option for individuals while Copilot provides a 30-day trial for individuals before requiring moving into a paid option. Both products offer teams/business and enterprise options as well. Codeium comes in at a lower price point for those, while Copilot offers strong integration with Enterprise GitHub features such as pull requests.

Overall, if you just need an individual option that handles code completion suggestions and chat, the free option from Codeium should work well for you and be all you need. However, if you or your team or company are working with GitHub and especially if you’re using GitHub Enterprise, you may be better off with Copilot. (Currently, Copilot’s individual offering does not offer a lot of extra value with GitHub instances, but that could well change.) The other reason to use Copilot is if you are expecting to focus on extensibility in the future such as a specialized workflow or partner program.

In case it’s useful, here’s a table summarizing some of the thoughts in this article.

GitHub Copilot vs. Codeium

Category	GitHub Copilot	Codeium
Training models	GPT 3.5 and 4 – in conjunction with OpenAI	Proprietary LLMs and GPT 4 at the Teams level
IDE integration	Visual Studio Code, Visual Studio, Vim, Neovim, the JetBrains suite of IDEs, GitHub Codespaces, and Azure Data Studio	Same as supported by Copilot plus many more including Jupyter, Colab, Databricks, Chrome, Eclipse, Sublime Text
Language and framework support	Ones represented in public GitHub	70+ languages and frameworks
Inline chat	Dialog invokable from editor that is easy to work with	Dialog invokable from editor that is easy to work with
Language and framework use cases	Works well for SQL, K8s manifests, getting APIs, translation of code	Works well for SQL, K8s manifests, getting APIs, translation of code
Pricing and plans	Paid plans for individual, business, and enterprise	Free plan for individuals, paid plans for teams and enterprise
Support	GitHub support channels	Discord
Privacy and security policies	Safeguards and options to avoid data use beyond basic functionality needed to function. Option to block/identify suggestions matching public code	Safeguards and options to avoid data use beyond basic functionality needed to function. Trained on permissive licenses, SOC-2 compliant
New features	Extensibility – workspaces, extensions, GitHub integration at Enterprise level	Core – increasing ease of use, interface features, context selection, etc.

---

Of course, this is only one user’s opinion and not a formal analysis. Your mileage may vary. If you are considering an AI coding assistant, I would suggest taking advantage of the free options to try these and do your own comparison for your particular use case. Also, keep in mind that there are many other AI coding assistants available besides just these two. Good luck and happy coding!