GitHub Copilot vs. ChatGPT: How they work

Although ChatGPT and GitHub Copilot have distinct user experiences and workflows, they share some similarities in terms of their underlying technology. Both systems utilize large language models (LLMs) to generate responses and suggestions.

While ChatGPT's LLM is trained on human language data, GitHub Copilot's LLM, called Codex, is fine-tuned on a massive data set of source code and natural language text, explained Nikita Povarov, data analytics and machine learning (ML) team lead at JetBrains.

"This fine-tuning process allows Codex to understand the syntax and structure of code," Povarov said.

Here is the response from OpenAI's Codex to a request by the author:

In contrast, ChatGPT is a general-purpose conversational AI platform that uses natural language processing to respond to user input. It is better suited for broader, more complex tasks.

Agur Jõgi, CTO at Pipedrive, a CRM platform, said the most common application development use case for ChatGPT is understanding and writing code based on the task description. Here is an example of ChatGPT explaining code submitted by the author:

"Overall, Copilot is a great tool for quick, tactical tasks, and ChatGPT is better suited for broader tasks. Like any early technology, they both have areas of improvement and will get better and more advanced with time," Jõgi said.

Comparing the strengths and weaknesses of GitHub Copilot and ChatGPT

Here is a rundown of the tools' respective strengths and weaknesses.

GitHub Copilot

Strengths. GitHub Copilot excels at generating code snippets and suggestions based on the context of the code being written, said Jorge Torres, co-founder & CEO of MindsDB, a platform designed for democratizing ML.

Copilot can suggest lines of code, variables and function names relevant to the context of the code, and it can assist with granular snippets. GitHub Copilot also acts as a code completion agent that can complete tasks that usual code completion tools built into the IDE cannot.

As such, it can save developers time and help them code more efficiently. Additionally, GitHub Copilot can learn from the code developers write, thereby improving its suggestions and accuracy over time.

Another strength of GitHub Copilot for code writing is its integration into the IDE, making it more efficient for real-time coding than ChatGPT and proving a better user experience overall for programmers, Jogi said. "Since it's embedded into the GitHub ecosystem, developers can use it without switching contexts or opening additional tools," he said.

Weaknesses. GitHub Copilot's weaknesses include generating incorrect or inefficient code suggestions, Torres said. Additionally, it may not be suitable for complex programming tasks that require extensive knowledge and expertise.

ChatGPT

Strengths. ChatGPT's strength is automating customer service interactions and providing virtual assistance for various tasks, Torres said. It can also generate text-based content, such as articles, stories and summaries, making it useful for content creation.

Povarov also finds ChatGPT helpful for generating a wide range of code responses and explaining code concepts. It is also more suitable for nontechnical stakeholders and more flexible: Users communicate through a chat-like interface with a helper who can answer virtually any question. This flexibility lets users clarify and reformulate questions, leading to more refined answers and distinctions.

Jõgi also uses ChatGPT to generate function and unit tests to validate outcomes. This can streamline test-driven development practices.

Weaknesses. In terms of shortcomings, Torres finds that ChatGPT may struggle with complex or technical language, and it may not always generate accurate or relevant responses. Additionally, it may not be suitable for applications that require real-time interactions, such as gaming or trading.

Similarities between Copilot and ChatGPT

Both tools take advantage of OpenAI's GPT LLMs to generate results in different ways. They also suggest different paths for improving all kinds of tasks besides just programming. For example, Microsoft has rolled out a range of Copilots for various Office applications and is also integrating ChatGPT capabilities into its Bing search engine.

"They both represent the future of generative AI experience design, where AI-powered assistants can help us accomplish tasks more efficiently and effectively," Torres said. As AI technology continues to improve, Torres said we will see evermore sophisticated AI-powered assistants that can understand human language and context more accurately and generate more complex and sophisticated outputs.

Both tools have demonstrated ways that generative AI can boost developer productivity by automating routine tasks. Both kinds of capabilities could eventually be integrated directly into the development tools.

"In the future, we may see the integration of an AI assistant similar to ChatGPT directly into IDEs, enabling developers to communicate with it and request it to perform routine tasks, such as checking out code from repositories, running tests or building code," Povarov said.

If something goes wrong, such as a test failure or a build crash, the developer could ask the assistant to identify the error and suggest how to fix it. The assistant could do so, thereby reducing the time and effort required to maintain the codebase.

Preview of the future: Copilot X in the wings

GitHub recently introduced Copilot X, a technical preview of its vision for the future of AI-assisted software development. The new tool combines chat and terminal interfaces directly into the IDE. It promises to automate more aspects of the development experience. For example, it can detect code changes and automatically suggest descriptions, called pull requests, to accompany software updates.

The new offering promises context-aware conversations to explain code, find bugs and suggest fixes. It will also personalize answers that are linked to official documentation. It also promises to write software tests and identify missing tests that may be required.

GitHub reported that the X is a placeholder for new services, such as security, pull requests, docs and testing. The X also suggests the magnitude of impact it could have on developer productivity.

For the moment, Copilot X represents GitHub's vision for the future but is not available as a product. The company said it wants to take the time to determine the best way to provide these new features to customers.