Microsoft Research launched an experimental project for Visual Studio and the Web in 2014. Bing Code Search Add-on released. Bing Code Search, a code fragment search tool for C#, has a natural language interface, uses code indexes from Stack Overflow, MSDN, Dotnetperls, and CSharp411, and runs on Azure based on Bing. The tool also had a feature for changing variable names in a fragment to variable names in your own code, but it didn’t work very well. When there was only one variable to replace, the accuracy was 70-80%, and as the number of variables that had to be renamed increased, the accuracy dropped sharply.
This experimental Microsoft Bing Code Search add-on from 2014 has more evolutionOne result is ‘GitHub Co-Pilot’. The author of GitHub Co-Pilot in 2021 Review the preview versionThe quality and accuracy of the code generated at the time of implementation varied, and there were cases where it did not run, but it was somewhat useful. At the time, GitHub Co-Pilot’s engine was OpenAI Codex based on GPT-3.5 LLM, and Co-Pilot only considered the current file in the context.
Two years later, in 2023, I released a collection of technology preview features that further expanded the existing GitHub Co-Pilot with a chat and terminal interface, pull request support, and early adoption of Open AI GPT-4. Review GitHub CoPilotdid it The GitHub CoPilot X Preview is a significant improvement over the original GitHub CoPilot. We were able to easily generate the right functions and test suites without much human help. Mistakes and hallucinations (creating false information) still occurred, but not as severely as before.
Even after that, GitHub Co-Pilot continued to develop. It improves accuracy and stability, and adds new features, including command line support, code editing, code reviews, and the ability to generate change descriptions in pull requests. Additionally, other models other than the Open AI GPT model have also begun to be supported.
GitHub Co-Pilot Features
Current GitHub CoPilot features include generating code suggestions as you type in the IDE, chatting about code and related topics such as algorithms or data structures, and support for command line usage. If you’re using an enterprise subscription, CoPilot can generate change descriptions from pull requests and maintain a knowledge base to use as context for chat. Additionally, several features for CoPilot Workspace are included in a preview version, which we will cover in a moment.
If the integrated development environment (IDE) you are using is supported (see the next section for a list of supports), you can start using GitHub Co-Pilot with your IDE today. You can use CoPilot as a chat interface on GitHub Mobile for Android, iOS, and iPadOS. It is also available through a chat interface on the GitHub website, and is currently marked as ‘beta’. If you have a business or enterprise subscription, administrators will be provided with additional controls, logs, and reports.
GitHub CoPilot is generally updated monthly, but improvements are not always made with each update. There have been several cases where the utility measured by benchmarks has fallen in reverse. This phenomenon appears to occur when the model is trained with more programming languages or frameworks, and when it is trained to eliminate some paths that deviate from the normal track. In some cases, changes resulting from updates may be noticeable in daily use, and in other cases, they may not. Sometimes big improvements are made. For example, when the context originally included only the active file and was changed to include all open IDE files, significant improvements were made when OpenAI upgraded its underlying model to a new generation of GPT.
GitHub Co-Pilot and editor integration
GitHub CoPilot is integrated and officially supported with Azure Data Studio, JetBrains IDE, Vim/Neovim, Visual Studio, and Visual Studio Code. Emacs, Eclipse, and Xcode are supported unofficially. Official support for Apple Xcode was announced on GitHub Universe in October 2024.
GitHub Co-Pilot works in several ways. suggest inline codeYou can do it. Giving a descriptive function name creates a function that actually works, at least in some cases. It doesn’t work well when you don’t have much context to draw on, but it usually works well when you have a lot of similar code available in open files or training corpora.
The same conditions apply when creating code blocks from inline comments. Also, being specific about what you want can be quite helpful. If you say something vague like “sort the list,” you can choose to use any known sorting algorithm, such as bubble sort. If you say “sort the list in memory using the quicksort algorithm with random pivot points, switching to insertion sort for short run intervals,” it will most likely execute exactly as requested, and will be much more efficient than bubble sort.
Test generation is generally easier to automate than initial code generation. GitHub Co-Pilot often generates a usable suite of unit tests in just one or two attempts from an obscure comment containing the word “test”, especially if you have an existing test suite open elsewhere in the editor. You’ll also usually receive hints for additional unit tests. However, there is a lot of repetitive code that needs to be refactored. Refactoring often works better in co-pilot chat. Also, co-pilot integration testingbut this may require providing hints about the scope, mock objects, specific functions to be tested, and any validations needed.
A downside to using GitHub CoPilot to generate code or tests is the lack of consistency or reliability. The code that a co-pilot (or any other LLM) generates requires careful scrutiny, such as when looking at a pull request from an unknown programmer or a check-in from a new programmer with a drinking habit, and you should also plan on doing a lot of code reviews and testing. .
The latest open AI model o1 familyis much better in this regard than the existing GPT family, but it is not completely free from hallucinations.
GitHub Co-Pilot Chat
The editor’s GitHub CoPilot Chat lets you suggest code, explain code, create unit tests, and suggest code modifications. You can also iterate together like a real pair programmer.
In the latest GitHub Co-Pilot Chat version, @workspaceLike in front @Keywords with #fileas #with variableChat using “participant” can be specified. When submitting a query, you can select the model you want to use in Copilot Chat in the “Ask Copilot” box. Current options are GPT 4o, o1-mini (preview), and o1-preview (preview), which are all OpenAI models. In the manual, Antropic’s Claude 3.5 Sonnetis also mentioned, and it came out as a preview while writing this article. GitHub discussed the new model with the slogan “Developer Choice” Universe 2024 eventGoogle’s Gemini 1.5 Pro will also be added in the coming weeks, the speaker said.
As of Visual Studio Code version 1.95, the location of the GitHub Co-Pilot Chat sidebar has been moved from the left to the right secondary sidebar. When on the left, it had to share space with many other commonly used functions. Also available in smart actions inline, quick chat dropdown menu, and context menu. Call Co-Pilot ChatYou can do it.
In Co-Pilot Chat /help slash commandIf you request , you currently receive the following response:
GitHub Co-Pilot
You can ask general programming questions or chat with the following participants who have expertise and can help you get the job done.
- @workspace – Ask about workspace
- /explain – explains how the code in the active editor works
- /tests – Generate unit tests for selected code
- /fix – Suggest fixes for problems in selected code
- /new – Scaffold code for new files or projects in your workspace.
- /newNotebook – Create a new Jupyter notebook
- /fixTestFailure – Suggest fixes for failed tests
- /setupTests – Set up tests in your project (experimental)
- @vscode – Ask about VS Code
- /search – Create query parameters to search your workspace.
- /runCommand – Retrieves and executes commands in VS Code
- /startDebugging – Create a launch configuration and start debugging in VS Code (experimental)
- @terminal – Ask about how the terminal works
- /explain – explains something in the terminal
- @github – Search the web, search code, and get answers based on your company’s knowledge base.
You can use the following variables to provide additional context to help you understand the question.
- #selection – Current selection in the active editor
- #codebase – Search the codebase to retrieve query-related information
- #editor – source code visible in the active editor
- #terminalLastCommand – Last executed command of the active terminal
- #terminalSelection – Selection of the active terminal
- #file – Select a file in the workspace
To have a fruitful conversation, ask questions as if you were a real programmer.
- Open the file and select the most important lines you want to talk about. please show me the code.
- By asking follow-up questions, adding clarifications, providing errors, etc. Refine your conversation.
- Review the code I suggestedIf you tell us any problems or improvements that need to be made, we can create it again.
You can also ask questions about your editor choice by starting an inline chat session (⌘KI).
Learn more about GitHub Co-Pilot in Visual Studio Code.
GitHub Co-Pilot Edit (Preview)
My Visual Studio Code (both Standard and Insider versions) has GitHub Copilot Edits, but it’s not listed in the documentation. This feature was demonstrated during the GitHub Universe 2024 keynote.
Basically, GitHub CoPilot Edit asks you to define a set of files you want to work with and then define the changes you want to make. Co-Pilot Edit, like Co-Pilot Chat, runs on the right sidebar. The biggest difference between the two is that Co-Pilot Edit allows you to change multiple files, while Co-Pilot Chat does not. However, Co-Pilot Chat can use multiple files in the context.
GitHub CoPilot Code Review (Preview)
GitHub Copilot Code Reviews offers two ways to review code and provides feedback after review. One way is to review selected code (Visual Studio code only, open public previews, all programming languages), and the other is a more in-depth way to review all changes (VS Code and GitHub websites, waiting list). There is a public preview). In an in-depth review, Custom coding guidelinescan be used. Additionally, it is currently limited to C#, Go, Java, JavaScript, Markdown, Python, Ruby, and TypeScript.
GitHub CoPilot on CLI
GitHub CLIYou can use GitHub Co-Pilot with , to get help with shell commands. step, gh The command must be installed and up to date. When you ask this command for help, it returns something like this:
Your AI command line copilot.
Usage:
copilot (command)
Examples:
$ gh copilot suggest “Install git”
$ gh copilot explain “traceroute github.com”
Available Commands:
alias Generate shell-specific aliases for convenience
config Configure options
explain Explain a command
suggest Suggest a command
Flags:
-h, –help help for copilot
–hostname string The GitHub host to use for authentication
-v, –version version for copilot
Use “copilot (command) –help” for more information about a command.
GitHub CoPilot programming language support
GitHub CoPilot’s proposed features support many programming languages and frameworks, but the best supported languages are Python, JavaScript, TypeScript, Ruby, Go, C#, and C++. This is because it is the language most frequently found in learning corpora. GitHub CoPilot can also help generate queries against databases and generate proposals for APIs, frameworks, and infrastructure as code.
GitHub CoPilot Extension (Public Preview)
Add it to your current account and enter its name in the GitHub Co-Pilot Chat with a prefix @ (e.g. @modelsclass @perplexityai) that can be called using GitHub Copilot ExtensionsThere are 27. I’ve successfully installed and authenticated these two extensions and a few others, but haven’t found anything very useful yet.
You can create a new extension if you want. The GitHub CoPilot extension is essentially a GitHub app, has additional read access to GitHub CoPilot Chat, integrates with the GitHub CoPilot API, and optionally integrates with other LLMs. Extensions can only be posted if the owner is an organizational account with Verified Creator status. In order for an app’s paid plan to be published on the GitHub Marketplace, it must be owned by an organization that is a verified publisher.
GitHub Co-Pilot Workspace (private technology preview)
GitHub Co-Pilot WorkspaceIs “AI Native” Development Environmentallows you to collaborate with GitHub Co-Pilot on coding tasks across repositories using natural language and integrated cloud computing. Co-Pilot Workspace is “task-centric.” This means you can start with a GitHub issue, a temporary task from the CoPilot Workspace dashboard, or a temporary task from the repository page. For GitHub issues, they are already defined, so you can use the “Open in Workspace” button to let CoPilot Workspace find a solution to the issue. For the other two, you must first define the draft issue yourself and then forward it to Co-Pilot Workspace for resolution.
GitHub Co-Pilot’s learning method
GitHub Co-Pilot originally used the OpenAI Codex model. The Codex model is basically GPT-3, and it is a model that has been additionally trained with many open source codes in the GitHub repository, especially Python code. Later versions used GPT-4, then GPT 4o, and now use multiple models trained using different methods.
Concerns about GitHub Co-Pilot
Early public concerns about GitHub CoPilot can be seen in a class action lawsuit filed in 2022. The plaintiffs argued that GitHub Co-Pilot violated the contract with GitHub users and violated privacy by sharing personally identifiable information. Although it was dismissed by the U.S. District Court in San Francisco in July 2024, the judge did not dismiss the plaintiff’s lawsuit for violation of the open source license agreement by all defendants.
Apple is Research report published in October 2024concluded that LLM cannot perform truly logical reasoning. Because programming requires logical reasoning, this conclusion suggests that an LLM cannot do coding. “LLM”stochastic parrotIt is in the same context as referring to it as “. Additionally, concerns were raised that students may overly rely on OpenAI Codex and similar models and plagiarize. Some argue that programming with models will make programmers stupid.
GitHub Co-Pilot’s competitors
There are currently about 10 competing GitHub Co-Pilot solutions. If we limit ourselves to models built into code editors, representative models include Tabnine, Codeium, CodeGeeX, Replit Ghostwriter, Devin AI, and Jetbrains AI ( JetBrains AI), Sourcegraph Cody, and Amazon Q Developer. If we broaden the definition of competition to include chat models that can generate code, models from OpenAI, Antropic, Google, Mistral, Meta, and many other companies are also considered. There are also Visual Studio Code alternatives like Zed and Cursor, and “next-generation” AI coding products like Solver and Zencoder.
Leading up to the GitHub Universe 2024 conference, I was wondering whether GitHub CoPilot was being overtaken by new, more powerful alternatives, including Tab9, Amazon Q Developer, or Zed, Cursor, Solver, and Zencoder. Now, I’m curious whether among these alternatives, there is a product that can surpass VS Code and GitHub Co-Pilot. Of course, this does not mean that it will not be a competitor, and we will have to continue to monitor it in the future.
editor@itworld.co.kr
Source: www.itworld.co.kr
