GitHub announced last week that it would release Copilot, its AI pair programming tool, to the general public. Copilot uses AI to provide a wide range of assistance functions, including auto-complete instructions, generating entire functions, and translating docstrings and descriptions into functional source code.
Copilot was launched as a technical preview in 2021. All developers may now apply for the program, which extends into integrated development environments (IDEs) such as Visual Studio, VS Code, Neovim, and JetBrains.
There was excitement about Copilots' incredible coding abilities. However, there were also concerns about how reliable its capabilities are and whether it has an impact on developers' productivity. After a year and billions of lines of code, Copilot is finally ready to go into the hands of every developer.
Here's what we know about Copilot's effect on real programming tasks, told by its creators and developers who have used it in their everyday work.
Join us at the leading conference on applied AI for enterprise business and technology decision makers on July 19 and 20-28.
With Copilot, how much code is written?
Copilot is built on OpenAIs Codex, a transformer that has been trained on tens of millions of code repositories. It provides suggestions based on existing code and cues, such as the names of your functions and classes and the comments in your code.
Copilot does not think and code like a programmer. However, as it has been exposed to huge amounts of source code, it can provide very useful code suggestions, especially on frequently encountered tasks.
More than 1.2 million people have used Copilot's technical preview in the past 12 months, according to GitHub. In files where Copilot is enabled, it accounts for nearly 40% of popular programming languages like Python.
Distractions are reduced by Github Copilot.
Salva noted that developers often emphasize the importance of staying in the flow and the positive impact [Copilot] has on their productivity, creativity, and overall happiness.
According to Salva, over 75% of developers self-reported that when using Copilot, they spend less time on repetitive programming tasks, focus on more satisfying work, and stay in the flow.
Salva said that by minimising distractions and focusing on work, we not only get work done, but we also have better and less stressful days. Weve heard stories of developers using Copilot to learn new programming languages, quickly generate boilerplate code for common tasks, write regular expressions, or simply recall the syntax for an API without needing to consult documentation.
When it comes to programming, I want the least distractions. Copilot has been a major help in that regard. Thakur said it has reduced the time I might spend looking for solutions on the web rather than having them at my fingertips in my favorite IDE.
Many programmers are looking for solutions to minor issues on search engines and StackOverflow, a web forum where programmers may exchange code snippets for specific tasks.
After using Copilot, I rarely visit these websites and can rather concentrate on code, according to Thakur.
Thakur noted that whenever I am in doubt, I try to write comments and allow Copilot to help me finish the code chunks. The rest is up to the developer to modify and reuse. Same applies to StackOverflow: It doesnt always have the answer, but it does have a lot of good answers that might work for your application.
With Copilot, you can tackle repetitive tasks.
Louis Castricato, a research assistant at Hugging Face and previously at EleutherAI, has used Copilot for scientific computing, where functions are often cumbersome and difficult to use. One of these functions is PyTorchs einsum, which requires complicated parameters about the dimensions of tensors you want to compute.
Castricato said Einsum is a complex concept for people who have never used it before, and it requires you to pay close attention to the shape that your tensors are taking at every instruction within a call to your model. Copilot is particularly adept at inferring the shape of a tensor and automatically writing einsum operations, as well as writing observations explaining the choices it made in writing the einsum expression.
Copilot is used by Aporia's R&D director for over a year. Given suitable naming conventions, Copilot is able to complete the whole function/next block of code. After that, I only need to review to see that everything is okay (and possibly add more specific logic).
Copilot's performance is particularly noticeable when it comes to long-running tasks. An example is launching an HTTP server, which typically requires several lines of code and adjustments, depending on the language it is being written in. In one study, GitHub required half of the participants to write the HTTP server code manually and the other half to complete the task using Copilot.
According to preliminary research, developers are not only more likely to complete their task when using Copilot, but they also do it in around half the time.
Pushing developers to better document code
When programmers provide more detailed descriptions, Copilot works better. Interestingly, using it has pushed developers to better document their code.
Copilot has helped people improve their behaviors when it comes to writing code, namely by using better, more succinct comments, according to Salva. This is not only because Copilot suggestions are improved, but also because it makes it easier to read for others.
Copilot has also become a handy tool for documenting software code, a task that is often overlooked, especially when programmers are striving for deadlines.
Castricato utilizes Copilot to document his code, automatically compiling docstrings and typing suggestions in Python. This improves the readability of the code and makes it easier for him and other developers to manage the code later.
According to Copilot, the amount of documentation I need to write for my code has increased by at least two or three times.
Castricato noted that [Copilot] is often a disaster at implementing complete algorithms. For example, when I first purchased Copilot, I wanted to see if it could implement basic forms of dynamic programming without much help. It failed miserably, and I very quickly realized that in order to use Copilot to its full potential, you must explain (through comments) exactly how to implement a particular algorithm.
Copilot will require plenty of input to perform its basic functions. In some instances, it will require a fully structured code file to offer useful suggestions.
Copilot is a long way from supplanting even the most modest of software engineers, according to Castricato.
Salva acknowledged that Copilot is still a work in progress and a new developer experience. The product team continues to learn lessons from how developers utilize it and are changing the AI model that powers it.
Copilot is constantly looking to understand your intent and to develop the best code it can, but the code it suggests may not always work or even make sense, according to Salva. While we are striving to improve Copilot, ideas should be carefully tested, reviewed, and vetted, like any other code.
Is Copilot worth the price?
Copilot will be offered for the time being for $10 per month, or $100 per year, with a 60-day free trial, which appears to be a bargain for software developers. In addition, the tool will be free for students and maintainers of popular open-source projects.
Thakur stated that the cost of constructing such software is worthwhile. I know that a lot goes into designing such programs, particularly Copilot, which offers ideas with a low-interval of time. In the real world, it is impossible to construct an infrastructure that would provide these kinds of services for free.
Thakur speculated that the price of Copilot may decrease in the future as AI infrastructure costs decline.
But at this point, in my opinion, it is definitely worth the price, especially for someone like me, who has been using Copilot since the preview, according to the author.
Castricato claims that Copilot has saved him many hours a week, as well as perhaps a day or two of troubleshooting per week.
Copilot allows me to test many experiments in an instant, often without having to spend significant time debugging. A set of tests that normally would take me days to implement instead takes me one day, according to the author. It certainly makes me more than $10 a month.
Salva believes that this is only the beginning of AI-augmented programming and sees Copilot as the next step in a long line of development tools.
According to him, tooling advancements have increased the amount of recognition and interaction developers have in our world. At the same time, those same tools are no substitute for a developer's expertise, skill, and creativity.
With enhanced tools, industry demand for developers has steadily increased. Were optimistic that GitHub Copilot would have similar effects, complementing the effort of developers and empowering them to write code more easily with greater focus and creativity, according to the speaker.