I've worked with teams that used Copilot. They claim it's great "Hey, now I don't have to actually spend any time writing all this boilerplate!" while for me, the person who has to review their code before releasing stuff, easier ways of writing boilerplate is not a positive, it's a negative.
If writing boilerplate becomes effortless, then you'll write more of it, instead of feeling the pain of writing it and then trying to reduce it, because you don't want to spend time writing it.
And since Copilot was accepted as a way to help the developers on the teams, the increase of boilerplate have been immersive.
I'm borderline pissed, but mostly at our own development processes, not at Copilot per se. But damn if I didn't wish it existed somehow, although it was inevitable it would at one point.
I feel ya. If your job is to kick back bad code, and now there is a tool that generates bad code, how does this not make your job more important?
Why not get some of the freed up, Copilot augmented developer labor budget moved to testing and do more there or build more tools to make your personal, boilerplate, repetitive tasks more efficient?
If the coders are truly just dumping bad code your way, that's an externality and the cost should be called out.
I use github copilot on a daily basis and it improves my time from thinking to code.
Often I have times where I'm think about a specific piece of code that I need and I have it partially in my head and github copilot "just completes" it. I press tab and that's it.
I'm not talking about writing entire functions where you have to mentally strain yourself to understand what it wrote.
But I've never seen any autocompleter do it so good then github copilot. Even for documentation purposes like JSdoc and related commenting system it's amazing.
It's a tool I pay for now since it's proven to be a tool that increases my productivity.
Is it gonna replace us? I hope not, but it does look promising as one of those tools people will talk about in the future.
It would be helpful if people could include in their assessment roughly how much time they've personally spent using these tools.
Helping write boilerplate is to Copilot what cropping is to Photoshop.
Some of the ways I've found Copilot a powerful tool in my toolbox: Writing missing comments (especially unfamiliar code bases), "translating" parts of unfamiliar code to a more familiar language, suggesting ideas for how to implement a feature (!) in comments.
Has it really? Or are you worried that this is something that will happen?
Of course I don't know how other people use it but I find that it's very much like having a fairly skilled pair programmer on board. I still need to do a lot of work but I get genuine help. I don't find that I personally write more boilerplate code than before, every programming principle applies as it always has.
I wrote it in past tense, it's based on actual situations :) If you don't believe what I write, I guess it doesn't matter what I write now. Regardless.
One simple example that I've had to reject more than once.
- Function 1 does something
- Developer needs something like Function 1 but minor change
- Developer starts typing name of function which has a similar name to Function 1, but again, minor difference
- Copilot helpfully suggests copy-pasting Function 1 but with the small change incorporated
- Developer accepts it, commits and sends the patch my way
Rather than extracting the common behavior into it's own function and call that from both of them, refactors which Copilot doesn't suggest, the developers is fine with just copy-pasting the function.
Now we have to maintain two full slightly different functions, rather than 1 full functions + 2 minor ones.
Obviously a small example, and it wouldn't be worth extracting it the first time it happens or on a smaller scale. But once you have entire teams doing something like this, it becomes a bit harder to justify copy-paste approach, especially when you want the codebase not to evolve to complete spaghetti.
And finally, I'm not blaming the tool, it's not Copilots fault. But it does seem to have made developers who rely on it think less, compared to the ones that don't.
What I have seen about it ranged from things that can be nearly just as well handled by your $EDITOR's snippet functionality to things where my argument kicked in - I have to verify this generated code does what I want, ergo I have to read and understand something not written by me. Paired with the at least somewhat legally and ethical questionable source of the training data, this is not for me.
So stop evangelizing about stuff you haven’t used. Understanding code is easier than writing it from the scratch. That’s why code review doesn’t take as much time as writing code and you still need to prove your code works, even if you wrote it yourself.
Understanding code is only easier for simple tasks. I've definitely had copilot spit out complex algorithms that looked right at first glance but actually had major issues that required me to write it from scratch.
I contend understanding the semantics of code is harder than writing the syntax. Reading the syntax without thinking deeply (to the level needed to write it, or deeper) seldom helps you realize unexpected corner cases. This is why stochastic testing is so valuable.
ad.: Code review takes less time than writing code for the same reason reading a book takes less time than writing one. Distillation and organization of ideas requires expertise gained through experience and long thought. Reading a book requires reading ability.
Understanding a book (and the intricacies underlying it) takes effort on the order of the original writing, but most people don't seek that level of understanding. The same is true of code.
Code review often takes me longer than writing code. More generally, reading other people's code is more difficult for me than writing (or reading) my own.
I've used it quite a lot and I agree with the original post. It seemed really useful at first but then it started introducing several bugs in large blocks of code. I've stopped using it in the end since the small snippets on the one line size is trivial enough to write myself (with just vim proficiency) and the larger blocks on the order of a function autocomplete is too bug prone (and kills too much willpower budget to fix).
Yep. I’m personally skeptical of so many other use cases for LLMs but CoPilot is fantastic and basically just autocomplete on rocket fuel. If you can use autocomplete, you can use CoPilot super effectively.
I almost always turn autocomplete off except in circumstances where the API has bad documentation. I also found that copilot was an aggravation more than a help after using it for a couple weeks.
We programmers enjoy writing code. We derive satisfaction when a code is perfect and elegant. But its going to end very soon. Artists are freaking out because things that takes them days to create now only take 2 seconds. We are next.
The writing is on the wall. Programming as we know it is going to end. We should be embracing these tools and should start moving from software developers to software architects role.
This is such a bullshit answer. No, I don't use it daily because I tried it for a couple hours and it suggested nothing useful and several harmful. Why would I keep using it?
I've used it and it did nothing helpful. I also find autocomplete slows me down. The code it suggested always needed enough reworking I would have been faster writing it out from scratch. It's just not that helpful for me. Maybe if I didn't know the apis that well but I suspect even then it would be as much a liability as a benefit
90% of the time CoPilot-bros stop at this point without giving any good examples of how this post-autocomplete monster helped them. Autocomplete works quite well in most use cases - it is low effort, free and most importantly, ethical. CoPilot on the other hand jumps through so many hoops to generate something marginally and arguably better, but at the cost of what? This is exactly like the Search Vs ChatGPT problem - do you want a deterministic, algorithmic, fine tuned experience or some random probabilistic, overconfident crap.
I don't find basic autocomplete useless, although I probably use it less than most people. (For example, I rarely write Java and when I do I don't use long names or deeply nested structures and I don't implement equals etc. by default.) I think people use autocomplete in two ways; when I use it I know what I want to be in the code and it's a way to type 30 characters by pressing 3 keys or whatever. But I also see a lot of people use it like "I don't know what to do next, what methods are available?" And this is usually to the detriment of the code quality.
Copilot is not like autocomplete. It only works in the second mode, because any nontrivial code it generates needs to be read, considered, and understood. (And any trivial code it generates can be done by autocomplete or long-existing non-AI tools.) This is especially true given LLMs' hallucinatory behavior - by definition it will often spit out something that "looks right" even if it's absolutely not - and such code is harder to review than code that looks obviously wrong.
So if you do use autocomplete in the second mode, maybe you find Copilot a super-powered version of that. And if you have the same weaknesses as Copilot, reviewing its code after it's done writing it is probably not any different than reviewing your own code after writing it, so for you it takes the same amount of time. For me, that's not the case.
When I used it, and when I see others use it, Copilot is like an impossibly overenthusiastic junior developer I will never be able to teach better habits to.
> They replace the default autocomplete in a way which largely unnoticeable, but surprisingly effective at complex autocomplete tasks.
I've yet to see it. It's barely above IDEAs autocomplete in the rare cases when it manages to trigger on my code, and it has already been wrong more than once in the few times it did deign to provide autocompletion.
I've worked with teams that used Copilot. They claim it's great "Hey, now I don't have to actually spend any time writing all this boilerplate!" while for me, the person who has to review their code before releasing stuff, easier ways of writing boilerplate is not a positive, it's a negative.
If writing boilerplate becomes effortless, then you'll write more of it, instead of feeling the pain of writing it and then trying to reduce it, because you don't want to spend time writing it.
And since Copilot was accepted as a way to help the developers on the teams, the increase of boilerplate have been immersive.
I'm borderline pissed, but mostly at our own development processes, not at Copilot per se. But damn if I didn't wish it existed somehow, although it was inevitable it would at one point.
I feel ya. If your job is to kick back bad code, and now there is a tool that generates bad code, how does this not make your job more important?
Why not get some of the freed up, Copilot augmented developer labor budget moved to testing and do more there or build more tools to make your personal, boilerplate, repetitive tasks more efficient?
If the coders are truly just dumping bad code your way, that's an externality and the cost should be called out.
I use github copilot on a daily basis and it improves my time from thinking to code.
Often I have times where I'm think about a specific piece of code that I need and I have it partially in my head and github copilot "just completes" it. I press tab and that's it.
I'm not talking about writing entire functions where you have to mentally strain yourself to understand what it wrote.
But I've never seen any autocompleter do it so good then github copilot. Even for documentation purposes like JSdoc and related commenting system it's amazing.
It's a tool I pay for now since it's proven to be a tool that increases my productivity.
Is it gonna replace us? I hope not, but it does look promising as one of those tools people will talk about in the future.
It would be helpful if people could include in their assessment roughly how much time they've personally spent using these tools.
Helping write boilerplate is to Copilot what cropping is to Photoshop.
Some of the ways I've found Copilot a powerful tool in my toolbox: Writing missing comments (especially unfamiliar code bases), "translating" parts of unfamiliar code to a more familiar language, suggesting ideas for how to implement a feature (!) in comments.
>the increase of boilerplate have been immersive
Has it really? Or are you worried that this is something that will happen?
Of course I don't know how other people use it but I find that it's very much like having a fairly skilled pair programmer on board. I still need to do a lot of work but I get genuine help. I don't find that I personally write more boilerplate code than before, every programming principle applies as it always has.
I wrote it in past tense, it's based on actual situations :) If you don't believe what I write, I guess it doesn't matter what I write now. Regardless.
One simple example that I've had to reject more than once.
- Function 1 does something
- Developer needs something like Function 1 but minor change
- Developer starts typing name of function which has a similar name to Function 1, but again, minor difference
- Copilot helpfully suggests copy-pasting Function 1 but with the small change incorporated
- Developer accepts it, commits and sends the patch my way
Rather than extracting the common behavior into it's own function and call that from both of them, refactors which Copilot doesn't suggest, the developers is fine with just copy-pasting the function.
Now we have to maintain two full slightly different functions, rather than 1 full functions + 2 minor ones.
Obviously a small example, and it wouldn't be worth extracting it the first time it happens or on a smaller scale. But once you have entire teams doing something like this, it becomes a bit harder to justify copy-paste approach, especially when you want the codebase not to evolve to complete spaghetti.
And finally, I'm not blaming the tool, it's not Copilots fault. But it does seem to have made developers who rely on it think less, compared to the ones that don't.
you could stop doing code reviews and do something else
I haven't. Now you know for a fact :)
What I have seen about it ranged from things that can be nearly just as well handled by your $EDITOR's snippet functionality to things where my argument kicked in - I have to verify this generated code does what I want, ergo I have to read and understand something not written by me. Paired with the at least somewhat legally and ethical questionable source of the training data, this is not for me.
So stop evangelizing about stuff you haven’t used. Understanding code is easier than writing it from the scratch. That’s why code review doesn’t take as much time as writing code and you still need to prove your code works, even if you wrote it yourself.
Understanding code is only easier for simple tasks. I've definitely had copilot spit out complex algorithms that looked right at first glance but actually had major issues that required me to write it from scratch.
2 replies →
I contend understanding the semantics of code is harder than writing the syntax. Reading the syntax without thinking deeply (to the level needed to write it, or deeper) seldom helps you realize unexpected corner cases. This is why stochastic testing is so valuable.
ad.: Code review takes less time than writing code for the same reason reading a book takes less time than writing one. Distillation and organization of ideas requires expertise gained through experience and long thought. Reading a book requires reading ability.
Understanding a book (and the intricacies underlying it) takes effort on the order of the original writing, but most people don't seek that level of understanding. The same is true of code.
I am not evangelizing, I am just stating why this is not for me and my way to write software.
Code review often takes me longer than writing code. More generally, reading other people's code is more difficult for me than writing (or reading) my own.
I've used it quite a lot and I agree with the original post. It seemed really useful at first but then it started introducing several bugs in large blocks of code. I've stopped using it in the end since the small snippets on the one line size is trivial enough to write myself (with just vim proficiency) and the larger blocks on the order of a function autocomplete is too bug prone (and kills too much willpower budget to fix).
Yep. I’m personally skeptical of so many other use cases for LLMs but CoPilot is fantastic and basically just autocomplete on rocket fuel. If you can use autocomplete, you can use CoPilot super effectively.
I almost always turn autocomplete off except in circumstances where the API has bad documentation. I also found that copilot was an aggravation more than a help after using it for a couple weeks.
We programmers enjoy writing code. We derive satisfaction when a code is perfect and elegant. But its going to end very soon. Artists are freaking out because things that takes them days to create now only take 2 seconds. We are next.
The writing is on the wall. Programming as we know it is going to end. We should be embracing these tools and should start moving from software developers to software architects role.
1 reply →
This is such a bullshit answer. No, I don't use it daily because I tried it for a couple hours and it suggested nothing useful and several harmful. Why would I keep using it?
[flagged]
I've used it and it did nothing helpful. I also find autocomplete slows me down. The code it suggested always needed enough reworking I would have been faster writing it out from scratch. It's just not that helpful for me. Maybe if I didn't know the apis that well but I suspect even then it would be as much a liability as a benefit
90% of the time CoPilot-bros stop at this point without giving any good examples of how this post-autocomplete monster helped them. Autocomplete works quite well in most use cases - it is low effort, free and most importantly, ethical. CoPilot on the other hand jumps through so many hoops to generate something marginally and arguably better, but at the cost of what? This is exactly like the Search Vs ChatGPT problem - do you want a deterministic, algorithmic, fine tuned experience or some random probabilistic, overconfident crap.
I don't find basic autocomplete useless, although I probably use it less than most people. (For example, I rarely write Java and when I do I don't use long names or deeply nested structures and I don't implement equals etc. by default.) I think people use autocomplete in two ways; when I use it I know what I want to be in the code and it's a way to type 30 characters by pressing 3 keys or whatever. But I also see a lot of people use it like "I don't know what to do next, what methods are available?" And this is usually to the detriment of the code quality.
Copilot is not like autocomplete. It only works in the second mode, because any nontrivial code it generates needs to be read, considered, and understood. (And any trivial code it generates can be done by autocomplete or long-existing non-AI tools.) This is especially true given LLMs' hallucinatory behavior - by definition it will often spit out something that "looks right" even if it's absolutely not - and such code is harder to review than code that looks obviously wrong.
So if you do use autocomplete in the second mode, maybe you find Copilot a super-powered version of that. And if you have the same weaknesses as Copilot, reviewing its code after it's done writing it is probably not any different than reviewing your own code after writing it, so for you it takes the same amount of time. For me, that's not the case.
When I used it, and when I see others use it, Copilot is like an impossibly overenthusiastic junior developer I will never be able to teach better habits to.
> They replace the default autocomplete in a way which largely unnoticeable, but surprisingly effective at complex autocomplete tasks.
I've yet to see it. It's barely above IDEAs autocomplete in the rare cases when it manages to trigger on my code, and it has already been wrong more than once in the few times it did deign to provide autocompletion.
Hey, maybe don't outright call people liars about their own lived experiences just because they don't agree with yours...?
I can say with a higher degree of confidence that you haven't actually used CoPilot daily for any respectably sized project.
oops, you are wrong :)