I write automation code for devops stuff. I’ve tried to use ChatGPT several times for code, and it has never produced anything of even mild complexity that would work without modification. It loves to hallucinate functions, methods, and parameters that don’t exist.
It’s very good for helping point you in the right direction, especially for people just learning. But at the level it’s at now (and all the articles saying we’re already seeing diminishing returns with LLMs) it won’t be replacing any but the worst coders out there any time soon.
It’s great for Pseudo code. But I prefer to use a local LLM that’s been fine tuned for coding. It doesn’t seem to hallucinate functions/methods/parameters anywhere near as much as when I was using ChatGPT… but admittedly I haven’t used ChatGPT for coding in a while.
I don’t ask it to solve the entire problem, I mostly just work with it to come up with bits of code here and there. Basically, it can partially replace stack overflow. It can save time for some cases for sure, but companies are severely overestimating LLMs if they think they can replace coders with it in its current state.
I can believe that they manage to get useful general code out of an AI, but I don’t think that it’s gonna be as simple as just training an LLM on English-code mapping. Like, part of the job is gonna be identifying edge conditions, and that can’t be just derived from the English alone; or from a lot of other code. It has to have some kind of deep understanding of the subject matter on which it’s working.
Might be able to find limited-domain tasks where you can use an LLM.
But I think that a general solution will require not just knowing the English task description and a lot of code. An AI has to independently know something about the problem space for which it’s writing code.
Context-aware AI is where it’s at. One that’s
integrated into your IDE and can see your entire codebase and offer suggestions with functions and variables that actually match the ones in your libraries. Github Copilot does this.
Once the codebase gets large enough, a lot of times you can just write out a comment and suddenly you’ll have a completely functional code block pop up underneath it, and you hit “tab” to accept it and move on. It’s a very sophisticated autocomplete. It removes tediousness and lets you focus on logic.
I write automation code for devops stuff. I’ve tried to use ChatGPT several times for code, and it has never produced anything of even mild complexity that would work without modification. It loves to hallucinate functions, methods, and parameters that don’t exist.
It’s very good for helping point you in the right direction, especially for people just learning. But at the level it’s at now (and all the articles saying we’re already seeing diminishing returns with LLMs) it won’t be replacing any but the worst coders out there any time soon.
It’s great for Pseudo code. But I prefer to use a local LLM that’s been fine tuned for coding. It doesn’t seem to hallucinate functions/methods/parameters anywhere near as much as when I was using ChatGPT… but admittedly I haven’t used ChatGPT for coding in a while.
I don’t ask it to solve the entire problem, I mostly just work with it to come up with bits of code here and there. Basically, it can partially replace stack overflow. It can save time for some cases for sure, but companies are severely overestimating LLMs if they think they can replace coders with it in its current state.
Could you recommend one?
I use this model for coding: https://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF I would recommend the one with the Q5_K_M quant method if you can fit it.
I can believe that they manage to get useful general code out of an AI, but I don’t think that it’s gonna be as simple as just training an LLM on English-code mapping. Like, part of the job is gonna be identifying edge conditions, and that can’t be just derived from the English alone; or from a lot of other code. It has to have some kind of deep understanding of the subject matter on which it’s working.
Might be able to find limited-domain tasks where you can use an LLM.
But I think that a general solution will require not just knowing the English task description and a lot of code. An AI has to independently know something about the problem space for which it’s writing code.
It’s good for doing the boilerplate code for you that’s about it… you still need a human to do the thinking on the hard stuff.
Context-aware AI is where it’s at. One that’s
integrated into your IDE and can see your entire codebase and offer suggestions with functions and variables that actually match the ones in your libraries. Github Copilot does this.
Once the codebase gets large enough, a lot of times you can just write out a comment and suddenly you’ll have a completely functional code block pop up underneath it, and you hit “tab” to accept it and move on. It’s a very sophisticated autocomplete. It removes tediousness and lets you focus on logic.