Preferred form of modification

> But (as I understand it) coding assistants are not just an LLM with an input string and an output string.

Right, because directly using the output of an LLM is like recording the output of someone's talking after asking them a question. You get the requests for clarification, side-tangents, ums and ahs, etc. You don't record someone talking, you ask them to write down their answer and let them refine it a few times. And you record that.

There are frameworks that do this, where you have the output stream of "talking" from the LLM and it also has an editor where it can write code, and rewrite it as required. But that's just making the LLM one cog in a larger machine, which makes the whole discussion about only LLMs kind of pointless.

For me LLMs feel like what happens in my mind between forming a thought and opening my mouth to produce grammatically correct sentences to convey a thought. They are a State -> Words transformer, that's all.