AI review?

What I think would be far more useful is an LLM-as-LSP setup where, instead of conforming to a chat interface, I can have an LLM "looking over my shoulder" and adding annotations directly to my editor as I'm working. If it then feeds suggested changes as LSP code actions, I don't have to pause any editing for the LLM to crunch its numbers so that its suggestions don't conflict. Even if the suggestions are just comments like "this codepath should be tested", I can later prompt it to try and create a skeleton test case for it.

But I really don't know how to do this in a token-efficient manner when forced into a chat-shaped API. Do you send the file and some kind of patch sequences on a timer so you're not tossing the entire context at it for each edit? Pare it down to `ed` commands and send that as the patch sequence?

At least to me, it sounds far more interesting than an LLM puking out hunks of unreviewed code at me at least (as someone who cares about long-term maintenance at least).