Comment by arjie
14 hours ago
The advantage of a turn-by-turn flow (honestly the chat interface was genius) is that with a little human intervention you can get much better results out of even silly models. The model of choice is strange, though. Not a particularly good one.
People apparently like Granite for RAG specifically. I'm not aware of any evaluations with numbers though, so I'm not sure if it's a real difference or just vibes.
Either it depends a lot on the model, or I’m bad at chat, or both, because in my experience once a chat starts going off the rails it’s pretty hard to correct the initial misconception. I find it works better to backtrack to just before the misconception and edit my previous response to head it off at the pass.
Definitely agreed that it's better to not have some text in the context than to have some that needs to be opposed. I suppose I meant steering prompts are a non-obvious innovation to LLM interaction.