Comment by sorenjan
3 years ago
This is really interesting. Like mentioned in the article, this is a kind of RLHF, and that's what takes GPT3 from a difficult to use LLM to a chat bot which is able to confuse some people into thinking is has consciousness. It makes it much more usable.
I don't know how these models are trained, but hopefully future models will include bad results as negative training data, baking it into the base model.
It's only mentioned in passing in the article, but apparently it's possible to merge LoRAs? How would you do that, I'd like to use one LoRA to include my own subjects, this LoRA to make the results better, and maybe a third one for a particular style.
Merging LoRAs is essentially taking a weighted average of the LoRA adapter weights. It's more common in other UIs.
diffusers is working on a PR for it: https://github.com/huggingface/diffusers/pull/4473