Comment by ByteAtATime
12 hours ago
> LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).
Now that a larger proportion of text is AI, I'm appreciating human-written text more and more; this got a good chuckle out of me.
>and whatever the hell Angular is
I've definitely seen LLMs compose jokes with this structure a lot where "whatever the hell ___ is" is in parenthesis. Not saying this is AI generated tho
I wouldn’t be so sure that that’s human text
I can reliably get my custom models to output a 95% perfect replication of my tone and voice
As somebody who attempted that recently, I seriously doubt that percentage :D
Maybe you just have a voice that's, shall we say, in sparse latent space? ;)
1 reply →
I have an insane amount of writing output as training data going back a loooong time so it all comes down to how much training data do you have
1 reply →
What do you mean by your custom models? Do you mean custom prompting?
Guessing - finetuned? Or LoRA?
EDIT: Nevermind, see below. I would definitely go with a LoRA though.
1 reply →
Couple things:
I have a “style guide” markdown that is pretty expansive
I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps
Then I will manually clean up that last 5% after a half dozen rounds
That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output
It’s been three years of tuning I’d suspect at this point
2 replies →