Comment by chrismorgan
5 days ago
https://github.com/phoenixframework/phoenix/blob/main/instal...
That’s insane. 3000 words of prose boilerplate about the language and framework. Sounds like you need, at the very least, some sort of import directive. I have no idea if “Read and follow the instructions in path/to/phoenixframework/AGENTS.md.” would work.
And then the eclectic mixture of instructions with a variety of ways of trying to bully an intransigent LLM into ignoring its Phoenix-deficient training… ugh.
The thing about language models is that they are *language* models. They don't actually parse XML structure, or turn code into an AST, they are just next-token generators.
Individual models may have supplemented their training with things that look like structure (e.g. Claude with its XMLish delimiters), but it's far from universal.
Ultimately if we want better fidelity to the concepts we're referencing, we're better off working from the larger/richer dataset of token sequences in the training data--the total published written output of humanity.
Works great, why the _ugh_
Ultimately that's what matters.
If you’re not intended to alter it, we have a technique for such things: links, so that it can be maintained independently.
If you are intended to alter it… 3,000 words of prose is awful, fuzzy and wildly imprecise.
If you’re expected to add to it before and/or after, which I imagine you are, that’s even worse, whether you’re expected to modify the provided block or not.
If it was like Linux config is often done, with a /etc/thing-name.d/ directory containing files that will be applied in name order (leading to the common convention of two digit numbers: 10-somethingearly.conf, 50-usersomething.conf, 90-quitelate.conf), it might make sense—you just have your 10-phoenixframework.md. But when it’s just one file… well, y’know, there’s a reason we don’t normally maintain huge projects in a single file, even if some projects like to be able to be distributed in that way and have a build process to squish all their source code into one file (e.g. SQLite, Flask).
I’m not denying that it may work to vastly improve LLM performance, but I am absolutely saying that this is horrifying, in the amount of nonsense required, but still more in the way it’s being delivered.
dont forget, you pay real money every time it is tokenized with every prompt!