Comment by AdieuToLogic
2 days ago
A perspective which has helped me is viewing LLM-based offerings strictly as statistical document generators, whose usefulness is entirely dependent upon their training data set plus model evolution, and whose usage is best modeled as a form of constraint programming[0] lacking a formal (repeatable) grammar. As such, and when considering the subjectivity of natural languages in general, the best I hope for when using them are quick iterations consisting of refining constraint sentence fragments.
Here is a simple example which took 4 iterations using Gemini to get a result requiring no manual changes:
# Role
You are an expert Unix shell programmer who comments their code and organizes their code using shell programming best practices.
Create a bash shell script which reads from standard input text in Markdown format and prints all embedded hyperlink URL's.
The script requirements are:
- MUST exclude all inline code elements
- MUST exclude all fenced code blocks
- MUST print all hyperlink URL's
- MUST NOT print hyperlink label
- MUST NOT use Perl compatible regular expressions
- MUST NOT use double quotes within comments
- MUST NOT use single quotes within comments
EDIT:
For reference, a hand-written script satisfying the above (excluding comments for brevity) could look like:
#!/usr/bin/env bash
perl -ne 'print unless /^```/ ... /^```/' |
sed -e 's/`[^`]*`//g' |
egrep -o '\[.+?\]\(.+?\)' |
sed -e 's/^.*(//' -e 's/)$//'
Doesn't look like your handwritten example follows the direction on commenting.
(While we're at it, there's no need for an apostrophe when pluralising an initialism, like "URLs".)
> Doesn't look like your handwritten example follows the direction on commenting.
Please read posts fully before replying:
One hack is to end the prompt with: following solid architecture principles.
EG: Step 1: Define problem in PROBLEM.md Step 2: Ask agent to gather scope from codebase and update PROBLEM.md Step 3: Ask agent to create a plan following design and architecture best practices (solid, etc) and update PROBLEM.md Step 4: Ask agent to implement PROBLEM.md
Do you get tangibly different results if you don't capitalize MUST (NOT)?
> Do you get tangibly different results if you don't capitalize MUST (NOT)?
My use of MUST/MUST NOT is from experience with various web specification documents and is capitalized more for visual recognition than any other reason. Having only used this form, I cannot answer your question as to document generation impact. I would imagine use of Markdown emphasis, such as:
Would have some effect though. Perhaps both even more?
I asked because I've seen other folks mention capitalizing things for emphasis so the LLM follows instructions more closely (which is one of those things that make me feel like the UX around LLMs is bananas).