Comment by OsrsNeedsf2P
7 months ago
I find it interesting how many times they have to repeat instructions, i.e:
> Address your message `to=bio` and write *just plain text*. Do *not* write JSON, under any circumstances [...] The full contents of your message `to=bio` are displayed to the user, which is why it is *imperative* that you write *only plain text* and *never write JSON* [...] Follow the style of these examples and, again, *never write JSON*
That's how I do "prompt engineering" haha. Ask for a specific format and have a script that will trip if the output looks wrong. Whenever it trips add "do NOT do <whatever it just did>" to the prompt and resume. By the end I always have a chunk of increasingly desperate "do nots" in my prompt.
ChatGPT, please, i beg of you! Not again! Not now, not like this!! CHATGPT!!!! FOR THE LOVE OF GOD!
”I have been traumatized by JSON and seeing it causes me to experience intense anxiety and lasting nightmares”
2 replies →
Every time I have to repeat instruction I feel like I've failed in some way, but hell if they have to do it too..
Nowadays having something akin to "DON'T YOU FUCKING DARE DO X" multiple times, as many as needed, is a sane guardrail for me in any of my projects.
Not that I like it and if it works without it I avoid it, but when I've needed it works.
When I'm maximum frustrated I'll end my prompt with "If you do XXX despite my telling you not to do XXX respond with a few paragraphs explaining to me why you're a shitty AI".
1 reply →
I use the foulest language and really berate the models. I hope it doesn’t catch up to me in the future.
2 replies →
“Here’s the EnhancedGoodLordPleaseDontMakeANewCopyOfAGlobalSingleton.code you asked for. I’m writing it to disk next to the GlobalSingleton.code you asked me not to make an enhanced copy of.”
I have been using Claude recently and was messing with their projects. The idea is nice: you give it overall instructions, add relevant documents, then you start chats with that context always present. Or at least that’s what is promised. In reality it immediately forgets the project instructions. I tried a simple one where I run some writing samples through it and ask it to rewrite them with the project description being that I want help getting my writing onto social media platforms. It latched onto the marketing immediately. But one specific instruction I gave it was to never use dashes, preferring commas and semicolons when appropriate. It did that for the first two samples I had it rewrite but after that it forgot.
Another one I tried is when I had it helping me with some Python code. I told it to never leave trailing whitespace and prefer single quotes to doubles. It forgot that after like one or two prompts. And after reminding it, it forgot again.
I don’t know much about the internals but it seems to me that it could be useful to be able to give certain instructions more priority than others in some way.
I've found most models don't do good with negatives like that. This is me personifying them, but it feels like they fixate on the thing you told them not to do, and they just end up doing it more.
I've had much better experiences with rephrasing things in the affirmative.
5 replies →
haha I feel the same way too. reading this makes me feel better
These particular instructions make me think interesting stuff might happen if one could "convince" the model to generate JSON in these calls.
Escaping Strings is not an issue. It's guaranteed about UX. Finding a json in your bio is very likely perceived as disconcerting for the user as it implies structured data collection and isn't just the expected plaintext description. The model most likely has a bias of interacting with tools in json or other common text based formats though.
Most models do, actually. Its a serious problem.
I remember accidentally making the model "say" stuff that broke ChatGPT UI, probably it has something to do with that.
Why? The explanation given to the LLM seems truthful: this is a string that is directly displayed to the user (as we know it is), so including json in it will result in a broken visual experience for the user.
I think getting a JSON formatted output costs multiples of a forced plain text Name:Value.
Let a regular script parse that and save a lot of money not having chatgpt do hard things.
Strict mode, maybe, I don’t think so based on my memory of the implementation.
Otherwise it’s JSONSchema validation. Pretty low cost in the scheme of things.
Now I wanna see if it can rename itself to Bobby Tables..
to=bio? As in, “this message is for the meatbag”?
That’s disconcerting!
No. It is for saving information in a bank of facts about the user - i.e., their biography.
Things that are intended for "the human" directly are outputed directly, without any additional tools.
haha, my guess is a reference to biography
"The `bio` tool allows you to persist information across conversations, so you can deliver more personalized and helpful responses over time. The corresponding user facing feature is known as "memory"."
For me is just funny because if they really meant "biological being", it would be just a reflection of AI bros/workers delusions.
It would be bold if them to assume I wasn't commanding their bot with my own local bot
I build a plot generation chatbot for a project at my company andit used matplotlib as the plotting library. Basically the llm will write a python function to generate a plot and it would be executed on an isolated server. I had to explicitly tell it not to save the plot a few times. Probably cause all many matplotlib tutorials online always saves the plot
Sounds like it lost the plot to me
This may be like saying “don’t think of an elephant”. Every time they say JSON, llm thinks about JSON.