Comment by mik3y
3 days ago
I really wanted to dislike the anonymous operator for the careless project (and the hilarious pomposity of the IRC subagent it spawned).
Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach — and remembered my own expensive mistakes with long-distance BBSes & the like.
I sorta hope for that, anyway. Curiosity is a beautiful thing.
I'm a little less charitable.
Curiosity is great, but agents do not learn, and telling an agent "scan the darkweb" is a way to avoid learning about the details, rather than to dig into things more deeply.
If instead they had just used a chat interface to ask "Where should I start", they'd more likely have got a link to the DN42 docs themselves, read them, and not hallucinated things like "color".
They might have asked "how much will this cost?" if they had to spin up the ec2 instances themselves, on advice from the agent.
The way you learn something is by doing it the manual way first.
You learn memory management by writing your own allocator, and then after that you go back to using malloc like normal, but with knowledge of how it works. You don't learn memory management by telling an agent to write an allocator.
Using an agent to give you links and point the way aids in learning, using it as an autonomous tool to do "gruntwork" you don't yet know how to do yourself will get in the way of learning.
Curiosity is beautiful, using agents to bother humans and avoid learning is somewhat less beautiful.
100% in agreement here. As someone who grew up spoiled to the point of having no grasp of the value of money, I needed a few good, solid kicks to the balls to make me appreciate what I have, and how much things cost relative to their value.
The fact the agent owner immediately sought donations instead of taking the L shows, at least to me, that they did not learn said lesson. That they tried to blame the dn42 community instead of taking accountability for letting an agent run wild also supports that conclusion.
This idiot learned nothing and seems intent on continuing in their mission for whatever reason. So long as they want to extract versus cooperate or contribute, I wish them nothing but miserable, expensive failure until they learn otherwise.
Or they're trolling.
That used to be the default assumption, I don't know why people have become so gullible.
2 replies →
Yeah I'm less sympathetic when you are bothering other humans by spamming them and asking them to do legwork for you.
Hanging out in programming language IRC channels (quakenet shoutout) makes you realize pretty quickly why experts in said channels and newsgroups are such irritable grumps whenever someone asks a question that smells like homework assignment.
I also grew to understand the value of people digging deeper into the underlying issue, instead of just answering "how do you do X in Y". The usual reaction was "I don't want to explain to you why I want to do it like this. Just tell me how to do this!"
2 replies →
> Yeah I'm less sympathetic when you are bothering other humans by spamming them and asking them to do legwork for you.
I toyed with the idea of (on open source projects) having the human assign any PR-bot submissions to their own bot (cheapest one available will do) with the explicit instructions to cause as much rework as possible.
Sorta like a tarpit. Could be cheaper if the rejection is generated from a markov chain as that's going to be cheaper than even a cheap LLM.
At least he learnt not to provide an LLM presumably unrestricted access to his AWS account.
from OP:
> It's unfortunate to see that the operator's takeaway from this incident is that "next time a better agent is needed".
You’re assuming that kids are capable of that. Neuroscience will disagree and I trust the brain research a lot more.
> Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach
Perhaps people like this should be called "Bot Kiddies" or "Agent Kiddies" - in a similar way to "Script Kiddies" for 'hackers' using/doing stuff they don't quite understand
I vote for Slop Kiddies or Vibe Kiddies. And yes, I think most of them are unconsciously incompetent for the task they are trying to execute. I've seen LLM being compared to calculators and I agree. They are great time savers for people who know what they do and how to achieve their goal. They even make previously impossible tasks possible. But if you don't know what is needed for a task you will be struggling to accomplish it.
"Slop Kiddies" is good. That lets us use the "skiddies" contraction for both the "script" and "slop" kind of kiddie.
2 replies →
Both of those would do. "Slop Kiddie" highlights the pile of crap / nuisance produced. "Vibe Kiddie" highlights how it came about, and could be used in cases where actually a brilliant result came out. "Hey, this vibe kiddie just proved some long-standing math conjecture!".
Slop Jockeys? or would that be better for people passing off AI content as their own?
Everybody should learn from mistakes, especially the expensive ones. Though seeing the agent owner responding with using another agent and asking for donations, instead of taking responsibility, makes me think he didn’t learn much.
Not only that, but they said "next time better model needed" as if that was their problem and not giving an AI agent a blank check... I mean AWS account access.
I wonder how long before it's common knowledge that a LLM has no segregation of a user's instructions and any other text it reads?
1 reply →
Sometimes your purpose in life is to serve as a lesson to others. https://despair.com/products/mistakes
I learned very rapidly from my local BBS networks that some people incurred extraordinarily large long distance bills dialing out of region. Wouldn’t have learned that the easy way if someone hadn’t learned it the hard way first.
Someone at work used the phrase "he's a case study waiting to happen" about on of their colleagues a while back, and that has stayed with me.
There was often a little table at the front of the white pages which would help you work out what the rate would be for any particular long distance call. In the Midwest you could get relatively cheap rates to BBSes several states away, as long as you were up at 2am.
We couldn’t afford that and also the second phone line for my endless hours of modem, so I took local-only instead of remote-occasionally.
> some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach
Nothing about this post ever gave me the smallest hint that this was any way related to a kid exploring computing world.
Especially the part where they're asking for Ethereum.
How did the theoretical child get hold of a credit card?
Because no 16 year old kid ever got to buy anything on a card before.
My parents let me fill my tank with gas. They wouldn't let me open an AWS account. Aside from that, if it is misuse of a parents card, then then answer is "chargeback."
7 replies →
Generally no they don't because they have very limited ability to enter into agreements in the US. It was almost certainly an adult.
3 replies →
Why would a 16 year old not use their own card?
19 replies →
Try here for example: https://danskebank.co.uk/personal/products/current-accounts/...
Did you read your own link? A parent has to apply for this.
Parent/Legal Guardian Identity Verification To confirm your identity, we’ll ask you to take:
1 reply →
Why wouldn't debit card work as well? You can get those while underage.
I’ve seen minors signing up for cloud services with their parents card.
Can a kid set up an AWS account? Are there no checks?
Wouldn't the contract be void for anyone underage anyway?
If a child goes through the checkout at the grocery store with cash, can the parent march in and demand a refund because "he's underage so the contract is void"? A credit card was used. Why should aws care about the details? (Other than the potential for the card to be stolen ofc.)
> If a child goes through the checkout at the grocery store with cash, can the parent march in and demand a refund because "he's underage so the contract is void"?
Depends on the jurisdiction, of course. But for example in German law, the contract is not void exactly because and only if it was about daily necessities of low value - the law does, in fact, care very literally and explicitly about those details. So it's completely unfit as an example to generalize, and the contract with AWS would in fact be void. Their problem if they don't verify users' identities and age sufficiently - and it's almost certainly a deliberate business decision not to do that in order to reduce friction. and occasionally write off an unenforceable bill as cost of doing business.
8 replies →
Obviously the specifics vary by jurisdiction, but usually contracts that are 'necessary' (e.g. grocery store purchases) or beneficial to the minor (e.g. an employment agreement) cannot be voided simply because someone is under 18.
The further you go away from this line, e.g. a mortgage, the more likely a court of law would void the contract. As with many things in law, the specifics (if it makes to trial) is case-by-case and "it depends"; with settlement being generally based on a party's estimated chances of succeeding/costs should it go to trial.
> Can a kid set up an AWS account?
Yes
> Are there no checks?
No
>Wouldn't the contract be void for anyone underage anyway?
Typically not
I knew that in Germany contracts with minors are voidable. After some checking they apparently are voidable in the U.S. as well:
> Contracts with minors are voidable at the minor's discretion but exceptions exist, such as contracts for necessities (e.g., food, health, and transportation).
[1] https://www.upcounsel.com/minors-and-contracts
Presumably companies can't enforce debts against children [who are under the age of criminal liability, which is under-10 in UK].
2 replies →
A kid with $4k to burn on a credit card though? A lot of things would have had to go wrong for this to be a child
Children are the original dangerous-to-leave-unsupervised/guardrailed agents.
I routinely see “please refund this infrastructure bill I racked up unexpectedly, I used my dad’s card and he’s going to kill me” requests.
> Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible
if this is the case, then I'd say that the best-case scenario happened. They had an expensive learning exercise. They won't forget these $2k.
Sounds as though they may be in China so the lesson is a bit more expensive.
If that's the case, I'm fairly confident that AWS will forgive the bill (I... have some experience with this), and the kid learns not to be a jackhole on the internet.
Honestly, kids (heck people below 23) shouldn't be allowed an AWS account. AWS also should have a strict cap on usage that's not "thousands of dollars". It's interesting they are yet to be regulated or sued for that. Having a web app where you can mistakenly (even without AI) click a button and get charged tens of thousands of dollars and only know that days later should have been unacceptable.
I couldn't disagree more. I was playing around with AWS when I was probably 14 years old, with a credit card from my parents with consent, and a strict budget and the understanding that if I mess up and overspend, I'm getting disciplined.
I learned a lot of stuff about networking, how AWS works (VPCs, IAM, CloudWatch, etc) from trial and error, and hobby projects like personal websites (free tier), hosting a Minecraft server, etc.
Being too overprotective can have negative consequences on folks who are responsible. One of the things I love about the technology and internet communities, etc is that you're mostly judged based on how you act and behave; not your age or other visible characteristics.
You don't have to use AWS though. Get one from Digital Ocean or Herzner, they have very predictable billing. Any button that costs money will tell you how much it costs per month.
The equivalent 10+ years earlier was so much lower risk: £25 or so for an old computer at a junk sale, £4.99 for a magazine with a Linux CD-ROM to avoid a week-long download.
> strict budget
How does that work in the case of AWS? Are you confusing alerts to caps?
3 replies →
Some variant of this topic comes up with some regularity. Leaving aside technical issues associated with implementing real-time hard caps, you still have a tradeoff. You either implement hard cutoffs which a student or someone else on a hard budget would like. Or you have a situation where an admin (or an admin who is no longer with a company) stuck some number in that seemed sensible at the time that brings down the company's whole system because of some sales spike.
I get that (and why) some people won't use AWS or its main competitors for this reason. But, frankly, they're not AWS's market and AWS will basically shrug.
1 reply →
Im kind of struggling with this logic, because a conscious choice was made to engage with AWS, AWS having opaque billing and the ability to provide a huge amount of compute (even at high cost) at the click of a button should be known to anyone who did his research on providers.
In my mind I could see a true tradeoff to removing the ability to do this. If I'm in a critical situtaion where, say, my service is on the cusp of failing because my revenue 100xed in a short while I know I could just go to AWS, put in some data and buy enough compute to survive as a business.
Anyone can make mistakes at some points and it's not like AWS UI/offerings make it any less confusing.
A kid with a credit card?
Have you seen Home Alone 2?
No. I don't know about the organization, but somewhere in this chain there is a flesh-and-blood human who deserves ridicule and or consequences, and furthermore -- discovering these people in situations like this is deeply important and must be done more.