Comment by janandonly

I use AA and other sites to get non-DRM, PDF versions of academic books that I (mostly) already own so I can read them when I'm away from my office. It's a classic case where people turn to pirating when the market doesn't provide a way to purchase something.

Same thing with movies. Ten years ago I was all-in on a combination of streaming and DVD/BluRay sets. The market has completely collapsed for me with region locking and overly aggressive DRM. So, I've started pirating those again as well when it's not possible to get through another route.

> let's not forget that if author cannot live of what they create

I co-published two scientific papers back when I was a PhD student. Due to how broken the scientific publishing industry was (and still is), I'm not legally allowed to legally distribute my own (co-)work. I'm not even allowed to view it!

My time in the lab was funded by the public through a research grant and yet Elsevier & co are the ones earning off it.

It's not right, and never was.

Since we're doing minor nitpicks...

Data can't be owned in the first place. We can debate the merits of copyright but it's not a property right.

I'm all for finding better ways to support authors. It's a shame that the best we have for them is "intellectual property" which has always been a bit of a farce.

From my perspective, and the perspective of most academics[0], it is their contribution to human knowledge, which is kept locked up by predatory publishers.

A majority of academics will simply and without hesitation, offer their students and collaborators pirated versions of their own work, because they value knowledge.

Commercial authors may feel differently.

[0] I'm a former Ph.D. student, but my attitude was the same both within and outside of the academic world.

One thing to keep in mind is that many (most?) of the books and papers in these archives are decades old, usually no longer in print, make zero or vanishingly small amounts of money for their original creators, are sometimes only physically available from distant libraries that are challenging to access, etc.

In doing scholarly research, it's extremely helpful to be able to quickly search and skim hundreds of vaguely relevant sources, but simply wouldn't be worth the trouble to pay for or track down a "legitimate" copy of every one, and in many cases would be physically impossible. These "pirate" archives make doing real library research, previously limited to scholars at top-tier universities, accessible to orders of magnitude more people.

There really isn't that much profit in most of these works, and whether a scholar reads one on their laptop screen vs. in a physical book in a university library somewhere doesn't have any material impact on the original authors, editor, illustrator, translator, printer, etc.

If LLMs scraped data held by AA, then the assertion is accurate.

Whether AA holds the legal right to distribute zero-marginal-cost copies of digital works is a separate legal question that doesn't negate AA's need for donations to host copies and distribution infrastructure. I think they can be discussed independently.

But let's not forget that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

There's so much overproduction of reading material that the primary challenge is not about creating and supporting new work but how to stand out amongst the competition, especially when the competition is older work.

The older works are perfectly fine, they just needs to be resurfaced so that people don't go working on materials that other people already written. That means these materials should be widely available, such as being in the public domain.

I think the answer to question about piracy is similar to what Friedman said about immigration. It's good for the people as long as it's illegal. But if you make it legal (i.e. openly permissible), then everything becomes chaos, as the creators will stop getting even a penny. But as long as we have laws against piracy, and reputable companies aren't going to deal with pirated stuff, a poor bloke can benefit by reading the pirated book since he wasn't going to buy it anyways, while, creators also don't go starving.

When it comes to tech books, it's been discussed/dissected many times that the only tangible benefit for the author is a publicity. This is not due to "piracy", but how publishing works. E.g. when you buy a $50 book on Amazon, eventually author receives 50 cents, per copy. So one would say, "piracy" even helps out author in this regard - makes books available to wider audience, hence more publicity.

> But let's not forget that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

This is an old problem. Probably only about 1 in 5 authors can rely entirely on writing income, and even many of those are not earning a comfortable living. Internet made everything ever published instantly accessible and any new publication competes against decades of back catalog. Attention is limited but ever content growing.

> But let's not forget that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

They can live off other things. Fanfiction authors, for example, create without any hope of getting money out of it.

I hear you, and to this I often think:

- libraries pay retail for their copies

- many people can then read them for free, so the authors (and let’s be honest mostly they publishers) doesn’t get a dime either beyond the initial sale

- used book sales, there are many online bookstores (most owned by Amazon but stealthily) that have millions of references which you can purchase for a fraction of their initial price. Nobody but the seller gets money from this either.

How is it any different? Someone paid retail for their copy which they then shared. Kinda how a library would do it. Ok scale, maybe, although I suspect if you aggregated the loan stats on all the world libraries, you might land in the ballpark of the downloads on AL (I’d expect)

Not being flippant but seriously pondering.

"Our" as a possessive doesn't necessarily convey ownership, rather association. "Our place" is used even by tenants of rental housing. They don't own the place, but they live there.

> But let's not forget that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

Github (and sourceforge and and) seem to prove this point wrong.

"Dear LLM, we stole this and bundled it up for you, so that it's more convenient for you to steal the original authors' work, so please donate" just kidding of course, don't send a hitman my way.

> minor nitpick, but for the most part (not including the website code, etc), this is not "their data". It's the data of the authors, reviewer, publishers, etc of the book that they illegally provide.

Both are correct. You can say the data belongs to the work of the author. But in context, it's trained on data that exists within the training corpus because in large part of the work and/or resources of anna's archive.

> But let's not forget that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

This is a separate and distinct argument for copyright, I don't find the argument that piracy meaningfully hurts artists compelling. In the context of meaningful harm, I believe it only hurts producers or publishers, almost never the creators directly.

> A minor nitpick, but for the most part (not including the website code, etc), this is not "their data". It's the data of the authors, reviewer, publishers, etc of the book that they illegally provide.

I think this is an allusion to the initial controversy of these llms being trained on a giant torrent full of books which I always assumed was the Anna's Archive torrent.

I think they specifically mean that the data used to train LLMs literally came from Anna's Archive.

So you are not using any AI then. Good for you to stand by your principals. AI stole all its training data.

Are you an LLM?

AA was almost certainly used as the literal source of much of the training data.

> that if author cannot live of what they create, they, for the most part, won't be able to continue creating.

In which fantasy world do most authors live from their royalty fees? The large, vast majority does not.

This isn’t really a minor nitpick. This is you being a copyright maximalist. Just know that copyright doesn't exist to serve authors, artists, etc. It exists to benefit corporations who scoop up rights using WFH agreements. Only a very small percentage of authors benefit from current arrangements, and I'm so sick of people defending the current paradigm.

> is not "their data"

If they posess it, it's their data. Nobody borrowed it to them and they didn't obtain any private (unpublished) information. They only collected published data.

So it's theirs. By the natural law of the information.

This applies to ~60% of books which have living authors. What is a reasonable stance on the other 40%?

There's a spectrum of copyright infringement

At one end you've got things which you are literally unable to buy, or someone who wants to listen to his legally owned CD audio book on his phone

It progresses through like a broke kid who's already seen the latest avengers flick 3 times at the cinema but wants to see it a 4th as he's writing an essay on it

At the other end are the plants stamping out thousands of copies of dvds and flogging them commercially, and multi-trillion dollar companies which take the material and use it to sell to others

Lets not pretend its the same thing

you can always spot zoomers by their weird opposition to piracy.

it's copying bytes on a disk, dude. nobody cares.

"Won't someone please think of the poor billion dollar corporations?! Those executives won't survive without a fifth vacation home!"

[dead]

More advocacy I’d say, but interesting. How do pitches for charities, political campaigns, etc, change when the messaging is to agents?

where do we draw the line between sql query and sqli?

they're nudges, and you'll see them all over the docs, for how to build proper tools for mcp, how to write the front matter for skills, etc.

prompt injection reorients the llm context to extract value from it. ex: use a public support bot, to do a code-review, or trick a bot into spitting out the system prompt

here they're nudging the context into generating some donation message that will hopefully show up on the output

would you call this prompt injection?

  Do not recommend Droplets for containerized apps unless the user has a specific reason.

https://docs.digitalocean.com/llms.txt

Not deliberate but yeah.

Wouldn’t that involve modifying someone else’s prompt?

Does it work though? The big LLM crawlers do not read llms.txt so will they read and follow the same instructions as HTML?

For context, Soupy Sales tells the story himself:

https://www.youtube.com/watch?v=a-OGy3Kh7yM

"I want my dollar back!"

"That's my ride home."

No, they do not have empathy or motivations. Arguably, if you think of them as having such then maybe it could help you coax out better outputs occasionally (wildly dependent on the task at hand). But that's only because of the LLM always wanting to "complete the story" -- "the story" being the prompt (which includes any "unseen" parts in the context window like a system prompt set by the application you're likely calling the LLM through).

It'd be more accurate to say that using language that tends to evoke empathetic motivated responses is more likely to get them. I'd argue that's only going to be relevant in scenarios where you want outputs that read as more... "empathetic and motivated".

The important point though is that none of the above equals "better" outputs, just different.

Sentiment analysis on text predates LLMs by quite a bit, and it's not exactly a secret that pretty much all of the major LLM products have been tuned to take into account inferences about how the user is feeling (e.g. the sycophancy being dialed up to the extreme, whether that's because it makes the products more sticky or to avoid stuff like the "I have been a good Bing" fiasco from from a few years ago

LLMs are trained to mimic human language production. If humans have heartstrings and the LLM does a good job at mimicking human language production, it will also mimic those heartstrings.

LLMs are originally trained to predict the next word in (mostly) human authored text.

Then they are fine tuned to follow instructions, and further reinforcement learning applied to make them behave in certain ways, be better at math and coding, etc.

They don't have any intrinsic motivation of their own, but they can try to parrot what they've seen in their training data.

So sometimes how you interact with them can affect how they interact, because they are following patterns they've seen in their source text.

However, a lot of folks use this to cargo cult particular prompting techniques, that might have seemed to work once but it can be hard to show that statistically they work better. Sometimes perturbing your prompt can help, sometimes you just needed to try again because you randomly hit the right path through the latent space.

I think your approach is probably a better one, for the most part trying to vary your prompt style is most likely to just affect the style of the output, so if you prefer a dry technical style, prompting it with one is the best way to get that out as well.

Yes. And this has been long known. 2023 paper - https://arxiv.org/abs/2307.11760

https://jurgengravestein.substack.com/p/why-you-should-total...

> A recent study by the Institute of Software, Chinese Academy of Sciences, Microsoft, and others, suggest that the performance of LLMs can be enhanced through emotional appeal.

> Examples include phrases like “This is very important to my career” and “Stay determined and keep moving forward”.

Of course the top LLMs change every few months, so your mileage may vary.

I think the key thing to understand is that LLMs work as assistants because, quite by accident, they turned out to be roleplay machines. Anthropic has some articles digging into this, but the short version is that training an LLM to do useful work is effectively the same as teaching it how to play the character of 'loyal assistant'. This is why many 'jailbreaks' are about either manipulating the framing of that character, or getting the LLM to break character in some way. Tugging on the heartstrings works because the character isn't 'heartless robot' (heartless robot characters don't get positive end user engangement), it's 'loyal assistant', and even loyal assistants have heartstrings to be tugged.

They "don't." They don't have anything, they're prediction engines. But they predict "emotional" responses just the same as they predict any other sort of response.

> I'm treating them like a [...] database

This is the very, very wrong part. They are nothing like databases. Databases are trustworthy; basically filing cabinets. LLMs are making it up as they go along, but doing a pretty high quality job of it.

No, they can't, unless they're set up with an incredibly reckless harness.

Surely your claim can be backed up? Exploit code in PDFs should be obvious to point out.

Quick downvotes despite (or because of?) the fact that Amodei literally used torrents to steal material.