Comment by diggan
8 days ago
> If you want to run LLMs locally then the localllama community is your friend: https://old.reddit.com/r/LocalLLaMA/
For folks new to reddit, it's worth noting that LocalLlama, just like the rest of the internet but especially reddit, is filled with misinformed people spreading incorrect "facts" as truth, and you really can't use the upvote/downvote count as an indicator of quality or how truthful something is there.
Something that is more accurate but put in a boring way will often be downvoted, while straight up incorrect but funny/emotional/"fitting the group think" comments usually get upvoted.
For us who've spent a lot of time on the web, this sort of bullshit detector is basically built-in at this point, but if you're new to places where the group think is so heavy as on reddit, it's worth being careful taking anything at face value.
LocalLlama is good for:
- Learning basic terms and concepts.
- Learning how to run local inference.
- Inference-level considerations (e.g., sampling).
- Pointers to where to get other information.
- Getting the vibe of where things are.
- Healthy skepticism about benchmarks.
- Some new research; there have been a number of significant discoveries that either originated in LocalLlama or got popularized there.
LocalLlama is bad because:
- Confusing information about finetuning; there's a lot of myths from early experiments that get repeated uncritically.
- Lots of newbie questions get repeated.
- Endless complaints that it's been took long since a new model was released.
- Most new research; sometimes a paper gets posted but most of the audience doesn't have enough background to evaluate the implications of things even if they're highly relevant. I've seen a lot of cutting edge stuff get overlooked because there weren't enough upvoters who understood what they were looking at.
> Most new research; sometimes a paper gets posted but most of the audience doesn't have enough background to evaluate the implications of things even if they're highly relevant. I've seen a lot of cutting edge stuff get overlooked because there weren't enough upvoters who understood what they were looking at.
Is there a good place for this? Currently I just regularly sift through all of the garbage myself on arxiv to find the good stuff, but it is somewhat of a pain to do.
I am not aware of any central, public location; I use a combination of arxiv, Google Scholar, Huggingface posts, and private Discords. The problem is that most of the public space has been poisoned with AI hype, and it's nearly impossible to find more than surface-level introductions for a lot of topics because the substandard Medium posts and YouTube hypemongers drowns them out.
Having a background in machine learning helps, because at least I can search for terminology that hasn't been picked up by the hype machine yet.
There's some communities that are more niche; more academically focused Discords or groups where there's better discussion going on, basically. Those are intermittent enough that you can't expect ongoing general discussion and for most I'd have to go back and check if they're still worth reading past the one discussion I found useful.
But for the wider internet, the hype train has forced most of the informed discourse off the road.
This is entirely why I can't bring myself to use it. The groupthink and virtue signaling is intense, when it's not just extremely low effort crud that rises to the top. And yes, before anyone says, I know, "curate." No, thank you.
Friend, this website is EXACTLY the same
I understand that the core similarities are there, but I disagree. The comparisons have been around since I started browsing HN years ago. The moderation on this site, for one, emphasizes constructive conversation and discussion in a way that most subreddits can only dream of.
It also helps that the target audience has been filtered with that moderation, so over time this site (on average) skews more technical and informed.
16 replies →
Strongly disagree.
Scroll to the bottom of comment sections on HN, you’ll find the kind of low-effort drive-by comments that are usually at the top of Reddit comment sections.
In other words, it helps to have real moderators.
3 replies →
> > . . . The groupthink and virtue signaling is intense . . .
> Friend, this website is EXACTLY the same
And it gnows it: https://news.ycombinator.com/item?id=4881042
It happens in degrees, and the degree here is much lower.
5 replies →
Strong disagree as well, this is one of the few places on the Internet which avoids this. I wish there were more
Lol this is true but also a TON of sampling innovations that are getting their love right now from the AI community (see min_p oral at ICLR 2025) came right from r/localllama so don't be a hater!!!
Poster: https://iclr.cc/media/PosterPDFs/ICLR%202025/30358.png?t=174...
Well the unfortunate truth is HN has been behind the curve on local llm discussions so localllama has been the only one picking up the slack. There are just waaaaaaaay to many “ai is just hype” people here and the grassroots hardware/localllm discussions have been quite scant.
Like, we’re fucking two years in and only now do we have a thread about something like this? The whole crowd here needs to speed up to catch up.
There are people who think LLMs are the future and a sweeping change you must embrace or be left behind.
There are others wondering if this is another hype juggernaut like CORBA, J2EE, WSDL, XML, no-SQL, or who-knows-what. A way to do things that some people treated as the new One True Way, but others could completely bypass for their entire, successful career and look at it now in hindsight with a chuckle.
And like those technologies it will find its own niches (like XML and no-SQL being used heavily in the publishing, standards, and other similar industries using document formats such as JATS) or fade away to be replaced with something else that fills the void (like CORBA and WSDL being replaced by other technologies).
I think LLMs will find their uses, it just takes time to distil what they are really useful for vs what the AI companies are generating hype for.
For example, I think they can be used to create better auto-complete by giving them the context information (matching functions, etc.) and letting them generate the completion text from that.
2 replies →
I use it as a discovery tool. Like if anybody mentions something interesting I go and research install/start playing with it. I could care less if they like it or not I'll make my own opinion.
For example I find all comments about model X be more "friendly" or "chatty" and model Y being more "unhinged" or whatever to be mostly BS. Like there's gazillion ways a conversation can go and I don't find model X or Y to be consistently chatty or unhinged or creative or whatever every time.