Comment by dafi70
10 hours ago
Honest question: how can VCs consider the 'star' system reliable? Users who add stars often stop following the project, so poorly maintained projects can have many stars but are effectively outdated. A better system, but certainly not the best, would be to look at how much "life" issues have, opening, closing (not automatic), and response times. My project has 200 stars, and I struggle like crazy to update regularly without simple version bumps.
> Honest question: how can VCs consider the 'star' system reliable?
They don't.
I've helped with due diligence on a couple projects. VCs know that metrics can be gamed because they see it all the time. Stars, followers, views, clicks, likes. A portion of entrepreneurs have been gaming every metric since before you and I learned how to program. It has always been this way and always will.
Most of the VC-related comments have interpreted this article to mean that VCs are so dumb that they haven't realized that stars can be faked, but in reality VCs spend so much time sorting through fake metrics that they understand this probably better than most here.
If you've ever gone through due diligence for an acquisition or big investment round it's amazing how much work you have to do in order to prove that your metrics are real. When things got crazy after COVID there was a short time when VCs were trying to move so fast that they skipped this, but it resulted in some high profile fraud cases.
During normal times, you will get grilled on metrics. They might see stars as a signal for rising stars, but they're not throwing money at projects based on star count like many commenters assume. They will do a deeper dive before investing and they will call it off if things aren't adding up. The amount of diligence scales with the investment, so someone getting a $10K check can get away with a lot of fraud but that $2mm funding round isn't going to cross the finish line based on star count.
The stars have fallen to the classic problem of becoming a goal and stopping being a good metric. This can apply to your measure just as well: issues can also be gamed to be opened, closed and responded to quickly, especially now with LLMs.
Was it ever a good metric? A star from another account costs nothing and conveys nothing about the sincerity, knowledge, importance or cultural weight of the star giver. As a signal it's as weak as 'hitting that like button'.
If the number of stars are in the thousands, tens of thousands, or hundreds of thousands, that might correlate with a serious project. But that should be visible by real, costly activity such as issues, PRs, discussion and activity.
There was a time when total number of hyperlinks to a site was an amazing metric measuring its quality.
2 replies →
I remember talking to some of the folks running UIUC's hackathon (probably ten years ago) and they'd built a sort of page-rank for Github - hand-identifying the most prominent and reputable projects/individuals and then using follows and stars to transfer that reputation. I don't know how well it worked in practice or if it was every published, but it might be more effective than pure star count.
(This was for admissions iirc - they had limited slots and a portion of them were allocated to people with a strong github rank.)
There isn't just "good metric" in vacuum - it was a good metric of exactly the popularity that you mentioned. But stars becoming an object of desire is what killed it for that purpose. Perhaps now they are a "good metric" of combined interest and investment in the project, but what they're measuring is just not useful anymore.
1 reply →
A repository with zero stars has essentially no users. A repository with single-stars has a few users, but possibly most/all are personal acquiantances of the author, or members of the project.
It is the meaning of having dozens or hundreds of stars that is undermined by the practice described at the linked post.
I especially love issues automatically "closed due to inactivity" just to keep the number of issues down :V
Sometimes people open issues without proper information. It cant be replicated and nobody else is jumping in that it affects them. You may suspect its something else, maybe with their environment, but if they don't engage what else can you do? Tell them you are closing it and specify what kind of info you need if they ever get around to providing it and it can be reopened.
1 reply →
"When a measure becomes a target, it ceases to be a good measure"
https://en.wikipedia.org/wiki/Goodhart%27s_law
I wonder if there's a more graph oriented score that could work well here - something pagerank ish so that a repo scores better if it has issues reported by users who themselves have a good score. So it's at least a little resilient to crude manipulation attempts
Sounds like 'advogato' for github... Social trust metrics
https://web.archive.org/web/20170715120119/http://advogato.o...
It would be more resilient indeed, I think. Definitely needs a way to figure out which users should have a good score, though - otherwise it's just shifting the problem somewhat. Perhaps it could be done with a reputation type of approach, where the initial reputation would be driven by a pool of "trusted" open source contributors from some major projects.
That said, I believe the core problem is that GitHub belongs to Microsoft, and so it will still go more towards operating like a social network than not - i.e. engagement matters. It will still take a good will to get rid of Social Network Disease at scale.
Reputation doesn’t equal good taste in judging other projects.
There are much better ways of finding those who have good taste.
GitHub has all kinds of private internal metrics that could update the system to show a much higher signal/quality score. A score that is impervious to manipulation. And extremely well correlated with actual quality and popularity and value, not noise.
Two projects could look exactly the same from visible metrics, and one is complete shell and the other a great project.
But they choose not to publish it.
And those same private signals more effectively spot the signal-rich stargazers than PageRank.
You are looking for different things to VCs. You are looking for markers that show software quality over the long-term. They are looking for markers that show rapidly gaining momentum over the short-term. These are often in opposition to one another.
Making the conversations about VCs expecting thousands of stars, is thinking too big. It's probably more often someone pays $20 to make one of their projects look good, for their CV, for vanity, thinking this will get them the push that they need to get clicks on reddit, or noticed over some other open source project. If there is someone offering a 10k star project an investment over 8k without looking at the project or revenue potential, I can only think they are clueless, or picking a student project to fund each summer.
The fake accounts often star my old repos to look like real users. They are usually very sketchy if you think for a minute, for example starring 5,000 projects in a month and no other GitHub activity. One time I found a GitHub Sponsor ring, which must be a money laundering / stolen credit cards thing?
Unless something has changed in the last ~3 years, I think the article vastly overstates the credibility with VC's.
Even 10 years ago most VCs we spoke to had wisened up and discarded Github stars as a vanity metric.
Agree that sophisticated funds don't, but the ecosystem hasn't caught up. StarHub/GitStar pricing pages still sell to "seed-stage founders pre-fundraise"
Much more important is who starred it. And are they selective about giving out stars or bookmarking everything. Forks is a closer signal to usage than stargazing.
Indeed, GitHub should set up a monthly quota for available stars to give and correlate the account age with it: either make something up like a "trusted-age-factor" that multiplies any given star by that factor, or scale the available quota accordingly by that factor (and let users star repos repeatedly).
GitHub should also introduce a way to bookmark a repo, additional to the existing options of sponsor/watch/fork/star-ing it.
With the advent of AI, these "life" events are probably even simpler to fake than AI though, and unlike the faking of stars not against the ToS.
Stars are a simple metric even someone like a VC investor can understand. Your "better system" sounds far too complicated and time consuming.
because VC don't care about anything being legitimate, if it can fool VCs it can also fool market participants, then VC can profit off of it.
one VC told me, you'll get more funding and upvotes if u don't put "india" in your username.
Many VCs are only doing one thing: how to use some magical quantitative metrics to assess whether a project is reliable without knowing the know-how. Numbers are always better than no numbers.
Honestly I don't know if that's true. Picking up on vibes might be better than something like GitHub stats.
When a partner decides to recommend a startup to the investment committee, he needs some explicit reasons to convince the committee, not some kind of implicit vibe
2 replies →
This is a good idea, but from my experience most VCs (I’m not talking about the big leagues) aren’t technical, they tend to repeat buzzwords, so they don’t really understand how star systems works.
Because VCs love quantifiable metrics regardless of how reliable they actually are. They raise money from outside investors and are under pressure to deploy it. The metrics give them something concrete to justify their thesis and move on with their life.
>Honest question: how can VCs consider the 'star' system reliable?
Founders need the ability to get traction, so if a VC gets a pitch and the project's repo has 0 stars, that's a strong signal that this specific team is just not able to put themselves out there, or that what they're making doesn't resonate with anyone.
When I mentioned that a small feature I shared got 3k views when I just mentioned it on Reddit, then investors' ears perked right up and I bet you're thinking "I wonder what that is, I'd like to see that!" People like to see things that are popular.
By the way, congrats on 200 stars on your project, I think that is definitely a solid indicator of interest and quality, and I doubt investors would ignore it.
> how can VCs consider the 'star' system reliable
I think VCs just know that there are no reliable systems, so they go with whatever's used.