Show HN: StackScope – I crawled over 40k indie launches to see what they ship

7 days ago (stackscope.dev)

Hey all, I built StackScope, a crawler/catalogue that looks at new product launches and shows what they were built with.

It watches launches from Product Hunt, Show HN, and PeerPush, then crawls the public site behind each one. The goal is to show what people actually launched with: hosting, frameworks, analytics, DNS, security headers, legal pages, AI-builder signals, and other public clues.

I started building it because most stack-detection sites look at the web as a whole. I was more interested in the current indie launch scene: what people are choosing right now, at the point they first put something in public.

A few implementation details: it runs on .NET, uses Playwright for rendered pages, and has a first-party fingerprint catalogue rather than one copied from Wappalyzer/etc. robots.txt is honoured, and the bot identifies itself.

Frustratingly, I am still waiting for verified bot status from Cloudflare and currently that knocks out about 10% of all sites.

There is also a private readiness check: paste a URL, get the same style of report, fix things, and recrawl. No account or email needed.

I'd be interested in feedback on the usefulness of this, the methodology, and any obvious false positives.

Jonathan.

26 comments

datafreak_

Reply
pixel_popping  7 days ago

Images/layout is broken - tried both chromium/ff. GET https://stackscope.dev/css/fonts.css?v=LZFlL4O9GYH_vtKkLpJwV... NS_ERROR_CORRUPTED_CONTENT

GET https://stackscope.dev/css/stackscope.css?v=qkmEKPTNqImS7aeD... NS_ERROR_CORRUPTED_CONTENT

GET https://stackscope.dev/css/utilities.css?v=l7--cPr2cfIn6iDRB... NS_ERROR_CORRUPTED_CONTENT

GET https://stackscope.dev/css/page-extras.css?v=m-VKj9bQg7fMpFK... NS_ERROR_CORRUPTED_CONTENT

GET https://stackscope.dev/js/site.js?v=SxUp8zU6gnAArRwBXr22wIBS... NS_ERROR_CORRUPTED_CONTENT

  • datafreak_  7 days ago

    What's your response header content? Reads like it's being cf-mitigated.

    Edit: loosened a cf rule, should be good now.

addedlovely  7 days ago

Nice one.

I've been doing the same bit wider scope, for the whole Crux list, pruned to apex domains, and looking for CMS signals - how's your throughput?

I'm not doing any headless browser stuff, or many requests, so hyper optimised for speed.

I do grab robots.txt - didn't really see much in llms.txt or humans.txt in the wild, does yours?

schamack  6 days ago

That looks very impressive! And really insightful as well. And to my surprise, I even found my startup on the landing page :D (possibly due to fresh HN post here as well).

I will have a thorough look on the insights later, as there might either be some small false positives, or my projects still needs some grunt work :).

So from my perspective, being part of the check, I find it extremely useful.

  • datafreak_  6 days ago

    Let me know if anything is detected wrong!

    • schamack  4 days ago

      Alright, so I went through all the potential issues and already implemented the changes where applicable :).

      My app is Tekyous (didn't go through this option to claim the launch): https://stackscope.dev/launch/xsa4cjwn/tekyous

      And turns out that actually there were almost no false-positives here at all. Some of the flagged things were rather minor, like missing files that are not really necessary in my case, or too long meta descriptions, but technically all is flagged correctly even if severity is low.

      But on the other hand, it has also correctly flagged my tracking settings or overly strict rules for AI crawlers, so this were genuinely useful for me to find out now.

      And the only real false-positive is actually a recognised AI builder signature for Bolt.new - not sure where that one comes from, as Bolt.new was not used here. I've only used Claude Code, mostly with Anthropic + GLM by Z.AI models.

      Hope that feedback helps you, as I have defnitely found it helpful on my end. Thanks!

8organicbits  6 days ago

Cool tool, I'm also surprised by how different the startup stacks are from the general Internet.

For HSTS, don't forget to check the preload list. Domains under .dev are all preloaded, for example, so they don't need to set the header for HSTS to apply.

aakresearch  1 day ago

Oh wow, "4851 unique technologies tracked" - isn't it an indictment on fragmentation of the industry? It is laughable what passes as "unique" "technology" these days. I know StackScope didn't start that, but it is still adding oil to the fire hollowing the industry out and leaving people stranded.

thetwopct  6 days ago

Interesting but Laravel usage seems so low. As Laravel uses React or Vue etc I feel like a site could be easily classified as React when it’s Laravel running the show.

cpursley  7 days ago

Makes me sad how Nextjs has become the default for so many startups…

  • sandeepkd  7 days ago

    Unfortunately the chances are that it would become cyclic with the increasing use of LLMs to generate code