Comment by zby

I am wrapping up https://github.com/zby/DayDreamingDayDreaming - it is a project to show that https://gwern.net/ai-daydreaming can work.

It is hard to show that AI can reimplement for example special relativity - because we don't even have enough text from 19th century to train an LLM on it - so we need a new idea something that was invented after an LLM was trained. I took the Gwern's essay and checked with deep search and deep research which ideas from that essay are truly novel and apparently there are some so reinventing them seemed like a good target: https://github.com/zby/DayDreamingDayDreaming/blob/main/repo... https://github.com/zby/DayDreamingDayDreaming/blob/main/repo...

So here it is - a system that can reliably churn essays on daydreaming AIs. On one level it is kind of silly - we already knew that infinite monkeys could write Shakespeare works. The generator was always theoretically possible, the hard part is the verifier. But still - the search space in my system is much smaller than the search space of all possible letter sequences - so at least I can show that the system is a little more practical.

Here are some results: https://github.com/zby/DayDreamingDayDreaming/tree/main/data...

You can modify it to reinvent any other new idea - you just need to provide it the inspirations and evals for checking the generated essays.

I am thinking about next steps - maybe I could do it a little bit more universal - but it seems that to build something that would work as needed would require scale.

I kind of like the software framework I vibe coded for this. It lets you easily build uniform samples where you can legitimately do all kinds of comparisons. But I am not so sure about using Dagster as the base for the system.