Comment by dbreunig
18 hours ago
A couple thoughts on the process here…
First off, Opus 5.5 built working weights and code with only two prompts:
1. Read this article and any relevant source code: https://divagr.com/blog/qwenputer. Then explore if you could emulate the chip and ram in a Nintendo Entertainment System (as people do today with FPGAs in MISTRs) using this technique. If so, what size model would you have to build and if you could use smaller faster ones, could you get it to be at the standard clockspeed.
It returned charts well illustrating any model would be _slow_. But it's still fun, so the second prompt was:
2. I think we should build it! Just for fun. It's okay if it's not fast. We want to be able to have the weights, and a cli to run them that we can point at a ROM.
Now while this was an _epic_ run by Opus (5.5 is a crazy capable model), this is a surprisingly well-structured challenge.
There is a long-lived and incredible community of classic gaming emulation work online for it to reference. And that community has created…conformance tests! Opus found them and built against one, that has >3k golden instructions: https://github.com/christopherpow/nes-test-roms/blob/master/...
Also, there is the original Qwenputer repo that Opus used to drive the weight writing. That was essential.
But this is a great example of what these models can do with proper reference materials, great specs, conformance tests, and clear direction.
> There is a long-lived and incredible community of classic gaming emulation work online for it to reference. And that community has created…conformance tests! Opus found them and built against one, that has >3k golden instructions
I'm doing something similar but not for an emulator, using tests other projects in a similar space use to validate things.
I also know a friend that feeds Claude Gherkin formatted test criteria, and apparently that yields insanely good Claude output.
I said last year to people that I swear BDD is going to make a comeback. Hasn’t yet, but I still reckon it might!
Any good recommended reading?
They are talking about Cucumber, a behavior driven development platform. Gherkin is a syntax for writing test cases:
https://cucumber.io/docs/gherkin/
Very neat, seems quite appropriate for the way people currently work with LLMs.
I'd just ask Claude about writing test specs in Gherkin. I'm going to try this Monday.
Truly amazing. Quite thought provoking how it was helped by having high quality oracles. Eg. Qwenputer & good NES reference materials.
I guess humans need their oracles too: teachers, encyclopaedias, degrees, phds.
Bravo, awesome project, really creative!
[flagged]