I've started to dabble in this only very recently. I have two iCEBreaker boards [0] in the mail right now that I'm excited for.
From what I understand, one of the big advantages of the up5k chip (the same chip on the webfpga and the icebreaker) is that there's an open toolchain that runs in seconds (at this chip size), which is not the case in the closed/Vivado/"typical fpga" part of the world. (feel free to correct me)
Purely in simulation this week, I had Claude (Fable) design a working Tetris out of 1400 relays - originally, this was for a Minivac web simulator that I/Fable had built [1]. Then, it compiled the relay netlist to Verilog and ran it on the simulated chip, comparing this (bezerk) port relay-for-relay against the original simulator. All matching (total llm time: less than an hour). I had it also make a viz of the netlist blinking through one simulated run [2] - sort of like visual6502.org..!
Today, I (Fable) ported tinystories [3], a very small language model to this exact same chip. It's currently "golfing" to get it to fit onto the real chip size (it's ~10% over, I'm hopeful). EDIT: it just golfed it! ~~7 tokens/sec at 12Mhz in simulation!
None of these are "use cases", but these boards seem small/cheap enough that you can do new deranged things with them, which is very very exciting to me. I see a great future.
A trick to fit language models of this size it to remove the word to embeddings from the NN, and have a database to look up a dictionary of words and their embeddings. This way the LLL only need the "core" and you do the reverse for loopup again (kind of Text->RAG->LLM->RAG->Text). Have an example here: https://punnerud.github.io/pyspell/
The example have a limited language of around 1000 words, but make it possible to do Python (like) programming with LLM on an ESP32.
Note, if you need a barebones CPU but want to save on gates, you can go with a serial (1-bit) CPU, e.g. SERV which only takes up 198 LUTs on ice40,[1] leaving plenty of space to do other things.
Presumably small FPGA boards, like the $47 one in the linked article
Small boards have less I/O pins than larger boards typically, and small/cheap FPGA chips on small boards are also more limited than larger, more expensive parts.
I've started to dabble in this only very recently. I have two iCEBreaker boards [0] in the mail right now that I'm excited for.
From what I understand, one of the big advantages of the up5k chip (the same chip on the webfpga and the icebreaker) is that there's an open toolchain that runs in seconds (at this chip size), which is not the case in the closed/Vivado/"typical fpga" part of the world. (feel free to correct me)
Purely in simulation this week, I had Claude (Fable) design a working Tetris out of 1400 relays - originally, this was for a Minivac web simulator that I/Fable had built [1]. Then, it compiled the relay netlist to Verilog and ran it on the simulated chip, comparing this (bezerk) port relay-for-relay against the original simulator. All matching (total llm time: less than an hour). I had it also make a viz of the netlist blinking through one simulated run [2] - sort of like visual6502.org..!
Today, I (Fable) ported tinystories [3], a very small language model to this exact same chip. It's currently "golfing" to get it to fit onto the real chip size (it's ~10% over, I'm hopeful). EDIT: it just golfed it! ~~7 tokens/sec at 12Mhz in simulation!
None of these are "use cases", but these boards seem small/cheap enough that you can do new deranged things with them, which is very very exciting to me. I see a great future.
[0] https://1bitsquared.com/products/icebreaker
[1] https://minivac.greg.technology/tetris/
[2] https://necessary-doom-48128.ondis.co/
[3] https://arxiv.org/abs/2305.07759
A trick to fit language models of this size it to remove the word to embeddings from the NN, and have a database to look up a dictionary of words and their embeddings. This way the LLL only need the "core" and you do the reverse for loopup again (kind of Text->RAG->LLM->RAG->Text). Have an example here: https://punnerud.github.io/pyspell/
The example have a limited language of around 1000 words, but make it possible to do Python (like) programming with LLM on an ESP32.
wow, that tailscale-in-browser demo is wild..!
re: language model - here, the embedding lookup, layers, logits all run on-chip and loops its own output back.
the only off-chip piece is the id to string table, i.e. the chip (once I get it!) will speak token ids and my laptop will print them as letters
Examples include:
* replacements for the C64 PLA, CIA, and potentially the SID
* Glue logic for a bread board project where it would require a lot of 74 or 4k series logic
* Translating TTL to VGA etc.
These are things people have used FPGAs for quite commonly
ICE40UP5K is large enough to synthesize a simple RISC-V CPU with some basic peripherals. It's not huge or fast, but it's capable enough.
Note, if you need a barebones CPU but want to save on gates, you can go with a serial (1-bit) CPU, e.g. SERV which only takes up 198 LUTs on ice40,[1] leaving plenty of space to do other things.
[1] https://github.com/olofk/serv
You mean it's large enough to synthesize several application specific RISC-V cores and let them form an embedded distributed system.
performing glitch attacks on embedded systems, consoles, phones etc.
small what boards? Do esp32 boards count?
Presumably small FPGA boards, like the $47 one in the linked article
Small boards have less I/O pins than larger boards typically, and small/cheap FPGA chips on small boards are also more limited than larger, more expensive parts.
Edit: precise price.
Jailbreaking consoles
https://a.co/d/05f9gq9A
Direct link: https://www.amazon.com/dp/B0D9Q8VSSM
“KooingTech 150MHZ Machine Pulse Chip Mode Chip ACE V3 for X360 Slim”