I am doing the initial work on many core dual CPUs. I already have systems with large system RAM and older GPUs with less VRAM, and large SAS HDs, fast SSDs running large open weight LLMs that do not really fit any GPU I own (they offload and/or run on CPU and system RAM, but slowly).
I have the MCUs and FPGAs (in a drawer) and I am retired, and this is my idea of fun.
I am trying to generalize an approach to use large MoE models (with possibly small quants) to run many agents in parallel without spending more on more or bigger GPUs.
I am also doing some edge ML (bird species recognition near the camera) using NPUs (in design phase, yet untested). I have an electronics lab, and I've emulated soft CPUs and built software that runs on FPGAs and in my emulators.
Instead of assuming my approach won't work or is too expensive, I choose to be optimistic. Also, failures are educational. I'm trying to gain more FPGA experience.
I am doing the initial work on many core dual CPUs. I already have systems with large system RAM and older GPUs with less VRAM, and large SAS HDs, fast SSDs running large open weight LLMs that do not really fit any GPU I own (they offload and/or run on CPU and system RAM, but slowly).
I have the MCUs and FPGAs (in a drawer) and I am retired, and this is my idea of fun.
I am trying to generalize an approach to use large MoE models (with possibly small quants) to run many agents in parallel without spending more on more or bigger GPUs.
I am also doing some edge ML (bird species recognition near the camera) using NPUs (in design phase, yet untested). I have an electronics lab, and I've emulated soft CPUs and built software that runs on FPGAs and in my emulators.
Instead of assuming my approach won't work or is too expensive, I choose to be optimistic. Also, failures are educational. I'm trying to gain more FPGA experience.