Raspberry Pi Drag Race: Pi 1 to Pi 5 – Performance Comparison

13 hours ago (the-diy-life.com)

I've got a brute force solver for the NYT Pips game. There's a particular puzzle that it takes 45.2 seconds on on my M2 Max Mac Studio. The solver is single threaded and doesn't use much memory so it is mostly limited by CPU and memory speed.

I ran it on my Pi 3, 4, 5, Intel iMac, and on my cheap Amazon Lightsail instance. Here are the results, in seconds:

  680.4  RPi 3
  274.5  RPi 4
  131.3  RPi 5
  108.5  Lightsail
   78.7  2017 iMac (3.4 GHz Intel Core i5)
   45.2  M2 Max Mac Studio

  • This reminds me of the Pi Digits benchmark, used back then, when Pentium III/IV were sold, to sell how CPU had improved back then

  • Very cool. I feel like Lightsail performed quite well. Feels like a local optimum of some kind. Mind specifying the tier?

    • It says "512 MB RAM, 1 vCPU, 20 GV SSD" and "General Purpose". It was whatever was the cheapest when I made it.

      Looking at the instance creation page it looks like it is no longer available. The cheapest now has 2 vCPU and the same other specs.

      If anyone would like to run it on something else for comparison I've put it Pastebin [1].

      Normally, a script converts the puzzle into a header file that describes the puzzle in terms of the data structures the solver uses, and then the solver is compiled with that header included via #include, but for benchmarking I manually included the header, so it is just a single C file that you simply compile and run. Add a -q flag to have it just find and count the solutions but not display them. For this particular puzzle it is fine to let it show the solutions. For some puzzles with a ton of solutions it displaying them can take significant time.

      If anyone actually wants a Pips solver to solve Pips puzzles, there's a copy of mine along with the scripts that convert puzzles from the JSON format the NYT provides for download to the header file for the solver and some other stuff as a shar archive on Pastebin [2]. Pastebin's download link messes up something with that, so click the raw link and copy/paste from the browser.

      Note: I'm still doing a lot of experimenting with it, both in the solving algorithm and in how the display the results, and am not at a point where I'm able to accept outside contributions. There are other Pips solvers that have been put on Github (and even posted on HN) that are (1) more sophisticated than mine, and (2) almost certainly more open to contributions. Just use mine if for some reason you want a fairly simple (aside from the damn display stuff...) brute force solver in C and aren't going to expect anything else from me :-)

      [1] https://pastebin.com/5HmymPXw

      [2] https://pastebin.com/SfSk3eNB

      1 reply →

    • If it's low RAM and single threaded then I would expect every tier to perform identically.

      (Also I was just looking at the pricing, you don't get any extra CPU cores until the $84 tier!)

      1 reply →

I think the Pi 3 range is a sweet spot for low cost, low power draw, decent-enough CPU. Newer models draw increasingly more power; going from 1.4W to 2.8W may not seem like much, but that's half your battery life. There's a few differences in Raspberry Pi 3 versions that may lead you to buy one or the other:

- The Pi 3B has 10/100 Ethernet, 802.11n (single-band) WiFi, Bluetooth 4.1. Power idled at 1.4W and peaked at 3.7W.

- The Pi 3B+ removed the 10/100 Ethernet in favor of USB Ethernet (~300Mbps w/USB2.0). CPU cores were overclocked from 1.2GHz to 1.4GHz (so a heatsink is more necessary), with ~15% increase in benchmark performance. It added 802.11ac (dual band) WiFi and Bluetooth 4.2 w/BLE. Power idled at 1.9W and peaked at 5.1W. This is also the only 3-model supporting PoE (w/ extra HAT).

- The Pi 3A+ removed Ethernet and reduced USB to a single port. The RAM was reduced from 1GB to 512MB. Power idled at 1.13W and peaked at 4.1W. The A+ form factor is more compact. Overall the 3A+ is smaller, cheaper, and less power draw than the 3B+ (but not as low as the 3B).

The lowest power draw with acceptable performance is the 3B. For slightly more power draw and more CPU performance, go with 3A+. For "everything" (including PoE) the 3B+ is it.

If you want the 3A+ but don't need the video, want a smaller form factor, and half the power draw, the Pi Zero 2 W is it. Though the Pi Zero 2 W is supposed to be cheapest, due to demand it's often sold out or more expensive. The 3A+ is still cheap (~$25) and available, with the downside of the higher power draw and larger form factor.

(disabling HDMI, LEDs, Wifi, Bluetooth, etc reduces power draw more. in testing, the 3A+ drew less power than the Zero 2 W with everything disabled. all of them draw ~0.1W when powered off)

  • > I think the Pi 3 range is a sweet spot for low cost, low power draw, decent-enough CPU. Newer models draw increasingly more power; going from 1.4W to 2.8W may not seem like much, but that's half your battery life.

    Is that with the same load? The chart in the article shows a Pi 3 and Pi 4 using the same idle power, with the 4 drawing more under full load. But the 4 can do more at full load, raising the question of what would be the 4's power usage running a load equal to the 3's full load?

    • The author's chart is incorrect. He has the idle power at "1" for the Pi 2, "2" for the Pi 1/3/4, and "3" for the Pi 5. No other published power draw numbers are whole numbers like this, they are floating points, like 1.2, 1.9, etc. Google around and you'll find several different power testing comparisons with more detail.

      Most reports show the Pi 4 drawing ~2.8 W in idle headless mode, and the Pi 3B+ drawing ~1.9-2.0 W in idle headless mode. With full load the Pi 4 draws more (6.4W, to the Pi 3B+'s 5.1W) with the same test procedure.

      But you do have to check the testing method; enabling/disabling hardware features changes the figure, and each additional USB peripheral draws more power. Otoh, to get a "max power draw" reading you have to enable everything and stress all CPUs at once, and then it will dip under thermal load.

      1 reply →

    • Right. In CPU scheduling this is the “race to idle” idea.

      Sometimes it’s more efficient to run a task at full CPU power for 25ms than low (non-idle) CPU power 100ms.

      It wouldn’t surprise me too much if the 4 could run the 3’s full load in less power than the 3 does.

About 7 years ago, we deployed a “gateway/orchestration” role device in ag tech. Power draw is a big concern for us (not a lot of free power out in the middle of fields). We used an SBC from Emtrion. I remember asking my EE counterpart at the time “why not a Pi? Surely someone makes hardened versions of those?” He was skeptical and I think the aura of “toy/hobby/maker” scared him off.

Fast forward. We’re getting ready to role out our next generation. It’s based on the Pi Compute Module 4 (the CMs are basically just the basic Pi and you put your own carrier board for peripherals under it). It is amazing. It has easily 20x the power, 20x the RAM, better temp specs and such, a great eco system, uses about 30% less power, and about 1/5 of the price. The only thing we’re not sure about yet, is the robustness of the BLE with the onboard radio chip.

It’s amazing how far these things have come. For low volume product builds, it’s hard to find a reason not to use one of the CMs.

  • Hah! And the value of the Pi for these kinds of ~industrial applications is why there was a shortage of Pis for hobbyists.

    It's funny how Raspberry Pi started out for an educational market, and accidentally revolutionized the embedded market.

    • Honestly, in all my life I've never seen the Pi being sold in EU for €35. The min. price I've found has always been around 45/50, with Pi5 never under 75, because of scalpers

    • Even funnier is the history. IIRC, the very first Raspberry Pi was an idea based on a bunch of stock of shitty SoCs for set-top boxes that Broadcom couldn't get sold, so Eben Upton got these for cheap for the foundation he and a few others had started to promote computer literacy.

I finally found a job for my Raspberry Pi 1 Model B from 2012. It’s been sitting in a drawer for years, but about a 2 years ago added it to my Tailscale network as an exit node.

It’s a single-core 700MHz ARMv6 chip with 512MB of RAM. It's a fossil—a Pi 5 is 600x faster (according to the video). But for the 'low-bandwidth' task of routing some banking traffic or running a few changedetection watches via a Hetzner VPS (where the actual docker image runs), it’s rock solid. There’s something deeply satisfying about giving 'e-waste' a second life as a weekend project.

  • As a fun weekend project in 2013, I stood up a weather station using Weewx and my RPI 2 with 1 GB RAM. I told myself if it ever crashes or the SD card gets corrupted, I'll just tear everything down.

    Well, it's still running today on the original SD card. At noon today it processed its 1,055,425th record in the database.

    Still, if it ever crashes, I'll just tear it down. :)

  • They'll run CUPS too! My B modernized some old, commercial Brother laser printers I was running.

    • That's a great idea - if I understood you right, you mean you used it to make a printer "wireless / wifi enabled" with it, right? Is there any guide you can recommend for that?

      2 replies →

  • I have a few older models lying around too, there's some other minor benefits as well:

      * They have full sized HDMI ports 
      * They will happily run using any random old USB charger and not overheat.

  • I mean in theory and practice a Pentium 2 300 could do full 1gpbs routing with Vyatta and I used that and other distros to do that for years

  • Well on the other hand, at which point does it become wasteful to run something when it gets less and less power efficient compared to newer devices? According to OP's benchmarks, the Pi 1 burns 2W constant to do essentially zero work and running that on a more modern device that's already running would use almost no extra power.

    Then again we use a kW or two to microwave things for minutes on a daily basis so who really gives a shit.

    • Yeah... 2W is just not that much energy.

      Enough energy to run that thing for an entire year in under 1/2 a gallon of gasoline.

      When you can pretty easily offset the entire yearly energy use by skipping a mow of your yard once, or even just driving slightly more conservatively for a few days... I'm not so worried about the power use.

      In my region - it's about $3.50 in yearly power costs.

  • > I finally found a job for my Raspberry Pi 1 Model B from 2012.

    Nice! Even though I've got a Proxmox serve at home running on a real PC (but it's not on 24/7), I do run my DNS, unbound, on a Pi 2. It's on 24/7 and it's been doing its job just fine since years.

Got as far as the cookie request, and this is one of those without a "Reject All" option where you have to scroll through dozens of options to deselect. I went no further.

Side note: that site has well over three hundred vendors listed for cookies! I thought they were generating fake outs without end but I did eventually reach the end

Pi 5 pricing gets harder to justify when you add up the extras. 8GB board + active cooler + good SD card + case + power supply = $150+ easily.

Used 1L mini PCs (EliteDesk, ThinkCentre Tiny, etc.) with i5-8400T/8GB/256GB go for $50-100 on eBay. You get x86 compatibility, NVMe support, real Ethernet, and no thermal throttling. Running an EliteDesk 800 G4 with 2.5GbE adapter and 2x2TB NVMe for home server duties. Draws ~15W idle, handles everything I threw at it. The Pi would need USB adapters for any of that. Pi still wins for GPIO projects and actual embedded use. But for "small Linux box" use cases, the used business mini PC market is tough to beat.

you should try running Linpack on them all (you can find the results here mixed in with other machines I own) https://web.eece.maine.edu/~vweaver/group/machines.html

When I did that on Pi3 when it first came out you could crash the system because the thermal throttling wasn't fast enough (the temp sensor was on the GPU not CPU). When I reported the issue on the pi forums the answer was essentially "why would anyone ever want to do that"

  • >"why would anyone ever want to do that"

    The more things change, the more things stay the same.

  • > When I reported the issue on the pi forums the answer was essentially "why would anyone ever want to do that"

    With all due respect to Raspberry Pi and everything they’ve accomplished in the educational and hobby space,

    I felt that one in my bones. I suspect a lot of people with embedded experience who worked with Raspberry Pi over the years feel it too.

Interestingly the original Pi had the same amount of memory as the PS3 which was available at the time of the Pi release. Still amazes me how much we did with only 512MB.

  • And you didn't even get the luxury of unified memory on the PS3 - the CPU and GPU had separate 256MB chunks.

    I can still barely believe they got Grand Theft Auto 5 running on that thing.

    • I think it’s more amazing it’s the latest GTA not only had been V for 3 console generations, but it’s still selling well.

Oh no, it appears to have received the hug of death?

I am interested in this, I have been using Raspberry Pis for various projects and home servers since the original - Currently one is hosting my navidrome music server, my password manager, and several other local network servers.

I feel the upgrade each time, and then get used to it, as I suppose we tend to do. I still remember the upgrade from 1 to 2 being the most impactful to me personally though. (I think maybe because that's when game emulation became viable?)

Do we think this page loads enough ads?

The graphs are interesting but, really, if you’re considering your readers rather than SEOing for last decade’s search engine technology, you should lead with them and discuss the findings afterwards.

I.e., get to the point quickly and then unpack the detail.

It’s interesting seeing where the incremental vs revolutionary improvements have occurred. CPU-wise, a huge leap with the 3 and then solid but steady improvement with 4 and 5. But the most meaningful jump in GPU performance seems to be 4 -> 5, and I’d be really interested in what that maybe opens up in terms of console emulation.

Anyway, fewers ads, please. Scanning through the article on mobile felt like playing hopscotch in a minefield.

  • I'll always agree that sites should be clear of ads, but ad blockers are the true solution. I can't stand browsing the internet without one.

    • And had I been on a desktop browser (thanks uBlock Origin), or sitting behind pihole, I'd never have noticed, but unfortunately with Safari on my iPhone over 5G... not so much.

> The Pi 2 managed to open the browser and actually started playing back a 1080P video, which was surprising, but playback was terrible.

A Youtube tab, web browser modern enough for YouTube, and OS modern enough for that web browser, all fit in 1GB of memory? Wow.

  • Worth noting: A Pi 1 will decode 1080p h264 video flawlessly. I used one as a media center for a long time, running OpenELEC (now LibreELEC). Moving around the UI was a bit sluggish, but video playback was smooth.

    YouTube is an absolute clown show. It's so bad that I'm certain Google devs are actively making it terrible on purpose. I use Newpipe on an older (but not that old) tablet. Whenever Google breaks Newpipe and I have to use a browser, it takes like 30 seconds just to load the page.

    Decoding video is trivial when you have hardware decoders.

    • Are they even pushing H264 to the pi? Or is this more of their “you’ll take WebP and like it” stuff.

    • I had this on my old laptop too, it would really struggle with HD YouTube, but if I copy pasted the url into VLC or MPV it would play just fine.

For $50 you can pick up a used mini PC with say, i5-6500T and 8GB ram, that'll be much faster than the Pi 5. And it'll be compatible with all Linux distros. Really the Pi 3 is good enough as an edge device where you want to hook up things to the GPIO pins.

  • But a PC like what you’re suggesting would result in a much higher energy bill compared to a Pi if you leave it on 24/7 to use as a server.

    • I've got a i5-7500T box running as a Proxmox Backup Server, and it idles at 6-7W. It runs at near idle most of the time (it's just running PBS and a few network services), so I'm not expecting it to cause much difference to my power bills. Even under full load, it only draws ~30W, so it's not _that_ much power.

The article shows how performance has always increased at a somewhat continually increasing level of inconvenience. Weird connectors, SUPER demanding power requirements, new case designs every generation, new cooling required every generation, etc.

My applications have remained the same for many years my octoprint and retropie don't require more FLOPs as time goes on but I'd really enjoy a modern board that has fewer headaches. Works on any normal USB port instead of requiring specialized power supplies, doesn't brown out and reset as much, doesn't heat up as much, etc. I suspect "a pi 3, but now with fewer headaches" would sell better than "a pi 3 but even more headaches and bigger numbers that you don't want".

  • I still use a PI3 as a daily driver. I never got around to the PI4 (too expensive, low availability), and when the PI 5 came along it was severely downgraded for my main usage purpose (x264/AVC playback) while much pricier too. I don't expect a further PI 6 will remedy this properly.

I suppose its unintentional comedy that they picked a 1080p H264 video playback as the benchmark. Because of course the chip in the Raspberry Pi 1 was literally designed for that! The only thing it asks of you is that you make use of the fixed function blocks that take up much of its silicon space. So no wonder that utterly fails with modern software - we need to go all the way to RPi 5 to smother the problem with enough generic computing power to overcome the careless people that spearhead much of browser development.

  • I don't think it is really a fair test because I doubt the browser is going to use any of the video decoding hardware that maybe available. I suspect if you used something native then any video decoding hardware would be used correctly.

  • Yeah.

    Except it's not even fixed function blocks, it's the 12 core VideoCore IV GPU running software that does the decoding.

    VideoCore is the real Raspberry Pi, the ARM block running Linux was just a subprocessor that VC controls.

  • Yep. A Pi 1 can almost play 1080p60 with a proper zero-copy decoding setup. Pi 2 and beyond have no issue with that. As you said: The Pi5 has enough CPU power, so even the H264 decoding itself now uses software as it no longer has a hardware decoder. Oh well.