Tech stack is a mix of things, but the big-ticket items:
- Dagster for data pipeline
- Parquet files for data format
- Cloudflare R2 for data storage
- DuckDB for data processing
- NodeJS for API
- NextJS + React for the app
- MapLibre for the base map functionality
- Deck.gl for the complex map layer (eg. wind particles)
- WeatherLayers (+ GFS) for the wind particles
- Cloudflare as a caching layer
The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.
This has just had a major update to include other renewables (eg hydro, solar) as well as non-renewable energy sources (gas).
It’s also added grid boundaries and cabling recently - and I think there’s also wind roses planned?
My one UI wish - the energy sources filters are currently displayed as options on the map - rather than controls for the entire UI - which meant I missed them entirely at first. Would be great if they could be given more prominence.
Definitely wind roses planned, I posted about them on my LinkedIn page if anyone is curious (linked on the main map).
That's a great point about the controls. Originally the map only showed wind farms and as I've expanded the features the control system (and really the entire UX) hasn't kept up. I need to sit down and have a proper think about how best to design the controls and documentation.
> New wind power records are set regularly, and between 5:30pm and 6:00pm on 5th December 2025 British wind farms averaged a record 23.94GW of generation.
Hey Robin, big fan of this map, congrats on getting front page on HN :).
Two suggestions/questions if you may:
1) Would be good to see how many MW each boundary can handle, not just %? Btw, I can't see the number for the south east england boundary.
2) Great job on the battery info. I'm seeing some battery storage is curtailed. How is that possible? Please don't tell me that we are paying batteries to _not_ export :/?
1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general)
2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today). You can dig into this yourself via the Detailed System Prices dataset [0] and looking at one of the batteries on the sell stack (eg. KILSB-5)
Love this! Would be super interested in any details the author could share on the data engineering needed to make this work. The vis is super impressive but I suspect the data is the harder thing to get working.
The most time and energy has been getting my head around the source data [0] and industry-specific nuances.
In terms of stack I have a self-hosted Dagster [1] data pipeline that periodically dumps the data onto Cloudflare R2 as parquet files. I then have a self-hosted NodeJS API that uses DuckDB to crunch the raw data and output everything you see on the map.
That's correct, it's using data from Open Infrastructure Map which is itself based on OpenStreetMap data. That's a good idea to zoom in further to show them off a bit more.
What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?
Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms.
Nice job dodging the awkwardness of whether to include Ireland, Northern Ireland, etc (IE and NI operate as a single, all-island wholesale electricity market). :)
I actually have plans to include Ireland and Northern Ireland once I get the GB side nailed down. The data seems to be mostly available though I'd have to really think about how to make it work well on the app as it'd be a pretty fundamental change to how things currently work (assumes a single market).
Some solar power assets are being tracked though unfortunately (for now) I'm only able to display data for assets that report on the balancing system in Great Britain. This means most solar has no data that I can easily access.
The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "balance" the various zones by turning up or down generators.
Yup! I have two ways to do it right now, the first is to click the plug-looking button on the map and this'll switch into curtailment mode to filter the assets that are currently being turned down due to transmission constraints, as well as the assets being turned-up elsewhere to balance things out. Also if you click on a curtailed assets you'll see the curtailment as a yellow area on the charts.
The other method is to visit my dedicated page [0] for tracking aggregate curtailment and turn-up, plus the costs for this.
Thanks for answering my curtailment question! I removed it from my comment as I commented before I found the data in your UX, but the extra context is useful. So much curtailment, much more transmission and battery storage required.
Oh wow, I made this! I didn't expect to see my map on here
Let me know if you have any questions or feedback and I'll do my best to respond to them.
Tech Stack used? Any specific non-technical difficulty getting this done ?
Tech stack is a mix of things, but the big-ticket items:
- Dagster for data pipeline
- Parquet files for data format
- Cloudflare R2 for data storage
- DuckDB for data processing
- NodeJS for API
- NextJS + React for the app
- MapLibre for the base map functionality
- Deck.gl for the complex map layer (eg. wind particles)
- WeatherLayers (+ GFS) for the wind particles
- Cloudflare as a caching layer
The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.
3 replies →
"rEnEwAbLeS"
Like the composites or even asbestos in wind turbine blades?
This has just had a major update to include other renewables (eg hydro, solar) as well as non-renewable energy sources (gas).
It’s also added grid boundaries and cabling recently - and I think there’s also wind roses planned?
My one UI wish - the energy sources filters are currently displayed as options on the map - rather than controls for the entire UI - which meant I missed them entirely at first. Would be great if they could be given more prominence.
Definitely wind roses planned, I posted about them on my LinkedIn page if anyone is curious (linked on the main map).
That's a great point about the controls. Originally the map only showed wind farms and as I've expanded the features the control system (and really the entire UX) hasn't kept up. I need to sit down and have a proper think about how best to design the controls and documentation.
FAQ also mentions it's only wind, so took me a while to notice :)
Would be nice to have a definition of 'Turn-up' and 'Curtailment' in that FAQ for us non grid savvy visitors.
Works great though, well done.
1 reply →
It's pretty windy here today! Around half of the UK's generation is currently coming from wind[1]
[1] https://grid.iamkate.com/
> New wind power records are set regularly, and between 5:30pm and 6:00pm on 5th December 2025 British wind farms averaged a record 23.94GW of generation.
Cool!
Semi-related: I also have a page where you can track the wind records as they happen
https://renewables-map.robinhawkes.com/records
This is really great. It’s fab to see this data being presented so clearly.
Been following your projects and writing since the Mozilla days, always impressed!
Oh wow that's a blast from the past, feels like a lifetime ago! And thank you.
Hey Robin, big fan of this map, congrats on getting front page on HN :).
Two suggestions/questions if you may:
1) Would be good to see how many MW each boundary can handle, not just %? Btw, I can't see the number for the south east england boundary.
2) Great job on the battery info. I'm seeing some battery storage is curtailed. How is that possible? Please don't tell me that we are paying batteries to _not_ export :/?
Hey! And thank you
1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general)
2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today). You can dig into this yourself via the Detailed System Prices dataset [0] and looking at one of the batteries on the sell stack (eg. KILSB-5)
[0] https://bmrs.elexon.co.uk/detailed-system-prices
> "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today).
Unfortunately you've got at least one negative wrong in this sentence and I'm still confused, and the linked dataset is currently blank?
Sorry for complaining, this is a great website!
2 replies →
Love this! Would be super interested in any details the author could share on the data engineering needed to make this work. The vis is super impressive but I suspect the data is the harder thing to get working.
The most time and energy has been getting my head around the source data [0] and industry-specific nuances.
In terms of stack I have a self-hosted Dagster [1] data pipeline that periodically dumps the data onto Cloudflare R2 as parquet files. I then have a self-hosted NodeJS API that uses DuckDB to crunch the raw data and output everything you see on the map.
[0] Mostly from https://bmrs.elexon.co.uk/ [1] https://dagster.io/
Just noticed the wind farms even have the individual units ( I assume ) and their interconnections mapped?!
Would show that off and zoom into them when selecting them from the list view!
That's correct, it's using data from Open Infrastructure Map which is itself based on OpenStreetMap data. That's a good idea to zoom in further to show them off a bit more.
What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?
Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms.
[0] https://openinframap.org/
Very cool. Thanks
Really nice project :)
Thanks!
Nice job dodging the awkwardness of whether to include Ireland, Northern Ireland, etc (IE and NI operate as a single, all-island wholesale electricity market). :)
I actually have plans to include Ireland and Northern Ireland once I get the GB side nailed down. The data seems to be mostly available though I'd have to really think about how to make it work well on the app as it'd be a pretty fundamental change to how things currently work (assumes a single market).
Great visualisation...but doesn't seem to have any display of the amount of Solar (though won't be much on such a cloudy day!)
Some solar power assets are being tracked though unfortunately (for now) I'm only able to display data for assets that report on the balancing system in Great Britain. This means most solar has no data that I can easily access.
I hope to work around this soon.
Can someone explain what the lines mean?
The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "balance" the various zones by turning up or down generators.
I note that one of those lines seems to take a big detour up and round Cruachan?
https://www.visitcruachan.co.uk/
3 replies →
Great work, this is so cool to see!
Thank you
Very well done, beautiful data communication.
Yup! I have two ways to do it right now, the first is to click the plug-looking button on the map and this'll switch into curtailment mode to filter the assets that are currently being turned down due to transmission constraints, as well as the assets being turned-up elsewhere to balance things out. Also if you click on a curtailed assets you'll see the curtailment as a yellow area on the charts.
The other method is to visit my dedicated page [0] for tracking aggregate curtailment and turn-up, plus the costs for this.
[0] https://renewables-map.robinhawkes.com/curtailment
Thanks for answering my curtailment question! I removed it from my comment as I commented before I found the data in your UX, but the extra context is useful. So much curtailment, much more transmission and battery storage required.