Comment by epakai
1 year ago
You can look at catalog.data.gov it shows totals. I'm comparing January 14th to today.
The biggest loss I see is Organizations - Department of Energy, 5473 to 3647. I also see under Bureaus - Energy Programs, 4347 to 2521. These are overlapping categories (-1826 on both).
There are others, but they seem smaller, a few hundred at most.
Looking closer, there was a major increase quite recently before this decrease. On January 8th there were 3617 DoE datasets. On the 14th it was up to 5473. Now we're back down to just 30 more than on the 8th.
Could it have been a publishing mistake, or some order to undo recent publications for review by the new admin?
This seems like a super important observation.
If the number of dataset saw a massive jump (50%), then back down a week later, that seems more like the correction of an error.
Maybe its related to climate change and advantages of renewables?
I know DoE does "other things", but I don't expect these to be public anyway.
I can't tell from the Internet Archive. The datasets that went away seem to have been added quite recently, between January 8th and 14th. There isn't a suitable capture between January 8th and 25th to see which tags or categories changed within the DoE datasets.
Common Crawl's latest crawl was Jan 12th-25th, and the index is available.