Comment by 7777777phil

You're right that a simple Python script would be more cost-effective for this kind of archiving. I went with workers because I was already familiar with the stack and wanted real-time processing, but for a research project focused on completeness rather than latency, your approach makes much more sense - please reach out if you want to offer your help. Initially I was planning on building a public realtime dashboard and might as well still do.