← Back to context

Comment by banku_brougham

3 years ago

I've seen this too. I guess 50% of query load were jobs that got deprecated in the next quarterly baseline.

It felt a system was needed to allocate query resource to teams, some kind of tradeable tokens that were scarce maybe, to incentivise more care and consciousness of the resource from the many users.

What we did was have a few levels of priority managed by a central org. It resulted in a lot of churn and hectares of indiscriminately killed query jobs every week, many that had business importance mixed in with the zombies.