Comment by CSDude

I feel like there is a great potential to be explored here by playing with cgroups dynamically, not in a machine learning way but allowing bursts, finding good ratios request/limit to pick up (1s/10s or 0.1s/1s ?) and voluntarily kicking out (eviction) stateless workloads.

I even pursued my PhD on it until I quit (unrelated reasons). There was a startup doing this with ML but forgot their name.