Comment by ssivark

TBC, I too did not really mean "AI" (as in LLMs and the like) which is often hosted/served with a very convenient interface. I do include more bespoke statistical / mathematical models -- be it hierarchical, Bayesian, whatever.

Since AWS/etc are quite complicated, there are now a swarm of startups trying to make it easier to take baby steps into the cloud (eg. Modal, Runpod, etc) and make it very easy for the user to get a small slice of that GPU pie. These services have drastically simpler server-side APIs and usage patterns, including "serverless" GPUs from Modal, where you can just "submit jobs" from a Python API without really having to manage containers. On the client side, you have LLM coding agents that are the next evolution in UI frontends -- and they're beginning to make it much much easier to write bespoke code to interact with these backends. To make it abundantly clear what target audience I'm referring to: I imagine they are still mostly using sklearn (or equivalents in other languages) and gradient boosting with Jupyter notebooks, still somewhat mystified by modern deep learning and stuff. Or maybe those who are more mathematically sophisticated but not software engg sophisticated (eg: statisticians / econometricians)

To inspire you with a really concrete example: since Modal has a well documented API, it should be quite straight-forward ("soon", if not already) for any data scientist to use one of the CLI coding agents and

1. Implement a decent GPU-friendly version of whatever model they want to try (as long as it's not far from the training distribution i.e. not some esoteric idea which is nothing like prior art)

2. Whip up a quick system to interface with some service provider, wrap that model up and submit a job, and fetch (and even interpret) results.

----

In case you haven't tried one of these new-fangled coding agents, I strongly encourage you to try one out (even if it's just something on the free tier eg. gemini-cli). In case you have and they aren't quite good enough to solve your problem, tough luck for now... I anticipate their usability will improve substantially every few months.