Comment by gbnwl
17 hours ago
Why are you using 2 seconds? The commenter you are responding to hypothesized being able to do 250/s based on "100 parallel inference at 5 at a time". Not speaking to the validity of that, but find it strange that you ran with the 2 seconds number after seemingly having stopped reading after that line, while yourself lamenting people don't read and telling them to "read again".
Ok, let me dumb it down for you: you have a cockroach in your bathroom and you want to kill it. You have an RPG and you have a slipper. Are you gonna use the RPG or are you going to use the slipper? Even if your bathroom is fine after getting shot with an RPG somehow, isn't this an overkill? If you can code and binary classifier train a classifier in 2 hours that uses nearly 0 resources and gives you good enough results(in my case way above what my targets were) without having to use a ton of resources, libraries, rags, hardware and hell, even electricity? I mean how hard is this to comprehend really?
https://deviq.com/antipatterns/shiny-toy
This thread is chock full of people who have no clue about what traditional AI even is. I'm sorry you have to deal with literal children
Sure, but this doesn't answer my question nor tie into your last comment at all. It's Saturday evening in much of the world, are you sober?
OP said 2 seconds as if that wasn't an eternity...
But then they said 250/second when running multiple inference? Again I don't know if their assertions about running multiple inference are correct but why focus on the wrong number instead of addressing the actual claim?
250/s is still nothing when compared to an actual NLP pipeline that takes a few ms per it, because you can parallelize that too.
I know it's hard to understand, but you can achieve a throughput that is a few orders of magnitude higher.