← Back to context

Comment by casper14

12 hours ago

What are some limitations you have found with using a smaller model like that?

As a developer, none related to the model. It’s faster, lighter, makes less assumptions.

For a casual consumer, it is no doubt a worse chat buddy. Knows less historical facts à la Wikipedia.

But I’m not really using LLMs for that kind of entertainment and I don’t rely on them for fact-checking anyway. It’s almost worse to rely on a smoother talker for something it can’t possibly know.

For feeding in thousands of daily updates and getting it to predict the next one based on a crude list of tasks, it’s great. For completing code files in my style, also great. It can also handle most small customer service issues and refunds related to my apps on App Store, the back and forth to collect data from users.

For a booking engine I run with a partner, it is great at confirming bookings and following up. It handles cancellations which are about 40% of confirmed bookings due to the nature of the business. I used to feel like I was wasting my time with them - there is no way we can charge for it. Having basically a smart bot do it makes everyone happier (even the customer who knows they aren’t hurting anyone’s feelings).

For invoices, I prefill notes and things based on previous ones. It’s fine. I don’t need a massive model that takes 30s to reply etc. it’s way overkill. Maybe worse than overkill - off in a direction I don’t need.