Comment by bricee98
2 years ago
I like the idea, but when I started practicing some Spanish, I noticed it gave me incorrect grammar advice (it claimed that the subjunctive form of “creer” is “creen” - it was right that the subjunctive was needed, but that would be “crean”). That makes me nervous to use it for a language that I know less well like Korean because I won’t be able to tell if the grammar feedback I’m getting is accurate.
If there’s a way to make it more robust on that front I could definitely see myself using it!
We're using LLMs and good ol'ML. These systems are never going to be 100% accurate. Then again humans are also not 100% correct and we're working hard on ironing out the kinks like the one you just discovered.
That is true of LLMs, but if you’re looking to replace iTalki, the standard to meet or beat on grammar correctness and explanation is that of a human instructor - this is not the type of mistake that my native tutor would make, and if he somehow did, I would find a new tutor that I could be confident is teaching me correct grammar.
I certainly don’t point it out to be discouraging - on the contrary, I feel like I am the exact target audience for a product like this and would happily pay for it if it can reach or be near the trustworthiness of a human teacher.
How are you guys assessing the accuracy? Sure, LLM's aren't going to be 100% accurate. But if I pick up a Spanish textbook, there is a right answer 100% of the time (okay, aside from publishing errors)
If you can't make any promises as to the quality of what you're offering, then how can you reasonably expect people to pay for it? At the end of the day you need to be able to tell consumers what they can reasonably expect from your product in terms of its capabilities and accuracy.
It could be somewhat embarassing for someone who trusts the app and then says the wrong thing to another human. As an off-the-cuff idea (not because you haven't already thought deeply but because I hate to find a problem without offering a solution), maybe use videos of real speakers for basic instruction, then the AI for responsive lessons, etc.?
As you no doubt have considered, but if you define you mission by the tool (AI) and not the outcome (accurate, faster language acquisition) you'll have a great tool and lesser outcomes. Whatever you prioritize, that's what you'll get! :)