Comment by eigenblake
3 hours ago
Woah this is so interesting. Confirmed it myself, you actually implemented elision!!! I really like your idea of capturing provenance in-document as well. This is humans speech, right? If you can do forced alignment on Latin, it gets me wondering if one could use a wav2vec especially turned to convert IPA into speech somewhat deterministically to compensate for how there is not heaps of training data in Latin.
Haha, thank you for using multiple exclamation marks!!! I'm thrilled that someone is as excited as I am about this feature, it made all the time I spent manually correcting/marking all of Gemini's elision hallucinations worthwhile :)
The recordings were made by my friend and Latin tutor. I used an Italian wav2vec2 model fine-tuned on Latin[0], which works pretty well for Latin only content, but falls down with mixed English / Latin. I found that some of the slower, larger Meta models worked fine for this use case though.
Some folks on the Latin language Discord said they got pretty good results using Kokoro TTS and X-Voice for creating synthetic Latin, as you suggested using IPA.
[0]: https://huggingface.co/lsb/wav2vec2-base-it-latin