Comment by erikschoster
4 hours ago
It does appear to synthesize the audio with a wasm module compiled from rust, presumably developed as a python script first based on the source comments.
https://github.com/celphase/duck-pond/blob/main/src/synth.rs
To the credit of the author there is also an attribution file which describes a process of deriving the models from recordings (none of the techniques are very novel, but it's fairly impressive to see automated -- edit: I assume the project is LLM-generated) https://github.com/celphase/duck-pond/blob/main/ATTRIBUTION....
The project was created with significant LLM use, I'd add claude as a co-contributor if I could do so manually. I don't personally like to have it added as a co-author on commits on my own projects, though I always make sure it's there if I submit PRs using it.
Honestly I can't take credit for any major input on it other than a few technical choices. Audio is entirely outside my wheelhouse beyond what happens at the platform API level.
The process involved running the LLM in a loop to gradually correct parameters of a synthesis stack to have the spectrogram of the output match the spectrogram of a real recorded sample. One funny part is where the synthesis was marked by a classification model as more duck than the real duck sample.
> the synthesis was marked by a classification model as more duck than the real duck sample
The quality of a recording can throw off most classifiers.
Models we have today are so dependent on the quality of training data, that it's questionable how we will sustain long term what we are currently calling "AI".
i.e. What happens when the cameras and microphones everyone has in their pockets are significantly higher quality and the AI model hasn't had a total refresh in a long time? How does that cause classification to drift enough to fail?