Comment by beeflet

1 year ago

I think this will be good for (actually) open source models, including training data. Because that will be the only way to confirm the model isn't hijacked

7 comments

beeflet

Reply
fl0id  1 year ago

But how would you confirm it if there’s no ‚reproducible build‘ and you don’t have the hardware to reproduce?

  • svachalek  1 year ago

    That's the point, there needs to be a reproducible model. But I don't know how well that really prevents this case. You can hide all kinds of things in terabytes of training data.

    • Imustaskforhelp  1 year ago

      Most ai models will probably shift to mixture of experts. Which has small models.

      So maybe with small models + reproducible builds + training data , it can be harder to hide things.

      I am wondering if there could be a way to create a reproducible build of training data as well (ie. Which websites it scraped , maybe archiving them as it is?) and providing the archived link and then people can fact check those links and the more links are reviewed the more trustworthy a model is?

      If we are using ai in defense systems. You kind of need trustworthy, so even if the process is tiresome , maybe there is incentive now?

      Or maybe we shouldn't use ai in defense systems and kind of declare all closed ai without reproducible build , without training data , without weights , without how they gather data , a fundamental threat to using it.

      1 reply →