Comment by Legend2440
1 year ago
Your neural network (LLM or otherwise) could be undetectably backdoored in a way that makes it provide malicious outputs for specific inputs.
Right now nobody really trusts LLM output anyway, so the immediate harm is small. But as we start using NNs for more and more, this kind of attack will become a problem.
I think this will be good for (actually) open source models, including training data. Because that will be the only way to confirm the model isn't hijacked
But how would you confirm it if there’s no ‚reproducible build‘ and you don’t have the hardware to reproduce?
That's the point, there needs to be a reproducible model. But I don't know how well that really prevents this case. You can hide all kinds of things in terabytes of training data.
2 replies →
well, not everyone has hardware to build large software anyway. like chrome requires 20+ cores and 64+ gb ram
- https://chromium.googlesource.com/chromium/src/+/main/docs/w...
This also incentivizes them to produce reproducible builds. So training data + reproducible build
maybe through some distributed system like BOINC?