Comment by jawiggins
2 years ago
The E.O. also requires that a model be reported if it:
> was trained using a quantity of computing power greater than 10^26 integer or floating-point operations, or using primarily biological sequence data and using a quantity of computing power greater than 10^23 integer or floating-point operations
However later it also says that reported is needed for, "Companies developing or demonstrating an intent to develop".
If I start training a CNN on an endless loop, do I become subject to these reporting requirements?
Also the fops requirement is not that high. An H100 does 3,958 fp8 flops. So it would take,
> >>> (10 * 23) / (3958 * (10 * 12)) / 86400 > 292.422
292 days until you have a regulated model.
CNN in an endless loop would hit the letter of the law (and not necessarily unfairly, biggish architecture/data combos seem to get better with more training far past what you'd expect). The spirit of the law and your adherence thereto will be decided by the courts and your individual circumstances.