← Back to context Comment by bob1029 10 months ago Where are the model parameters stored and how are they represented? 4 comments bob1029 Reply esafak 10 months ago In disk or memory as multidimensional arrays ("tensors" in ML speak). bob1029 10 months ago Do we agree that these memories consist of a finite # of bits? esafak 10 months ago Yes, of course.Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time. 1 reply →
esafak 10 months ago In disk or memory as multidimensional arrays ("tensors" in ML speak). bob1029 10 months ago Do we agree that these memories consist of a finite # of bits? esafak 10 months ago Yes, of course.Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time. 1 reply →
bob1029 10 months ago Do we agree that these memories consist of a finite # of bits? esafak 10 months ago Yes, of course.Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time. 1 reply →
esafak 10 months ago Yes, of course.Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time. 1 reply →
In disk or memory as multidimensional arrays ("tensors" in ML speak).
Do we agree that these memories consist of a finite # of bits?
Yes, of course.
Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time.
1 reply →