← Back to context

Comment by ggambetta

18 days ago

"Brute force" would be trying random weights and keeping the best performing model. Backpropagation is compute-intensive but I wouldn't call it "brute force".

4 comments

ggambetta

Reply
Ygg2  18 days ago

"Brute force" here is about the amount of data you're ingesting. It's no Alpha Zero, that will learn from scratch.

  • jazzpush2  18 days ago

    What? Either option requires sufficient data. Brute force implies iterating over all combinations until you find the best weights. Back-prop is an optimization technique.

    • Ygg2  17 days ago

      In context of grandparents post.

           > You determine the weights via brute force. Simply running a large amount of data where you have the input as well as the correct output

      Brute force just means guessing all possible combinations. A dataset containing most human knowledge is about as brute force as you can get.

      I'm fairly sure that Alpha Zero data is generated by Alpha Zero. But it's not an LLM.

      1 reply →