They were dealing with small datasets or infinite datasets, and double decent only really works when the patterns in your test set are similar enough to those in your training set.
While you do need to be mindful about some of the the older opinions, the fundamentals are the same.
For fine tuning or RL, the same problems with small datasets or infinite datasets, where concept classes for training data may be novel, that 1992 paper still applies and will bite you if you assume it is universally invalid.
Most of the foundational concepts are from the mid 20th century.
The availability of mass amounts of data and new discoveries have modified the assumptions and tooling way more than invalidating previous research. Skim that paper and you will see they simply dismissed the mass data and compute we have today as impractical at the time.
Find the book that works best for you, learn the concepts and build tacit experience.
Lots of efforts are trying to incorporate symbolic and other methods too.
IMHO Building breadth and depth is what will save time and help you find opportunities, knowledge of the fundamentals is critical for that.
Have not read the book, but only deep learning has had such wild advancement that a decade would change anything. The fundamentals of ML training/testing, variance/bias, etc are the same. The classical algorithms still have their place. The only modern advancement which might not be present would be XGBoost style forests.
This is one of the most acronym heavy discussions I've ever seen. I searched "AIMA/PRML/ESL" to find the books, and the first result is a Reddit thread with most upvoted comment "Can we use the names of the books instead of all acronyms, not everyone knows them lol".
Depends on what your goal is. If you’re just curious about ML, probably none of the info will be wrong. But it’s also really not engaging with the most interesting problems engineers are tackling today, unlike an 11 year old chemistry book for example (I think). So as interview material or to break into the field it’s not going to be the most useful.
I have read parts of it. It arguably was already "outdated" back then, as it mostly focused on abstract mathematical theory of questionable value instead of cutting edge "deep learning".
Even Russel and Norvig is still applicable for the fundamentals, and with the rise of agenic efforts would be extremely helpful.
The updates to even the Bias/Variance Dilemma (Geman 1992) are minor if you look at the original paper:
https://www.dam.brown.edu/people/documents/bias-variance.pdf
They were dealing with small datasets or infinite datasets, and double decent only really works when the patterns in your test set are similar enough to those in your training set.
While you do need to be mindful about some of the the older opinions, the fundamentals are the same.
For fine tuning or RL, the same problems with small datasets or infinite datasets, where concept classes for training data may be novel, that 1992 paper still applies and will bite you if you assume it is universally invalid.
Most of the foundational concepts are from the mid 20th century.
The availability of mass amounts of data and new discoveries have modified the assumptions and tooling way more than invalidating previous research. Skim that paper and you will see they simply dismissed the mass data and compute we have today as impractical at the time.
Find the book that works best for you, learn the concepts and build tacit experience.
Lots of efforts are trying to incorporate symbolic and other methods too.
IMHO Building breadth and depth is what will save time and help you find opportunities, knowledge of the fundamentals is critical for that.
Have not read the book, but only deep learning has had such wild advancement that a decade would change anything. The fundamentals of ML training/testing, variance/bias, etc are the same. The classical algorithms still have their place. The only modern advancement which might not be present would be XGBoost style forests.
Machine Learning concepts have been around forever, they just used to call them statistics ;0
Nope, and AIMA/PRML/ESL are still king!
Apart from these 3 you literally need nothing else for the very fundamentals and even advanced topics.
This is one of the most acronym heavy discussions I've ever seen. I searched "AIMA/PRML/ESL" to find the books, and the first result is a Reddit thread with most upvoted comment "Can we use the names of the books instead of all acronyms, not everyone knows them lol".
You're right.
AIMA is Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig.
PRM is Pattern Recognition and Machine Learning by Christopher Bishop.
ESL is Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman.
Depends on what your goal is. If you’re just curious about ML, probably none of the info will be wrong. But it’s also really not engaging with the most interesting problems engineers are tackling today, unlike an 11 year old chemistry book for example (I think). So as interview material or to break into the field it’s not going to be the most useful.
I have read parts of it. It arguably was already "outdated" back then, as it mostly focused on abstract mathematical theory of questionable value instead of cutting edge "deep learning".
Any recommendations?