Comment by ryzvonusef
13 hours ago
Karpathy's career arc feels similar to Jim Keller's; a butterfly flitting from one flower to another, gathering experiences and creating magic everywhere they go.
13 hours ago
Karpathy's career arc feels similar to Jim Keller's; a butterfly flitting from one flower to another, gathering experiences and creating magic everywhere they go.
I don't think Karpathy has nearly the portfolio of accomplishments. I think of him more as an educator.
> creating magic everywhere they go
Like specifically what has he done?
I can spare a minute :). This isn't exhaustive because this is just stuff I know of, obviously.
- At Stanford, Led research on the first (to my knowledge) crop of joint image/text models. Super widely cited work.
- At Tesla, led their whole self driving effort for a while, came up with critical techniques that allowed them to make progress (e.g., the concept of "auto labelling": using a much larger NN to generate training data with which to train smaller models that could fit in the on-device compute. IIRC, Elon said they would not have been able to make progress without this insight).
I'm not sure his educative efforts for the mold of what you're looking for, but if so, the course he designed at Stanford (and availed online):for neural networks, as well as his blog posts, (most famous of which, to my knowledge, is "the unreasonable effectiveness of LSTMs"), made a huge impact on educating a generation of tinkerers and researchers.
The auto labeling work (which has been partially described/presented at Tesla AI day events) seems more like engineering than research, a grab bag of techniques that I would guess the whole team must have contributed to. For example, they auto label low resolution/indeterminate objects (image segments) by temporal continuity... Something that is a low-res blob in the distance becomes a hi-res and easy to identify object when you drive by it, so by tracking objects backwards across frames you can learn how to more confidently label the lo-res blob. Things like this are useful, but it's the sort of stuff that engineers and developers are coming up with every day.
2 replies →
Karpathy is also badmephisto, a name you might have heard of if you're into cubing.
http://badmephisto.com/
Tesla still hasn't achieved their 2016 self-drive goal by their self imposed deadline of 2017, even now a decade later. So, politely, is that accolade merited?
2 replies →
Add microgpt to that list
It wasn't LSTMs, it was RNNs.
1 reply →
Thank you!
I was more looking for signal that him + Anthropic might yield something beyond a step-change from Opus 4.7 (disappointing so far). We have not gotten to use Mythos yet, I wonder if that will become Opus 5 or something.
[dead]