Comment by captainclam
13 hours ago
I completely understand where you're coming from. While understanding the math and architectures of these large models can be edifying, it lacks some of the liberating/empowering spirit of traditional software. You can build a bespoke to-do app that is better suited to your needs than anything produced by some productivity software company, but you're totally beholden to one of the few big players for access to an effective LLM.
But there is so, so much interesting work and research to do at a much smaller scale. I work on drug discovery models, and none of my work requires data or infrastructure that are not generally accessible to the public (it's nice to have the computing resources of my institution but there are other ways to access a GPU).
So if you are really interested in this stuff, I would suggest looking at the endless supply of interesting problems in domains that are not dominated by huge frontier labs.
Ive been interested in exploring other domains as well, including biology, genomics, etc. Do you have any recommendations on where to get started?
For genomics, look into viruses, in particular RNA viruses like influenza and HIV. You can fit the entire genome on your screen and learn to read it like code! There are some good text books about viruses that include genomics from scratch (I can't find the one I used now but could look later if you're interested). My advice would be to learn the biology first and understand why we care about genomes before looking into the computational stuff.