Comment by a_bonobo
17 hours ago
I've been using Claude Science a lot and it is VERY good at finding patterns in the DNA around my binding sites - quite often it went 'you could put your primer here but that looks like an Alu repeat, so better not, the primer won't be specific' - it seems like the press release is one step above that pattern recognition? I.e., 'there's a recurring motif here that hasn't been described before', which is probably straightforward to pick up when your context window is 1 million tokens, i.e. within the range of entire bacterial genomes...
Curious about this: do you expect that Claude science will also scoop interesting directions looking at anonymous usage, to publish posts like this before the original author (similar to the Navier Stokes debacle)?
Isn't finding this out from an LLM somewhat... complex and non reproducible.
Everyone who survived 7 rounds of multi model reviews and they still keep finding mediums in their PRs is not in the least surprised. These things are not oracles - they miss stuff all the time even when told to look.
I'm convinced that the LLMs are capable of finding everything in one shot but that's not good for token usage so they only report a few at a time.
1 reply →
Yep :) it's VERY non-reproducible - but I haven't asked it to look for Alu repeats in the first place, I can then go and reproduce the work it's doing
Do you find it a big improvement over the tools you used previously?
It's honestly harder - you have to do a lot of extra work to ensure your work is traceable. It's super fast in doing things you have no overview of - it makes pretty figures, it runs command line jobs etc. and I'm sure there are mistakes in there. For now I'm pretending Claude Science is an IDE, like Positron/VSCode, and I have to keep enforcing proper git usage etc. so I can reproduce this work
Edit: compared to my tools before, it generally uses the same tools in the same way, just 20x faster than me and I mostly struggle to keep up and verify what it's doing
Sounds exactly like Claude Code (and ilk) tbh. Just a lot of the stakes are lower and a lot of people are more comfortable with it (e.g. there were always people blindly copying and lasting from StackOverflow) I suppose.
1 reply →
> have to keep enforcing proper git usage etc. so I can reproduce this work
Depending on what you mean here, it might be worth looking at jj, which works with git repositories. One of the features is that everything gets committed at change time (kind of) which may or may not be helpful to you here.
always add rules for it to never touch git besides git log. you do the commits. make reviews so much easier