← Back to context

Comment by gavinray

18 hours ago

Did anyone read the blogpost? They did publish a pre-print:

  > Our work to understand the primary function of ARTs is ongoing. However, we think it is important to share such findings early, both to demonstrate Claude’s capabilities and to give the broader community insight into what we’re working on. We have released a pre-print (here) that discusses this in more detail.

https://www-cdn.anthropic.com/22573675ada52a8ca8a97a1a4b4326...

20 comments

gavinray

Reply
bonsai_spool  18 hours ago

I don't recall that being there when I first read their post a few hours ago... No way to confirm, sadly, as they've posted it to their own domain

epihelix  16 hours ago

I just looked at it. I really hope they're not thinking of sending that to an actual bioinformatics, computational biology or molecular biology journal! So embarrassing...

(I love how Anthropic boast about building a lab, but don't seem to realise that you have to test your hypothesis in the lab! Right now, all their "spectacular" assertions are untested and unproven.)

I realise that this will only improve from here, but gods Anthropic has no idea about the biological sciences right now.

  • dekhn  16 hours ago

    Um. Take a look at the authors. Every single one of them is an expert in this field. They did test this in a lab.

    If you want to complain about things like this, it really helps to be specific. Given the author list, it's unlikely they made any truly spectacular errors (and also possible the system they studied is not interesting).

beAbU  18 hours ago

What's with all this pre-print business. It became very prevalent during covid, where it felt like every week some new pre-print was published that discusses some new aspect of the virus. These papers would then be used in arguments and put forward as proof of whatever claim the arguer was making.

Every man and his dog can publish a pre-print and in my opinion it's academically worthless.

  • ACCount39  17 hours ago

    Preprints are just a way to sacrifice rigor for accessibility and velocity. You can throw out "here, this is what I'm working on, here are the quick and dirty findings" really fast and with little friction.

    This does skip the academic "checks and balances" like journal selection and peer review - but it can also help anyone else who's working on the adjacent topics.

    If a field is moving fast, and you think there can be some value in your work for others in the near term? Preprint. If your work is too incomplete or too minor to warrant trying to polish and publish it, but you don't want to table it? Preprint. Too deep in corporate structures to care about academic "street cred", and want your work to be accessible? Preprint. Have an exciting early finding that you want to push out there, and are willing to take the rep risks of being wrong about it? Preprint.

    There's a reason why preprints came to be the lifeblood of ML.

    • VBprogrammer  17 hours ago

      Academia isn't my thing but I also wonder if there isn't an aspect of putting a stake in the ground? So that if someone beats you to publishing you at least have some record of being on that track.

      1 reply →

  • sandeepkd  17 hours ago

    I have a similar to pagerank method I use to evaluate such papers. I look for the references to see how many authors are using their own references (past work), the idea being that people do not jump too far, they make incremental progress.

    For the pre-print I could only find only one author who has a single referenced article.

    • dekhn  12 hours ago

      The pre-print authors all have long publication histories.

      Pagerank was inspired by academic citation networks; it just turned it in a recursive matrix problem (of which there was some prior literature).

      1 reply →

  • howunfortunate  18 hours ago

    Journal articles are usually behind a paywall. Preprints are a fully open workaround allowed by most journals.

    > Every man and his dog can publish a pre-print and in my opinion it's academically worthless.

    Sure but if you look at the authors names and see they have 50 other published papers, you can get a rough idea that it's probably equivalently good to their other work.

    Until you've done it yourself, it's hard to grok just how bad the peer review process is. It's like...5% better than nothing.

    Honestly you could argue peer review is worse than nothing, as it also filters out actually quality work that violates some dogma of the field.

  • bpodgursky  17 hours ago

    The review time on top journals is multiple years now (your paper will go through many review loops each of which takes months). It's just totally unworkable for active research, whether you're a student or a corporation.