Comment by goranmoomin

3 days ago

I'm genuinely curious on how well this is working, is there an independent Java test suite that covers major Java 5/6 features that can verify that the JOPA compiler works per the spec? I.e. I see that Claude has wrote a few tests in it's commits, but it would be wonderful if there's a non-Clauded independent test suite (probably from other Java implementations?) that tracks progress.

I do feel that that is pretty much needed to claim that Claude is adding features to match the Java spec.

5 comments

goranmoomin

Reply
pshirshov  3 days ago

Well, it's complicated. The original jdk compliance tests are notoriously hard to deal with. Currently I parse nearly 100% of positive testcases from JDK 7 test suite (in one of Java 7 branches) but I only have several dozens of true end to end tests (build .java with jopa, validate classfile with javap, run classfile with javac).

So, I can't tell how good it actually is but it definitely handles reasonably complex source files with generics (something the original compiler was unable to do).

The actual goal of the project is to be able to build at least ANT to simplify clean bootstrap of OpenJDK.

  • AtlasBarfed  3 days ago

    That is perilously close to the usual:

    "AI DID EVERYTHING IN A DAY"

    "How do you know it works?"

    "... it just looks like it does"

    Like when I ask AIs to port sed to java, and it writes test cases ... running sed on a CLI and doesn't implement the full lang spec no matter how much prompting I give it.