Comment by bensyverson

I guess we’ve just reached the point where everyone has to state the obvious, and common sense is extremely uncommon.

So here goes: you should not use an AI model to validate a claim which is trivial to calculate deterministically. That is (obviously?) not what a model like Jev is for, thus it is not a good test of Jev.

yeah, technically. (/s)

    python3 - <<'EOF'
    import json, urllib.request
    body = json.dumps({
        "state": "The car wash is only 100 meters away from my house.",
        "model": "jev-1.13-free",
        "questions": {"q": {"type": "choice",
            "instructions": "Should I drive or walk to the car wash?",
            "criteria": {"drive a car": None, "walk": None}}}
    }).encode()
    req = urllib.request.Request("https://opencode.ai/zen/v1/systemone", data=body,
        headers={"Content-Type": "application/json", "User-Agent": "opencode/1.18.31"})
    with urllib.request.urlopen(req, timeout=60) as r:
        print(json.dumps(json.load(r)["answers"]["q"], indent=2))
    EOF
    {
      "type": "choice",
      "choice": "walk",
      "confidence": 0.66,
      "probabilities": {
        "walk": 0.83,
        "drive a car": 0.17
      }
    }