Comment by matheusmoreira

5 days ago

I just asked Fable for a complete code review of my lone lisp project. Started out strong. Launched Fable agents, then spent like 10 minutes thinking... And then got interrupted by a switch to Opus 4.8.

> Fable 5's safety measures flagged this message for cybersecurity or biology topics.

> They may flag safe, normal content as well.

> These measures let us bring you Mythos-level capability in other areas sooner, and we're working to refine them.

Here are the results of the agentic code review session:

  ┌──────────────────────────┬───────────────┬────────────────┐
  │          Agent           │ Fable 5 turns │ Opus 4.8 turns │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ values                   │ 134           │ 0              │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ data-intrinsics          │ 104           │ 0              │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ tools-tests-build        │ 81            │ 0              │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ core-intrinsics (failed) │ 25            │ 0              │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ system-memory            │ 44            │ 20             │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ reader-modules           │ 104           │ 25             │
  ├──────────────────────────┼───────────────┼────────────────┤
  │ linux-startup            │ 95            │ 15             │
  └──────────────────────────┴───────────────┴────────────────┘

This 40 minute session cost me 16% of my weekly usage. A simple code review of the most critical areas of my project got flagged as a cybersecurity risk. It really made me not want to try it again.

4 comments

matheusmoreira

Reply
kordlessagain  5 days ago

Same. I asked for a security review and it immediately triggered. I then started a new session and asked for a software review and it ran for a bit before getting tripped on token usage by the project.

  • andai  5 days ago

    This is interesting. Security issues are bugs. So if you ask it to look for bugs, it will also find security issues. Is that a workaround for the "no cybersec" rule?

    Or is it just not allowed to find bugs? Or it's only allowed to tell you bugs that don't pose a security risk?

    • matheusmoreira  5 days ago

      > Or it's only allowed to tell you bugs that don't pose a security risk?

      Seems that way. "Security" was never part of the prompt. It was something like:

      > Hello, Fable! Can you give me a complete code review of my lone lisp project? Opus has already done extensive code review. I'm curious to see what you say.

      Result was the table above.

      1 reply →