Thanks! We provide eval templates that can be applied on specific stages or the whole conversation. Users can specify their own evals that can be as granular as they'd like. We're also working on conversation simulation feature that lets users quickly iterate on evals via simulating previous real conversations and seeing if the eval output aligns with human judgement.
P.S. Arkadiy is locked out of his HN account due to the anti-procrastination settings. HN team, can you plz help? :)
Thanks! We provide eval templates that can be applied on specific stages or the whole conversation. Users can specify their own evals that can be as granular as they'd like. We're also working on conversation simulation feature that lets users quickly iterate on evals via simulating previous real conversations and seeing if the eval output aligns with human judgement.
P.S. Arkadiy is locked out of his HN account due to the anti-procrastination settings. HN team, can you plz help? :)