Comment by bulba4aur

I actually though about it multiple times over at this point.

You're right, this deserves more attention, and is a valid problem going forward with this app. And I had this problem when just started building, it either generated XSS tests for any user input validation method (even if it used other validators) or just 1 single test case.

For now I attempt to strictly limit the amount of tests for LLM to generate.

This is achieved with "Planner" that plans the tests for each function before any generation happens, that agent is instructed to generate a plan that follows the criteria:

- testCases.category MUST be one of "happy_path" | "edge_case" | "error_handling" | "boundary".

And it is asked to generate 2-3 tests for each category. While this may result in the unnecessary tests, it at least tries to limit the amount of them.

Going forward I believe the best approach would be to tune and tweak the requirements based on the language/framework it detects.