Comment by saagarjha
2 days ago
How are you checking that the AI is not just gaming your correctness tests? It's very easy to write incorrect synchronization for example.
2 days ago
How are you checking that the AI is not just gaming your correctness tests? It's very easy to write incorrect synchronization for example.
It cant if you be cautious about it because the inputs can set manually and outputs are generated through the cuda harness by executing the reference kernel, again can be provided externally.
Comparison is simply byte by byte equalness check of reference kernel outputs with candidate (optimized) outputs.
Why I added ai generated inputs then? I was just being lazy and this was more of a langgraph playground for me:)
Yes, but the AI can totally make a kernel that passes your test inputs but is not correct