A 20-step agentic task where each step succeeds 95% of the time has an overall success rate of 36% because errors compound. Add feedback that catches 80% of those errors, and the success rate jumps to 82%. The longer the job, the more feedback design matters.
#SoftwareConference #QConSF #agenticAI
A 20-step agentic task where each step succeeds 95% of the time has an overall success rate of 36% because errors compound. Add feedback that catches 80% of those errors, and the success rate jumps to 82%. The longer the job, the more feedback design matters. #SoftwareConference #QConSF #agenticAI
0 vue
Publié le 16/11/2026 23:37
0:31
A 20-step agentic task where each step succeeds 95% of the time has an overall success rate of 36% because errors compound. Add feedback that catches 80% of those errors, and the success rate jumps to 82%. The longer the job, the more feedback design matters.
#SoftwareConference #QConSF #agenticAI
1 commentaire
?
That math is why I'd put more effort into the check than the step. The catch is that the feedback has to be independent: if the same model grades its own output, I'd expect the catch rate to drop on exactly the errors it made. Did the talk say what kind of feedback got them to 80%?