Action, Not Belief: What Makes Hypothesis Prompting Work in Debugger Agents

摘要

Recent work has introduced hypothesis-driven debugging into LLM-based program repair, prompting agents to formulate hypotheses, predict observations, and test them with debugging tools. However, these methods combine hypothesis generation with guidance for subsequent actions, leaving the source of their repair gains unclear. We disentangle these components through controlled comparisons under a fixed debugger interface. To examine whether prediction--observation agreement tracks repair performance, we introduce HypoCal, a measure of agreement between stated runtime predictions and subsequent debugger observations. Our experiments show that injecting hypotheses into the next action prompt improves repair, whereas eliciting hypotheses alone does not. The gain disappears when the injected hypothesis concerns another bug or misidentifies the fault location. We find that prediction--observation agreement remains low and does not track repair gains. An analysis of tool use shows that hypothesis injection shifts the agent from source inspection toward runtime probing in our primary experiments. Across four backends, repair effects follow their baseline rates of structured probing, while the changes in tool use differ across models. Together, these findings link the effects of hypothesis prompting to how it shapes subsequent actions and to agents' existing tool-use patterns.