Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute

Recent advancements in software engineering agents have demonstrated promising capabilities in automating program improvements. However, their reliance on closed-source or resource-intensive models introduces significant deployment challenges in private environments, prompting a critical question: $\mathit{\text{How}}\mathit{\text{can}}\mathit{\text{personally}}\mathit{\text{deployable}}\mathit{\text{open-source}}\mathit{\text{LLMs}}\mathit{\text{achieve}}\mathit{\text{comparable}}\mathit{\text{code}}\mathit{\text{reasoning}}\mathit{\text{performance?}}$ To this end, we propose a unified Test-Time Compute scaling framework that leverages increased inference-time computation instead of larger models. Our framework incorporates two complementary strategies: internal TTC and external TTC. Internally, we introduce a $\mathit{\text{development-contextualized}}\mathit{\text{trajectory}}\mathit{\text{synthesis}}$ method leveraging real-world software repositories to bootstrap multi-stage reasoning processes, such as fault localization and patch generation. We further enhance trajectory quality through rejection sampling, rigorously evaluating trajectories along accuracy and complexity. Externally, we propose a novel $\mathit{\text{development-process-based}}\mathit{\text{search}}$ strategy guided by reward models and execution verification. This approach enables targeted computational allocation at critical development decision points, overcoming limitations of existing “end-point only” verification methods. Evaluations on SWE-bench Verified demonstrate our $\mathbf{\text{32B}}\mathbf{\text{model}}\mathbf{\text{achieves}}\mathbf{\text{a}}\mathbf{\text{46\%}}\mathbf{\text{issue}}\mathbf{\text{resolution}}\mathbf{\text{rate}}$, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1. Additionally, we provide the empirical validation of the test-time scaling phenomenon within SWE agents, revealing that $\mathbf{\text{models}}\mathbf{\text{dynamically}}\mathbf{\text{allocate}}\mathbf{\text{more}}\mathbf{\text{tokens}}\mathbf{\text{to}}\mathbf{\text{increasingly}}\mathbf{\text{challenging}}\mathbf{\text{problems}}$, effectively enhancing reasoning capabilities. We publicly release all training data, models, and code to facilitate future research. https://github.com/yingweima2022/SWE-Reasoner