Learning to Recommend Multi-Agent Subgraphs from Calling Trees

Multi-agent systems (MAS) increasingly solve complex tasks by orchestrating agents and tools selected from rapidly growing marketplaces. As these marketplaces expand, many candidates become functionally overlapping, making selection not just a retrieval problem: beyond filtering relevant agents, an orchestrator must choose options that are reliable, compatible with the current execution context, and able to cooperate with other selected agents. Existing recommender systems – largely built for item-level ranking from flat user-item logs – do not directly address the structured, sequential, and interaction-dependent nature of agent orchestration. We address this gap by $\mathbf{\text{formulating}}\mathbf{\text{agent}}\mathbf{\text{recommendation}}\mathbf{\text{in}}\mathbf{\text{MAS}}\mathbf{\text{as}}\mathbf{\text{a}}\mathbf{\text{constrained}}\mathbf{\text{decision}}\mathbf{\text{problem}}$ and introducing a generic $\mathbf{\text{constrained}}\mathbf{\text{recommendation}}\mathbf{\text{framework}}$ that first uses retrieval to build a compact candidate set conditioned on the current subtask and context, and then performs $\mathbf{\text{utility}}\mathbf{\text{optimization}}$ within this feasible set using a learned scorer that accounts for relevance, reliability, and interaction effects. We ground both the formulation and learning signals in $\mathbf{\text{historical}}\mathbf{\text{calling}}\mathbf{\text{trees}}$, which capture the execution structure of MAS (parent-child calls, branching dependencies, and local cooperation patterns) beyond what flat logs provide. The framework supports two complementary settings: $\mathbf{\text{agent-level}}\mathbf{\text{recommendation}}$ (select the next agent/tool) and $\mathbf{\text{system-level}}\mathbf{\text{recommendation}}$ (select a small, connected agent team/subgraph for coordinated execution). To enable systematic evaluation, we construct a unified calling-tree benchmark by normalizing invocation logs from eight heterogeneous multi-agent corpora into a shared structured representation.