Text2Model: Modeling Copilots for Text-to-Model Translation

There is growing interest in leveraging large language models (LLMs) for text-to-model translation and optimization tasks. This paper aims to advance this line of research by introducing $\text{textsc}Text2Model$ and $\text{textsc}Text2Zinc$. $\text{textsc}Text2Model$ is a suite of copilots based on several LLM strategies with varying complexity, along with an online leaderboard. $\text{textsc}Text2Zinc$ is a cross-domain dataset for capturing optimization and satisfaction problems specified in natural language, along with an interactive editor with built-in AI assistant. While there is an emerging literature on using LLMs for translating combinatorial problems into formal models, our work is the first attempt to integrate $\mathit{\text{both}}$ satisfaction and optimization problems within a $\mathit{\text{unified}}\mathit{\text{architecture}}$ and $\mathit{\text{dataset}}$. Moreover, our approach is $\mathit{\text{solver-agnostic}}$ unlike existing work that focuses on translation to a solver-specific model. To achieve this, we leverage $\text{textsc}MiniZinc$'s solver-and-paradigm-agnostic modeling capabilities to formulate combinatorial problems. We conduct comprehensive experiments to compare execution and solution accuracy across several single- and multi-call strategies, including; zero-shot prompting, chain-of-thought reasoning, intermediate representations via knowledge-graphs, grammar-based syntax encoding, and agentic approaches that decompose the model into sequential sub-tasks. Our copilot strategies are competitive, and in parts improve, recent research in this domain. Our findings indicate that while LLMs are promising they are not yet a push-button technology for combinatorial modeling. We contribute $\text{textsc}Text2Model$ copilots and leaderboard, and $\text{textsc}Text2Zinc$ and interactive editor to open-source to support closing this performance gap.