Ensemble learning for large language models in text and code generation: a survey
Figure 1: Ensemble generation and selection. Diverse models produce candidates (a); an aggregation stage — voting, ranking, or judged selection — yields output that outperforms any single member (b).∎
No single language model wins everywhere: models trade strengths across domains, styles, and failure modes. This survey maps the ensemble-learning landscape for text and code generation — voting, cascading, mixture-of-experts, and rank-and-select architectures — and the conditions under which each beats its best individual member.
For code generation the implications are practical: an ensemble that generates with several models and selects with an independent judge consistently outperforms single-model pipelines. That finding is the research base of Artemis's multi-LLM design — variants generated by many models, reviewed by a panel of different ones.
Key results
- Systematic taxonomy of LLM ensemble methods for generation tasks
- Ensembles reliably beat their strongest single member under diversity conditions
- Published in IEEE Transactions on Artificial Intelligence