Prompt Engineering Paper Accepted at ICML 2026 Sparks Heated Debate Among Netizens
A paper on prompt engineering, titled "Verbalized Sampling (VS)," has been accepted by the prestigious machine learning conference ICML 2026, sparking significant debate online.
The paper addresses the problem of "mode collapse" in large language models (LLMs), where models tend to produce repetitive, safe, and homogeneous outputs. Instead of proposing new training algorithms or model architectures, the authors introduce a simple yet effective prompt-based method. The core technique, Verbalized Sampling, instructs the model to generate multiple responses (e.g., five jokes) while also outputting a possible probability value for each. This prompt adjustment alone was shown to significantly increase output diversity by 1.6x to 2.1x in creative writing tasks, without compromising factual accuracy or safety.
The authors argue that the root cause of mode collapse lies not in optimization algorithms but in the "typicality bias" present in human preference data used for alignment. Human annotators naturally favor familiar and fluent text, which steers models toward conservative outputs. The VS method aims to counteract this by leveraging the model's inherent pre-training distribution during inference.
The paper's acceptance has led to polarized reactions. Critics argue that prompt engineering lacks the theoretical depth and algorithmic innovation expected from top-tier conferences like ICML, questioning its novelty, generalizability across models, and experimental scale. Some draw parallels to reproducibility crises in other fields, citing a potential over-reliance on empirical results.
Supporters, including an author who responded online, defend the work's rigor. They emphasize its comprehensive problem analysis, theoretical grounding, mathematical derivation, and extensive quantitative experiments. Proponents compare VS to seminal techniques like Chain-of-Thought (CoT) prompting, suggesting that inference-stage methods are becoming a core part of ML research capable of expanding model capabilities without retraining.
The research was conducted by a team from Northeastern University, Stanford University, and West Virginia University, with Jiayi Zhang, Simon Yu, and Derek Chong as co-first authors.
marsbit07/15 07:56
