Prompt engineering can get a paper published in ICML these days???
Recently, a guy shared a paper just accepted by ICML 2026 on Reddit, and the post instantly went viral, with the number of comments skyrocketing.
However, everyone was left scratching their heads: Is this for real?
The authors didn't propose any new optimization algorithms or train any new large models. They simply did one thing—
Modified the Prompt.
The paper proposes a method called Verbalized Sampling (VS), which significantly enhances the diversity of large language model outputs and alleviates the long-standing issue of Mode Collapse in LLMs, solely by adjusting the prompt.
It sounds quite practical, but is it right for a mere prompt trick to get into a top-tier conference?
Let's take a look at the paper first before passing judgment.
A Highly Controversial ICML Paper
Have you ever felt that AI is becoming increasingly homogeneous?
Ask it ten times to "Tell me a joke," and the answers you get are often highly similar. This is not only true for creative tasks but also for Q&A, code generation...
This phenomenon is collectively referred to as mode collapse in academia.
Simply put, models increasingly prefer to output the highest probability, safest, and most classic answers, while rejecting alternative creative ideas.
In the past, to solve this model problem, most researchers would first think of adjusting sampling parameters, modifying decoding algorithms, retraining, etc. However, this paper takes a different approach: directly asking the model to output its own sampling process along with the answer.
For example, using the earlier joke-telling scenario, the authors would modify the prompt, requiring the model to:
Generate 5 jokes, while also assigning a possible probability value to each joke.
Then the model can produce more varied and less repetitive answers.
It sounds very simple, doesn't it? In fact, this is the core contribution of the paper—the Verbalized Probability Sampling method. No fine-tuning is needed; just by changing the way of asking, content diversity can be significantly improved.
In the paper, however, the authors provide a rigorous argumentation process for this.
First, they address the root cause of the model's uniformity.
Past academia attributed this problem to algorithmic aspects, such as insufficient reward models or improperly set KL penalty terms. This paper delves deeper, suggesting the real root lies in the preference data itself.
They propose a concept called prototypical bias. From a cognitive psychology perspective, human annotators naturally prefer familiar, fluent, conventional text, and instinctively give higher scores to stereotypical, mainstream answers.
Therefore, even if reward models and optimization algorithms are perfected, as long as the human preference data used for training inherently contains prototypical bias, post-alignment models will still suffer from mode collapse.
The authors tested this repeatedly on five preference datasets and different base models, and the conclusion remained consistent.
After realizing this, the authors argue that since the problem is rooted in the training data, one only needs to consider designing a prompt-based correction scheme during the inference stage. That is, by making the model output the complete probability distribution in the prompt, the model's inherent diverse output distribution from the pre-training phase can be awakened, restoring diversity.
The rest involved running experiments with this method in various scenarios. The results show that in creative writing tasks, diversity was 1.6~2.1 times that of ordinary prompts, without reducing factual accuracy or compromising model safety.
Furthermore, the stronger the model's capabilities and the larger the parameter count, the more pronounced the diversity improvement brought by VS.
So, while the final method presented in the paper is simple, ICML still passed it.
Reddit Users in a Heated Debate
Under the original post, however, opinions on this paper are polarized.
Many netizens expressed that in the past, ICML featured hardcore innovations like new models, new algorithms, and new theories. Merely optimizing prompts or inference processes doesn't qualify as serious machine learning research.
In comparison, the innovation of this work seems somewhat thin, and several issues exist:
First, similar instruction-writing methods are not unique; some even claimed they were writing prompts like this just yesterday. Second, the theory is difficult to verify because prompts might fail when models are changed, unlike algorithms which are more stable. Third, the experimental scale is limited, insufficient to prove this is a universal law.
Some netizens directly compared the current state of the machine learning field to the academic crisis in psychology over a decade ago.
Back then, many researchers had weak statistical foundations and misused statistical tools, leading to many papers with unreproducible conclusions and a severe trust crisis in the field. Similarly, the machine learning industry now heavily relies on empirical experiments and undervalues rigorous theoretical support.
The industry is internally competitive, chasing new methods, but there's a widespread atmosphere of excessive hyperparameter tuning and benchmark score chasing. Many so-called innovative algorithms offer little practical value compared to mature baseline models, being packaged as innovative achievements based on tiny metric improvements.
Essentially, these are issues in paper publishing caused by unclear professional standards following the rapid expansion of the discipline.
But supporters argue that scientific research is not about whose method is more complex. As long as the hypothesis is clear, experiments are sufficient, and results are stable and reproducible, it can still be excellent research.
For example, this paper thoroughly explains what mode collapse is and proposes that the real problem lies in prototypical bias—a viewpoint more important than the prompt itself.
One of the authors also replied in the comments, stating that while the paper appears simple, it actually involves a great deal of complex processing.
The entire work includes complete problem tracing, new theoretical attribution, mathematical derivation, and multi-dimensional quantitative experiments—it's not a shallow, trivial prompt-tuning effort.
Many also mentioned Chain-of-Thought (CoT). When CoT first appeared, it was essentially a one-line prompt:
Let’s think step by step.
But now, almost all reasoning methods can be traced back to CoT. This precisely indicates that prompt engineering is no longer just about writing prompts; it is becoming a new method for studying model behavior.
Over the past decade, machine learning research has almost revolved around training. But now, some usage techniques during the inference stage are gradually moving toward the core of machine learning research.
Perhaps in the coming years, we will see more and more papers like this. They don't add a single line of training code or an extra model parameter, yet they can still change the capability boundaries of large models.
Research Team Introduction
Finally, let's take a look at the research team.
This work was completed by Weiyan Shi's team at Northeastern University (USA) in collaboration with Stanford's Manning Lab and West Virginia University. Jiayi Zhang, Simon Yu, and Derek Chong are listed as co-first authors.
Jiayi Zhang completed her undergraduate studies at the University of Michigan, earning triple bachelor's degrees in Computer Science, Mathematics, and Linguistics. She then pursued a Master's in Computer Science at Northeastern University (USA).
Another paper of hers, "Analyzing the Role of Semantic Representations in the Era of Large Language Models," accepted by the NLP top conference NAACL 2024, also revolves around semantic representation and large models.
Simon Yu is currently pursuing a Ph.D. at Northeastern University (USA), with his main research direction focusing on alignment and reinforcement learning mechanisms in large models. He completed both his bachelor's and master's degrees at the University of Edinburgh and has published several top conference papers.
Besides this paper, another one of his papers, "Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents," was also accepted by ICML 2026.
Derek Chong earned his Master's from Stanford University and is currently a researcher at the Stanford Artificial Intelligence Laboratory. His research primarily focuses on large language models in NLP.
He previously had three years of experience as a founder-entrepreneur and worked at Ello as an Applied Scientist, participating in industry-end AI implementation R&D. He possesses both solid theoretical research skills and rich hands-on practical experience.
References:[1]https://www.reddit.com/r/MachineLearning/comments/1uv1xb3/promptengineering_paper_accepted_to_icml_r/
[2]https://www.linkedin.com/in/jiayizx/[3]https://simonucl.github.io/[4]https://www.linkedin.com/in/derekch/
This article is from the WeChat public account "Qubit" (量子位), author: Focus on Frontier Technology
