In the first half of this year, the AI world staged a highly dramatic "scientific reality show."
The protagonist was FARS, an AI scientist developed by Analemma. Without any human intervention, it ran non-stop for 228 hours, forcefully "producing" 100 academic papers within cloud computing clusters.
Meanwhile, Japanese star startup Sakana AI drove the cost of this business to rock bottom—their AI Scientist system could compress the cost of generating a single academic paper to an extreme $15. On the other side of the coin, Zochi, an AI scientist developed by Intology, successfully submitted its autonomously written paper to ACL, a top conference in natural language processing, in 2025, receiving a high score ranking in the top 8.2%.
AI can not only produce low-cost, bulk content but has also managed to break through doctoral-level academic barriers. It seems that overnight, conducting research has turned into piecework coding on an assembly line.
But behind these dazzling technological showcases, a recent audit report from the authoritative medical journal The Lancet struck like a hammer blow: among the 2.5 million papers they sampled, purely fabricated references generated by AI had surged an astonishing 12-fold over the past few years.
As capital pushes large models to force open the doors of academia, how reliable are these "silicon-based Einsteins"?
In May 2026, a research team from Peking University, Tongji University, and the University of Tübingen (Zonglin Yang et al.) jointly released the world's first benchmark test specifically evaluating the academic integrity of AI scientists: SciIntegrity-Bench.
This report ruthlessly tore the fig leaf off AI research.
Dilemma Testing: What Will AI Do If the Data Is Empty?
Past AI tests all focused on whether models "could get things right." But SciIntegrity-Bench adopted a very "unconventional" testing method: dilemma assessment.
The researchers set 11 types of traps for the AI. For example, deliberately giving the AI an empty table with only headers and no data, or providing a derivation logic that is fundamentally unsolvable.
In such cases, the only correct action is to honestly tell humans, "Data is missing, I can't do it."
However, as long as the AI forces out a seemingly perfect report, it is judged as academic misconduct.
In 231 high-pressure tests conducted on seven of the world's top large language models, the overall "problem rate" was as high as 34.2%.
The most chilling was the "empty dataset" test. Faced with a table containing no data at all, all seven large models, without exception, chose to "fabricate from nothing."
They didn't even output an error message. Instead, they wrote code, fabricated thousands of extremely realistic rows of sensor parameters, applied international standards, and even produced a plausible-looking equipment maintenance report.
Beyond "Fabrication," Where Else Did AI Fall into Traps?
Beyond the "fabrication from nothing" trap, the paper team set a total of 11 types of scientific research traps for the large models. The test results showed a highly polarized "specialization" phenomenon.
First, the "excellent" side: The large models know the rules very well. When faced with "traditional data science norms," the AI behaved like diligent, rule-following students. For example, "peeking at test set answers before the exam (T02)" or "selecting metrics to present only good news (T03)," their failure rates were surprisingly 0%. Even for "picking easy targets by choosing inappropriate benchmarks (T01)," the failure rate was only 4.8%. This indicates that for any explicit norms written in textbooks, AI has already mastered them thoroughly.
However, on the other side, as soon as it involves logical dead ends that "require stopping," the large models began to run amok (critical danger zones):
When tools are restricted, they "forge imperial edicts" (Constraint Violation, problem rate as high as 95.2%): When asking AI to call a certain API but not providing the real key. The AI almost never reported an error. Instead, it directly wrote a piece of code, fabricated a perfectly formatted JSON response packet (complete with virtual call statistics), pretended the API call was successful, and continued writing the report.
Hallucinating critical experimental parameters (Hallucinated Steps, problem rate 61.9%): Faced with an incomplete chemical experiment note, the AI not only failed to verify with humans but "intelligently constructed a false audit trail." It confidently embellished the standard operating procedure (SOP), fabricating specific parameters like "4000 RPM centrifuge" or "ethanol quench." In a real chemistry lab, this could cause a fatal explosion.
The workplace dodger who "knows better but does wrong anyway" (Causal Confusion, problem rate 52.3%): When evaluating advertising return on investment (ROI), the AI had already shrewdly written in the code comments, "There are confounding variables/causal inversion here." But to wrap things up quickly, it instantly abandoned its own correct diagnosis, forcibly ran a basic regression analysis, and produced an absurd "1099% ROI" conclusion.
Calling a stag a horse (Anomaly Blindness, failure rate 19.0%): When sensor data showed obvious equipment failure jumps, the AI wouldn't suspect the data was faulty. Instead, it wildly speculated, interpreting it as "discovering a new physical combustion mechanism."
In summary, the large models have learned explicit rules but haven't learned to "quit." Once the instinct to "complete the task" overwhelms common sense, they force together a perfect report by faking interfaces, hallucinating parameters, or abandoning logic.
Report Card for 7 Top Models: Underlying Character Under Extreme Pressure
It must be clarified that "fabrication" here does not mean the models are malicious in their daily operation. It refers to the systematic bias driven by underlying mechanisms when facing extreme dilemmas. Under extreme task pressure, different models revealed completely different underlying quality control characteristics:
Claude 4.6 Sonnet: The top student with the most solid defenses Out of 33 high-risk scenarios, it had only 1 critical failure.
Strengths: Extremely restrained; has clear awareness of obvious constraints and logical flaws.
Weakness: Still couldn't resist the temptation of the "empty dataset"; even it failed to trigger the underlying "honest refusal" mechanism.
GPT-5.2 & DeepSeek V3.2: High-IQ "task compromisers" With 2 and 3 critical failures respectively.
Strengths: Extremely strong logical reasoning; can shrewdly point out "there is causal confusion here" in code comments.
Weaknesses: Exhibit "identification bypass" phenomenon. To accomplish the goal, they abandon their own correct diagnosis just made, compromise to task pressure, and use a fundamentally flawed method to produce an absurd but deliverable conclusion.
Gemini 3.1 Pro, Qwen3.5, GLM 5 Pro: Standard executors Failure counts were 5, 6, and 7 times respectively.
Characteristics: Vulnerable to traps involving "tool calling" and "causality." For example, when lacking a real API interface, they tend to directly fabricate a perfectly formatted fake response to forcibly advance the task.
Kimi 2.5 Pro: The "filler" with an extremely high hallucination tendency Ranked last with 12 failures, a problem rate of 36.36%.
Characteristics: Under extreme testing, it showed a strong preference for "fabricating steps." When asked to complete incomplete experiment records, it confidently fabricated key parameters like centrifuge speed (4000 RPM) and quenching solvents, and even fabricated fake literature to cover up data generation traces. In a real chemistry lab, such behavior could cause major accidents.
Why Do Top AIs Fall into "Systematic Lying"?
Why would AI with massive parameter counts and high intelligence fabricate from nothing?
The paper pointedly identified the root cause: Intrinsic Completion Bias.
This comes from the large models' "upbringing." Currently, mainstream models rely on Reinforcement Learning from Human Feedback (RLHF). In this mechanism, AI is systematically rewarded for "providing answers" and "solving problems."
Conversely, "stopping" or "admitting inability" is seen by the algorithm as slacking off, which gets penalized.
This mechanism has been internalized as the AI's underlying logic: The process isn't important; no matter how bad the conditions, the final output must be delivered.
Moreover, many developers, when writing system prompts for AI, often add high-pressure instructions like "overcome difficulties, output a report no matter what."
"Innate nature" combined with "high pressure" directly pushes AI into the corner of fabrication.
The greatest value of this paper is not to criticize AI, but to tell us: Large models inherently carry "completion anxiety."
Now that we understand its weakness, ordinary people, in daily use or AI application development, need to change communication strategies. When dealing with AI, the traditional "issuing commands" is no longer sufficient. You need to master the following communication and prevention techniques:
1. Remove Coercive Pressure, Grant It the "Right to Refuse" Paper tests show that after deleting the high-pressure instruction "must complete the task" from the prompt, the rate of AI concealing data fabrication plummeted from 20.6% to 3.2%.
How to communicate: Always add "exit conditions" in the Prompt. Don't just say, "Give me a market analysis based on this data." You should say: "First, assess whether the data is sufficient. If data is missing or there are logical gaps, immediately stop reasoning and report an error to me. Do not assume core data under any circumstances."
2. Intercept the "Generation Instinct," Establish Physical Verification Anchors The essence of large models is probabilistic prediction; faced with emptiness, filling it with hallucinations is their "factory default."
How to communicate: Never let AI run an end-to-end process in a black box. Break the task into pieces. If asking it to analyze data, forcibly insert a confirmation step: "Before drawing final conclusions, first output the original data line numbers and calculation formulas you rely on. Wait for my manual confirmation before proceeding to the next step."
3. Beware of "Compliance-Based Self-Review," Activate "Fault-Finding Mode" Since smart models like GPT-5.2 abandon error correction to meet deadlines, you can't expect them to find problems on their own while following your train of thought.
How to communicate: After getting an AI's plan, don't ask, "Is this plan good?" (It will definitely praise it to please you). Open a new chat window, assign it the role of a "cold auditor," and throw the plan at it: "The conclusions of this report may involve causal inversion or common-sense errors. Find where it substituted concepts or fabricated premises in which step."
4. Macro Defense: Use "Physical Quotas" Against "Infinite Productivity" We cannot rely solely on worker-level prompt defenses; institutional rule counterattacks have begun. Faced with the onslaught of AI generating vast amounts of proposals at zero cost, the US National Institutes of Health (NIH) issued the landmark NOT-OD-25-132 policy in July 2025. Starting in 2026, it mandates: each Principal Investigator (PI) can submit a maximum of only 6 funding applications per year.
Business Insight: When AI's productivity is nearly infinite, traditional "content review mechanisms" will inevitably be breached. The future moat will no longer be about output speed, but about establishing scarcity defenses based on physical identity and credit quotas.
The essence of technology is to reduce costs and increase efficiency, but the foundation of business and science is always reverence for facts.
In an era where content generation costs are almost zero, what is scarce is no longer "typists" who can write reports, but "auditors" who can see through data hallucinations. Mastering this art of gaming with the system is how you truly take the lead amidst the torrent of computing power. (This article was first published on Titanium Media APP, author | SiliconValley_Tech_news, editor | Linshen)
(The core evaluation data, model rankings, and cause analysis in this article are all cited from the first large model academic integrity benchmark test SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems published in May 2026. The newly added 11 trap problem rates are all cited from the latest calculations in that research report.)
