When asked to give an image a "quality score," large models often get it wrong.
A clean photo and its subtly degraded version—with added noise, compression, or slight blurring—look drastically different in quality to human eyes. But in the "eyes" of a multimodal large model (MLLM), these two images look almost identical.
Consequently, the score it provides is often an estimate based on a feeling, not derived from examining evidence.
Now, a research team from Northwestern Polytechnical University and The Hong Kong University of Science and Technology has proposed a new framework—IQA-T1—for the first time enabling multimodal large models to learn to assess image quality like a professional inspector:
No longer relying on intuitive "glances," but actively calling upon a set of professional analysis tools to generate structured "visual evidence" such as noise residual maps, gradient distributions, and Fourier spectrums, then making judgments step-by-step based on this evidence.
Transitioning from "scoring based on feeling" to "judging with evidence"—IQA-T1 achieved state-of-the-art (SOTA) overall performance on 7 image quality assessment benchmarks, with an average PLCC/SRCC of 0.795/0.784. Its evaluation process is now interpretable and traceable.
The paper and code are open-sourced.
△ Existing MLLM scoring methods are either "pure text reasoning" (prone to semantic bias) or "region-cropping reasoning" (a collection of unexplained image patches); IQA-T1 introduces a new paradigm: "tool-generated visual evidence."
Why Do Large Models Score "Based on Feeling"?
The goal of Image Quality Assessment (IQA) is to make machine-given scores align as closely as possible with human subjective perception. It serves as the "referee" for a series of visual tasks like image restoration, enhancement, and compression, directly determining how well and reliably visual systems perform in real-world, open environments.
With the rise of multimodal large models, more work has started using their "reasoning capabilities" for IQA, hoping to provide both a score and an explanation of "why this score." It sounds promising, but the research team points out a common fundamental flaw:
Existing MLLM scoring relies on internally learned visual representations with semantic bias, rather than verifiable low-level degradation evidence.
In simpler terms: The visual encoders of large models are naturally adept at understanding high-level semantics like "this is a cat, this is a beach," but are extremely insensitive to low-level degradations like noise, blur, or compression artifacts.
The paper provides an intuitive illustration: Given an original image
and its noise-added degraded version
, humans would give vastly different scores, but their visual representations within the large model are nearly identical—
The representations remain almost unchanged, yet quality differs noticeably. This is the so-called Semantic Bias: the model simply lacks sufficient information to perceive degradation, so it can only "guess."
Existing methods attempt two patches, but neither hits the mark:
- Pure text reasoning (e.g., Q-Insight, VisualQuality-R1): Generates a structured quality description before scoring, but the entire reasoning still relies on the model's own "imagined" semantic priors, lacking verifiable visual anchors;
- Region-cropping reasoning (e.g., Zoom-IQA, Q-Probe): Enlarges and crops potentially problematic areas to feed to the model. However, these regions are essentially just unexplained image patches; even if the model sees them, it can't clarify "whether this area contains noise, blur, or artifacts."
In summary: Neither provides "evidence" for the reasoning process.
Core Idea: Equipping the Large Model with a "Detection Toolkit"
The solution of IQA-T1 can be likened to the evolution of a doctor's diagnosis:
Past models were like traditional doctors relying solely on the experience of "observation, auscultation, inquiry, and palpitation" for diagnosis; IQA-T1 is like a modern doctor who actively orders CT scans, blood tests, and X-rays—first obtaining objective examination results (evidence), then making judgments accordingly.
Specifically, the team prepared a Perception Tool Library for the model. During reasoning, the model autonomously decides which tools to call. Each tool is specialized to "visualize" a specific type of image quality degradation, transforming problems invisible to the naked eye (and to standard model representations) into structured evidence maps:
- NoiseResidualMap: Subtracts a denoised version from the original, "extracting" sensor noise or compression noise;
- FourierMagnitudeSpectrum: Identifies periodic artifacts in the frequency domain;
- GradientOrientationCoherenceMap: Reveals structural inconsistencies caused by blurring or oversmoothing;
......
These evidence maps are gradually integrated into the reasoning chain, becoming "physical evidence" for each step of the model's judgment. Thus, IQA transforms from a reasoning task relying purely on semantic speculation into evidence-driven reasoning that can be verified.
The tool library design is also meticulous. The team identified 7 key perceptual attributes in IQA—structure, sharpness, noise, artifacts, brightness, color, and naturalness—and accordingly designed 15 types of visual evidence. All tools also meet three engineering constraints: unified calling interface, appropriately downsampled outputs (saving tokens), and completely deterministic results (ensuring consistency across data construction, SFT, and RL phases).
From "Knowing How" to "Knowing When": Two-Stage Training
To truly master this capability, the model needs two abilities: knowing how to use the tools, and knowing when to use them. IQA-T1 uses a two-stage training pipeline to solve this.
△
Stage One: Supervised Fine-Tuning (SFT) — First Learn "How"
Supervised fine-tuning on the self-built Q-Tool dataset teaches the model three things: following a structured reasoning template, using the standardized tool-calling syntax, and associating visual evidence with quality judgments. Note, the model does not need to generate the evidence maps itself (those are deterministically produced by the tools); during training, loss masking is applied to tokens corresponding to the evidence maps.
Stage Two: Reinforcement Learning (RL) — Then Learn "When"
SFT teaches "how," but does not optimize the "calling strategy." Therefore, the team uses GRPO for reinforcement learning, specifically refining the model's adaptive tool-calling strategy. The reward function is carefully composed of four parts:
- Format Reward
: Whether the output conforms to the specified structure (including tool-call labels and final score);
- Scoring Reward
: Higher reward for predictions closer to ground truth (exponential decay);
- Tool Usage Reward
: Encourages calling an "appropriate number" of tools—too few means insufficient evidence, while too many incur penalties for redundancy;
- Repetition Penalty
: Directly penalizes repeatedly calling the same tool to inflate presence.
These four are weighted and summed (R_score has the highest weight of 5), ultimately teaching the model to use as few, as correct tools as possible to obtain sufficient evidence.
Dataset Q-Tool: The First IQA Dataset for "Evidence Reasoning"
No existing IQA dataset supports "visual evidence reasoning." The team decided to build their own—Q-Tool, containing 11k high-quality multimodal reasoning chains. Each chain binds tool-generated visual evidence with human-aligned textual analysis.
△
It was constructed through a three-stage pipeline:
- Tool Library Construction:
The aforementioned 7-attribute / 15-evidence toolset;
- Evidence Reasoning Chain Synthesis:
First, human experts wrote a reasoning template (breaking down quality assessment into stages like "check structure → analyze noise → evaluate color..." and mapping attributes to tools). Then, GPT-4o was used to generate complete reasoning chains under the template's constraints, preventing the model from hallucinating freely;
- Reasoning Chain Validation:
Each chain was reviewed from three dimensions: whether tools were used correctly, whether evidence was genuinely referenced, and whether conclusions were supported by evidence. Unqualified chains were deleted, and borderline cases were manually refined.
Experimental Results: SOTA on 7 Major Benchmarks
The team trained on KonIQ and evaluated cross-distribution generalization on 7 benchmarks covering real-world distortions, synthetic distortions, algorithmic degradations, and AI-generated content. Competitors included traditional handcrafted methods, deep learning methods, and the latest MLLM methods.
The result: IQA-T1 achieved comprehensive first place with an average PLCC of 0.795 / SRCC of 0.784 across the 7 datasets:
Compared to MLLM methods that only score without explanation (e.g., Q-Align, DeQA), IQA-T1 maintains high accuracy while providing interpretable, evidence-based reasoning chains;
Compared to methods that also include reasoning (e.g., Q-Insight, VisualQuality-R1, Zoom-IQA), it set new records on the tough synthetic distortion (CSIQ) and algorithmic degradation (PIPAL) datasets, while remaining highly competitive on real-world and AI-generated content.
A vivid example best illustrates the point:
△
More interesting are the ablation studies on "how tools are called":
That is, using more tools is not necessarily better—irrelevant tools can introduce redundant visual tokens and interference. IQA-T1's dynamic calling uses an average of only 2.34 tools per image, achieving the best overall results while saving time and VRAM. This indicates the model is truly "reasoning based on evidence", not simply fitting scores.
△
Furthermore, applying the framework to different base models like Qwen3-VL-2B and InternVL3.5-4B maintained stable performance trends—proving that "tool-based visual evidence reasoning" is a general mechanism, not a trick specific to a particular model.
The Team
This work is a collaboration between Northwestern Polytechnical University and The Hong Kong University of Science and Technology. The first authors are Jinjian Wu from Northwestern Polytechnical University and Jiaqi Tang from The Hong Kong University of Science and Technology (co-first authors). The corresponding authors are Professor Wei Wei from Northwestern Polytechnical University and Professor Qifeng Chen from The Hong Kong University of Science and Technology.
The team hopes this work will inspire more research: introducing structured visual evidence into multimodal reasoning systems to make large models' "perception" more reliable and interpretable.
Paper: https://arxiv.org/abs/2607.12375v1
Code: https://github.com/zibuyu-02/IQA-T1
Model: https://huggingface.co/zibuyu-02/IQA-T1
Data: https://huggingface.co/datasets/zibuyu-02/Q-Tool
Demo: https://huggingface.co/spaces/Jiaqi-hkust/IQA-T1
This article comes from the WeChat public account "QbitAI," by the IQA-T1 team
