The "source data" in transfer learning doesn't necessarily have to be images, text, or audio.
A set of noise randomly sampled from a Gaussian distribution, devoid of any semantics, can also help models learn better with limited annotations.
This work is called Semi-Supervised Noise Adaptation (SSNA), and the paper has been published at ICML 2026.
The SSNA team further proposes the Noise Adaptation Framework (NAF). It uses randomly generated noise to construct discriminative class structures and transfers this structure to real target data, thereby improving the learning effectiveness of models in scenarios with limited annotations.
△NAF projects the noise domain and the target domain into a shared representation space and performs alignment at the class level. Image from the paper.
With only 4 labeled samples per class, based on the ResNet-18 backbone network, NAF achieved accuracy improvements of 12.35, 7.61, 4.38, and 2.74 percentage points on CIFAR-10, CIFAR-100, DTD-47, and Caltech-101 respectively, compared to the standard supervised learning baseline ERM (Empirical Risk Minimization) trained using only labeled samples. The paper's source code is now open-sourced; see the end of the article for details.
Replacing the Real Source Domain with Noise
Traditional transfer learning typically requires a source domain with abundant labels. However, real source data is not always easy to obtain. Privacy, confidentiality, and copyright restrictions can all prevent the sharing of source domain data. SSNA attempts to remove this prerequisite: the source domain no longer contains real samples but is instead replaced by noise generated from a simple probability distribution.
The specific method is not complicated. Assuming the target task contains C classes, the research team first randomly samples a mean vector for each class in a 1024-dimensional space, using the identity matrix as the covariance, thereby constructing C Gaussian distributions. They then sample 50 noise vectors from each distribution. The noise domain and the target domain share the same set of class indices; each noise class is pre-assigned a target class index, but this correspondence itself contains no semantic information.
The indices themselves have no semantics. Noise class 0 does not inherently represent "cat"; it is simply fixed to correspond to a certain class in the target domain before training begins. What is truly transferred is not visual knowledge of cats or dogs, but the discriminative structure formed in the noise domain.
Noise of the same class is clustered together, while noise of different classes is pushed apart. As long as this structure can align with the target domain, it can provide clearer classification boundaries for real target samples.
Limited Labels Are Still Necessary
Noise can replace real source data but cannot completely replace target domain labels. The reason is straightforward: Noise comes from a different space, and the class indices are artificially assigned. The model must rely on a small number of labeled target samples to determine which real class noise class 0 should align with.
When the number of labeled samples per class is reduced to 0 on CIFAR-100, the accuracy rates of ERM and NAF are only 0.97% and 1.34% respectively, both close to random guessing. Once a few labels are provided, NAF consistently outperforms ERM.
Therefore, the conclusion of this work is: In the semi-supervised classification setting, a real source domain is not a necessary condition for achieving positive transfer.
NAF Primarily Does Three Things
Based on the generalization bound given in the paper, NAF breaks down the training objective into three parts.
First, Learn Well from a Few Real Labels
The target domain encoder maps real samples into the shared representation space, and the classifier computes the cross-entropy loss using the few labeled samples. This part is consistent with ordinary supervised learning, used to build a bridge between noise classes and real classes.
Then, Let Noise Form a Clear Class Structure
The noise projector is responsible for mapping random vectors into the same representation space, and the classifier identifies these noises according to the pre-assigned class indices. After training, noise of the same class gradually clusters together, while noise of different classes separates from each other.
Finally, Align Both Sides
NAF also calculates the distribution divergence between the noise domain and the target domain. The distribution alignment module is not limited to a specific implementation; the paper compares various schemes and empirically adopts Negative Domain Similarity (NDS) as the default mechanism in the experiments. NDS simultaneously compares the global means and the per-class means of the two domains, using cosine similarity to push them closer. The class means for unlabeled target samples are iteratively estimated using pseudo-labels generated by the model.
The overall objective can be written as
. Where,
is responsible for classifying the few real labeled target samples,
is responsible for classifying noise, and
is responsible for aligning the distributions of the target and noise domains. The generalization bound provided in the paper also corresponds to these three terms: the target domain empirical error, the noise domain empirical error, and the distribution divergence between the two domains in the shared representation space.
From CIFAR to ImageNet, Noise Can Bring Gains
The main experiments cover 8 visual datasets and 1 text classification dataset. Except for ImageNet-1K, visual tasks use 4 labeled samples per class, with the remaining training samples used as unlabeled data.
On ResNet-18, NAF's Top-1 improvements over ERM reach: CIFAR-10 +12.35, CIFAR-100 +7.61, DTD-47 +4.38, Caltech-101 +2.74 percentage points.
△
It is also effective for fine-grained classification. Using ResNet-18, NAF improved over ERM by 8.94, 5.51, and 7.74 percentage points on CUB-200, Oxford Flowers-102, and Stanford Cars-196 respectively.
On the larger-scale ImageNet-1K, the research team retained 100 labeled samples per class. NAF achieved 37.10% accuracy, which is 0.99 percentage points higher than ERM. On the text task AG News-4, NAF using BERT reached 82.82%, 4.18 percentage points higher than ERM's 78.64%.
NAF can also be directly plugged into existing semi-supervised methods. The paper integrated it into UDA, FixMatch, FlexMatch, DebiasMatch, DST, LERM, and SA-FixMatch, with gains observed overall. Taking the 20-epoch training results on CIFAR-10 as an example, after adding NAF, the accuracy of UDA and FixMatch improved by 20.83 and 9.91 percentage points respectively.
What's Truly Useful Is Not "Randomness", But Structure
Why can noise help? The paper's ablation experiments point to the same factor: Whether there is a separable structure between classes.
The team first collapsed all classes of noise into a single point. In this way, the noise domain completely lost its discriminative structure, and NAF not only failed to provide gains but also showed significant negative transfer. CIFAR-10 accuracy dropped from ERM's 58.15% to 33.34%, and CIFAR-100 dropped from 42.24% to 6.79%.
Conversely, when gradually increasing the distance between noise class centers, the CIFAR-100 accuracy increased from 43.80% to 49.78%. This indicates that the easier the noise classes are to distinguish, the stronger the structural guidance they can usually provide.
The amount of noise is not as critical. When the number of noise samples per class increased from 10 to 100, the accuracy remained stable at around 50%; it even slightly decreased after increasing to 200. Based on this, the paper argues that as long as a separable pattern can be formed, a small amount of noise is sufficient to be effective.
△
The research team also simplified the noise domain to a single center point per class. Whether the centers were fixed or learnable, the results were higher than ERM; among them, learnable centers performed better than fixed centers but still lower than the full NAF.
In the transfer experiment from Amazon to Caltech-10, real source data still slightly outperformed overall, but the noise source domain was already able to increase accuracy from ERM's 83.51% to about 88% to 89%. This provides a more realistic positioning: when real source data is unavailable, synthetic noise can be a very low-cost alternative.
It also differs from common data augmentation. Data augmentation typically performs rotations, crops, interpolation, or generation around real samples; SSNA first constructs an independent noise domain and then completes cross-domain alignment in the representation space.
Summary: Even Without Semantics, Structure Can Still Be Transferred
This work provides a rather counter-intuitive new perspective for transfer learning: The ability of source data to help a target task does not necessarily rely entirely on its real semantics; the class structure formed in the representation space can also become transferable knowledge.
NAF leverages this very point, allowing random noise to form a discriminative structure in the shared representation space, and then using a few labeled samples to transfer it to real data. Experimental results show that even if the source domain contains no real images, text, or audio, as long as an appropriate class structure is retained, it can still have a positive effect on the target task.
In other words, what transfer learning transfers may not only be "what the data says" but also "how the data is organized in the representation space." This also opens a new research direction for scenarios with privacy restrictions, copyright sensitivities, or difficulties in obtaining real source data.
Paper Title: Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain
Paper Address: https://arxiv.org/pdf/2606.00558
Source Code Address: https://github.com/AIResearch-Group/SSNA
Video Explanation: https://www.bilibili.com/video/BV1UV7h61EvW/
This article is from the WeChat public account "Quantum Bits", author: SSNA Team
