A New Scaling Variable for Text-to-Image Generation, Discovered by ByteDance's Seed Team
ByteDance's SEED team investigated a crucial but often overlooked scaling variable in text-to-image diffusion models: the amount of image-grounded information in training captions. They found that simply increasing caption length with natural language does not improve model performance, as it often adds redundancy without new, usable visual supervision.
The core discovery is that the final training loss of a diffusion model can be predicted by the *information content* of its text condition, measured by two complementary metrics: Grounded Perplexity Gain (GPG) and Effective Detailness (ED). This establishes a scaling relationship for text conditioning.
To systematically increase information content, the team proposed **Structured Prompt (SP)**, a JSON-based representation that organizes visual variables (global scene, object attributes, spatial relationships) into clear fields, enhancing **Diffusability**—the model's ability to learn from captions.
For inference, an LLM **Prompter** is trained to convert user queries into detailed SP instances, defining **Promptability**. The overall generation quality is viewed as a product of Diffusability and Promptability. A three-stage training strategy (SFT, cold-start reasoning distillation, and verifier-guided reinforcement) significantly improves the prompter's capability. The structured format also enables efficient iterative refinement through a *refine-render-judge* loop.
In matched-control experiments using the same Qwen-Image backbone, data, and compute, the SP-based system substantially outperformed its natural-language counterpart, demonstrating that gains stem from the structured information interface, not just more training. The work shows that scaling text-to-image models requires scaling the *usable visual information* in conditions, not just model size or data volume.
marsbit08/12 03:17
