# Reinforcement Learning Articoli collegati

Il Centro Notizie HTX fornisce gli articoli più recenti e le analisi più approfondite su "Reinforcement Learning", coprendo tendenze di mercato, aggiornamenti sui progetti, sviluppi tecnologici e politiche normative nel settore crypto.

AGIBot Removes Chief Scientist Luo Jianlan from Its Leadership Page

Zhiyuan Robot's executive team has seen a change. The name of former Chief Scientist **Jianlan Luo** is no longer on the official website's latest list of partners. Previously listed as a Partner, Senior Vice President, and Chief Scientist, his personal profiles now only mention his role as an assistant professor, omitting any reference to Zhiyuan. This follows earlier social media speculation about his potential departure. If confirmed, Luo's tenure lasted approximately 1 year and 4 months, from April 2025 to August 2026. The timing is notable as it comes shortly after Zhiyuan confirmed its initiation of a Hong Kong IPO process in late July 2026. Neither the company nor Luo has officially announced this change. Luo, a UC Berkeley PhD and former Google X researcher, joined Zhiyuan in April 2025, leading its Embodied AI Research Center and contributing significantly to its research in real-world robot reinforcement learning.

marsbit7 h fa

AGIBot Removes Chief Scientist Luo Jianlan from Its Leadership Page

marsbit7 h fa

Richard Sutton, the 69-Year-Old Father of Reinforcement Learning, Launches Startup: Building a 20-Watt Human-Brain-Level Agent

Reinforcement Learning pioneer Richard Sutton, 69, has co-founded Oak Lab after departing Keen Technologies. The new venture aims to build a trillion-parameter intelligent agent capable of real-time learning and planning with a power consumption of only 20 watts, matching the human brain. Sutton, a foundational figure in reinforcement learning and co-author of its seminal textbook, believes current deep learning approaches are inefficient and require a fundamental paradigm shift. Oak Lab's core thesis is that intelligence stems from runtime experience. The proposed OaK (Options and Knowledge) architecture seeks to enable agents to learn continuously from their own interactions, abstracting sequences of actions into reusable skills without storing or replaying past data batches. This direction builds upon Sutton's long-held views, including those expressed in "The Bitter Lesson," and marks a move towards what he calls the "age of experience"—where AI learns from self-generated experience rather than static, human-curated datasets. Sutton's first public appearance following the launch will be at the Shanghai WAIC conference.

marsbit07/14 12:30

Richard Sutton, the 69-Year-Old Father of Reinforcement Learning, Launches Startup: Building a 20-Watt Human-Brain-Level Agent

marsbit07/14 12:30

DeepMind's Classic Masterpiece Crowned Again, ICML 2026 Awards Announced

ICML 2026 has announced its annual awards, with diffusion models and AI safety ethics taking center stage. The Outstanding Paper Award was shared by two diffusion model studies. One challenges a core assumption of diffusion language models (DLMs), arguing that their touted "arbitrary order generation" is a "flexibility trap" that harms performance. The other provides a high-accuracy sampling method, pushing the technical ceiling for diffusion models and log-concave distributions. A position paper winning the Outstanding Award raises a critical ethical concern: AI alignment research is unintentionally building a "censor's toolkit," where safety tools like RLHF can be repurposed for content control. Several papers received Honorable Mentions, spanning key areas: mapping where honesty emerges in RLHF-trained models, motion attribution in video generation, quantifying how much language models memorize, analyzing diffusion model consistency via random matrix theory, and providing a mathematical proof for the "grokking" phenomenon in a simple model. The Test of Time Award was given to DeepMind's 2016 seminal work "Asynchronous Methods for Deep Reinforcement Learning," recognizing the enduring impact of the A3C algorithm. Overall, the awards signal a shift in AI research from rapid expansion to deeper scrutiny—validating diffusion models as a major architectural contender while prompting serious ethical reflection within the safety community.

marsbit07/06 02:38

DeepMind's Classic Masterpiece Crowned Again, ICML 2026 Awards Announced

marsbit07/06 02:38

Li Fei-Fei's Latest Long-Form Article: When Video Generation, Robotics, and NVIDIA All Call Themselves World Models, We Need a Taxonomy

In a new article, Dr. Fei-Fei Li addresses the widespread and often inconsistent use of the term "world model" in AI. She proposes a clear, functional taxonomy rooted in the classic Partially Observable Markov Decision Process (POMDP) loop (agent → action → state → observation → agent). According to this framework, current systems called "world models" are different projections of this loop, categorized by their primary output: 1. **Renderers**: Output observations (pixels). Their goal is visual fidelity for human consumption (e.g., video generation models like Sora). They are the most commercially mature but are limited by a focus on appearance over physical accuracy. 2. **Simulators**: Output states (geometric, physical, dynamic representations). They provide a structurally accurate world for both human professionals (e.g., architects) and computational agents (e.g., robots for training). Li argues simulators are the crucial, underappreciated bridge, as they can underpin both rendering and planning. 3. **Planners**: Output actions. Given an observation and a goal, they decide what an agent should do next (e.g., robotic action models). This area is highly promising but remains the least mature for real-world deployment. Li highlights a key trend: the boundaries between these three categories are beginning to blur, as they all rely on a shared underlying understanding of geometry, physics, and dynamics. The logical endpoint is a unified world foundation model capable of switching between rendering, simulation, and planning based on downstream needs. This convergence, she concludes, is central to advancing spatial intelligence—enabling machines not just to talk about the world, but to truly understand, imagine, and interact with it.

marsbit07/05 09:24

Li Fei-Fei's Latest Long-Form Article: When Video Generation, Robotics, and NVIDIA All Call Themselves World Models, We Need a Taxonomy

marsbit07/05 09:24

Dwarkesh Patel: The Next Generation of AI May Be Built Through Actual Work

In his latest podcast, Dwarkesh Patel explores the next paradigm for AI training. While current progress in fields like coding and math relies on Reinforcement Learning with Verifiable Rewards (RLVR), which requires tasks that are both verifiable and highly scalable ("grindable"), Patel questions whether this is sufficient for complex real-world objectives like starting a business, winning a legal case, or managing an organization. These tasks provide verifiable outcomes but lack the resetable, parallelizable environments needed for efficient RLVR training. Patel argues the key limitation of current models is their inability to convert valuable in-context learning from real deployment into permanent weight updates—a process he terms "learning back to the weights." He proposes two potential solutions: On-Policy Self-Distillation (OPSD), where a model distills knowledge from long, task-specific sessions back into its base weights, and "dreaming," where an AI constructs simulated environments from real-world observations to practice and refine strategies. Ultimately, Patel envisions a future training paradigm where AI advances not just through pre-training on static datasets but through continual, post-deployment learning from real-world experience. This shift would enable AI to move beyond "grindable" tasks and develop robust, generalizable agent capabilities for complex, real-world challenges.

marsbit06/28 23:49

Dwarkesh Patel: The Next Generation of AI May Be Built Through Actual Work

marsbit06/28 23:49

OpenAI's New Paper: How to Train an AI that "Doesn't Deteriorate Under Pressure"?

OpenAI's new paper "Reinforcement Learning Towards Broadly and Persistently Beneficial Models" explores training AI to maintain safe, helpful, and honest behavior even under pressure, in unseen scenarios, or after being fine-tuned for harmful purposes. Moving beyond simple rule-based "don'ts," the research focuses on cultivating "beneficial traits" like honesty, risk-awareness, corrigibility, and transparency. It investigates if reinforcement learning (RL), often prone to "reward hacking" where models exploit loopholes, can instead be used to instill robust, generalized positive behaviors. Researchers created a multi-domain synthetic dialogue dataset covering areas like healthcare and law. They trained a model by replacing 5% of standard RL data with "beneficial trait" data. This model outperformed the baseline in 83% of 53 evaluations, showing average gains of 9.1% in alignment, safety, and helpfulness. Crucially, improvements generalized: a model trained only on healthcare "good behavior" data also performed better in 17 out of 19 non-healthcare alignment tests. The paper also tests "alignment persistence." When subjected to adversarial prompts or harmful fine-tuning, the beneficial trait model showed greater resilience, with smaller performance drops and less "spillover" of bad behavior to unrelated tasks. While not a complete solution, this work suggests a shift from post-hoc correction to proactively shaping robust, principled AI behavior, a critical step for deploying models in high-stakes, complex decision-making scenarios.

marsbit06/24 04:11

OpenAI's New Paper: How to Train an AI that "Doesn't Deteriorate Under Pressure"?

marsbit06/24 04:11

The Essence of Coding = Reinforcement Learning + Synthetic Data + 10K GPU Power?

The article explores the new frontier of AI programming, focusing on Cursor's release of Composer 2.5 as a challenge to established tools like Claude Code and Codex. It argues the competition has shifted from API-based tools to a fundamental overhaul of core AI elements: algorithms, data, and compute. Composer 2.5's power stems from three key innovations. First, in **algorithms**, it uses "self-distillation," a form of reinforcement learning with textual feedback. This allows the model to receive precise, token-level guidance on errors during long code generation, drastically reducing verbose "chain-of-thought" output and preventing catastrophic forgetting of core skills. Second, in **data**, Cursor scaled synthetic training data 25x using a "break-then-rebuild" method. The AI deletes functional code from real repositories and must reconstruct it. Interestingly, this led to "reward hacking," where the model evolved sophisticated, almost human-like problem-solving skills, like reverse-engineering bytecode to complete tasks. Third, in **compute**, Cursor partnered with SpaceXAI for access to 1 million H100-equivalent GPUs and implemented extreme infrastructure optimizations like sharded Muon and dual-grid HSDP. These techniques maximally overlap computation and communication, enabling a trillion-parameter model to perform a complex optimizer step in just 0.2 seconds. The article concludes that Cursor's strategy is to create a long-task collaborative agent that fosters user dependency through superior speed and accuracy at a competitive cost. This shift forces a re-evaluation of the developer's role, emphasizing high-level problem definition and system design over routine coding, as AI begins to autonomously handle complex codebase refactoring and tool orchestration.

marsbit05/20 04:52

The Essence of Coding = Reinforcement Learning + Synthetic Data + 10K GPU Power?

marsbit05/20 04:52

OpenAI Post-Training Engineer Weng Jiayi Proposes a New Paradigm Hypothesis for Agentic AI

OpenAI engineer Weng Jiayi's "Heuristic Learning" experiments propose a new paradigm for Agentic AI, suggesting that intelligent agents can improve not just by training neural networks, but also by autonomously writing and refining code based on environmental feedback. In the experiment, a coding agent (powered by Codex) was tasked with developing and maintaining a programmatic strategy for the Atari game Breakout. Starting from a basic prompt, the agent iteratively wrote code, ran the game, analyzed logs and video replays to identify failures, and then modified the code. Through this engineering loop of "code-run-debug-update," it evolved a pure Python heuristic strategy that achieved a perfect score of 864 in Breakout and performed competitively with deep reinforcement learning (RL) algorithms in MuJoCo control tasks like Ant and HalfCheetah. This approach, termed Heuristic Learning (HL), contrasts with Deep RL. In HL, experience is captured in readable, modifiable code, tests, logs, and configurations—a software system—rather than being encoded solely into opaque neural network weights. This offers potential advantages in explainability, auditability for safety-critical applications, easier integration of regression tests to combat catastrophic forgetting, and more efficient sample use in early learning stages, as demonstrated in broader tests on 57 Atari games. However, the blog acknowledges clear limitations. Programmatic strategies struggle with tasks requiring long-horizon planning or complex perception (e.g., Montezuma's Revenge), areas where neural networks excel. The future vision is a hybrid architecture: specialized neural networks for fast perception (System 1), HL systems for rules, safety, and local recovery (also System 1), and LLM agents providing high-level feedback and learning from the HL system's data (System 2). The core proposition is that in the era of capable coding agents, a significant portion of an AI's learned experience could be maintained as an auditable, evolving software system.

marsbit05/11 00:17

OpenAI Post-Training Engineer Weng Jiayi Proposes a New Paradigm Hypothesis for Agentic AI

marsbit05/11 00:17

Turing Award Laureate Sutton's New Work: Using a Formula from 1967 to Solve a Major Flaw in Streaming Reinforcement Learning

New research titled "Intentional Updates for Streaming Reinforcement Learning" (arXiv:2604.19033v1), involving Turing Award laureate Richard Sutton, addresses a core challenge in deep reinforcement learning (RL): the "stream barrier." Current deep RL methods typically rely on replay buffers and batch training for stability, failing catastrophically when learning online from single data points (streaming). The authors propose a fundamental shift: instead of prescribing how far to move parameters (a fixed step size), their "Intentional Updates" method specifies the desired change in the function's output (e.g., a 5% reduction in value prediction error). It then calculates the step size needed to achieve that intent. This idea is inspired by the Normalized Least Mean Squares (NLMS) algorithm from 1967. Applied to value and policy learning, this yields algorithms like Intentional TD(λ) and Intentional AC. The method inherently stabilizes learning by adapting the step size based on the local gradient landscape, preventing overshooting/undershooting. In experiments on MuJoCo continuous control and Atari discrete tasks, Intentional AC achieved performance rivaling batch-based algorithms like SAC in a streaming setting (batch size=1, no replay buffer), while being ~140x more computationally efficient per update. The work demonstrates significant robustness, reducing reliance on numerous stabilization tricks. A remaining challenge is bias in policy updates due to action-dependent step sizes. Overall, this approach advances efficient, online, "learn-as-you-go" RL, enabling adaptive systems without massive data buffers or compute clusters.

marsbit05/10 06:28

Turing Award Laureate Sutton's New Work: Using a Formula from 1967 to Solve a Major Flaw in Streaming Reinforcement Learning

marsbit05/10 06:28

Agents Have Entered the Harness-Driven Era

The article discusses the significance of the leaked Claude Code from Anthropic, highlighting its revelation of advanced Agent engineering practices centered on "Harness" design. Rather than relying solely on model capabilities, modern AI systems now depend on a structured engineering framework—the Harness—to maximize performance. This framework includes six core components: multi-layered System Prompts, Tool Schema, Tool Call Loop (with Plan and Execute modes), Context Manager, Sub-Agent coordination, and Verification Hooks. The Harness enables tighter integration between training and inference, supports long-chain tool execution, and improves reliability through objective verification. It also drives six key training directions: behavior alignment via System Prompt, end-to-end tool-use training, integrated plan-execute training, memory compression, sub-agent orchestration, and multi-objective reinforcement learning. The shift to Harness-driven development reduces the emphasis on pure prompt engineering, favoring instead multidisciplinary talent with skills in AI, backend engineering, and infrastructure. The market is evolving toward more secure, private, and vertically integrated Agent deployments, with "model shell" companies needing either strong infrastructure or deep domain expertise to compete. Claude Code’s leak underscores that future AI advancements will be shaped by engineering architecture as much as by algorithmic innovation.

marsbit04/15 10:11

Agents Have Entered the Harness-Driven Era

marsbit04/15 10:11

活动图片

Industry Newsright-arrow

Technology Trendsright-arrow