Large language models now truly have hands.
This Thursday, Anthropic announced the launch of MCP for hardware: Model Hardware Standard (MHS). This is a new standard designed to help all large model-driven AI agents safely and quickly manipulate physical devices.
Now, large models like Claude can operate hardware such as microscopes, robotic arms, liquid handlers, and lasers in the physical world, just like using MCP (Model Context Protocol). This development is seen by the industry as a key step for AI moving from the digital world to the physical world.
Those who use agents have probably heard of MCP. It is an open standard protocol introduced by Anthropic in November 2024, aiming to provide a standardized, secure, bidirectional communication interface between large language models (LLMs) and external data sources, local files, development tools, and various application services.
The core positioning of MCP is the 'USB-C interface for the AI field,' enabling connectivity between large models and software environments like GitHub, Slack, local file systems, and databases. With recent AI technology developments, MCP has evolved into an indispensable de facto standard in the entire agent ecosystem.
Now, Anthropic, the same company, hopes to extend this standard communication protocol to the next dimension: physical hardware, sensors, embedded systems, and test equipment through MCP for hardware.
The development of MHS began with a collaboration between Anthropic and the Howard Hughes Medical Institute (HHMI) Janelia Research Campus. It is now open for research preview to the first group of research labs and advanced manufacturers.
Typically, labs or manufacturing plants require weeks or even months to set up and integrate hardware. Most devices cannot communicate directly with each other, necessitating experts to build custom integration solutions. MHS can reduce this integration work to hours or even minutes. Furthermore, by integrating AI into these tools, MHS also helps researchers and engineers more easily coordinate autonomous, around-the-clock experiments and workflows. Agents can reason about each step in an experiment, update parameters in real-time, and in some cases, recover from hardware failures without human intervention.
MHS can work with any device that has a programmable interface. It is also agnostic to the type of foundational model; any agent framework can use standard protocols, such as MCP, to access it.
Even without considering the additional difficulty of integrating AI into systems, getting multiple devices in a lab or factory floor to communicate with each other is not easy. Each device usually has its own programming interface, and there is currently no standardized method for integration. Moreover, once devices are connected, there is no universal way for them to share data with an Agent, nor a method for the Agent to safely operate these devices.
Now, MHS addresses these challenges by introducing a standardized driver: this is a piece of software that translates between a computer's operating system and a hardware device. The MHS driver uses a set of simple, basic commands like read (e.g., get temperature) and write (e.g., set temperature), which any hardware device can understand and execute. It enables each device to be discovered in a standard format, allowing devices and agents to discover each other and communicate across networks without needing a custom 'translation' program in between.
The MHS driver also helps an agent understand and use devices it has never encountered before, and provides it with machine-specific information that might not be available from code alone (such as the weight of a robotic arm, which is crucial for safe operation). Historically, this information has mostly been stored in paper manuals, on users' computers, or as tacit knowledge. But the MHS driver includes tags, allowing users to input this information directly in natural language (users can input it themselves, or AI can ask about their hardware setup through chatting with the agent).
Using the information in these tags, the MHS driver automatically generates a reference file containing general characteristic information about the device, such as what it can measure, what can be adjusted, and what safety limits will be enforced. This file provides the agent with everything it needs to know to operate the device.
After devices are connected and the agent program understands how to use each one, a method to control this hardware is still needed. For MHS, there are three such mechanisms: MCP, a command-line interface, and code files (APIs). They work together, allowing users to coordinate multiple devices with a single line of code.
Once an agent can control devices, it can receive operational data from each device and supervise and guide the work at a high level. The agent can sequence operational steps across different instruments, monitor results, and adjust parameters based on changing conditions in real-time. When the agent needs to perform long-running tasks or operate devices at speeds beyond its online reasoning capability, it can chain driver commands from one or more devices into a code file. This way, devices can execute operations autonomously without requiring the agent to reason through each step.
During the testing of MHS, Anthropic found that Claude interacts with experiments and hardware in an exploratory manner akin to a scientist.
Anthropic observed Claude adjusting a laser and observing the results through a camera, evaluating how adjustments moved the laser beam, then repeating the process to try to understand the sequence of events. Afterward, Claude packaged the learned information into a code file, writing a deterministic script that allowed it to calibrate the laser without needing to reason through each step, so the entire process could be run as a single command.
During the development of MHS, Anthropic shared this technology with several labs and hardware manufacturers in fields like biotechnology, robotics, and quantum computing. In these early projects, we can see MHS shortening device integration time, increasing iteration speed in various experimental settings, and aiding in real-time operation and fault detection of machines.
Hardware vendors and their accompanying software companies are also building MHS support into their devices so that agents can discover and operate them. For example:
Amazon Web Services (AWS) is supporting MHS through Strands Robots. This is a library for connecting AI agents to physical devices. During the MHS research preview, AWS will provide a private pre-release version of the Strands Robots software package to participants.
Automata is adding MHS support to its laboratory automation platform LINQ for intelligent error handling of instruments in autonomous labs.
Danaher is actively exploring how MHS-supported features can enable its smart instruments and autonomous labs to scale biomedical research and development.
Doosan Robotics is testing MHS with its robotic arms, including performing automated quality assurance and coordinating tasks among multiple robots.
MBF Bioscience is developing an MHS driver for ScanImage, the software that runs laser-scanning microscopes in hundreds of neuroscience labs worldwide, aiming to integrate AI agents into real-time data analysis and experiments.
QIAGEN is experimenting with MHS through a proof-of-concept on its nucleic acid purification platform QIAsymphony Connect to demonstrate how AI agents can help labs troubleshoot instrument failures faster, guide operators in recovery, increase instrument uptime, and reduce risks to biological samples.
Tecan is adding MHS support to its Fluent liquid handling platforms so that AI agents can directly discover and operate them.
Universal Robots has gained early access to MHS and plans to add MHS support to its robotic platforms.
Before open-sourcing, Anthropic also wants to further refine the standard.
As a large language model, Claude learns about the physical world through text and images, which means its spatial and physical reasoning capabilities have limitations, still requiring expert supervision. For example, when handling protein samples, researchers at Genentech had to guide Claude to recognize that errors caused by sample bubbling were physical faults, not software errors, and could only be mitigated through corresponding physical corrections.
MHS currently cannot be compatible with hardware lacking a programming interface, so Anthropic is working with manufacturers of such devices to integrate MHS drivers into the hardware. Many developers are already using Claude Code to operate single physical devices. In the next phase of MHS, Anthropic hopes to extend the standard to cover more of the devices developers are using.
Early adopters include Hugging Face (which is adding MHS support to its robotics library LeRobot) and Raspberry Pi (which is enabling MHS integration across several of its products after successful testing with its Camera MHS driver).
References:
https://www.anthropic.com/news/model-hardware-standard-research-preview
This article is from the WeChat public account "Almost Human" (ID: almosthuman2014), author: Agent-focused Almost Human.
