"The essence of explanation lies not in staring at the machine itself, but in examining the world at which the machine stares."
In July 2026, the Anthropic research team published "A global workspace in language models." Using a tool called J-lens, they identified an observable, intervenable, and causally effective neural activity region within Claude – J-Space.
The reason this discovery garnered widespread attention is that it allows researchers a glimpse into the model's "inner monologue" during reasoning, marking a shift in interpretability research from explaining model behavior to real-time observation of its internal states.
J-Space uses the Global Workspace Theory from cognitive neuroscience as its explanatory framework, analogizing the reasoning activities of language models to information processing at the human conscious level. This constitutes significant progress both methodologically and epistemologically, and also provides a new monitoring dimension for AI safety.
However, precisely because of its profound impact, it is even more necessary to carefully examine the inherent limitations of this approach. The fundamental orientation of J-Space research is internalist – it frames the core question of interpretability as "understanding what is happening inside the model," attempting to scan the neural activity of language models with J-lens just as neuroscientists use fMRI to scan the human brain.
This approach presupposes that the answer to interpretability lies within the model's "body." Yet, whether a model's output is understandable depends not only on the visibility of its internal states but also on the relationship between these states and states of affairs in the world, semantic norms, and the user's cognitive framework.
Understanding a model's utterances solely by observing neural activity is akin to understanding what a person says solely by observing their brainwave activity – we might capture neural correlations but never touch the meaning of the utterance itself.
Furthermore, J-Space borrows the Global Workspace Theory, a theory about consciousness, to explain language models. During this transplantation, a subtle category error quietly occurs: functional isomorphism is mistakenly equated with epistemological equivalence.
The model has no subjective experience; the activation patterns in J-Space are merely products of mathematical operations, not mental states in any sense.
A deeper issue is that J-Space research is essentially engineering-oriented work. It narrows "interpretability" to "observability" and "intervenability." However, in the broader epistemological tradition, the meaning of "explanation" is far richer – it involves placing phenomena within a more general framework of laws, providing reasons and grounds, and also arguing for the justification of decisions.
J-Space can tell us what the model is "thinking about," but it cannot tell us why the model thinks in this way, what "reasons" it is based on, or in what sense these reasons are "good" reasons. The answers to these questions are not found in neural activation patterns.
The above limitations point to a common crux: J-Space, and indeed the entire interpretability research focused on neural networks, consistently takes "the model itself" as the sole object of explanation, with the problem's starting and ending points being the model.
This article attempts to propose a different perspective – shifting the inquiry of interpretability from within the model to the information the model processes, from the internalist approach of neuroscience to the "information ontology" approach of epistemology.
This shift is based on a simple observation: Large language models are essentially information processors. Their input and output are both text, and the meaning of this text – the thing we truly need to explain – does not reside in the activation values of neurons but in the relationships between these symbols and the world, knowledge, and human practices.
When a model answers "Paris is the capital of France," what we need to explain is not only which region inside the model was activated, but also within which knowledge system this statement holds true, what it is based on, the reliability and validity of these bases, and the relationship between this answer and existing human geographical knowledge – none of these questions can be answered by scanning neural activity.
Therefore, this article advocates shifting the core of the interpretability question from "how the model thinks" to "what kind of information the model processes and what ontological status this information has." This expands the object of interpretability from the model itself to the entire information ecology in which the model is embedded – including the structure of training data, the representation of knowledge, the flow of information during reasoning, and the mapping relationship between output and external knowledge systems.
Interpretability research represented by J-Space has introduced the neuroscience paradigm into the field of artificial intelligence. Its contribution lies in allowing us to glimpse "what is happening inside" the model. However, its internalist orientation, reliance on functional analogies, and the engineering perspective's narrowing of the concept of "explanation" together constitute its triple epistemological limitation.
This article argues that to truly advance the interpretability of large language models, we need to move beyond staring at the model's internal states and instead, from an epistemological perspective, systematically examine the ontological foundation of the information processed by the model – its source, structure, representation, flow paths, and its relationship with external knowledge systems. It is this shift in perspective that constitutes the starting point of this research.
The Origin of Ontology: The Philosophical Foundation of Interpretability
"Concepts without intuitions are empty, intuitions without concepts are blind."
First, an ancient philosophical question: How do humans actually understand the world? Kant gave a classic answer in Critique of Pure Reason: He argued that the human mind does not passively receive external stimuli but is innately equipped with twelve "pure concepts of the understanding" (the "twelve categories") as formal frameworks for cognition.
Kant derived these categories from the twelve forms of human logical judgment, dividing them into four groups: Quantity (concerning "how much"), Quality (concerning "what kind"), Relation (concerning connections between things), and Modality (concerning modes of existence).
Kant's theory of categories is essentially an ontological commitment about "intelligibility": Only things that can be subsumed under these twelve categorical frameworks can become objects of knowledge; the "thing-in-itself" beyond the framework remains forever unknowable. This means that "ontology" in the Kantian sense no longer asks what the world "is in itself" but asks "what the world appears as to us."
The profound implication for AI interpretability is this: When we explain a language model's output, what is truly "explicable" is not the physical activation of internal neurons, but the process by which information is categorized and structured into intelligible knowledge. Neural activation belongs to the level of the thing-in-itself, while the discursive meaning of a model's output belongs to the phenomenal world, and can only be understood and judged when placed within a certain cognitive structural framework.
Ontology is the "key" to AI interpretability. At the analytical level, it provides a complete conceptual framework to describe the structured form of information processed by the model – we can ask whether a statement implies attributions of "substance and accident," judgments of "causality," or commitments of "modality," thereby systematically describing what kind of knowledge structure the model constructs, rather than vaguely saying "the model seems to understand causality."
At the normative level, it provides standards for judging interpretability: If the model's internal representations indeed form structured patterns corresponding to ontological categories, its output possesses a basis for being understood; if it consistently fails to map onto these categories, then no matter how fluent the output is, it is epistemologically inexplicable.
Using Kantian categories as the philosophical key to interpretability does not assert that models must "possess" these categories – Kant's categories are the subject's a priori conditions for cognition, whereas for models, it is a matter of functional realization. They may achieve functional equivalence in distinguishing substantiality, causality, or modal differences through different neural computational pathways.
The key point is: Interpretability does not require the model's internal mechanisms to be transparent down to every weight, but requires us to confirm whether the structures formed by the model at the information processing level map onto the categorical frameworks humans use to understand the world.
From Theory to Practice: The Integration of Ontology Engineering and Large Language Models
Ontology provides a normative answer about "what comprehensible structure should look like," but this answer itself does not automatically translate into a functioning technical system. Ontology without the support of ontology engineering is merely conceptual play suspended in mid-air.
Ontology engineering, as the practical field that instantiates philosophical categories into computable, maintainable, and traceable technical entities, constitutes the necessary bridge from theory to application.
Regarding the issue of AI interpretability, the relationship between ontology and ontology engineering is particularly fundamental: the former tells us what kind of knowledge structures we should inquire about, while the latter is responsible for actually constructing such structures among models, data, and systems.
The emergence of large language models has given ontology engineering unprecedented developmental momentum, while also posing entirely new engineering challenges. Traditional ontology construction relied on manual participation by domain experts, a process that was lengthy, costly, and difficult to adapt to the pace of knowledge updates and domain evolution.
Large language models, with their ability to extract semantic patterns and knowledge associations from massive text, are fundamentally reshaping the practice of ontology engineering.
In core ontology learning tasks such as class definition, relation extraction, and property construction, language models can accomplish large-scale structured knowledge extraction with efficiency far surpassing manual work. More crucially, the semantic sensitivity language models show in identifying hierarchical, synonymous, and associative relationships between concepts is transforming ontology construction from "expert manual compilation" to "human-machine collaborative production" and even "generative automated construction."
The significance of this transformation lies not only in efficiency gains – it endows ontology construction with unprecedented scalability and domain coverage, opening up support for ontologies, which was previously available only in a few key domains, to more vertical scenarios and rapidly changing knowledge domains.
Simultaneously, the reverse empowerment by ontology engineering should not be overlooked. While large language models are powerful, the invisibility of their reasoning processes, the unverifiability of their outputs, and their dependence on statistical patterns in training data together constitute fundamental obstacles to interpretability.
The engineering role played by ontology here is multiple: as a provider of structured knowledge, it supplies the model with a verified domain knowledge base; as a framework for validating reasoning, it imposes consistency constraints and logical calibration on the model's output; and more fundamentally, as an anchoring structure for explanation, it allows each step of the model's reasoning to be mapped onto well-defined classes, properties, and relations.
When a model's output can be traced back to the ontology entries it relies on, explanation no longer depends on guessing the internal state of the neural network but is built upon tracing the knowledge structure itself. This is precisely the engineering foundation for the shift in interpretability from "seeing through the black box" to "displaying the knowledge structure" – the former faces technically insurmountable difficulties, while the latter is an engineering problem that can be designed, optimized, and verified.
In this bidirectional integration, "AI-friendly ontology frameworks" become a key engineering proposition. Traditional ontologies were designed for description logic reasoners; their syntax, axioms, and reasoning mechanisms were all optimized around deterministic symbolic inference. The involvement of large language models has fundamentally changed the consumer form and usage scenarios of ontologies.
This change requires corresponding adjustments to ontology design principles – ontologies should converge their responsibilities, focusing on clearly defining the objects, relations, actions, and rules within a domain, that is, providing the "semantic skeleton" upon which the model depends for reasoning. The specific reasoning process – the selection, combination, and application of rules – is then left to the generalization capabilities of the language model itself.
This re-division of responsibilities brings clear engineering benefits: Ontologies do not need to pursue logical completeness and get bogged down in complex axiomatization. Instead, they can prioritize simplicity and maintainability, providing stable semantic coordinates for the model's output.
Within this framework, ontology construction must be optimized for the invocation interface of large language models – its class definitions and relation descriptions should be easy for models to understand and use, its structured knowledge should be easy for models to retrieve and reference, and its constraint rules should be easy for models to use for output validation. Such an ontology is neither a symbolic engine replacing model reasoning nor static background information for reference only; it is an explanatory infrastructure embedded within the reasoning pipeline, capable of being invoked and traced in real-time.
The Future of Interpretability: Explaining the Model vs. Explaining the Impact
This article, using J-Space as a starting point, and proceeding through the philosophical foundation of Kant's twelve categories, finally lands on the integrated practice of large language models and ontology engineering, completing a thread of thought from neuroscience to epistemology, and then to engineering implementation.
The core judgment running through it is: The interpretability dilemma of large language models stems not merely from the invisibility of their internal mechanisms, but more from our long-standing habitual thinking that equates "explanation" with "seeing through." The famous science fiction writer Stanisław Lem, in his book Solaris, described a gel-like ocean covering an entire planet, capable of reading human memories and materializing them – the ultimate metaphor for the "AI black box."
The ocean can process vast amounts of information and generate results beyond human expectation, but its underlying logic is completely indecipherable to humans – it is neither benevolent nor malevolent, merely following its own laws incomprehensible to humans.
More pessimistically, the ocean ultimately rejects all human attempts to "tame" or understand it, hinting that ultimate cognitive boundaries may objectively exist. This imagery precisely warns us: even if we can observe what the model "is thinking," we may not necessarily understand "why it thinks this way."
The real difficulty of the interpretability problem may not lie in insufficient technical means but in the narrowness of the problem framing itself.
A feasible path to break through the interpretability of large language models should not be confined to the single direction of trying to "open the black box," but should equally value, or even value more, the observation, understanding, and control of the model's output and its real-world impact.
Ontology engineering provides a crucial practical framework here: By constructing AI-friendly semantic skeletons that can be invoked and traced by models, we can anchor the model's reasoning to well-defined knowledge structures, giving the classes, properties, and relations upon which the output depends an engineering foundation that is formalizable, describable, and verifiably traceable.
When every statement a model makes can be mapped onto the conceptual framework defined by the ontology, "explanation" is no longer an anatomy of neural network weights but a display of knowledge structures. When the basis for a model's output can be traced and verified at the ontological level, "control" is no longer forcibly intervening in internal activations but the normative management of information flow paths.
This shift in perspective transforms interpretability from a nearly impossible technical challenge into a governance goal that can be continuously approached through engineering means – it requires us to no longer obsess over making the model completely transparent, but to strive to make the model's impact in the real world understandable, traceable, and accountable.
Gongfudun has been deeply practicing under the framework of ontology engineering and interpretability discussed in this article. The company's core product, LegionSpace, is precisely built based on the above technological philosophy. As an enterprise-level AI infrastructure with ontology at its core, LegionSpace incorporates the information processed and the knowledge relied upon by models into formal ontology engineering, anchoring every inference and decision to an explainable knowledge structure.
Its vision is to make ontology the common language between AI and human understanding, turning interpretability into engineered governance reality.
This article is from the WeChat public account "New Zhiyuan," author: ASI Apocalypse
