A wise society should not allow invisible systems to influence people's choices, rewards, and behaviors without granting them effective means to observe, question, and correct this influence. As artificial intelligence develops, society is sliding down a dangerous slope, rapidly moving from experimenting with and integrating AI to depending on it, and ultimately even becoming addicted. However, one of the most important issues is whether policymakers are aware of this shift.
Generally speaking, asymmetry means the two parties in a relationship are not equal. In digital life, "algorithmic asymmetry" describes a deeper imbalance between the two parties: one can observe, model, test, and improve its algorithms, while the other primarily bears the consequences of the algorithms. This imbalance has now permeated areas such as hiring, lending, insurance, education, policing, media, and the architecture of daily attention. Its consequence is the asymmetry of algorithmic agency, meaning users cannot identify and resist the improper influence of algorithms on their own situations.
The Three Layers of Algorithmic "Cognitive Shackles"
This algorithmic asymmetry can be explained on three levels.
The first level is opacity, which refers to the fact that organizations that design, deploy, or purchase algorithmic systems usually understand the system's goals, thresholds, incentives, and weaknesses better than the people interacting with them. The "opacity problem" explains why this gap persists: some systems are deliberately hidden to protect intellectual property, some require professional training to understand, and others are difficult to interpret even for experts. When a system is hard to inspect, its outputs often appear more objective than they actually are, leading to the "black box fallacy."
The second layer of algorithmic asymmetry is the amplification of historical bias. Algorithms learn from the past world, including past biases or exclusions. Even seemingly neutral systems may replicate existing patterns of inequality present in the data. The biased past is input as training material and ultimately output in the form of predictions, scores, or recommendations, which appear neutral because they are computational results. In reality, this is just older hierarchical structures reappearing in a more modern, streamlined interface.
The third layer is recursive systems. Systems are usually not deployed once and for all; instead, users continuously train these systems. Every click, pause, prompt, path choice, purchase behavior, and hesitation becomes data. Recommendation systems are designed to learn from these signals and adjust, but this is not the end of the cycle. Based on these learnings, the systems shape what we see next, determine what feels normal, what seems relevant, and sometimes even what feels desirable, while their goals remain obscure to the end-user. In other words, we train the systems, and the systems train us in return. "Algorithmic drift" refers to this co-evolutionary relationship between users and platforms.
When Algorithms "Live" for You
The agency of artificial intelligence refers to the ability to judge, choose, and act in meaningful ways, understanding the various forces affecting one's own choices.
Asymmetry of agency arises when organizations use digital systems—such as personalized pushes, targeted advertising, dynamic pricing, recommendation engines, risk scoring, etc.—to test, measure, and optimize influence and outcomes on a massive scale. Marketing has always tried to shape behavior; the difference now lies in precision and feedback mechanisms: organizations can observe individual behavior in real-time, segment populations into increasingly finer categories, continuously run A/B tests, and adjust what each person sees, pays, or the offers they receive. In contrast, individuals typically only see the surface of the system: a push notification, a score, a price, a recommendation, or a rejection, without knowing how their data is being used, which objective is being optimized, or how their choices are being steered.
This is crucial because people adapt to what the system rewards. In hiring, it's no longer just about whether job seekers meticulously polish their resumes to please recruiters; automated screening tools and AI ranking systems may reward certain specific signals while hiding the logic behind them. A University of Washington study found that after ranking over 550 real resumes, large language models favored resumes with names associated with white individuals in over 85% of cases, while never favoring resumes with names associated with Black men. In education, the UK's 2020 grading controversy showed how algorithmic models can translate school-level history into individual grades: the Office of Qualifications and Examinations Regulation (Ofqual) downgraded the school-assessed grades of about 40% of students, sparking public outcry and ultimately leading the government to withdraw the decision.
Furthermore, newer AI tools bring more risks. Stanford University researchers tested seven widely used AI detectors using samples from both native and non-native English speakers. The results showed that in samples from non-native speakers, the AI detectors incorrectly classified 61.22% of the essays as AI-generated, indicating that some students are more likely to be suspected or penalized because of their writing style. Similar phenomena appear in digital life and work. Facebook's famous News Feed experiment in 2014 on 689,003 users showed that changes in users' exposure to positive or negative posts affected the emotional language they used afterward. In retail, Amazon warehouse workers have also reported having to meet speed-based metrics without knowing how those metrics are calculated. Reports and research on algorithmic management in Amazon warehouses have explored this phenomenon. These cases reveal a deeper problem: digital systems are not just categorizing behavior after the fact. They also teach people which words to use, which risks to avoid, which emotions to express, and which metrics to pursue. When organizations shape the conditions under which people think, behave, and make decisions, while individuals merely experience these conditions as scores, grades, information, targets, or prices, the asymmetry of algorithmic agency takes on political significance.
Policies Must Go Beyond Slogans
Therefore, policies must rebalance this relationship. First, legislators should require meaningful notification and explanation when influence occurs. Users should know when they are interacting with AI, when content is synthetic, and when an important decision is affected by an automated system. The logic behind the European transparency obligations in Article 50 of the EU's AI Act points in the right direction. The OECD AI Principles also express the same view from a broader perspective: people need sufficient information to understand outcomes and, if necessary, challenge them.
Second, governments should require enforceable impact assessments before algorithmic systems enter high-risk areas such as employment, education, housing, insurance, healthcare, welfare, and policing. Some existing methods provide a basis for this, such as Canada's Algorithmic Impact Assessment, Ontario's Human Rights AI Impact Assessment, and Europe's Fundamental Rights Impact Assessment for high-risk AI systems. Recent failures indicate that stronger safeguards are crucial. In the UK, the Court of Appeal ruled in "R (Bridges) v. The Chief Constable of South Wales Police" that South Wales Police's use of live automated facial recognition technology was unlawful. In Detroit, Robert Williams was wrongly arrested due to a facial recognition error match, documented by the ACLU. Therefore, before deployment, agencies should assess the potential impacts of AI systems, such as rights infringements, harm to vulnerable groups, and error distribution, and also assess the need for human oversight, complaint mechanisms, and remedies, with public reporting where possible.
Third, human oversight must be genuine, effective, trained, and protected. In many institutions, the power of "human-in-the-loop" is often limited when employees face pressure to trust the system's outputs. Australia's "Robodebt scheme" showed how automated welfare debt calculations could harm people when officials treated system-generated claims as authoritative. In the R (Bridges) v. South Wales Police case, the UK Court of Appeal ruled that the use of live facial recognition was unlawful partly due to insufficient safeguards around discretion, data protection, and equitable impact. The UK Post Office's "Horizon" scandal exposed similar failures: people believed flawed software outputs over the lived experiences of hundreds of sub-postmasters. The value of Article 14 of the EU AI Act lies in its requirement that personnel overseeing high-risk AI systems must understand, monitor, interpret, override, or interrupt the system. Any institution using AI with significant impact should designate responsible reviewers, train them to recognize automation bias, and grant them real power to block harmful outputs.
Fourth, regulation should not stop at system release. Models drift, environments change, and incentives shift. A system that seems acceptable in testing may become discriminatory or manipulative once interacting with real people. Therefore, post-deployment monitoring, logging, independent auditing, and incident reporting should become legal obligations. The U.S. National Institute of Standards and Technology's "AI Risk Management Framework" and the post-market monitoring provisions in the AI Act acknowledge this. The Prosocial AI Index can be used to map, measure, and monitor the impact of AI systems on humans and their environments.
Fifth, certain practices should be prohibited. Systems designed to exploit weaknesses, distort behavior through deceptive design, or manipulate children and other vulnerable groups should be banned, not merely given soft guidance. Article 5 of the EU AI Act, which prohibits certain manipulative and exploitative uses, draws a necessary, firm line. A healthy digital society cannot rely solely on disclosure; it must also consider whether the underlying design aims to undermine judgment.
Algorithmic literacy should be seen as civic infrastructure. If only developers, vendors, and compliance teams understand how these systems operate, power asymmetry persists even under good regulation. Citizens, teachers, judges, journalists, clinicians, and public managers all need practical literacy regarding synthetic media, ranking systems, behavioral nudges, the right to question, and the limitations of model outputs. Article 4 of Europe's AI literacy clause is a beneficial signal and should evolve into a broader public mission. Beyond AI literacy, it is time to invest in dual literacy to ensure users are aware of the interaction between personal perception, behavior, and the influence of artificial agents.
Ultimately, the asymmetry of algorithmic agency is not an isolated technical problem but a structural imbalance in who can perceive, shape, and resist algorithmic power. One side learns faster, continuously tests, and quietly intervenes; the other adapts under partial information opacity. Good policy cannot eliminate this asymmetry entirely, but it can narrow the gap in the most critical areas by making automated influence visible, questionable, auditable, and governable.
This article is from the WeChat public account "Internet Law Review", author: Cornelia Waller
