Deliberation Deficit and Black Box Justice: Why AI Cannot Yet Replace the Arbitrator’s Mind

Imagine an AI ruling on your dispute—flawless data, zero explanation. Justice? Justice isn’t just a verdict, it’s who decides, and how. In the realm of arbitration, particularly international commercial arbitration, the rise of artificial intelligence (AI) presents both opportunities and profound challenges. While AI-enhanced tools such as legal research engines, contract review systems, and procedural drafting assistants are becoming commonplace, the temptation to delegate deeper adjudicative functions to machines must be approached with caution. As someone who has watched AI reshape industries, I find its promise in arbitration tantalizing yet deeply troubling. This article explores two central limitations of such delegation: the phenomenon of Deliberation Deficit and the problem of Black Box Justice.

Deliberation Deficit—a concept developed by the author in ongoing scholarly research on ethical AI integration in dispute resolution—describes the cognitive and moral limitations of AI systems that prevent them from replicating the nuanced, context-sensitive reasoning that human arbitrators bring to complex disputes. Deliberation Deficit refers to AI’s inability to understand human emotions, cultural differences, or moral nuances. A human senses a witness’s fear—say, a trembling voice hinting at coercion—while AI sees only words, missing the story. In a family feud, this could mean everything. Arbitration hinges on more than rules—it requires sensitivity to people, cultures, and the spirit of agreements, areas where AI falls short.

Black Box Justice—a term developed by the author in ongoing scholarly research on AI-assisted arbitration—captures the problem of opacity in algorithmic reasoning. While drawing inspiration from Frank Pasquale’s foundational work in The Black Box Society (2015), which critiqued the secrecy of algorithmic governance in financial and social sectors, this term applies the concern specifically to arbitral awards and judicial legitimacy. It raises questions about explainability, accountability, and enforceability under legal instruments like the New York Convention. It occurs when AI delivers rulings without showing its work—leaving everyone guessing how it decided. Even accurate outcomes lose legitimacy if they are inaccessible to scrutiny.

To see these concepts in action, consider this scenario: Picture a Japanese supplier and a US buyer clashing. The Japanese witness paused—respect in his culture—but AI, Western-trained, flagged uncertainty. This didn’t just dent credibility; it threatened millions and a decade-long bond, all because AI misread silence. Like healthcare AI misdiagnosing minorities (Hao, 2020), arbitration AI misses cues it wasn’t built for—Deliberation Deficit in stark relief.

Beyond ethical concerns, AI’s use in arbitration raises serious legal challenges. Article V(1)(e) of the New York Convention allows for the refusal of enforcement where an award is not reasoned or is contrary to public policy. If a tribunal relies on AI-generated reasoning that cannot be independently justified or understood, a national court may justifiably set aside the award or refuse enforcement (UNCITRAL, 2006, Article 31(2); United Nations, 1958, Article 5 (1) (e)). Explainability is no longer a luxury—it is a legal necessity. Consider the scenario: Picture an AI award upheld in Paris but struck down in Dubai—its logic a mystery. One party celebrates, the other fumes, trust in arbitration frays. Courts won’t tolerate that (Zalnieriute & Moses, 2020).

Proponents of AI in arbitration often argue that machines can minimize human biases by offering neutral, pattern-based decision-making. While this may hold true for limited applications like document review, extending this logic to final judgment ignores a key fact: machines inherit the prejudices of their makers—prejudices harder to spot and impossible to cross-examine. Can we trust that? An AI fed Western cases might skew against Middle Eastern norms. Hao (2020) warns of data-driven bias; here, it’s Synthetic Neutrality—a term coined by the author to describe the deceptive appearance of impartiality in AI outputs that, despite seeming neutral, may encode structural bias due to data training, model design, or default legal assumptions. An AI steeped in US precedent might dismiss a handshake deal vital in Riyadh. Humans adapt to such norms; AI’s rigid lens distorts justice.

Efficiency is often touted as AI’s greatest contribution to arbitration. Yet justice is not a race to resolution. Speed without sensitivity can amplify harm, especially in complex, emotionally charged disputes. A system that delivers a prompt outcome but overlooks cultural nuance, power imbalance, or unspoken meaning risks being efficient—but unjust. Arbitration must remain a deliberative process where context matters. Delegating too much to machines risks redefining fairness through the narrow lens of computation. In pursuit of quicker awards, we must not sacrifice the foundational principles of listening, empathy, and meaningful party participation that make arbitration a just alternative (Wagner, 2021).

To responsibly integrate AI, arbitration must move toward auditable algorithms—systems that retain records of how and why decisions were reached. Ethical infrastructure should include AI declarations in procedural orders, oversight panels within institutions, and opt-out mechanisms for parties uncomfortable with algorithmic intervention. These steps are not bureaucratic hurdles; they are essential to preserving trust. Just as procedural fairness underpins legitimacy in human-led arbitration, algorithmic transparency is vital for machine-assisted systems. If parties are to accept outcomes, they must first be able to understand how those outcomes emerged. Arbitrators cannot defer that responsibility to code (Burrell, 2016; European Parliamentary Research Service, 2020).

How do we keep arbitration just in a digital age? AI can scan files, detect inconsistencies, and manage timelines, but it cannot judge. It cannot discern when silence is sorrow or when a glance carries the weight of a broken promise. Mandating AI logs or requiring human sign-off isn’t just good practice—it’s an ethical imperative. Without transparency, explainability, and accountability, we risk outsourcing judgment to machines that lack understanding.

I’ve seen cases turn not on the law alone, but on intangibles that AI cannot compute: empathy, cultural fluency, the unspoken hesitation before a reply. Arbitration is not just a process—it’s a human encounter. And that human core must remain intact.

Institutions, too, must rise to the challenge—by adopting ethical frameworks, defining boundaries for AI tools, and ensuring enforceability does not depend on the whims of opaque algorithms. Jurisdictions should not compete in a regulatory race to the bottom but converge around shared standards of fairness and human oversight.

The real question isn’t whether AI will evolve—it’s whether we will demand that it evolve in service of justice, not in defiance of it. Until then, AI belongs in the assistant’s chair, not at the tribunal’s bench.

About the author: Rishabh Gandhi is an Arbitration lawyer and former trial court Judge. Gandhi is also the founder of Rishabh Gandhi and Advocates.

Disclaimer: The opinions expressed in this article are those of the author(s). The opinions presented do not necessarily reflect the views of Bar & Bench.

If you would like your Deals, Columns, Press Releases to be published on Bar & Bench, please fill in the form available here.