"The attempt to make heaven on earth invariably produces hell." - Karl Popper
"There is no algorithm for justice. There is only the ongoing work of deciding, again and again, what justice requires." - adapted from Hannah Arendt
AI systems discriminate. This is not in dispute. Resume-screening tools penalize women's names. Credit-scoring models extract higher interest rates from Black borrowers with identical credit profiles. Predictive policing sends officers to the same neighborhoods over and over, generating the arrest data that justifies sending officers to the same neighborhoods over and over. Facial recognition fails more often on dark-skinned faces. Healthcare algorithms direct fewer resources to sicker Black patients. Content moderation flag homoerotic images as "sexually suggestive" while leaving equivalent heterosexual images untouched.
These are facts. They are documented. They are not controversial among people who have looked at the evidence.
What is controversial is what to do about it. And the controversy, as usual, is more revealing than the facts.
I. The Standard Response and Why It Fails
The standard response to algorithmic bias has four moves:
Measure it. Define fairness metrics - demographic parity, equalized opportunity, calibration - and audit the algorithm against them.
Explain it. Require the algorithm to be interpretable or explainable, so that when it discriminates, we can understand why.
Remove the offending attributes. Strip race, gender, and other protected categories from the training data, so the algorithm cannot use them as the basis for discrimination.
Add more data. Diversify the training set so that underrepresented groups are better represented, reducing the gaps in the model's knowledge.
Each of these sounds reasonable. Each has generated an enormous literature, dozens of conferences, hundreds of papers, and a thriving industry of fairness consultants and responsible AI teams. And each fails - not because it is badly implemented, but because the problem it addresses is the wrong problem, or rather, a smaller problem than it appears to be.
Fairness metrics are mathematically incompatible. This was demonstrated by Chouldechova (2017) and Kleinberg, Mullainathan, and Raghavan (2016), and it is one of the most important results in the fairness literature that most people outside the field do not know. It says, roughly: you cannot simultaneously satisfy three intuitively necessary conditions for fairness in a classifier. You can pick any two. You cannot have all three. This is not an engineering limitation. It is a mathematical theorem. It means that every fair algorithm involves a choice about which dimension of fairness to sacrifice, and that choice is - not technical, not neutral, not discoverable by further research - political. The algorithm designer is doing what the book has been describing from the beginning: compressing a plural, contested moral landscape into a single, machine-encodable specification, and in the compression, losing exactly the complexity that made the question morally significant in the first place.
Explainability is not accountability. Knowing why an algorithm made a decision does not change the fact that the decision was made, does not provide a mechanism for contesting it, and does not transfer power from the institution that deployed the algorithm to the person affected by it. Explanation without recourse is confession without absolution - it makes the explainer feel better without making the explained-to any less harmed. And as Chapter 7 argued, the symbol grounding problem applies here: the explanation the algorithm gives is expressed in formal symbols that do not connect to the lived experience of the person being classified. "Your loan was denied because your debt-to-income ratio exceeded 0.43" is an explanation. It is not an answer to the question the denied applicant is actually asking, which is: "Why does a system designed by people unlike me, trained on data that underrepresents people like me, get to determine whether I can buy a home?"
Removing attributes doesn't remove proxies. This has been known since at least 2016. If you remove race from the dataset, the algorithm discovers zip code, which correlates with race. If you remove zip code, it discovers surname patterns. If you remove those, it discovers purchasing habits, social networks, language patterns, browser history - a thousand surrogate variables that collectively reconstruct the very categories you tried to erase, just without the transparency of having them named. The result is not a less biased algorithm. It is a more insidious one - one that discriminates on the basis of race while maintaining plausible deniability about doing so, because race is "not in the data." The algorithm does not need the attribute. It needs the pattern, and the pattern is embedded in the structure of social life in ways that no data scrubbing can erase.
More data is not better data. Adding more examples of underrepresented groups improves the model's accuracy on those groups. But accuracy is not fairness. A model that is equally accurate at predicting loan defaults for Black and white borrowers is not fair if the underlying economic system that produces the default data is itself discriminatory - if Black borrowers are defaulting because they were steered into subprime loans by the last generation of biased algorithms, or because they live in neighborhoods starved of investment by the last generation of discriminatory policy. Training a better model on a discriminatory world produces a better model of discrimination. The model becomes more accurate at reflecting the world as it is, not the world as it should be. And the accuracy - presented as objectivity, as data-driven decision-making - becomes a laundering mechanism: a way of taking a morally contested outcome and presenting it as the neutral verdict of mathematics.
II. The Real Problem
The real problem with algorithmic fairness is not that our algorithms are biased. It is that our world is biased, and algorithms are exceptionally good at learning the world's biases and reproducing them at scale, with the additional feature of appearing objective while doing so.
This is the laundering problem, and it is the one that the standard fairness approaches cannot address because they presuppose what they should be questioning: the legitimacy of the underlying decision-making process that the algorithm is automating.
Consider: before algorithms, loan decisions were made by human loan officers, who were themselves biased - consciously and unconsciously - in ways that produced discriminatory outcomes. The algorithms did not introduce discrimination into lending. They formalized the discrimination that already existed, making it more consistent, more scalable, and - crucially - more legible as objective. The algorithm's decision can be presented as the output of a mathematical process, free from the prejudices of individual decision-makers, even though the mathematical process was trained on data generated by a system shaped by those very prejudices.
This is the core mechanism: algorithmic objectivity conceals social bias by translating it from the register of human judgment (where it is visible and contestable) into the register of mathematical optimization (where it is invisible and appears inevitable). The loan officer who denies a Black applicant because of personal prejudice can be confronted, sued, shamed, retrained. The algorithm that denies the same applicant because of a debt-to-income ratio that was itself produced by a history of discriminatory lending cannot be confronted - it has no face, no conscience, no capacity for shame. It can only be audited, and the audit will reveal that the algorithm is doing exactly what it was designed to do: minimize default risk based on the data it was given.
The problem is not the algorithm. The problem is the decision to algorithmatize - the choice to take a decision that was previously made by humans accountable to other humans and transfer it to a system that is accountable only to the metrics it was optimized for. This choice is always presented as efficiency, consistency, scale. It is never presented as what it actually is: a transfer of power from the people affected by the decision to the people who control the algorithm.
III. What Would Actually Help
If the standard approaches fail, what might work? Not a solution - the book has argued consistently that the problems of alignment and its kin do not have solutions in the engineering sense. But practices, structures, and orientations that might make the situation less bad. Less bad is the honest goal. Perfect fairness is not available, and the pursuit of it produces its own pathologies - most notably, the endless multiplication of fairness criteria, each one correct in isolation and incompatible with the others, generating a literature that is mathematically sophisticated and politically paralysed.
What follows are not solutions. They are interventions - ways of altering the relationship between algorithmic power and the people subject to it that do not depend on the impossible dream of a perfectly fair algorithm.
1. Algorithmic decisions should be contestable in practice, not just in theory.
The right to contest an algorithmic decision is meaningless without the means to contest it. Currently, when an algorithm denies you a loan, a job, or parole, you receive a form letter - maybe - that tells you the outcome and, if you are lucky, a vague reference to the factor that produced it. You do not receive the data the algorithm used, the model that processed it, the training set that shaped the model, or the decision boundary that classified you. You cannot test the model on hypothetical versions of yourself to see what would need to change for the decision to flip. You cannot compare your treatment to the treatment of similarly situated individuals. You cannot argue with the algorithm - because there is no one to argue with, and the system is designed to make argument impossible.
Contestability requires: access to the input data and the model's output for your specific case. The right to submit corrections and have the decision re-evaluated. A human decision-maker, not employed by the institution that deployed the algorithm, with the authority to override the algorithmic decision. A time limit on algorithmic decisions - they should expire and require re-evaluation, preventing the permanent entrenchment of a single classification.
This is not explainability. Explainability tells you why the algorithm decided what it decided. Contestability gives you the power to challenge the decision and have it changed. The difference is the difference between a diagnosis and a cure.
2. Affected communities should have structural power over algorithmic deployment, not just advisory input.
The current model of algorithmic governance is: a company builds a model, deploys it, and then - if forced by regulation or public pressure - convenes an "ethics board" or publishes a "model card" or conducts an "audit" that is performed by people chosen and paid by the company, using methodologies approved by the company, with results that the company can interpret favorably. This is not accountability. This is the performance of accountability - a ritual that produces the appearance of oversight without the substance of constraint.
What would actual accountability look like? Affected communities - racial minorities subject to predictive policing, women excluded by resume screeners, poor people denied credit - should have veto power over the deployment of algorithms that affect them. Not advisory power. Not the power to be consulted and then ignored. Veto power: the ability to say no and have it stick.
This sounds radical. It is. But it is no more radical than the current arrangement, in which communities have no power at all over the algorithms that shape their lives, and the companies that deploy those algorithms have effectively unchecked power to classify, rank, and exclude. The current arrangement is also radical; it is just radical in a direction that concentrates power rather than dispersing it.
The mechanism could take many forms: elected algorithmic oversight boards with the power to suspend deployment. Community-controlled data trusts that require informed consent before data can be used for algorithmic training. Mandatory impact assessments conducted by independent bodies chosen by the affected community, not by the deploying institution. The specifics matter less than the principle: the people who bear the costs of algorithmic decision-making should have structural power over whether and how it is deployed.
3. Competing algorithms, not singular ones.
Monopoly is the enemy of fairness - not because monopolists are evil but because monopoly eliminates the possibility of exit. When there is one credit-scoring algorithm, and every lender uses it, the people it misclassifies have nowhere to go. When there are many algorithms, with different architectures, different training data, different fairness tradeoffs, and different blind spots, the people misclassified by one have the option of being classified by another.
This is not a naive market solution. Markets do not naturally produce algorithmic diversity - they produce algorithmic convergence, as firms copy each other's features and optimize for the same metrics on the same data. Diversity requires design: regulatory mandates for interoperability and portability (so that switching between algorithms is easy), prohibitions on exclusive contracts (so that institutions cannot lock in a single algorithm), and public investment in alternative algorithmic architectures (so that the field is not dominated by the approaches that happen to be most profitable).
The goal is not a "fair" algorithm. The goal is a plurality of algorithms, each imperfect, each making different mistakes, such that the aggregate system is less brittle and less totalizing than any single algorithm could be. This is the Madisonian principle applied to AI: ambition counteracting ambition, faction counteracting faction, imperfection counteracting imperfection. Not justice through a single wise decision-maker. Resilience through the controlled fragmentation of decision-making power.
4. Friction as fairness.
The fastest algorithm is the most dangerous one - not because speed is inherently harmful, but because speed eliminates the time required for reflection, contestation, and correction. Algorithmic decisions that are made and implemented in milliseconds - ad targeting, content moderation, dynamic pricing - leave no space for the affected party to notice, understand, or respond. The decision is made and executed before the person knows they have been decided about.
The response is not to slow down all algorithms. It is to mandate deliberation in high-stakes contexts: a waiting period between algorithmic decision and execution for decisions about credit, employment, parole, housing, and medical care. During the waiting period, the affected party is notified, given the information they need to understand the decision, and offered the opportunity to contest it before it takes effect.
This is friction, and it is the same friction the book has been defending since Chapter 20. Not the friction of inconvenience for its own sake, but the friction that creates time - time for the human being to exercise the agency that the algorithm would otherwise bypass. Time is the resource that algorithmic speed confiscates. Requiring time - building it into the process, mandating it as a feature rather than a bug - is the simplest and most direct way to restore the balance of power between the deciding system and the decided-upon person.
5. Stop laundering.
The most important intervention is also the most difficult: refusing to treat algorithmic outputs as objective when they are produced by systems trained on data that reflects historical injustice.
This is not a technical intervention. It is a normative one: the insistence that every algorithmic decision carries a footnote - a permanent, prominent, unremovable acknowledgment that the decision was produced by a system trained on data generated by a world that is not just, and that the decision therefore cannot claim the authority of objectivity.
The footnote does not fix the decision. It does not make the algorithm fair. What it does is prevent the algorithm from being used as a laundering device - a way of transforming "this is how the world is" into "this is how the world should be." The footnote insists on the gap between is and ought, and it refuses to let the algorithm close that gap by pretending that its outputs are anything other than a reflection of the data that produced them.
This is, in a sense, the negative solution: not making the algorithm better, but preventing it from being used as a justification for the status quo. The algorithm can still make decisions. It cannot claim that those decisions are just, or natural, or inevitable. It can only claim that they are consistent with the data. And the data, everyone now knows, is consistent with injustice.
IV. The Uncomfortable Truth
The uncomfortable truth about algorithmic fairness is that it is not primarily a technical problem. It is a political problem - specifically, the problem of who gets to decide how power is distributed, who benefits from the current distribution, and who bears the costs of changing it.
The technical approaches fail not because they are poorly designed but because they attempt to solve a political problem with technical means. Fairness metrics cannot tell you which dimension of fairness to prioritize, because that is a values question. Explainability cannot give you power over the algorithm, because explanation without recourse is just information. Removing attributes cannot eliminate the patterns that reproduce bias, because those patterns are woven into the structure of social life. And more data cannot produce justice from an unjust world, because the data is the world.
The interventions proposed here - contestability, structural community power, algorithmic pluralism, mandatory friction, anti-laundering norms - are not technical. They are institutional and political. They redistribute power from the deployers of algorithms to the people affected by them. They do not promise fairness. They promise less unfairness - a reduction in the total amount of unaccountable power exercised through algorithmic systems, and an increase in the capacity of affected communities to resist, contest, and refuse.
This is modest. It does not satisfy the engineering impulse, which wants a metric to optimize and a target to hit. But the engineering impulse is the problem, not the solution. The desire to reduce fairness to a measurable quantity, to find the algorithm that satisfies all the axioms simultaneously, to make justice a matter of computation - this desire is itself a form of the alignment dream: the faith that the right specification, correctly implemented, can make the world conform to our values.
It cannot. The world does not conform. Injustice does not compute its way out of existence. The best we can do - and it is not nothing - is to build structures that make injustice visible, contestable, and expensive. Structures that deny the algorithm the authority of objectivity. Structures that give the people classified by the machine the power to classify the machine in return.
Not fairness. Just less unfairness, achieved through politics, not mathematics.
That is the honest offer. It is the only one available.