Notes

Strategies:

• Participation by stakeholders
• Transparency measures
• Forecasting of system impacts
• Auditing of what actually happens once the system is turned on

4 lenses through which to view algorithm creation and management processes

Technical experts can’t take over moral choices. e.g. a physician can usefully provide advice for treatment, esp if they’ve witnessed many past patients go through pain through the treatment, but their moral opinion shouldn’t be the only

Bureaucracy itself can serve as a way of converting hard moral problems into boring technical ones, a process that long predates computers. But software-based systems can accelerate and amplify this trend. Quantification can be a moral anesthetic, and computers make that anesthetic easier than ever to administer. In the book’s final chapter, I’ll briefly consider the claim that such anesthesia might sometimes be for the best, even though it is seldom chosen openly. But for now I’ll say that the pattern of abdicating important moral choices to technical experts often seems both accidental and unwise.

Making software = making laws

In a certain light, the question of how best to make software code that will govern people is just a special case of how best to make laws (pg 30)

Participation

Putative (reptuable) consultation isn’t always great: might just be a fig leaf for deferring to experts without asking for people who are actually affected

Robinson offers general argument for participatory deliberative practices because they allow people to unite and empathize with each other, but also they tend toward finding the best solution

• Democratic instrumentalism is the idea that democracy is only useful insofar as it finds the best solution — and that a best solution exists outside of process?
• Meh. The reality is that process is conducive to product: sometimes the best solution emerges from people compromising but being more willing to do so because they were involved firsthand in confronting the hard choices

Representational democracy is common: they don’t always have to be resource-intensive; the art of finding and empowering the right representatives is crucial

• The art could manifest as randomness (sortition): randomly getting people who will help forge the decision. Ex) Iceland constitution was made this way
  • Resolves bias of passion in self-selected representation if the product is also affecting people who aren’t necessarily obvious or passionate stakeholders
• Participatory models are the trend, but there is also some skepticism: the problem is not software’s technical decisions, but injustice at large. Himmelreich: we don’t need people to get involved in software making, but overarching decision-making processes for high-stakes policy situations, i.e. court and social services office and not just the software they use
• Different methods to participation: notice-and-comment approach, in-person forums, stakeholder meetings, etc