Computer Error: What do people blame for problems with technological systems?

Computer Error:
What do people blame for problems with technological systems?

Software	None
Covers	Computer ethics, psychology and sociology of computer use
Skills Required	Interest in computer ethics, interest in the social impact of technology
Challenge	Conceptual ???? Technical ?? Programming

Brief Description

"To err is human, to really foul up requires a computer."

"At the source of every error which is blamed on the computer you will find at least two human errors, including the error of blaming it on the computer."

As technology becomes more complex it becomes more opaque and often less predictable. The designers of complex systems can have difficulty predicting behaviour - sometimes systems can malfunction when all the subsystems perform to specification but they interact in unforeseen ways. Designing to predict and avoid error behaviour in software gets to be a complicated business, where there is no direct relationship between the physics of how machines work and what they will do (to learn about it, see the works of Nancy Leveson of MIT).

Normal human assumptions about how things work and where to attribute blame for failures may simply be too naive to cope with the complexities of the computerized real world. Moreover people differ in how far they believe that someone must be at fault for any problem, or that sometimes shit happens without any individual having done anything particularly culpable. Nevertheless, we still need ways to decide on degrees of blame for the malfunctioning of complex systems, for example when judging legal cases or considering how far to trust computer systems for, say, medical diagnosis or driving driverless vehicles, or finding social and legal procedures for safeguarding sensitive information and preventing computer crime.

The challenge of this project is to look at how people think about the attribution of responsibility for computer systems' malfunctions, errors, security breaches, or inappropriate behaviour, and consider how the ways people think about this depends on degree of technical knowledge or on ethical beliefs, and whether the way people think fits the social and technological realities of the situation.

Variants

These questions might best be considered in the context of a particular class of complex technological system, such as expert systems for medical diagnosis, or autonomous fighting vehicles, or data storage for banks or other possessors of sensitive information.

This project could focus on a systematic treatment of the ethical considerations, and whether there is a feasible way to decide blame and responsibility that doesn't place too harsh a judgement on particular human beings who are at the point of failure but not necessarily at the root of the problem. Alternatively it could focus on the psychology of attributions of blame and responsibility, looking at the influence of people's beliefs about the nature of computers and software and complex technological systems, as well as about engineering companies and other complex social systems, on the nature of their attributions of responsibility.

Back to