• Goto Meditations main page
  • Sleepwalking with Our Computers

    The Machine's Threat Lies in Our Absence

    Stephen L. Talbott (stevet@netfuture.org)

    The old science fiction chestnut has failed as a prognostication. Computers, we now recognize, will never "rebel against their masters."

    The danger of the computer lies rather in its ability to mimic human intelligence -- but only those aspects of our intelligence that can run automatically and "unthinkingly." However elaborate and lifelike the computerized telephone answering system, no conscious presence expresses itself through the stitched-together voices. No one is really there.

    Similarly, every computer program is a mass of congealed logic calcu- lated in advance to fill the human being's absence. The computer, you might say, thrives upon the suppression of our highest functioning, supplanting it with automatisms. We have not yet reckoned with the peril in this.

    The mechanical devices of the Industrial Age also "ran on automatic." What the computer adds is a dimmed-down, mechanical -- yet technically effective -- shadow of our mental life. That's what software is.

    And so the power of our computer-based institutions to sustain them- selves in a semisomnambulistic manner, free of any present human con- trol, is increasing to a degree we have scarcely begun to fathom. Everything runs with a certain internal, logical coherence, but it is the maddening, stifling coherence of "The System." We are rapidly fashioning a diffuse, planetary mechanism that far exceeds the tradi- tional devices of power in its ability to incarnate an autonomous and anti-human spirit.

    This is not a rebellion of machines. Rather, it is our own willing- ness to erect the machinery of society upon those processes in our- selves that are automatic, deterministic, semiconscious. The widespread debate about whether computers foster or discourage cen- tralized, despotic authority misses the crucial point. The danger today is the danger of things that run by themselves. The dangerous face of oppression today is the faceless one, the nameless one, the one for which none of us seems to be responsible. So we pay it no heed -- which is exactly what lets it go its own way.

    Of course, we gain something by empowering the machines in our midst: we avoid having to deal with each other. Suppose I run into the sort of trouble that, in an earlier age, would have forced me to my neigh- bors for help, thereby throwing me into the most vexing -- but also the highest -- dimensions of human relationship. Today, I instead seek a bank loan where not even a personal interview is asked for. It is a mere "transaction," captured by transaction processing software and based solely upon standard, online data.

    So everything that once followed from the qualities of a present meet- ing between two people -- everything, for example, that could make for an exceptional case -- has now disappeared from the picture. The loan applicant is wholly sketched when the data of his past have been sub- jected to automatic logic. Any hopeful glimmer, filtering toward the sympathetic eye of a supportive fellow human from a personal destiny only now struggling toward birth, is lost in the darkness between bits of data. The software says "yes" or "no"