NETFUTURE
Technology and Human Responsibility
--------------------------------------------------------------------------
Issue #55 Copyright 1997 Bridge Communications September 9, 1997
--------------------------------------------------------------------------
Editor: Stephen L. Talbott (stevet@netfuture.org)
On the Web: http://netfuture.org
You may redistribute this newsletter for noncommercial purposes.
CONTENTS:
*** Editor's Note
*** Quotes and Provocations
How To Kill and Have Fun
Big Brother in the Bathroom?
Is the Productive Computer Still AWOL?
Sick Campus Networks
*** The Executioner's Motto (Stephen L. Talbott)
The empty destination of a computational society
Departments
*** Announcements and Resources
Education Technology: Asking the Right Question
Erratum
*** About this newsletter
"The Executioner's Motto" in this issue represents the best summary statement I have been able to come up with about the concerns underlying NETFUTURE. I hope you will read it.
SLT
Now, however -- well ... let me just get it out of my system: What is this world coming to? An English correspondent forwarded a news clipping (newspaper not identified) describing a new computer game in which players gain points by mowing down as many pedestrians as possible while driving their cars at high speed. For a presumably more splattery diversion, one variation of the game grants extra points for swerving off the road and killing cows.
The game is called "Carmageddon," and its collisions involve "grotesquely realistic graphics." A spokesman for the manufacturer (London-based Sales Curve Interactive) claimed the company had made "slight cosmetic changes" to the color palette for palatability's sake, giving the characters zombie white faces and red eyes, and causing them to emit "green gunk" instead of blood. But the spokesman, protective of the company's honor, did want us to know that "we don't have any regrets with the original version."
Despite expressions of concern from some quarters, the game appears to be doing well for Sales Curve.
A demonstration of Carmageddon on the Internet has attracted 500,000 users to the site and Jon Evans, editor of PC Power magazine, said that the game had enjoyed good reviews from computer game magazines. "We liked it. Mowing down pedestrians clearly isn't something we would like to encourage, but it is so obviously over the top as to be pure fantasy. It may not be morally correct, but it is fun. There is no question that it will sell well.
An electronic system called Hygiene Guard (produced by Net/Tech International in New Jersey) is now being installed in restaurants, hospitals, and other facilities where personal hygiene is a matter of public concern. A "smart badge" worn by untrustworthy employees
communicates with sensors in the bathroom that are connected to a computer in a manager's office .... Unless an employee uses the soap dispenser and stands for a required amount of time in front of a sink with running water, an infraction will be recorded on the computer. In some instance the employee's badge will flash. (Albany Times Union, August 31)Civil libertarians, worried about increasing electronic intrusion in the workplace, cite Big Brother. I won't quarrel with them -- nor will I quarrel with those who point out that personal hygiene in these situations is a matter that transcends the isolated individual and his precious rights, with grave implications for other people. No, what impresses me -- apart from the clownishness of the situation -- is the impossibility of solution along primarily technical lines.
According to the Times Union report, a recent study shows that "only two-thirds of all people wash their hands after using the bathroom, even though more than nine of the ten report in interviews that they do so." Clearly, then, the problem includes issues of honesty and social responsibility.
Does anyone really believe that coercive measures will meaningfully improve the situation? Will the reduced risk from newly obedient employees count more in the final equation than the radically increased risk from the inevitable few who respond to the system in a spirit of rebellion? Personally, for my own safety, I'll make it a point never to dine in a facility where I know Hygiene Guard is installed.
You will recognize in this an echo of my earlier remarks about technologically protected privacy (NF #28): The increasing technical mediation of our social checks and balances weakens the social bonds that are the only real guarantee of the values we are trying to protect. To the degree we employ technology, we must redouble our commitment to the strength of the underlying social matrix.
I don't understand how it could possibly occur to a manager that a several-thousand-dollar piece of equipment (and don't forget the maintenance, the support, the training ... ) could prove more effective than actually confronting the problem in a straightforward, mutually empowering way within his group. The choice of a high-tech solution doesn't say much for how the human side of these organizations is being managed.
"Contrary to recent expectation," writes Business Week (August 25), "the Commerce Department's revisions to gross domestic product back to 1993 gave no support to the argument that productivity growth in the 1990s is accelerating." Rising corporate profits appear to be a gift bestowed, not by higher productivity, but rather by employees, whose share of corporate income continues a long decline.
The same issue of Business Week carried commentary by Mike McNamee, who remarked:
The biggest puzzle -- the productivity paradox -- remains. Corporations have poured billions into technology, with no sign of a return on that investment in the productivity stats. It's still likely their bet will pay off someday.Personally, I don't have the foggiest idea about what will eventually show up in the productivity stats. But I do find interesting the extraordinarily powerful conviction that productivity gains somehow must come. Computerized technology alters, obviously or otherwise, just about everything in a business. So far as I know, no one has a handle on all these issues, ranging from support and maintenance, employee training, online distractions, and accelerating rates of obsolescence to the less remarked but potentially more important ways in which computers subtly affect our habits of thought and responsibility.
Given the scale of the unknowns, the oddly persistent optimism after years of refractory data seems all too much like the blind confidence of those who are convinced that, whatever information their terminals display, it just must be right because "it's in the computer."
Productivity gains must come, because computers are such efficient things. Right?
And the Chronicle of Higher Education speaks of a "user-support crisis" in campus computing. As the director of academic computing at Seton Hall University puts it: "People are pushed to use the technology, but most are not given the help to do so." There's worry that today's huge investments in computers will be wasted. Of course, the cost of adequate support can only come from other educational departments.
As I've been saying for awhile, the backlash against mindless investment in educational technology is coming this year, and will surely gather momentum during the autumn. Who knows? -- it may even progress from pocketbook worries about technical support to an actual consideration of the relation between computers and the ends of education.
SLT
From Stephen L. Talbott (stevet@netfuture.org)
There's a slogan among artificial intelligence (AI) researchers that runs this way:
If you take care of the syntax, the meaning will take care of itself.Dubbed the Formalist's Motto by philosopher John Haugeland, this turns out to be a formula for erasing the human being.
Stated simply, the idea runs something like this: if you put the computer through the motions of human behavior, it will in fact mean and intend what we would mean and intend by such behavior. So the AI programmer should concentrate on abstracting the formal structure of our tasks in the world without worrying about the inner qualities of consciousness, feeling, and will with which we invest those tasks. After all, our subjective illusions notwithstanding, nothing is really "there" in either man or machine beside formal structure, or syntax. The meaningful, inner content of our lives is a kind of syntactic epiphenomenon, the mystery of which need not concern us.
On this premise the hope for true, human-like artificial intelligence now rests.
You may never have heard of the Formalist's Motto, but I venture to predict that it accurately circumscribes a substantial part of your thought world, as it does the thought world of nearly everyone in our culture. For the motto does not apply only to AI. Here, for example, is what you might call the Physicist's Motto:
If you take care of the equations, their meaningful relation to the world will take care of itself.One might wonder about the truth of this at a time when the equations have become almost mystically esoteric and remote from the world of our experience. The wondering is justified, but we also need to realize that the equations succeed remarkably well as shorthand prescriptions for the effective manipulation of the world (and especially of experimental apparatus). The problem lies in how easily and dangerously we forget that manipulating things is not the same as understanding them.
Then there is the Economist's Motto, blossoming from an unshakable faith in the power of the Invisible Hand to smooth over our own neglect of what really matters:
If you take care of the economic numbers, the value for society will take care of itself.Or, as Adam Smith originally put it in his Wealth of Nations (1776), "By pursuing his own interest [the individual] frequently promotes that of the society more effectually than when he really intends to promote it." And "It is not from the benevolence of the butcher, the brewer, or the baker, that we expect our dinner, but from their regard to their own interest. We address ourselves, not to their humanity, but to their self-love." So a quantitative concern for the bottom line results automatically in a wider social good, regardless of one's base intentions.
In this case, not only does the syntax of the formal (market) mechanism take care of the meaning, it skillfully negates any unsavory meanings that mere humans try to inject!
One could go on. Probably the most fundamental version of the motto is that of communications theory, as it has seeped into the popular consciousness:
If you take care of the transmission of bits, the meaning of the text will take care of itself.Nearly all misconceptions about the Information Age trace back to this formula, including what we might call the Educationist's Motto:
If you take care of the flow of information, the education will take care of itself.What's going on here? Clearly we're not just talking about computers or education or information or business. We're talking about us. What is at issue is the common style of thinking we bring to these various areas. The most decisive fact about the age of the computer is a fact about our own minds: we are, without being fully aware of it, leaking meaning and content at an alarming rate. And what is replacing them? Empty, computationally manipulable abstractions.
Each of the mottos I have cited directs us toward a mathematical or logical calculus that can easily be read from, or impressed upon, a mechanism. We may have begun with meaning -- the meaning of a proposition, the meaning of a business activity, the meaning of an animal's behavior -- but we are driven by our predilections toward empty form without content -- the p's and q's of the logician, the cost analyses of the financial officer, the DNA structure of the geneticist. For these can be arranged in a sequence whose logic can drive an automaton.
Our exquisite ability to reduce content to usable abstraction is one of our rightly prized achievements. But we cannot abstract from the content of a thing unless we are given the thing in the first place -- given it, that is, in all its qualitative and meaningful presence. Otherwise there is simply nothing there. You cannot arrive at the concrete object from its dimensions alone, you cannot arrive at a product from a set of cost specifications alone, and you cannot arrive at the organism from its DNA alone.
We are powerfully one-directional in our intentions. We want to abstract the mathematical law of things, but we do not know how to get the things back once we have found ourselves holding nothing but a set of pure abstractions. Once a business becomes a smoothly humming calculator of the bottom line, its resistance as an complex, integrated, and programmed mechanism to intrusive questions like "What is the good of this product?" becomes almost impossible to overcome.
The difference between the two directions of movement -- toward abstraction and toward meaning -- can be painfully hard to grasp amid the actual affairs of life. It is the difference between a business that uses economic controls to discipline its pursuit of ends independently judged to be worthy -- and a business that pursues profit for its own sake, without regard for the human worth of its products.
It is the difference between a science that began as a passionate insistence upon observing the actual world instead of relying upon the subtle cerebrations of the medieval schoolmen -- and a science whose developing abstractions have encouraged it first to ignore and then (as an inevitable consequence of the ignoring) to ride roughshod over the natural environment.
It is the difference between an education that enables students to inquire, "What does this mean?" -- and an education bent upon shoveling inert facts into cranial "databases."
It's no use talking about the risks of technology without also talking about our styles of thinking. If computerized technology is pivotal for the modern era, it's not because of some wholly inherent capacity, but rather because we have fashioned in the computer a perfectly adapted tool for the expression of our preferred modes of thought. Toss the machine without altering the thought, and not much will change. Transform the thought, on the other hand, and we just might be able to wrestle the machine toward profoundly humane ends.
Unfortunately, there's not much in all this talk about "modes of thought" that wired folks, including many social activists, care to bother about. We all too instinctively want a program first. Perhaps I do not stretch the matter too far when I offer the Involved Citizen's Motto:
If you take care of the program of action, its meaning will take care of itself.But it's not true. Actions considered apart from their inner, expressive gesture degenerate into empty formalisms (like computer-orchestrated "grassroots" campaigns). Or else they carry meanings we are simply unaware of.
We have no constructive choice except to consider what we ourselves will become -- which is another of saying: except to consider whether we will transcend our currently "executing" syntax in a way that formal mechanisms never can. The various mottos I have listed, after all, capture a historical movement of just the past few hundred years. In becoming aware of that movement, will we disown responsibility for it as if it were an unalterable given, while at the same time embracing with exhilarated anticipation the wondrous changes our machines are bringing about?
In this way we would forget ourselves precisely at the moment when the "spirit" of technology is making a nearly irresistible offer: "You can drop out of the picture and I'll keep all the formal mechanisms humming along just fine. Don't worry; everything else will take care of itself."
It's a genuine offer -- and one we look too much like accepting.
These observations will sound, audibly or inaudibly, as the leitmotiv in much of NETFUTURE's wide-ranging content over the coming year. All of the suggestions I have offered need filling out. For example, I will try to show how the hunger for formal, computer-like mechanisms leads both to the conservative's denial of limitations in laissez-faire capitalism and the liberal's excessive faith in governmental programs to counter those limitations. I will also make some suggestions about a third way that arises only from the free and responsible, socially embedded human being.
In addition, there will be some 75 papers read by other participants in the conference (of whom I am privileged to be one). For more information, contact Chris Dufour at 814-863-5110, or send email to ConferenceInfo1@cde.psu.edu. (Thats the number one, not the letter el, before the at-sign.)
At the Open Software Foundation, he managed software engineering groups and established liaisons between OSF and companies, consortia, and governments in the U.S., Europe, and the Far East. He spent his last year at OSF assisting the president and board of directors in managing a merger with a foreign consortium.
Copyright 1997 by The Nature Institute. You may redistribute this newsletter for noncommercial purposes. You may also redistribute individual articles in their entirety, provided the NetFuture url and this paragraph are attached.
NetFuture is supported by freely given reader contributions, and could not survive without them. For details and special offers, see http://netfuture.org/support.html .
Current and past issues of NetFuture are available on the Web:
http://netfuture.org/
To subscribe or unsubscribe to NetFuture:
http://netfuture.org/subscribe.html.Steve Talbott :: NetFuture #55 :: September 9, 1997