NETFUTURE
Technology and Human Responsibility
--------------------------------------------------------------------------
Issue #35 Copyright 1996 O'Reilly & Associates December 5, 1996
--------------------------------------------------------------------------
Editor: Stephen L. Talbott (stevet@netfuture.org)
On the Web: http://netfuture.org
You may redistribute this newsletter for noncommercial purposes.
CONTENTS:
*** Quotes and Provocations
Beyond Surfing
Software Meltdown in the Year 2000?
Sex on the Internet
*** Imprisoning Criminals in Software (Stephen L. Talbott)
Where are the deepest risks of intelligent technology?
*** About this newsletter
Headlines move dynamically across the screen, the colors pop and all you have to do is keep your eyes open. Effortless. No surfing required.I guess they were right: computers will deliver us from the restrictive burdens of television. No more bothersome remote control. But do we have to keep our eyes open?
Gates says that sysadmin and programmer types of her acquaintance "are all adamant about the inevitability of this issue." One of them claims it is now "mathematically impossible" to rewrite or convert the old COBOL software that drives the banking system -- not, at least, in time for the bewitching hour at midnight, December 31, 1999, when your nest egg will turn into a pumpkin.
If I have responded to this story over the past few years with more of a curiosity than an engaged keyboard, it is because (1) I've never looked into the problem; (2) I tend to have boundless faith in the technical ingenuity of programmers; (3) whatever the potential for a two-digit disaster in the year 2000 (and I don't dismiss it at all), I haven't been able to connect it in any noteworthy way to the fundamental, underlying challenges of machine intelligence that I believe will prove decisive for our future. The main article in this issue of NETFUTURE, dealing with a piece of software used for law enforcement, may suggest what I mean by "fundamental."
Maybe, on the other hand, I'm just being obtuse, and someone will wake me up to the deeper issues here. In any case, Gates supplies a url for reference: www.year2000.com. She also mentions a CNN story dated October 13, 1996, "Experts Bemoan Denial of '2000 Bug.'"
I have, however, now seen one of those infamous graphic images that college students supposedly spend their time trying to download behind the backs of system administrators. It was in the Sunday Boston Globe (Dec. 1), with a strategically placed blackout stripe. The accompanying story by the Globe's Anthony Flint was a front-page feature about the burgeoning sex industry in this country. It's gist:
the transformed sex trade of today is rendering places like [Boston's] Combat Zone or New York's Times Square obsolete. Consumers can view material without ever leaving their homes, much less sneaking into a seedy bookstore....Internet advertising, beepers, call-forwarding, cellular phones...bring prostitution into the private realm. The customer need not risk arrest on the streets; the technology has removed the world's oldest profession from sight, and from easy detection.Flint also points to the widespread effort "to bring sex into the mainstream: to get beyond the blushing...and operate more like any other business." There are stock offerings, trade associations, more health benefits, journals, even congressional lobbyists. On the Net, he might have added, the selling of sex already is pretty much like any other business: largely depersonalized and anonymous. Here one runs into the sort of question that surfaces everywhere in a world being re-made by high tech: has the sex industry been finding its way toward the mainstream, or has the mainstream been finding its way toward the sex industry?
Flint, as near as I can tell, offers a sober assessment of the scale of the electronically-based portion of the industry. His interviews with some of the players in the business yield comments like this one from Sue Freeman, a spokesman for Virtual Dreams, an interactive service based in San Diego: "We honestly can't keep up with the volume. We just hired 60 more models, and we have 12 studios going all the time." The service offers models who, with the aid of video conferencing technology, disrobe according to the instructions of paying customers.
Apparently the Internet sex business is growing explosively. Flint cites Internet analysts who peg the current value of the business (excluding under-the-table transactions) at $100 million -- already making it "the third-largest sector of sales in cyberspace, behind computers and travel." In addition, CD-ROM sales for the sex industry are estimated at $300 million. By comparison, sales of adult videos are said to be $3.1 billion, comprising 30 percent of the video market. This business, too, could change dramatically as the need to walk into a video store is removed.
Finally according to Flint, "analysts say [the Internet] is the engine driving the business to greater accessibility." Peep shows aside, the Net is a decisive marketing tool. "Adult entertainment web sites have become clearinghouses, offering quick links to all the services that make up the modern industry -- whether in cyberspace or in the physical realm."
It is slightly ironic to speak of "greater accessibility" for a business now flourishing on the basis of increased anonymity and lack of genuine contact. But, of course, the two things do go together.
To the extent that the sex industry is one of the Net's major driving forces, those who believe the Net will reinvigorate social relationships and community have a lot of explaining to do. After all, the sexual dimension of a human relationship carries us to the core of the relationship; everything else gets tangled up in it. If the Net is lending itself so effectively to the continued wholesale depersonalization of the most intimate basis of human exchange, where do we see equally powerful indications that the Net is lending itself to a strengthening and deepening of human exchange?
Actually, I suspect there is a positive story to tell. But, all the glib talk about "virtual community" notwithstanding, we haven't even figured out where to begin looking for it, let alone how to put the kind of energy into it that is driving up the stock price of CyberLust, Inc.
Postscript: among the people Flint quotes in his article, the record for logic gone south gets awarded to Lisa Stetor, an exotic teleconference stripper and also editor of an industry journal. "I'm told I am selling my body," she laments. "In what other industry are skills characterized in that way? Do you tell a typist she's selling her hands?"
Ah, Lisa, how very nice to meet you. May I shake your hand? But wait. Since it's all the same to you, maybe I'll just jiggle your.... No, excuse me. I didn't say that. News stories like this always do put me a little off-balance. See, that's why I'd best stay away from "Sex on the Internet."
SLT
From Stephen L. Talbott
The Los Angeles County Sheriff's Department has begun using new software designed to "assess how dangerous a domestic abuser is to his household and help authorities determine what action should be taken." The program, called Mosaic-20 and developed by Gavin de Becker, Inc., profiles known batterers and then compares these profiles to a database of more than 4000 abusers whose behavior has led to homicide.
The Christian Science Monitor, which reported this story in late October (unfortunately, I have misplaced the exact date), goes on to note that Mosaic-20 will "aid prosecutors and judges in determining court strategies and sentencing." According to Deputy District Attorney Scott Gordon, the software is "a triage tool helping courts and cops figure out which cases are worth working on."
Oddly, this story about assessing the risks posed by abusers is completely silent about the risks of the software. It is true that the latter are subtle and difficult to lay hold of. But they are nonetheless critically important.
Suppose that I related to my wife by receiving her words and gestures as representative of this or that profile drawn statistically from collections of other women. She would rightly complain that I was ignoring her -- and, in fact, my method would be the perfect means to avoid meeting the person standing in front of me. She is none of those other women, nor is she the abstraction that we encounter in a statistical profile.
I think it is fair to say that almost everything, not only in a marriage but in all human relationships, comes down to our ability to make contact with the unique person who is expressing himself through -- but is not reducible to -- all those particular, statistically classifiable actions. If anything, this is more rather than less true in dealings with social misfits. Redemption from our failings very often comes in the form of another human being who notices us. In attending to me rather than merely to my symptoms, the Samaritan administers healing and helps me overcome my symptoms.
Mosaic-20 will not force law officers and case workers to deal inhumanely with their clients. But the distinctive contribution of the software in identifying the likely "tendencies" of particular batterers and in flagging cases for official attention, is based upon analysis that is hopelessly incapable of reaching through to the unique individual. So far as policies and actions begin to be organized around the software, case workers will be powerfully encouraged to view their clients through the programmatic lens. It is worth considering what sort of lens this is.
The profile as a summation of statistical relations is a perfect tool for treating people as collections of symptoms. This means, first, that a manipulative stance is encouraged. Our symptoms, after all, arise from precisely those places within us where we are not yet free. So in every human transaction we must choose between the manipulation of unfree tendencies -- of symptoms -- and the ennobling appeal to nascent freedom. Huge social resources -- for example, the hundreds of billions of dollars deployed by Madison Avenue -- are already committed to the lower route.
Advertisers profile individual consumers and market groups, subjecting the acquired data to sophisticated statistical analysis. The results of advertising campaigns based on such analysis can sometimes be fairly accurately predicted. But behind every successful prediction there is an inescapable fact: the advertising achieves its results through coercion -- by working upon symptoms. The entire aim is to outflank the consumer's conscious assessment of what he needs, what is healthy, what is good for society, and so on. Concern for the individual's welfare disappears amid enthusiasm for statistically predictable results.
But a manipulative stance is only one consequence of reducing others to collections of symptoms. A second consequence is that we cut people off from their future potential. What we can profile and analyze statistically are things already done. But the essence of human life is to be continually leaving ourselves behind, becoming what we are not yet. When we sum people up in terms of their past actions instead of reaching through and making contact with the self who is passing through those experiences, we freeze the future.
This is illustrated by the remark a hotshot software engineer once made to me. "When I take a job with a new company, I work day and night for the first few months, impressing everyone with what I can do. I've learned that once your initial reputation is established, it doesn't matter how little work you do afterward. People still think of you as the overachieving hero."
He was right. And, if anything, a negative reputation sticks even more stubbornly than a positive one. It is every manager's job -- and every probation officer's -- to help those in his charge escape the curse of their reputations. This means recognizing not only the past that has already been made, but also the future now in the making. This recognition is itself a creative power that helps an unexpected future to be realized.
There is a third consequence of the sort of profiling we've been discussing, but it is really just the deeper meaning of the other two: we lose sight of the individual human being. To take a manipulative stance toward someone and to ignore the potentials of his future is, at the same time, to ignore the person. This "loss of the other" is implicated in virtually every social problem we face. Can we afford to accentuate the loss by adding software filters to the barriers already separating the criminal from the rest of humanity?
All this exemplifies the sort of tortured choice we face everywhere in society today. On the one hand, the proponents of Mosaic-20 can claim that the software will save lives. And, given the terms of the claim, they may be fully as justified in their confidence as the advertising executive launching a new, sure-fire ad campaign.
But, on the other hand, the software is a nearly irresistible invitation toward the kind of manipulation and depersonalization that helps to produce criminal behavior in the first place. This, however, is not so easy to demonstrate. The saved lives will be measured -- in good scientific fashion -- by comparing situations that are as identical as possible except for the presence or absence of the software. But the real question is how, over the long term, the software helps to redefine that underlying situation.
In assessing the risks of software, it is at least as important for us to look at the slowly evolving, qualitative background of its use as to tabulate the immediately calculable, foreground results. I may, on some occasions, gain a tactical advantage over my wife by reading her statistical profile in detail, predicting her next move successfully, and countering it in advance. If I do it well, I may even help to smooth over what would otherwise have been a rough time. But this pattern of behavior on my part can only corrode the foundations of the relationship over the long term. I am, for my own advantage, manipulating her in terms of the past rather than discovering what she (and we) might become in the new situation.
I am not arguing flatly that Mosaic-20 should never be used. What needs saying, I think, runs exactly parallel to what I said about the various technologies of anonymity in my earlier series on privacy. The depersonalized conditions of modern society may require our use of tools like Mosaic-20. But for every step we take in that direction we must take three steps in the pursuit of the unique human individual. Otherwise, we will only contribute further to the depersonalization whose consequences drove us to the software in the first place.
Copyright 1996 by The Nature Institute. You may redistribute this newsletter for noncommercial purposes. You may also redistribute individual articles in their entirety, provided the NetFuture url and this paragraph are attached.
NetFuture is supported by freely given reader contributions, and could not survive without them. For details and special offers, see http://netfuture.org/support.html .
Current and past issues of NetFuture are available on the Web:
http://netfuture.org/
To subscribe or unsubscribe to NetFuture:
http://netfuture.org/subscribe.html.Steve Talbott :: NetFuture #35 :: December 5, 1996