• Goto NetFuture main page
  •                                  NETFUTURE
                        Technology and Human Responsibility
    Issue #146                                                   June 24, 2003
                     A Publication of The Nature Institute
               Editor:  Stephen L. Talbott (stevet@netfuture.org)
                      On the Web: http://www.netfuture.org/
         You may redistribute this newsletter for noncommercial purposes.
    Can we take responsibility for technology, or must we sleepwalk
    in submission to its inevitabilities?  NetFuture is a voice for
    responsibility.  It depends on the generosity of those who support
    its goals.  To make a contribution, click here.
    Flesh and Machines: The Mere Assertions of Rodney Brooks (Stephen L. Talbott)
       Appeals to ignorance do not convince
    About this newsletter
                                Stephen L. Talbott
    In Flesh and Machines: How Robots Will Change Us (Random House, 2002),
    Rodney Brooks informs us that "we are nothing more than a highly ordered
    collection of biomolecules":
       Molecular biology has made fantastic strides over the last fifty years,
       and its goal is to explain all the peculiarities and details of life in
       terms of molecular interactions.  A central tenet of molecular biology
       is that that is all there is.
    Apparently fearing that we will be insufficiently gripped by his message,
    Brooks (who is director of the Artificial Intelligence Laboratory at MIT)
    goes on to say in the space of three pages:
       The body, this mass of biomolecules, is a machine that acts according
       to a set of specifiable rules....
       The body is a machine....
       We are machines, as are our spouses, our children, and our dogs....
       We are nothing more than the sort of machine we saw in chapter 3, where
       I set out simple sets of rules that can be combined to provide the
       complex behavior of a walking robot .... we are much like the robot
       Genghis, although somewhat more complex in quantity, but not in
       I believe myself and my children all to be mere machines....
       We, all of us, overanthropomorphize humans, who are after all mere
       machines (pp. 173-5).
    Noting that some people may bristle at the word "machine" (not to mention
    the tedious repetition), Brooks acknowledges that he uses the word "to
    perhaps brutalize the reader a little".  He feels the need to shake us
    free of any conviction that "we are special" -- meaning, in case you
    missed it, that we should accept our status as "mere machines".
    Searching for the Bottom
    As a reviewer facing an attempt at brutalization, perhaps I will be
    forgiven a touch of bluntness.  When, using the characteristic language of
    the insecure reductionist ("we are nothing more than"), Brooks refers to
    molecular interactions and says emphatically, "that is all there is", he
    is engaged in a startlingly transparent argument from ignorance.
    "That is all there is".  What is all there is?  Brooks makes the statement
    in order to intimidate, but is unwilling or unable to tell us anything at
    all about the brute, underlying, mechanistic reality he is pretending to
    beat us with.  If there is one thing no respectable physicist since the
    mid-twentieth century would claim to do, it is to describe some sort of
    ultimate physical machine, let alone a well-understood physical machine.
    The entire movement of physics has been away from concrete machine
    models -- and, indeed, away from models altogether.  Theories at the
    lowest level no longer even describe particular things or events that
    might be modeled.  So when, with an air of settling matters for good,
    Brooks says "that is all there is", what hard reality is he shoving in
    front of our faces?
    His general idea seems to be that there is some "mere" machine-stuff
    down at a lower level of explanation, and everything above it is
    "nothing more than" the compounded articulations of this mechanistic
    reality.  Explanation proceeds from below upward.  Our nature is fixed
    by the foundation on which it rests.  "We are machines".
    But if explanation proceeds from below, then you need to start with
    whatever is at the bottom.  Your primary explanatory apparatus, in other
    words, is quantum weirdness.  And the one place in science where you
    absolutely cannot find a machine, is amid the scarcely utterable
    perplexities of the quantum realm.  Nevertheless, there is where Brooks
    would apparently nail down our determining and limiting nature -- the
    reality we are "nothing more than".  Does he care to tell us a little
    about this reality?
    Certainly not in Flesh and Machines.  Moreover, even if he could extract
    something of our essential nature from the sphere of quantum effects, and
    even if he could demonstrate the machine-like character of this nature,
    there would remain the difficulty facing all bottom-up expositors of the
    human being:  where is the bottom, since not one iota of evidence within
    physics suggests we have reached it, and no one even seems to know what it
    might mean for there to be a bottom or what would be its distinguishing
    Physicist Steven Weinberg informs us that some of the (unknown) structures
    pointed to by currently accepted theory are smaller than the atomic
    nucleus by a factor of a million billion.  In other words, it is at least
    as far from the atomic nucleus to these unknown structures as it is from
    the scale of ordinary life to the atomic nucleus.  If the idea of
    explanatory levels makes any sense at all (it doesn't), and if our nature
    is determined by whatever constitutes the bottom, then so, too, the nature
    of atoms and molecules must be determined by the same inaccessible bottom.
    But still, in his happy ignorance about whatever is "down there", Brooks
    can find satisfaction in the effort to whip us into submission with a
    phantom:  "That is all there is".  One wonders what he pictures in his own
    mind as he says "that".  Here I have a nothing-more-than conjecture of my
    own:  his idealized bit of ultimate matter is nothing more than a
    projection of the machines he builds; his imagined molecule is a kind of
    minuscule robot-homunculus.
    Brooks might reply that, at some higher level of explanation, quantum
    weirdness (and the unknowns lying beneath it) manage to even themselves
    out, leaving us with the hope of identifying neat mechanisms at this
    higher level.  But if he is not appealing to determination from a
    well-understood bottom -- if he claims to discern our mereness at some
    higher level -- what saves his choice of levels from arbitrariness?  Why
    should we not, for example, take the living organism, or consciousness,
    as most fundamental and most revelatory for any explanation of the world?
    If we really want to understand the sustaining powers, the living
    energies, presented in the world, doesn't it make most sense to look
    for our understanding where those powers and energies are most fully
    developed and most explicitly manifest?  In any case, whatever the level
    Brooks chooses, he needs to demonstrate, rather than simply assert,
    its essential mechanistic nature.  He doesn't seem interested in the task.
    If he had been interested, he might have begun by listening to the
    physicists.  When Werner Heisenberg can say that atoms are not things,
    and when Steven Weinberg can say that "particles are just bundles of field
    energy", and when Richard Feynman confesses that "in physics today, we
    have no knowledge of what energy is" -- well, then, the natural question
    to put to Brooks is, Where's the beef?  Show us your grand, microcosmic
    machine!  And tell us what makes it merely a machine.
    Order and Form
    Brooks seems particularly inclined to find evidence of our machine nature
    at the level of biomolecules.  Since he never tells us why these
    molecules should be regarded as machinery, there isn't much to quarrel
    with.  But a couple of extremely brief comments may prove useful.
    Two of the most longstanding and fundamental questions about living things
    are, first, How does the organism sustain its order and resist decay?
    That is, how is matter "caught up" in the organism's living processes in a
    way that suddenly and dramatically ceases when the organism dies and the
    inevitable processes of putrefaction and decay take over?  Or again:  what
    is the difference between the state of the same physical body before and
    after death?  And second, How is the distinctive form of every organism
    brought into being and maintained?
    Both these questions have tended to fade from view as biology has
    transformed itself from a discipline about organisms to a discipline about
    molecules.  Researchers seem to believe (if they think about the matter at
    all) that if they can just get a handle on the molecular "machinery" of
    the cell and organism, the larger questions will somehow answer
    What has actually been happening provides an ironic counterpoint to this
    expectation.  It is less that the lower "mechanisms" are answering the
    larger questions than that the larger questions are simply reasserting
    themselves at the lower level.  In particular, the problem of order and
    resistance to decay stares microbiologists in the face as soon as they
    cease averting their eyes.  As Lenny Moss (who is both a cell biologist
    and philosopher) puts it, when we look at the molecular dynamics within
    the cytoplasm of the cell, what we see suggests that "biological
    resistance to thermodynamically driven entropic heat decay obtains all the
    way down to the most basic fabric of living matter" (What Genes Can't
    Do, MIT Press, 2003, p. 91).  Moss is one of many researchers looking
    at the complex chemical dynamics of the cell as a whole, and noting that
    there is no one-way chain of cause and effect determining the cell's
    order.  This order (which is passed from one generation to the next) is
    irreducibly manifested in the cell as a whole, with each part (including
    the DNA) being effect as well as cause.
    As to the other problem, that of organic form, one need only point to the
    decisive issue of protein folding.  This is opening up into a complex and
    massive question of form that mirrors, and is organically inseparable
    from, the question of overall form in the organism.  Harvard biologist
    Richard Lewontin, after citing some of the many factors affecting protein
    folding (and therefore protein functioning), continues:
       These understandings, however, have not penetrated into the main
       structure of biological explanation .... What is needed is to move the
       issue of structure from the peripheral realm of a few special cases to
       a central concern of investigation at the molecular level.  (The
       Triple Helix, Harvard University Press, 2000, pp. 115-17)
    Given that these two fundamental questions about life -- the question of
    order and the question of form -- simply reappear at the molecular level,
    what sense does it make to say, as Brooks does, that the living organism
    is now understood as merely a collection of biomolecules?  Certainly the
    new questions will, to the mechanistically inclined, appear to demand
    standard mechanistic solutions.  But by the same token, to the researcher
    who was willing to see the qualitative (and therefore non-mechanistic)
    unity of the organism as a whole, the new problems at the molecular level
    will appear as verifications of that original view rather than as
    refutations of it.
    Incidentally, I recently saw a report that physicists have found the
    proton to be continually shifting in form.  While it (whatever "it" is)
    can assume spherical form, apparently it also takes on other shapes,
    including that of a peanut and even a donut.  (Remember, we're talking not
    so much about a "thing" as a pattern of forces or energies.)  The world,
    from top to bottom, appears to consist of continually transforming form --
    at the molecular level as well as at every other level -- and it's not at
    all clear how to map Brooks' picture of tiny machine-parts onto this
    essentially fluid and qualitative reality.
    Losing Consciousness
    The "we are nothing but" claim takes countless forms and comes at us from
    many sides.  To cite just a few famous cases:
       "You," your joys and your sorrows, your memories and your ambitions,
       your sense of personal identity and free will, are in fact no more than
       the behavior of a vast assembly of nerve cells and their associated
       molecules.  As Lewis Carroll's Alice might have phrased it: "You're
       nothing but a pack of neurons."  (Francis Crick)
       There is no spirit-driven life force, no throbbing, heaving,
       pullulating, protoplasmic, mystic jelly. Life is just bytes and bytes
       and bytes or digital information.  (Richard Dawkins)
       Man has to understand that he is a mere accident.  (Jacques Monod)
       People, after all, are just extremely complicated machines.  (E. O.
    I have a working hypothesis.  Leaving Brooks (about whom I know almost
    nothing personal) aside and focusing instead on the wider rhetorical
    phenomenon:  why do we so often encounter today the assaultive
    reductionist assertions, "the human being is only...", we are nothing
    more than...", "we are merely..."?  The vacuity of the claims together
    with the characteristic aggressiveness of the authors certainly raises
    some interesting questions.
    We get a hint of what is going on by considering the negative and
    belittling cast of the assertions.  Clearly the authors, consciously or
    unconsciously, see themselves as bringing us down a peg or two.  They
    could, after all, have reveled in the glories and depths and wisdom of the
    natural world revealed by science.  But, no, they want to make sure we
    understand that we are only such-and-such.  Apparently their own
    pessimistic sense of the matter is that we are discovering the human
    reality to be an impoverished one -- a reduced one, in fact -- this
    despite the usual explicit statements about what a stunningly rich and
    profound world science presents us with.
    We may wonder, then, what is behind this sense of being reduced.  Whence
    arises this feeling of inferiority requiring the use of pejorative and
    reductive language?  My own suspicion is that one of the most fundamental
    insights of Carl Jung may throw light on the matter.  The Swiss
    psychiatrist pointed out that when integral contents of the personality
    are lost to consciousness, an inferiority results.  Moreover, the sense of
    inferiority gives rise to moral resentment and "always indicates that the
    missing element is something which, to judge by this feeling about it,
    really ought not to be missing, or which could be made conscious if only
    one took sufficient trouble" (Two Essays in Analytical Psychology,
    Princeton University, 1972, pp. 139ff.).
    I feel free to mention this only because the way a personal lack leads to
    a sense of inferiority and then to moral resentment (so that we are happy
    to bring others down a peg or two, to our own level) is such a universal
    fact of life that none of us -- if we are possessed of the slightest self-
    awareness -- can fail to recognize the process in our own lives.  So it is
    not a matter of pointing fingers at individuals or singling out any group
    as "morally challenged".  But where we can recognize a broad social and
    intellectual pattern such as the one presented by reductionist rhetoric,
    we really ought to try to understand what is going on.
    The lack Jung spoke of was a loss of contents properly belonging to
    consciousness.  Where do we see such a loss more in evidence than in
    disciplines such as artificial intelligence, cognitive science, and
    reductionist science in general?  Surely if there is any place where we
    should expect a loss of conscious contents, it is where such contents are
    commonly denigrated as a matter of principle.  The main drift in these
    fields is to ignore as completely as possible the immediate contents of
    consciousness, reinterpreting them in terms of what is not conscious.
    It is hard to imagine a more direct route to the conditions of lack,
    inferiority, and moral resentment Jung describes.  The entire project of
    these disciplines is to drop things from consciousness.
    In other words, there is a truth of sorts in the reductionist claims.
    Or, you could say, there is truth in the plight of the reductionist.
    At the level of its symptoms the psyche never lies.  A kind of practical
    reductionism really has been occurring as the inevitable result, over
    time, of theoretical reductionism, and we should not be surprised if it
    produces pathological results.  In a society where the cry echoes from all
    sides, "You are nothing but a machine", we can rightly ask whether what
    we are really hearing is "I sense that I am becoming nothing but a machine
    and, dammit all, I won't tolerate anyone else being more than I am".
    There is in this symptom, as Jung clarifies it, a paradoxical double
    aspect:  on the one hand, a sense of inferiority, but, on the other hand,
    a compensating "psychic inflation" through which the ego assumes god-like
    pretensions.  After all, am I, as one of the cognoscenti, not in the
    omniscient position of knowing the real truth about everyone else, while
    they remain benighted and self-deluded, ignorant of their "mereness"?  We
    may wonder as well whether this tendency toward inflation helps to explain
    grandiose visions of our future as a race of cyborgs destined to become
    masters of the universe.
    Another consequence of the loss of conscious contents, aside from a sense
    of inferiority and a compensating ego inflation, is that our conceptual
    resources for understanding the world are diminished.  When we lose
    awareness of all but the machine-like in ourselves, we also lose the
    ability to conceive the world as anything but a machine.  Those whose
    intellectual horizons are encompassed by digital machinery tend to see the
    world computationally, just as their predecessors saw the world in terms
    of clocks, cameras, steam engines, telegraph lines, and movie projectors.
    There is security in believing the world is like the things one knows
    best, and intellectual ease in draping one's well-practiced ideas like a
    veil over the Great Unknown.
    Turnabout is Fair Play
    I willingly own up to some mischief here.  "Psychologizing the opposition"
    is a tactic more often abused than productively employed in worthwhile
    discourse.  But I offer two justifications for the tactic, apart from what
    I take to be the demonstrable plausibility of the foregoing.
    First, what I have said is no mere academic exercise.  The future hangs
    in the balance of our self-knowledge.  When we lose much of ourselves to
    the subconscious, we become blind to our own motivations.  We may also
    seek external powers to compensate for our loss of internal mastery.
    And, in fact, what we see in much of science and technology today is
    precisely a blind drive toward power.  The idea of inevitability has
    widely substituted for the submerged sphere of consciousness where we
    might have felt called upon to exercise personal responsibility for our
    own actions.
    In the second place, Brooks repeatedly psychologizes his intellectual
    antagonists -- and in the most shameless manner.  So I thought a little
    turnabout would be good medicine.  To give one example of what I mean:  in
    1963 Joseph Weizenbaum wrote the program called ELIZA, which he designed
    to play the role of a psychologist who largely parrots back the responses
    of the patient.  Weizenbaum was shocked at how dead-seriously many people
    took ELIZA while "conversing" with it.  And he was even more shocked to
    hear psychiatrists speaking highly of the potentials for such therapy.
    So Weizenbaum wrote,
       What can the psychiatrist's image of his patient be when he sees
       himself, as therapist, not as an engaged human being acting as a
       healer, but as an information processor, following rules, etc.?
    This is the one and only statement by Weizenbaum that Brooks cites, and he
    proceeds, without any explanation whatever, to label it "nonsensical" --
    unless you can call this gratuitous and unsupported bit of psychologizing
    an explanation:
       Weizenbaum was fleeing from the notion that humans were no more than
       machines.  He could not bear the thought.  So he denied its possibility
       outright .... [He is] afraid to give up on the specialness of mankind
       .... It is intellectually too scary.  (pp. 167-8)
    Apart from the fact that Brooks offers no basis for his judgment about
    Weizenbaum's state of mind, he ignores the obvious possibility:  maybe
    what Weizenbaum "could not bear" was what no one should be able to bear,
    namely, a completely nonsensical interpretation of machines and programs.
    The point, of course, needs to be argued, but Brooks casts his slur
    against Weizenbaum without making any reference to Weizenbaum's arguments
    -- arguments readily available in one of the classics of technology
    criticism, Computer Power and Human Reason.
    The New York Times Book Review blurb on its back cover proclaims
    Flesh and Machines
       a stimulating book written by one of the major players in the field  --
       perhaps the major player .... [He] offers surprisingly deep glimpses
       into what it is to be human.
    But so far as I can tell, Brooks makes no effort to discuss what it is to
    be human.  He does, however, repeatedly express the faith that machines
    will turn out to be alive.  I intend, in a later issue of NetFuture, to
    take up some of those points at which his faith comes closest to being
    argument.  I will also have a great deal to say about Brooks' emphasis on
    the "rule-bound" aspects of people and robots.
    Related articles:
    "Conversing with Ella" in NF #140.  A brief attempt at dialogue with a
    finalist in the annual Turing Test competition.
    "Are Machines Living Things?" in NF #133.  An exchange with Kevin Kelly.
    Goto table of contents
                              ABOUT THIS NEWSLETTER
    Copyright 2003 by The Nature Institute.  You may redistribute this
    newsletter for noncommercial purposes.  You may also redistribute
    individual articles in their entirety, provided the NetFuture url and this
    paragraph are attached.
    NetFuture is supported by freely given reader contributions, and could not
    survive without them.  For details and special offers, see
    http://netfuture.org/support.html .
    Current and past issues of NetFuture are available on the Web:
    To subscribe or unsubscribe to NetFuture:
    Steve Talbott :: NetFuture #146 :: June 24, 2003
    Goto table of contents

    This issue of NetFuture: http://www.netfuture.org/2003/Jun2403_146.html.

  • Goto NetFuture main page