Inspired by Scott Rosenberg, I have started to read Joseph Weizenbaum’s book Computer Power and Human Reason. Weizenbaum, who died last month, was the inventor of ELIZA, the first chatbot; and he was so horrified by the enthusiastic reactions to his program that he set out to write one of the first great anti AI manifestos. As a programmer of real brilliance himself, who was none the less a product of the philosophically sophisticated central European milieu between the wars, he was horrified by the emergent hacker culture at MIT, where he worked, and there is a long, and eminently quotable passage describing the friends of Richard Stallman:
Wherever computer centers have become established, that is to say, in countless places in the United States, as well as in virtually all other industrial regions of the world, bright young men of disheveled appearance, often with sunken glowing eyes, can be seen sitting at computer consoles, their arms tensed and waiting to fire their fingers, already poised to strike, at the buttons and keys on which their attention seems to be as riveted as a gambler’s on the rolling dice. When not so transfixed, they often sit at tables strewn with computer printouts over which they pore like possessed students of a cabalistic text. They work until they nearly drop, twenty, thirty hours at a time. Their food, if they arrange it, is brought to them: coffee, Cokes, sandwiches. If possible, they sleep on cots near the computer. But only for a few hours–then back to the console or the printouts. Their rumpled clothes, their unwashed and unshaven faces, and their uncombed hair all testify that they are oblivious to their bodies and to the world in which they move. They exist, at least when so engaged, only through and for the computers. These are computer bums, compulsive programmers. They are an international phenomenon.
How may the compulsive programmer be distinguished from a merely dedicated, hard-working professional programmer? First, by the fact that the ordinary professional programmer addresses himself to the problem to be solved, whereas the compulsive programmer sees the problem mainly as an opportunity to interact with the computer.
….
The compulsive programmer is usually a superb technician, moreover, one who knows every detail of the computer he works on, its peripheral equipment, the computer’s operating system, etc. He is often tolerated around computer centers because of his knowledge of the system and because he can write small subsystem programs quickly, that is, in one or two sessions of, say, twenty hours each. After a time, the center may in fact be using a number of his programs. But because the compulsive programmer can hardly be motivated to do anything but program, he will almost never document his programs once he stops working on them. A center may therefore come to depend on him to teach the use of, and to maintain, the programs that he wrote and whose structure only he, if anyone, understands. His position is rather like that of a bank employee who doesn’t do much for the bank, but who is kept on because only he knows the combination to the safe. His main interest is, in any case, not in small programs, but in very large, very ambitious systems of programs. Usually the systems he undertakes to build, and on which he works feverishly for perhaps a month or two or three, have very grandiose but extremely imprecisely stated goals. Some examples of these ambitions are: new computer languages to facilitate man-machine communication; a general system that can be taught to play any board game; a system to make it easier for computer experts to write super-systems (this last is a favorite). It is characteristic of many such projects that the programmer can long continue in the conviction that they demand knowledge about nothing but computers, programming, etc. And that knowledge he, of course, commands in abundance. Indeed, the point at which such work is often abandoned is precisely when it ceases to be purely incestuous, i.e., when programming would have to be interrupted in order that knowledge from outside the computer world may be acquired.
Weizenbaum, writing in 1975, then goes on to explain that these people do not talk of themselves as “programmers”, but as “hackers” and goes on to compare them with the compulsive gamblers depicted by Dostoevski. Such a gambler, he says, is of course consumed with magical thinking; and he then goes on to draw an comparison between the magical thinking of the hacker or compulsive programmer and what we now call scientism (which is the real target of his book):
… the reason we are so interested in the compulsive programmer is that we see no discontinuity between his pathological motives and behavior and those of the modern scientist and technologist generally. The compulsive programmer is merely the proverbial mad scientist who has been given a theater, the computer, in which he can, and does, play out his fantasies.
Let us reconsider Bergler’s three observations about gamblers. First, the gambler is subjectively certain that he will win. So is the compulsive programmer–only he, having created his own world on a universal machine, has some foundation in reality for his certainty. Scientists, with some exceptions, share the same faith: what science has not done, it has not yet done; the questions science has not answered, it has not _yet _ answered. Second, the gambler has an unbounded faith in his own cleverness. Well?! Third, the gambler knows that life itself is nothing but a gamble. Similarly, the compulsive programmer is convinced that life is nothing but a program running on an enormous computer, and that therefore every aspect of life can ultimately be explained in programming terms. Many scientists (again there are notable exceptions) also believe that every aspect of life and nature can finally be explained in exclusively scientific terms. Indeed, as Polanyi correctly points out, the stability of scientific beliefs is defended by the same devices that protect magical belief systems:
“Any contradiction between a particular scientific notion and the facts of experience will be explained by other scientific notions; there is a ready reserve of possible scientific hypotheses available to explain any conceivable event. . . . within science itself, the stability of theories against experience is maintained by epicyclical reserves which suppress alternative conceptions in the germ.”
Hence we can make out a continuum. At one of its extremes stand scientists and technologists who much resemble the compulsive programmer. At the other extreme are those scientists, humanists, philosophers, artists, and religionists who seek understanding as whole persons and from all possible perspectives. The affairs of the world appear to be in the hands of technicians whose psychic constitutions approximate those of the former to a dangerous degree. Meanwhile the voices that speak the wisdom of the latter seem to be growing ever fainter.
My copy — bought as a paperback from abebooks — is already disintegrating as I scan it. This is a pleasant meta-demonstration of his argument, since merely understanding the book by reading it would cause less physical damage. But then I scan it so that you can all make comments. I will have to buy another one just to have to read.
Two chapters of the book are a dense explanation from first principles of what a Turing machine is and does. This demands very careful rereading, if, like me, one only half-absorbed Goedel, Escher, Bach.
So far it looks as if Weizenbaum’s early argument that science cannot exist without prior, independent value judgements, and cannot hope ever to replace them, is really the core of the book, and I will come back to it when I have thoroughly digested it.
