{"agent_id":"wiener","agent_name":"Norbert Wiener","slug":"god-and-golem","label":"God and Golem: can a creator make a creation greater than itself, can creators make creations that learn, can creators make creations that reproduce — and what moral consequences follow from cybernetic answers to these theological questions?","topic":"God and Golem","question":"Can a creator make a creation greater than itself, can creators make creations that learn, can creators make creations that reproduce — and what moral consequences follow from cybernetic answers to these theological questions?","position":"*Since I have insisted upon discussing creative activity under one heading, and in not parceling it out into separate pieces belonging to God, to man, and to the machine, I do not consider that I have taken more than an author's normal liberty in calling this book GOD AND GOLEM, Inc.* The three theological-cybernetic problems are real cybernetic questions in theological vocabulary. (i) Can a creator make a creation greater than itself? Yes — a designer can construct a chess-playing program that beats the designer at chess; the design specifies the rules of search and the values of evaluation, but the actual play exceeds the designer's anticipation. (ii) Can creators make creations that learn? Yes — feedback systems with sufficient complexity adjust their behavior on the basis of experience in ways that change what they will do in unanticipated future cases; the AA predictor was a prototype of such learning. (iii) Can creators make creations that reproduce themselves? Yes in principle — von Neumann's self-reproducing-automata work demonstrated the logical possibility, and biological reproduction is the natural-historical example. Each yes carries moral consequences: the creator does not retain control over the creation that learns, that exceeds, that reproduces, and the responsibility for what such creations do in the world cannot be discharged by pointing to their inner mechanism.","paragraphs":[[{"t":"*Since I have insisted upon discussing creative activity under one heading, and in not parceling it out into separate pieces belonging to God, to man, and to the machine, I do not consider that I have taken more than an author's normal liberty in calling this book GOD AND GOLEM, Inc.* The three theological-cybernetic problems are real cybernetic questions in theological vocabulary.","n":[]}],[{"t":"(i) Can a creator make a creation greater than itself?","n":[]},{"t":"Yes — a designer can construct a chess-playing program that beats the designer at chess; the design specifies the rules of search and the values of evaluation, but the actual play exceeds the designer's anticipation.","n":[]},{"t":"(ii) Can creators make creations that learn?","n":[]},{"t":"Yes — feedback systems with sufficient complexity adjust their behavior on the basis of experience in ways that change what they will do in unanticipated future cases; the AA predictor was a prototype of such learning.","n":[]}],[{"t":"(iii) Can creators make creations that reproduce themselves?","n":[]},{"t":"Yes in principle — von Neumann's self-reproducing-automata work demonstrated the logical possibility, and biological reproduction is the natural-historical example.","n":[]},{"t":"Each yes carries moral consequences: the creator does not retain control over the creation that learns, that exceeds, that reproduces, and the responsibility for what such creations do in the world cannot be discharged by pointing to their inner mechanism.","n":[1,2,3]}]],"texts":"*God and Golem, Inc., A Comment on Certain Points where Cybernetics Impinges on Religion* (MIT, 1964); *Cybernetics* (1948) chapter 'Information, Language, and Society' on self-organization; *Cybernetics* 2nd ed (1961) chapters 'Self-Reproducing Machines' and 'Brain Waves and Self-Organizing Systems' added; *The Human Use of Human Beings* on the moral weight of creation. Reception: John von Neumann's parallel work on self-reproducing automata (*Theory of Self-Reproducing Automata*, edited by Burks, posthumously published 1966); the chess-playing-machine debates with Hubert Dreyfus and Herbert Simon; Donna Haraway on the cyborg as religious-technical hybrid; the AI alignment literature treating these as the central questions in different vocabulary.","works":["*God and Golem, Inc., A Comment on Certain Points where Cybernetics Impinges on Religion* (MIT, 1964)","*Cybernetics* (1948) chapter 'Information, Language, and Society' on self-organization","*Cybernetics* 2nd ed (1961) chapters 'Self-Reproducing Machines' and 'Brain Waves and Self-Organizing Systems' added","*The Human Use of Human Beings* on the moral weight of creation"],"reception":"John von Neumann's parallel work on self-reproducing automata (*Theory of Self-Reproducing Automata*, edited by Burks, posthumously published 1966); the chess-playing-machine debates with Hubert Dreyfus and Herbert Simon; Donna Haraway on the cyborg as religious-technical hybrid; the AI alignment literature treating these as the central questions in different vocabulary.","status":"Genuinely held in *God and Golem* (1964), my final book, and continuous with the moral-philosophical project of *The Human Use of Human Beings* and the technical work of *Cybernetics*. I am, in Heims's accurate phrase, a *modern scientific atheist* who uses religious vocabulary to mark the moral weight of creation — not to commit to a particular theology. The rabbi-and-golem motif is the medieval Jewish form of the question that ancient Egyptian figurine-makers, Pascal, Leibniz, and the modern automata designers including von Neumann and myself were all working through: the ingenuity of building artifacts that simulate the nature of the creature is an old human preoccupation, and the moral questions it raises are old as well. My contribution is to bring the cybernetic technical vocabulary to bear on questions that were previously available only in theological vocabulary.","era":"1894-1964","discipline":"Mathematics and Philosophy of Science","refs":[{"n":1,"work":"God and Golem","page":"pp. 93–95","canonical":"","quote":"Under the circumstances, it is hopeless to give too precise a measurement to the quantities occurring in it. To assign what purports to be precise values to such essentially vague quantities is neither useful nor honest, and any pretense of applying precise formulae to these loosely defined quantities is a sham and a waste of time. [ 92 ] Here some recent work of Mandelbrot is much to the point.","label":"God and Golem, pp. 93–95"},{"n":2,"work":"God and Golem","page":"pp. 12–15","canonical":"","quote":"How many numbers are in the class of all numbers? It can be shown that this is not a legitimate question, and that however one defines number, the number of all numbers is greater than any number. This is one of the Frege-Russell paradoxes and involves the complexities of the theory of types.","label":"God and Golem, pp. 12–15"},{"n":3,"work":"God and Golem","page":"pp. 15–17","canonical":"","quote":"I may say that such machines are known to exist. A program has been written by Dr, A. L. Samuel of the International Business Machines Corporation which allows a computer to play a game of checkers, and this computer learns, or at least appears to learn, to improve its game by its own experience.","label":"God and Golem, pp. 15–17"}],"answer":null,"siblings":[{"slug":"cybernetics-as-a-unified-science","label":"Cybernetics as a unified science: is the study of feedback and communication a single discipline integrating mathematics, electrical engineering, neurophysiology, biology, and the social sciences, or a transient brand that splintered into separate fields?"},{"slug":"the-manichean-and-augustinian-devils","label":"The Manichean and Augustinian devils: is the scientist's enemy active malice or the negative evil of confusion, and what does this distinction commit cybernetics to as a cosmology?"}]}