Theo Todman's Web Page - Notes Pages

1. On Feb. 15, 1965, a diffident but self-possessed high school student named Raymond Kurzweil₂ appeared as a guest on a game show called I've Got a Secret. He was introduced by the host, Steve Allen, then he played a short musical composition on a piano. The idea was that Kurzweil was hiding an unusual fact and the panellists — they included a comedian and a former Miss America — had to guess what it was.
2. On the show, the beauty queen did a good job of grilling Kurzweil, but the comedian got the win: the music was composed by a computer. Kurzweil got $200.
3. Kurzweil then demonstrated the computer, which he built himself — a desk-size affair with loudly clacking relays, hooked up to a typewriter. The panellists were pretty blasé about it; they were more impressed by Kurzweil's age than by anything he'd actually done. They were ready to move on to Mrs. Chester Loney of Rough and Ready, Calif., whose secret was that she'd been President Lyndon Johnson's first-grade teacher.
4. But Kurzweil would spend much of the rest of his career working out what his demonstration meant. Creating₃ a work of art is one of those activities we reserve for humans and humans only. It's an act of self-expression; you're not supposed to be able to do it if you don't have a self₄. To see creativity, the exclusive domain of humans, usurped by a computer built by a 17-year-old is to watch a line blur that cannot be unblurred, the line between organic intelligence₅ and artificial intelligence⁶.
5. That was Kurzweil's real secret, and back in 1965 nobody guessed it. Maybe not even him, not yet. But now, 46 years later, Kurzweil believes that we're approaching a moment when computers will become intelligent, and not just intelligent but more intelligent than humans. When that happens, humanity — our bodies, our minds, our civilization — will be completely and irreversibly transformed. He believes that this moment is not only inevitable but imminent₇. According to his calculations, the end of human civilization₈ as we know it is about 35 years away.
6. Computers are getting faster. Everybody knows that. Also, computers are getting faster faster — that is, the rate at which they're getting faster is increasing.
7. True? True₉.
8. So if computers are getting so much faster, so incredibly fast, there might conceivably come a moment when they are capable of something comparable to human intelligence¹⁰. Artificial intelligence¹¹. All that horsepower could be put in the service of emulating₁₂ whatever it is our brains are doing when they create consciousness — not just doing arithmetic very quickly or composing piano music but also₁₃ driving cars, writing books, making ethical decisions, appreciating fancy paintings, making witty observations at cocktail parties.
9. If you can swallow that idea, and Kurzweil and a lot of other very smart₁₄ people can, then all bets are off. From that point on, there's no reason₁₅ to think computers would stop₁₆ getting more powerful. They would keep on developing until they were far more intelligent than we are. Their rate of development would also continue to increase, because they would take over their own development₁₇ from their slower-thinking human creators. Imagine a computer scientist that was itself a super-intelligent computer. It would work incredibly quickly. It could draw on huge amounts of data effortlessly. It wouldn't even take breaks to play Farmville.
10. Probably. It's impossible to predict the behavior of these smarter-than-human intelligences¹⁸ with which (with whom?) we might one day share the planet, because₁₉ if you could, you'd be as smart as they would be. But there are a lot of theories about it. Maybe we'll merge with them to become super-intelligent cyborgs₂₀, using computers to extend our intellectual abilities the same₂₁ way that cars and planes extend our physical abilities. Maybe the artificial intelligences²² will help us treat the effects of old age and prolong our life spans indefinitely₂₃. Maybe we'll scan our consciousnesses₂₄ into computers and live inside them as software₂₅, forever, virtually. Maybe the computers will turn on humanity and annihilate₂₆ us. The one thing all these theories have in common is the transformation of our species into something that is no longer recognizable as such to humanity circa 2011. This transformation has a name: the Singularity₂₇.
11. The difficult thing to keep sight of when you're talking about the Singularity²⁸ is that even though it sounds like science fiction₂₉, it isn't, no more than a weather forecast is science fiction. It's not a fringe idea; it's a serious hypothesis about the future of life on Earth. There's an intellectual gag reflex that kicks in anytime you try to swallow an idea that involves super-intelligent immortal cyborgs³⁰, but suppress it if you can, because while the Singularity³¹ appears to be, on the face of it, preposterous, it's an idea that rewards sober, careful evaluation.
12. People are spending a lot of money trying to understand it. The three-year-old Singularity University, which offers inter-disciplinary courses of study for graduate students and executives, is hosted by NASA. Google was a founding sponsor; its CEO and co-founder Larry Page spoke there last year. People are attracted to the Singularity³² for the shock value, like an intellectual freak show, but they stay because there's more to it than they expected. And of course, in the event that it turns out to be real, it will be the most important thing to happen to human beings since the invention of language₃₃.
13. The Singularity³⁴ isn't a wholly new idea, just newish. In 1965 the British mathematician I.J. Good described something he called an "intelligence³⁵ explosion":
    Let an ultraintelligent₃₆ machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence³⁷ explosion," and the intelligence³⁸ of man would be left far behind. Thus the first ultraintelligent machine is the last invention₃₉ that man need ever make.
14. The word singularity is borrowed from astrophysics: it refers to a point in space-time — for example, inside a black hole — at which the rules of ordinary physics do not apply. In the 1980s the science-fiction novelist Vernor Vinge attached it to Good's intelligence-explosion⁴⁰ scenario. At a NASA symposium in 1993, Vinge announced that "within 30 years₄₁, we will have the technological means to create super-human intelligence⁴². Shortly after, the human era will be ended."
15. By that time Kurzweil was thinking about the Singularity⁴³ too. He'd been busy since his appearance on I've Got a Secret. He'd made several fortunes as an engineer and inventor; he founded and then sold his first software company while he was still at MIT. He went on to build the first print-to-speech reading machine for the blind — Stevie Wonder was customer No. 1 — and made innovations in a range of technical fields, including music synthesizers and speech recognition. He holds 39 patents and 19 honorary doctorates. In 1999 President Bill Clinton awarded him the National Medal of Technology.
16. But Kurzweil was also pursuing a parallel career as a futurist: he has been publishing his thoughts about the future of human and machine-kind for 20 years, most recently in The Singularity⁴⁴ Is Near, which was a best seller when it came out in 2005. A documentary by the same name, starring Kurzweil, Tony Robbins and Alan Dershowitz, among others, was released in January. (Kurzweil is actually the subject of two current documentaries. The other one, less authorized but more informative, is called The Transcendent Man.) Bill Gates has called him "the best person I know at predicting₄₅ the future of artificial intelligence⁴⁶."
17. In real life, the transcendent man is an unimposing figure who could pass for Woody Allen's even nerdier younger brother. Kurzweil grew up in Queens, N.Y., and you can still hear a trace of it in his voice. Now 62, he speaks with the soft, almost hypnotic calm of someone who gives 60 public lectures a year. As the Singularity⁴⁷'s most visible champion, he has heard all the questions and faced down the incredulity many, many times before. He's good-natured about it. His manner is almost apologetic: I wish I could bring you less exciting news of the future, but I've looked at the numbers₄₈, and this is what they say, so what else can I tell you?
18. Kurzweil's interest in humanity's cyborganic⁴⁹ destiny began about 1980 largely as a practical matter. He needed ways to measure and track the pace of technological progress. Even great inventions can fail if they arrive before their time, and he wanted to make sure that when he released his, the timing was right. "Even at that time, technology was moving quickly enough that the world was going to be different by the time you finished₅₀ a project," he says. "So it's like skeet shooting — you can't shoot at the target." He knew about Moore's₅₁ law, of course, which states that the number of transistors you can put on a microchip doubles about every two years. It's a surprisingly reliable rule of thumb. Kurzweil tried plotting a slightly different curve: the change over time in the amount of computing power, measured in MIPS (millions of instructions per second), that you can buy₅₂ for $1,000.
19. As it turned out, Kurzweil's numbers looked a lot like₅₃ Moore's. They doubled every couple of years. Drawn as graphs, they both made exponential₅₄ curves, with their value increasing by multiples of two instead of by regular increments in a straight line. The curves held eerily steady, even when Kurzweil extended his backward through the decades of pre-transistor computing technologies like relays and vacuum tubes, all the way back to 1900.
20. Kurzweil then ran the numbers on a whole bunch of other key technological indexes₅₅ — the falling cost of manufacturing transistors, the rising clock speed of microprocessors, the plummeting price of dynamic RAM. He looked even further afield at trends in biotech and beyond₅₆ — the falling cost of sequencing DNA and of wireless data service and the rising numbers of Internet hosts and nanotechnology patents. He kept finding the same thing: exponentially accelerating progress. "It's really amazing how smooth₅₇ these trajectories are," he says. "Through thick and thin, war and peace₅₈, boom times and recessions." Kurzweil calls it the law of accelerating returns₅₉: technological progress happens exponentially, not linearly.
21. Then he extended the curves into the future₆₀, and the growth they predicted was so phenomenal, it created cognitive resistance in his mind. Exponential curves start slowly, then rocket skyward toward infinity. According to Kurzweil, we're not evolved₆₁ to think in terms of exponential growth. "It's not intuitive. Our built-in predictors are linear. When we're trying to avoid an animal, we pick the linear prediction of where it's going to be in 20 seconds and what to do about it. That is actually hardwired in our brains."
22. Here's what the exponential curves told him. We will successfully reverse-engineer₆₂ the human brain by the mid-2020s. By the end of that decade, computers will be capable of human-level intelligence⁶³. Kurzweil puts the date of the Singularity⁶⁴ — never say he's not conservative — at 2045. In that year, he estimates, given the vast increases in computing power and the vast reductions in the cost of same, the quantity of artificial intelligence⁶⁵ created will be about a billion times the sum of all the human intelligence⁶⁶ that exists today.
23. The Singularity⁶⁷ isn't just an idea. It attracts people, and those people feel a bond with one another. Together they form a movement, a subculture; Kurzweil calls it a community. Once you decide to take the Singularity⁶⁸ seriously, you will find that you have become part of a small but intense and globally distributed hive of like-minded thinkers known as Singularitarians₆₉.
24. Not all of them are Kurzweilians, not by a long chalk. There's room inside Singularitarianism for considerable diversity₇₀ of opinion about what the Singularity⁷¹ means and when and how it will or won't happen. But Singularitarians share a worldview₇₂. They think in terms of deep time, they believe in the power of technology to shape history, they have little interest in the conventional wisdom about anything, and they cannot believe you're walking around living your life and watching TV as if the artificial-intelligence revolution were not about to erupt and change absolutely everything₇₃. They have no fear of sounding ridiculous; your ordinary citizen's distaste for apparently absurd ideas is just an example of irrational bias, and Singularitarians have no truck with irrationality. When you enter their mind-space you pass through an extreme gradient in worldview, a hard ontological shear that separates Singularitarians from the common run of humanity. Expect turbulence.
25. In addition to the Singularity⁷⁴ University, which Kurzweil co-founded, there's also a Singularity Institute for Artificial Intelligence⁷⁵, based in San Francisco. It counts among its advisers Peter Thiel, a former CEO of PayPal and an early investor in Facebook. The institute holds an annual conference called the Singularity⁷⁶ Summit. (Kurzweil co-founded that too.) Because of the highly interdisciplinary nature of Singularity theory, it attracts a diverse crowd. Artificial intelligence⁷⁷ is the main event, but the sessions also cover the galloping progress of, among other fields, genetics and nanotechnology.
26. At the 2010 summit, which took place in August in San Francisco, there were not just computer scientists but also psychologists, neuroscientists, nanotechnologists, molecular biologists, a specialist in wearable computers, a professor of emergency medicine, an expert on cognition in grey parrots and the professional magician and debunker James "the Amazing" Randi₇₈. The atmosphere was a curious blend of Davos and UFO convention. Proponents of sea-steading — the practice, so far mostly theoretical, of establishing politically autonomous floating communities in international waters — handed out pamphlets. An android⁷⁹ chatted with visitors in one corner.
27. After artificial intelligence⁸⁰, the most talked-about topic at the 2010 summit was life extension₈₁. Biological boundaries that most people think of as permanent and inevitable Singularitarians see as merely intractable but solvable₈₂ problems. Death is one of them. Old age is an illness₈₃ like any other, and what do you do with illnesses? You cure them. Like a lot of Singularitarian ideas, it sounds funny at first, but the closer you get to it, the less funny it seems. It's not just wishful thinking; there's actual science going on here.
28. For example, it's well known that one cause of the physical degeneration associated with aging involves telomeres, which are segments of DNA found at the ends of chromosomes. Every time a cell divides, its telomeres get shorter, and once a cell runs out of telomeres, it can't reproduce anymore and dies. But there's an enzyme called telomerase that reverses this process; it's one of the reasons cancer cells live so long. So why not treat regular non-cancerous cells with telomerase₈₄? In November, researchers at Harvard Medical School announced in Nature that they had done just that. They administered telomerase to a group of mice suffering from age-related degeneration. The damage went away. The mice didn't just get better; they got younger.
29. Aubrey de Grey is one of the world's best-known life-extension researchers and a Singularity Summit veteran. A British biologist with a doctorate from Cambridge and a famously formidable beard, de Grey runs a foundation called SENS, or Strategies for Engineered Negligible Senescence. He views aging as a process of accumulating damage, which he has divided into seven categories, each of which he hopes to one day address using regenerative medicine. "People have begun to realize that the view of aging being something immutable — rather like the heat death of the universe — is simply ridiculous," he says. "It's just childish. The human body is a machine₈₅ that has a bunch of functions, and it accumulates various types of damage as a side effect of the normal function of the machine. Therefore in principal that damage can be repaired periodically. This is why we have vintage cars. It's really just a matter of paying attention. The whole of medicine consists of messing about with what looks pretty inevitable until you figure out how to make it not inevitable."
30. Kurzweil takes life extension seriously too. His father, with whom he was very close, died of heart disease at 58. Kurzweil inherited his father's genetic predisposition; he also developed Type 2 diabetes when he was 35. Working with Terry Grossman, a doctor who specializes in longevity medicine, Kurzweil has published two books on his own approach to life extension, which involves taking up to 200 pills and supplements a day. He says his diabetes is essentially cured, and although he's 62 years old from a chronological perspective, he estimates that his biological age is about 20 years younger.
31. But his goal differs slightly from de Grey's. For Kurzweil, it's not so much about staying healthy as long as possible; it's about staying alive until₈₆ the Singularity⁸⁷. It's an attempted handoff. Once hyper-intelligent artificial intelligences⁸⁸ arise, armed with advanced nanotechnology₈₉, they'll really be able to wrestle with the vastly complex, systemic problems associated with aging in humans. Alternatively, by then we'll be able to transfer₉₀ our minds to sturdier vessels such as computers and robots. He and many other Singularitarians take seriously the proposition that many people who are alive today will wind up being functionally₉₁ immortal₉₂.
32. It's an idea that's radical and ancient at the same time. In "Sailing to Byzantium," W.B. Yeats describes mankind's fleshly predicament as a soul fastened to a dying animal. Why not unfasten it and fasten it to an immortal robot instead? But Kurzweil finds that life extension produces even more resistance in his audiences than his exponential growth curves. "There are people who can accept computers being more intelligent than people," he says. "But the idea of significant changes to human longevity — that seems to be particularly controversial₉₃. People invested a lot of personal effort into certain philosophies dealing with the issue of life and death. I mean, that's the major reason we have religion₉₄."
33. Of course, a lot of people think the Singularity⁹⁵ is nonsense — a fantasy, wishful thinking, a Silicon Valley version of the Evangelical story of the Rapture, spun by a man who earns his living making outrageous claims and backing them up with pseudoscience. Most of the serious critics focus on the question of whether a computer can truly become intelligent₉₆.
34. The entire field of artificial intelligence⁹⁷, or AI, is devoted to this question. But AI doesn't currently produce the kind₉₈ of intelligence⁹⁹ we associate with humans or even with talking computers in movies — HAL or C3PO or Data. Actual Ais tend to be able to master only one highly specific₁₀₀ domain, like interpreting search queries or playing chess. They operate within an extremely specific frame of reference. They don't make conversation at parties. They're intelligent, but only if you define intelligence¹⁰¹ in a vanishingly narrow way. The kind of intelligence¹⁰² Kurzweil is talking about, which is called strong AI or artificial general intelligence¹⁰³, doesn't exist₁₀₄ yet.
35. Why not? Obviously we're still waiting on all that exponentially growing computing power to get here. But it's also possible that there are things going on in our brains that can't₁₀₅ be duplicated¹⁰⁶ electronically no matter how many MIPS you throw at them. The neurochemical architecture that generates the ephemeral chaos we know as human consciousness may just be too complex and analog₁₀₇ to replicate¹⁰⁸ in digital₁₀₉ silicon. The biologist Dennis Bray was one of the few voices of dissent at last summer's Singularity Summit. "Although biological components act in ways that are comparable to those in electronic circuits," he argued, in a talk titled "What Cells Can Do That Robots Can't," "they are set apart by the huge number of different states they can adopt. Multiple biochemical processes create chemical modifications of protein molecules, further diversified by association with distinct structures at defined locations of a cell. The resulting combinatorial explosion₁₁₀ of states endows living systems with an almost infinite capacity to store information regarding past and present conditions and a unique capacity to prepare for future events." That makes the ones and zeros that computers trade in look pretty crude.
36. Underlying the practical challenges are a host of philosophical₁₁₁ ones. Suppose we did create a computer that talked and acted in a way that was indistinguishable from a human being — in other words, a computer that could pass the Turing test. (Very loosely speaking, such a computer would be able to pass as human in a blind test.) Would that mean that the computer was sentient, the way a human being is? Or would it just be an extremely sophisticated but essentially mechanical automaton₁₁₂ without the mysterious spark of consciousness — a machine with no ghost in it? And how would we know₁₁₃?
37. Even if you grant that the Singularity¹¹⁴ is plausible, you're still staring at a thicket of unanswerable questions. If I can scan my consciousness₁₁₅ into a computer, am I still me₁₁₆? What are the geopolitics and the socioeconomics₁₁₇ of the Singularity¹¹⁸? Who decides who gets to be immortal? Who draws the line₁₁₉ between sentient and non-sentient? And as we approach immortality, omniscience and omnipotence, will our lives still have meaning₁₂₀? By beating death, will we have lost our essential humanity₁₂₁?
38. Kurzweil admits that there's a fundamental level of risk₁₂₂ associated with the Singularity¹²³ that's impossible to refine away, simply because we don't know what a highly advanced artificial intelligence¹²⁴, finding itself a newly created inhabitant of the planet Earth, would choose to do₁₂₅. It might not feel like competing with us for resources. One of the goals of the Singularity¹²⁶ Institute is to make sure not just that artificial intelligence¹²⁷ develops but also that the AI is friendly. You don't have to be a super-intelligent cyborg¹²⁸ to understand that introducing a superior life-form into your own biosphere is a basic Darwinian₁₂₉ error.
39. If the Singularity¹³⁰ is coming, these questions are going to get answers₁₃₁ whether we like it or not, and Kurzweil thinks that trying to put off the Singularity¹³² by banning₁₃₃ technologies is not only impossible but also unethical and probably dangerous. "It would require a totalitarian system to implement such a ban," he says. "It wouldn't work. It would just drive these technologies underground₁₃₄, where the responsible scientists who we're counting on to create the defenses would not have easy access to the tools."
40. Kurzweil is an almost inhumanly patient and thorough debater. He relishes it. He's tireless in hunting down his critics so that he can respond to them, point by point, carefully and in detail.
41. Take the question of whether computers can replicate¹³⁵ the biochemical complexity of an organic brain. Kurzweil yields no ground there whatsoever. He does not see any fundamental difference₁₃₆ between flesh and silicon that would prevent the latter from thinking. He defies biologists to come up with a neurological mechanism that could not be modelled₁₃₇ or at least matched in power and flexibility by software running on a computer. He refuses to fall on his knees before the mystery of the human brain. "Generally speaking," he says, "the core of a disagreement I'll have with a critic is, they'll say, Oh, Kurzweil is underestimating the complexity of reverse-engineering₁₃₈ of the human brain or the complexity of biology. But I don't believe I'm underestimating the challenge. I think they're underestimating the power of exponential₁₃₉ growth."
42. This position doesn't make Kurzweil an outlier, at least among Singularitarians. Plenty of people make more-extreme predictions. Since 2005 the neuroscientist Henry Markram has been running an ambitious initiative at the Brain Mind Institute of the Ecole Polytechnique in Lausanne, Switzerland. It's called the Blue Brain project, and it's an attempt to create a neuron-by-neuron₁₄₀ simulation of a mammalian brain, using IBM's Blue Gene super-computer. So far, Markram's team has managed to simulate one neocortical column from a rat's brain, which contains about 10,000 neurons. Markram has said that he hopes to have a complete₁₄₁ virtual human brain up and running in 10 years. (Even Kurzweil sniffs at this. If it worked, he points out, you'd then have to educate₁₄₂ the brain, and who knows how long that would take?)
43. By definition, the future beyond the Singularity¹⁴³ is not knowable by our linear, chemical, animal brains, but Kurzweil is teeming with theories about it. He positively flogs himself to think bigger and bigger; you can see him kicking against the confines of his aging organic hardware. "When people look at the implications of ongoing exponential growth, it gets harder and harder to accept," he says. "So you get people who really accept, yes, things are progressing exponentially, but they fall off the horse at some point because the implications₁₄₄ are too fantastic. I've tried to push myself to really look."
44. In Kurzweil's future, biotechnology and nanotechnology give us the power to manipulate our bodies and the world around us at will, at the molecular₁₄₅ level. Progress hyperaccelerates, and every hour brings a century's worth of scientific breakthroughs. We ditch Darwin and take charge₁₄₆ of our own evolution¹⁴⁷. The human genome becomes just so much code to be bug-tested and optimized and, if necessary, rewritten₁₄₈. Indefinite life extension becomes a reality; people die only if they choose to. Death loses its sting once and for all. Kurzweil hopes to bring his dead father back₁₄₉ to life.
45. We can scan our consciousnesses into computers and enter a virtual₁₅₀ existence or swap our bodies₁₅₁ for immortal robots and light out for the edges of space as intergalactic godlings. Within a matter of centuries₁₅₂, human intelligence¹⁵³ will have re-engineered and saturated all the matter in the universe. This is, Kurzweil believes, our destiny as a species.
46. Or it isn't. When the big questions get answered, a lot of the action will happen where no one can see it, deep inside the black silicon brains of the computers, which will either bloom bit by bit into conscious minds or just continue in ever more brilliant and powerful iterations of nonsentience₁₅₄.
47. But as for the minor questions, they're already being decided all around us and in plain sight. The more you read about the Singularity¹⁵⁵, the more you start to see it peeking out at you, coyly, from unexpected directions. Five years ago we didn't have 600 million humans carrying out their social lives over a single electronic network. Now we have Facebook. Five years ago you didn't see people double-checking what they were saying and where they were going, even as they were saying it and going there, using handheld network-enabled digital prosthetics. Now we have iPhones. Is it an unimaginable step to take the iPhones out of our hands and put them into our skulls₁₅₆?
48. Already 30,000 patients with Parkinson's disease have neural implants₁₅₇. Google is experimenting with computers that can drive cars. There are more than 2,000 robots₁₅₈ fighting in Afghanistan alongside the human troops. This month a game show will once again figure in the history of artificial intelligence¹⁵⁹, but this time the computer will be the guest: an IBM super-computer nicknamed Watson will compete on Jeopardy! Watson runs on 90 servers and takes up an entire room₁₆₀, and in a practice match in January it finished ahead of two former champions, Ken Jennings and Brad Rutter. It got every question it answered right, but much more important, it didn't need help understanding the questions (or, strictly speaking, the answers), which were phrased in plain English. Watson isn't strong AI, but if strong AI happens, it will arrive gradually₁₆₁, bit by bit, and this will have been one of the bits.
49. A hundred years from now, Kurzweil and de Grey and the others could be the 22nd century's answer to the Founding Fathers — except unlike the Founding Fathers, they'll still be alive to get credit — or their ideas could look as hilariously retro and dated as Disney's Tomorrowland. Nothing gets old as fast as the future.
50. But even if they're dead wrong about the future, they're right about the present. They're taking the long view and looking at the big picture. You may reject every specific article of the Singularitarian charter, but you should admire Kurzweil for taking the future seriously. Singularitarianism is grounded in the idea that change is real and that humanity is in charge of its own fate and that history might not be as simple as one damn thing after another. Kurzweil likes to point out that your average cell phone is about a millionth the size of, a millionth the price of and a thousand times more powerful than the computer he had at MIT 40 years ago. Flip that forward 40 years and what does the world look like? If you really want to figure that out, you have to think very, very far outside the box. Or maybe you have to think further inside₁₆₂ it than anyone ever has before.

In-Page Footnotes:

• Kurzweil: See Wikipedia: Ray Kurzweil, Kurzweil: Accelerating Intelligence, and much else besides. I seem to have one of his books - "Kurzweil (Ray) - The Age of Spiritual Machines". Click here for Note.

• Creativity: there’s presumably a distinction between rules-based creativity, which is what (presumably) computers can do, and creativity of a less constrained sort, that we don’t know how to get computers to do (yet)? Click here for Note.

• Self: and “self-expression” – facon de parler? Click here for Note.

• Intelligence and Consciousness: there’s a sharp distinction between intelligence and consciousness. As far as we know, consciousness is the preserve of organic intelligence. We can presume that lots of rather dim animals are phenomenally conscious (even if not self-conscious … the distinction is important) so, there’s no link between getting smarter and smarter and then (as a result) getting phenomenally conscious. I’m not sure of the link between intelligence and self-consciousness. Click here for Note.

• Imminence of the “Singularity”: this is predicated on the assumption of continued exponential growth. Click here for Note.

• Human Civilization: So far, computers have only enhanced human civilisation. “Ending” it (“as we know it”) depends on delivering (out of control) the various promissory-notes of this article. Click here for Note.

• Faster Faster: Is this really so, and will it really continue to be so, if it is so? Note that Kurzweil's graph muddles together speed and cost. Click here for Note.

• Emulation: Two points here. Firstly, emulation isn’t the real thing. Models of hurricanes aren’t wet and windy, so why should emulations of consciousness be conscious? Secondly, digital computers are serial devices in which the components are (now) very quick, and brains are massively parallel devices whose components are very slow. Why should simulating one by the other produce the same (phenomenal) effect, and even be possible at all? Click here for Note.

• Intelligent Actions: The items on the list (“driving cars, writing books, making ethical decisions, appreciating fancy paintings, making witty observations at cocktail parties”) can all (presumably) be rules-based and situation-driven. No doubt this is true of human intelligence as well (ultimately) but modelling it is not straightforward, as we don’t know how the brain does it. The issue isn’t really (in this case) to do with “whether”, but “when”, as there are lots of major breakthroughs required before the promissory note can be delivered on. Also, all these functions can be delivered unconsciously (if they can be delivered at all). Click here for Note.

• Smart people: Does it matter how smart they are? Lots of equally smart people don’t share the optimism of the futurologists. Click here for Note.

• Increasingly Powerful Computers: are there really no reasons to doubt that their onward exponential growth is really never going to end? Miniaturisation of components has to stop soon due to QM effects. So, a radically-new technology is needed. Some ideas are there, but we might get “stuck” on their delivery, as has been the case for controlled nuclear fusion (Wikipedia: Fusion Power), which in the 1950s was expected soon, in the 1970s by 2000 and in 2006 “not within 100 years”. Click here for Note.

• Computing Power: There’s no doubt that computers will continue to get more powerful, as hardware and software continues to improve, as it always will. The issue is really over the rate of change (can exponential growth continue indefinitely) and can certain conceptual breakthroughs be made? Click here for Note.

• Bootstrapped Development: This is certainly an important point, as we certainly use computers to help manufacture computers. But the extrapolation to development may involve the solution of the real “machine creativity” problem. Click here for Note.

• Prediction: is this true? It would be true if machines became “smarter” than humans in every dimension of “smartness”. But “unpredictability” (ie. non-rules-based) is one of the aspects of machine-intelligence yet to be delivered by AI. Also, this argument sounds a bit like the “you can’t know the mind of God” (at all) arguments, which may or may not be sound. Click here for Note.

• Cyborgs: This sounds a more promising approach than simulation, and it’d relieve computers from having to realise consciousness. But any cognitive interlinking would still require a fuller understanding of how the brain works than is currently on the horizon. Click here for Note.

• Analogies: We don’t “integrate” with cars and planes any more than we integrate with computers. They are just tools. Prosthetics are the nearest analogues, but there’s a long way from that to true integration. Click here for Note.

• Nanotechnology: At this stage of the argument, it’s not clear how intelligent machines will help repair our bodies and brains (especially “indefinitely”). Usually nanotechnology is invoked at this stage (see Wikipedia: Nanotechnology for an overview). Now, it’s true that intelligent machines would be needed to manufacture, and probably program, these myriads of tiny, very specialised machines, but the possibilities are very schematic. There’s no evidence that anything workable is around the corner. Click here for Note.

• Consciousnesses: Just what is meant here? Is this just loose speaking? A thing (an animal) is conscious, and the animal can’t be scanned and downloaded anywhere. No-one really knows (at the theoretical level) what phenomenal consciousness is, though there are many theories. What’s probably intended here is that “the contents of our brains” would be read and uploaded to some device that can simulate our brains. This, of course, assumes that mind-body substance dualism is false (as it probably is), but even so – and admitting that whatever runs the downloaded software is at best a copy of the original, there’s a long way to go before this sort of thing becomes even a worked-out theoretical possibility. Click here for Note.

• Software: Well, philosophically-speaking, this is an outrageous idea. It depends on what we are, and we’re almost certainly not software, though software is important to us. And there are issues of identity – since software is easy to copy, and copies aren’t identical, what reason would an individual have for thinking any particular installed copy was (identical to) him? Click here for Note.

• Annihilation: Well, this is certainly something to watch out for, but I dare say it’s a way off. It’s more of a worry in genetic engineering or (if it gets going in the futurist mini-robot sense) nanotechnology. Click here for Note.

• The Singularity: This term is defined later, but see Wikipedia: Technological Singularity and Wikipedia: The Singularity Is Near (amongst much else). Click here for Note.

• Science Fiction: The difference, presumably, is that talk of the Singularity is intended as a prediction rather than as mere entertainment with no real concern with the facts. But the predictions don’t really seem to be worked out in any detail – it’s just the idea that throwing hardware at things will work, combined with the assumption of indefinitely-continued exponential growth. Click here for Note.

• Importance of the Singularity: It would certainly be important. Whether it’s as important as language is debateable. Why not choose for comparison some other technological development, like the use of agriculture, or an intellectual one like the invention of writing? Also, language isn’t something that was invented, is it? It arose, maybe as externalised inner thoughts – an external and public Language of Thought. Click here for Note.

• Ultraintelligence: This is a definition of ultraintelligent. It does not guarantee that there will ever be anything that falls under this category. Also, it seems a bit heavy-handed. Superintelligent machines – those that may not be ultimate, but will supplement human intelligence even more than current computers – might do the job. The idea is that there could be a human invention that obviates the need for any further human inventions, because any invention that a human could come up with, the machine could also come up with. Maybe all we need is that it (with human assistance) can come up with anything that a human can come up with (though a brick is such a “machine”), or that it (with human assistance) can come up with something that no unaided human can come up with (but this is already satisfied). More thought required. Click here for Note.

• Last Invention: No doubt Hollywood would disagree. After the machines have taken over, human beings would have to invent a way of defeating them. This aside, is it really clear what “surpassing all the intellectual activities of any man however clever“ really means? Click here for Note.

• Failed Predictions?: This is by 2023, now in 2011 – just 12 years away. While the prediction hasn’t yet failed, it will no doubt do so as super-human intelligence seems as far away as ever, and the human era shows no sign of ending. Click here for Note.

• Predicting the Future: One could be styled “good at predicting the future” if your predictions had a habit of coming true. Is this the case with Kurzweil’s predictions, or is it just that his predictions are the sort that Bill Gates likes? Click here for Note.

• The Numbers: As always, this is the extrapolation of exponential growth. What if Moore’s Law fails because we’ve reached QM-interference levels? What then? There was an article in Custom PC that made further progress look rather a struggle. Joining together microprocessors reduces miniaturisation and introduces light-speed effects. Compare with the stalled progress on nuclear fusion. Electricity “too cheap to metre” is still a way off after 60 years of research. Click here for Note.

• A Different Future: This is certainly true – products have to be placed in a context to be useful – both because fashions change, and they need to link in with other technology and people’s needs. Technology does become obsolete very quickly, and has been doing so for decades. But technologies eventually reach maturity, or have to await the development of other technologies to mature before they can move on further. Click here for Note.

• Moore's Law: See Wikipedia: Moore's Law. This article mentions Kurzweil and other futurologists, and the possible breakdown of Moore’s Law within the next 5 years or so (ie. well before 2045). It also notes that Moore’s Law is a self-fulfilling prophesy, in that the industry has taken it as a paradigm for R&D aims. Also, that the R&D costs of keeping up with Moore’s Law are also increasing exponentially. Click here for Note

• Hardware Costs: As any IT professional knows, the costs associated with any major development are almost all down to software; and residual hardware costs are mostly down to those of their minders. These costs aren’t going to exponentially decay. Click here for Note.

• Kurzweil's Graph: This graph intentionally muddles together speed and cost, but so-doing can lead others to draw the wrong conclusions from it. Currently, while there continue to be improvements in computing power, the current driver behind the continuing exponential growth of Kurzweil’s graph is economic – ie. computer hardware is being delivered cheaper, faster, not faster faster. Also, even if Kurzweil’s graph did continue for ever, it might still not lead to the singularity, in that the (infinitely cheap) computer hardware might still not deliver what Kurzweil needs. It might still be too slow. Click here for Note.

• Exponential Curve: Kurzweil’s graph is slightly more than exponential (an exponential curve would appear as a straight line given the Y-axis is logarithmic). Maybe the Timeeditor made the curve look exponential, lest we failed to get the message. But, this extra bit of hyper-exponentiality – which depends critically (it seems to me) on the last two points on the graph, has a huge impact on the date of the Singularity. If we were to fit a straight line to these points, the power in 2045 would be only 1/10,000,000,000 of that predicted by Kurzweil. But, such is exponential growth, that this would only defer the Singularity by 30 years or so. Unfortunately, while this is no-time in the grand scheme of things, this will be disappointing to those who are “waiting” for the Singularity, as it may come along too late given that this would imply it’s 64 years away. Click here for Note.

• Technological Indexes: It’s true that it’s not just micro-processor speeds that are important, and that other related technologies are always improving. The question is whether these will also hit the wall at some time. The trouble with exponentiation is that there are certain fundamental properties of the world that are not open to human manipulation. Click here for Note.

• Exponentiation Beyond IT: This should give us pause. Some of these indicators are clearly not open to indefinite exponential growth. Click here for Note.

• Smooth Curves: One would need to check this by investigating whether the smoothness is a point-selection effect. I suppose, however, that by choosing the “best of breed” at any date, the chosen points will be accurate. But dates without points may (were points to be supplied) show periods of stasis, and a less smooth curve. Click here for Note.

• Peace: This – that exponential growth continues irrespective of the state of the world – is a critical claim, as if (on my calculations) the Singularity is (even assuming all Kurzweil’s miracles take place) still 64 years away, that assumes some sort of stability is maintained for a period comparable to that between the rise of Nazism and the present. Now, traditionally wars have been stimuli for technological change – but whether this will remain so is open to doubt. Terrorism is more destructive of technological development than carpet bombing, as it can get anywhere (imagine the situation if the Nazis could have reached Los Alamos, or the industrial centres of America). Click here for Note.

• Law of Accelerating Returns: Whether returns continue to accelerate depends on the maturity of a product. In the “green fields” situation, exponentiation is possible, but eventually stasis kicks in. Consider the railways. The wonder is that exponentiation in IT has continued for so long. But it cannot last indefinitely. Click here for Note.

• Future Extrapolation: As noted passim, it’s the extrapolation of indefinite exponential growth (rather than linear growth) that causes cognitive dissonance here. Kurzweil thinks he has an answer to the dissonance, but I don’t believe it. Click here for Note.

• Evolutionary Psychology: There are two issues here. Arguments aren’t won or lost by what we’ve evolved to think. Scientists (presumably) over-ride whatever their evolved prejudices might be all the time. We’re not exactly evolved to favour curved space-time. Secondly, we may be right to intuit a suspicion of exponential growth as, in general, the environment can’t cope with it. This is at the centre of Malthusian accounts of the practical necessity of natural culls of exponential population growth. Finally, we might note that digital computers are linear, and what Kurzweil needs (ultimately) for his continued exponential growth is massive parallelism, which hasn’t been invented yet. Click here for Note.

• Reverse-engineering: Where does this claim come from? This is not a problem that can be solved by throwing hardware at it. The human brain has billions of neurons with billions of connections – fine – this might be simulated. But the contents of the brain relates to just what these connections are, and no-one has the vaguest idea how the wiring works, so how could this be simulated – especially in the next 15 years? Click here for Note.

• Singularitarian Subculture: One can a thoroughgoing naturalist, and admit that the naturalist programme will eventually get there (as it will with controlled nuclear fusion) but claim that there are numerous technical saltations between now and the Singularity that we have no warrant for supposing a near-immediate solution is available. Click here for Note.

• xxx: Click here for Note.

Previous Version of this Note:

Summary of Notes Referenced by This Note

To access information, click on one of the links in the table above.

Summary of Notes Citing This Note

To access information, click on one of the links in the table above.

Authors, Books & Papers Citing this Note

Author	Title	Medium	Extra Links	Read?
Todman (Theo)	Thesis - Androids	Paper		Yes
Todman (Theo)	Thesis - Chapter 03 (What is a Person?)	Paper		Yes
Todman (Theo)	Thesis - Chapter 10 (Thought Experiments)	Paper	2, 3, 4	Yes
Todman (Theo)	Thesis - Chapter 11 (Resurrection)	Paper		Yes
Todman (Theo)	Thesis - Cyborgs	Paper		Yes
Todman (Theo)	Thesis - Immortality	Paper		Yes
Todman (Theo)	Thesis - Intelligence	Paper		Yes
Todman (Theo)	Thesis - References	Paper		Yes
Todman (Theo)	Thesis - Transhumanism	Paper		Yes
Todman (Theo)	Thesis - Uploading	Paper		Yes

---

References & Reading List

Author	Title	Medium	Source	Read?
Kurzweil (Ray)	The Age of Spiritual Machines	Book - Cited	Kurzweil (Ray) - The Age of Spiritual Machines	No
Regis (Ed)	Great Mambo Chicken and the Transhuman Condition: Science Slightly over the Edge	Book - Cited	Regis (Ed) - Great Mambo Chicken and the Transhuman Condition: Science Slightly over the Edge	Yes

Text Colour Conventions

1. Blue: Text by me; © Theo Todman, 2026
2. Mauve: Text by correspondent(s) or other author(s); © the author(s)