Why are we so good at accumulating more information and power, but far less successful at acquiring wisdom? (Location 77)
The lesson to the apprentice—and to humanity—is clear: never summon powers you cannot control. (Location 92)
What should we do, then? The fables offer no answers, other than to wait for some god or sorcerer to save us. This, of course, is an extremely dangerous message. It encourages people to abdicate responsibility and put their faith in gods and sorcerers instead. (Location 96)
The tendency to create powerful things with unintended consequences started not with the invention of the steam engine or AI but with the invention of religion. Prophets and theologians have summoned powerful spirits that were supposed to bring love and joy but occasionally ended up flooding the world with blood. (Location 98)
human power is never the outcome of individual initiative. Power always stems from cooperation between large numbers of humans. (Location 103)
Our tendency to summon powers we cannot control stems not from individual psychology but from the unique way our species cooperates in large numbers. (Location 108)
The main argument of this book is that humankind gains enormous power by building large networks of cooperation, but the way these networks are built predisposes us to use that power unwisely. Our problem, then, is a network problem. (Location 109)
Even more specifically, it is an information problem. Information is the glue that holds networks together. But for tens of thousands of years, Sapiens built and maintained large networks by inventing and spreading fictions, fantasies, and mass delusions—about gods, about enchanted broomsticks, about AI, and about a great many other things. While each individual human is typically interested in knowing the truth about themselves and the world, large networks bind members and create order by relying on fictions and fantasies. (Location 111)
We should not assume that delusional networks are doomed to failure. If we want to prevent their triumph, we will have to do the hard work ourselves. (Location 123)
The naive view argues that by gathering and processing much more information than individuals can, big networks achieve a better understanding of medicine, physics, economics, and numerous other fields, which makes the network not only powerful but also wise. (Location 128)
This view posits that in sufficient quantities information leads to truth, and truth in turn leads to both power and wisdom. Ignorance, in contrast, seems to lead nowhere. (Location 132)
However, the naive view assumes that the antidote to most problems we encounter in gathering and processing information is gathering and processing even more information. While we are never completely safe from error, in most cases more information means greater accuracy. (Location 142)
The remedy to racism is therefore to provide people with more biological and historical facts. It may take time, but in a free market of information sooner or later truth will prevail. (Location 156)
Given enough information and enough time, we are bound to discover the truth about things ranging from viral infections to racist biases, thereby developing not only our power but also the wisdom necessary to use that power well. (Location 159)
“the biggest of Big Brothers is increasingly helpless against communications technology…. Information is the oxygen of the modern age…. It seeps through the walls topped with barbed wire. It wafts across the electrified, booby-trapped borders. Breezes of electronic beams blow through the Iron Curtain as if it was lace.”[2] (Location 163)
in 1858 an editorial in The New Englander about the invention of the telegraph stated, “It is impossible that old prejudices and hostilities should longer exist, while such an instrument has been created for an exchange of thought between all the nations of the earth.”[4] (Location 171)
“the reality is that nearly every aspect of life is getting progressively better as a result of exponentially improving technology.” (Location 177)
while a single apprentice pilfering his master’s secret spell book is likely to cause disaster, when a lot of apprentices are given free access to all the world’s information, they will not only create useful enchanted brooms but also learn to handle them wisely. (Location 183)
Every smartphone contains more information than the ancient Library of Alexandria[13] and enables its owner to instantaneously connect to billions of other people throughout the world. (Location 208)
He concluded, “The development and proliferation of AI—far from a risk that we should fear—is a moral obligation that we have to ourselves, to our children, and to our future.”[14] (Location 219)
Ray Kurzweil concurs, arguing in The Singularity Is Nearer that “AI is the pivotal technology that will allow us to meet the pressing challenges that confront us, including overcoming disease, poverty, environmental degradation, and all of our human frailties. We have a moral imperative to realize this promise of new technologies.” (Location 221)
No matter where we live, we might find ourselves cocooned by a web of unfathomable algorithms that manage our lives, reshape our politics and culture, and even reengineer our bodies and minds—while we can no longer comprehend the forces that control us, let alone stop them. (Location 243)
AI is the first technology in history that can make decisions and create new ideas by itself. All previous human inventions have empowered humans, because no matter how powerful the new tool was, the decisions about its usage remained in our hands. Knives and bombs do not themselves decide whom to kill. They are dumb tools, lacking the intelligence necessary to process information and make independent decisions. In contrast, AI can process information by itself, and thereby replace humans in decision making. AI isn’t a tool—it’s an agent. (Location 249)
Homo Deus hypothesized that if humans aren’t careful, we might dissolve within the torrent of information like a clump of earth within a gushing river, and that in the grand scheme of things humanity will turn out to have been just a ripple within the cosmic dataflow. (Location 267)
In the words of the 1848 Communist Manifesto, “The history of all hitherto existing societies is the history of class struggles. Freeman and slave, patrician and plebeian, lord and serf, guildmaster and journeyman, in a word, oppressor and oppressed stood in constant opposition to one another, carried on an uninterrupted, now hidden, now open, fight.” This binary interpretation of history implies that every human interaction is a power struggle between oppressors and oppressed. (Location 300)
Traditional religions like Christianity, Islam, and Hinduism have typically characterized humans as untrustworthy power-hungry creatures who can access the truth only thanks to the intervention of a divine intelligence. (Location 330)
If we wish to avoid relinquishing power to a charismatic leader or an inscrutable AI, we must first gain a better understanding of what information is, how it helps to build human networks, and how it relates to truth and power. (Location 348)
History isn’t the study of the past; it is the study of change. History teaches us what remains the same, what changes, and how things change. (Location 377)
by making informed choices, we can prevent the worst outcomes. (Location 385)
Information is increasingly seen by many philosophers and biologists, and even by some physicists, as the most basic building block of reality, more elementary than matter and energy. (Location 418)
When is a pigeon just a pigeon, and when is it information? (Location 470)
The naive view of information argues that objects are defined as information in the context of truth seeking. Something is information if people use it to try to discover the truth. This view links the concept of information with the concept of truth and assumes that the main role of information is to represent reality. (Location 472)
While different people, nations, or cultures may have competing beliefs and feelings, they cannot possess contradictory truths, because they all share a universal reality. Anyone who rejects universalism rejects truth. (Location 488)
Truth and reality are nevertheless different things, because no matter how truthful an account is, it can never represent reality in all its aspects. (Location 490)
The point is that even the most truthful accounts of reality can never represent it in full. There are always some aspects of reality that are neglected or distorted in every representation. Truth, then, isn’t a one-to-one representation of reality. Rather, truth is something that brings our attention to certain aspects of reality while inevitably ignoring other aspects. No account of reality is 100 percent accurate, but some accounts are nevertheless more truthful than others. (Location 531)
This idea, sometimes called the counterspeech doctrine, is associated with the U.S. Supreme Court justice Louis D. Brandeis, who wrote in Whitney v. California (1927) that the remedy to false speech is more speech and that in the long term free discussion is bound to expose falsehoods and fallacies. If all information is an attempt to represent reality, then as the amount of information in the world grows, we can expect the flood of information to expose the occasional lies and errors and to ultimately provide us with a more truthful understanding of the world. On this crucial point, this book strongly disagrees with the naive view. (Location 539)
If only people were given more information about the universe, surely they would abandon astrology altogether. (Location 548)
DNA and adrenaline thereby help to connect trillions of cells in the heart, legs, and other body parts to form a functioning network that can do remarkable things, (Location 580)
Information is something that creates new realities by connecting different points into a network. This still includes the view of information as representation. (Location 592)
New highlights added October 9, 2024 at 9:39 AM
To conclude, information sometimes represents reality, and sometimes doesn’t. But it always connects. This is its fundamental characteristic. Therefore, when examining the role of information in history, although it sometimes makes sense to ask “How well does it represent reality? Is it true or false?” often the more crucial questions are “How well does it connect people? What new network does it create?” (Location 635)
While information always connects, some types of information—from scientific books to political speeches—may strive to connect people by accurately representing certain aspects of reality. But this requires a special effort, which most information does not make. This is why the naive view is wrong to believe that creating more powerful information technology will necessarily result in a more truthful understanding of the world. If no additional steps are taken to tilt the balance in favor of truth, an increase in the amount and speed of information is likely to swamp the relatively rare and expensive truthful accounts by much more common and cheap types of information. (Location 639)
Contrary to what the naive view believes, Homo sapiens didn’t conquer the world because we are talented at turning information into an accurate map of reality. Rather, the secret of our success is that we are talented at using information to connect lots of individuals. Unfortunately, this ability often goes hand in hand with believing in lies, errors, and fantasies. (Location 646)
We Sapiens rule the world not because we are so wise but because we are the only animals that can cooperate flexibly in large numbers. (Location 658)
The Catholic Church has about 1.4 billion members. China has a population of about 1.4 billion. The global trade network connects about 8 billion Sapiens. (Location 663)
more than a few hundred individuals.[1] It takes many years and common experiences to get to know someone’s unique character and history and to cultivate ties of mutual trust and affection. Consequently, if Sapiens networks were connected only by personal human-to-human bonds, our networks would have remained very small. (Location 665)
A story can thereby serve like a central connector, with an unlimited number of outlets into which an unlimited number of people can plug. (Location 679)
People think they connect to the person, but in fact they connect to the story told about the person, and there is often a huge gulf between the two. (Location 703)
While branding campaigns are occasionally a cynical exercise of disinformation, most of the really big stories of history have been the result of emotional projections and wishful thinking. (Location 731)
Family is the strongest bond known to humans. One way that stories build trust between strangers is by making these strangers reimagine each other as family. The (Location 740)
reality. As far as we know, prior to the emergence of stories the universe contained just two levels of reality. Stories added a third. The two levels of reality that preceded storytelling are objective reality and subjective reality. (Location 765)
The information humans exchange about intersubjective things doesn’t represent anything that had already existed prior to the exchange of information; rather, the exchange of information creates these things. (Location 773)
Of all genres of stories, those that create intersubjective realities have been the most crucial for the development of large-scale human networks. (Location 812)
This Marxist view, however, is not only cynical but wrong. While materialist interests certainly played a role in the Crusades, World War I, the Iraq War, and most other human conflicts, that does not mean that religious, national, and liberal ideals played no role at all. (Location 850)
History is often shaped not by deterministic power relations, but rather by tragic mistakes that result from believing in mesmerizing but harmful stories. (Location 878)
When it comes to uniting people, fiction enjoys two inherent advantages over the truth. First, fiction can be made as simple (Location 911)
as we like, whereas the truth tends to be complicated, because the reality it is supposed to represent is complicated. (Location 912)
Telling a fictional story is lying only when you pretend that the story is a true representation of reality. Telling a fictional story isn’t lying when you avoid such pretense and acknowledge that you are trying to create a new intersubjective reality rather than represent a preexisting objective reality. (Location 930)
The genius of the U.S. Constitution is that by acknowledging that it is a legal fiction created by human beings, it was able to provide mechanisms to reach agreement on amending itself and remedying its own injustices (Location 944)
unlike the U.S. Constitution, the Ten Commandments failed to provide any amendment mechanism. There is no Eleventh Commandment that says, “You can amend commandments by a two-thirds majority vote.” (Location 951)
Having a lot of information doesn’t in and of itself guarantee either truth or order. It is a difficult process to use information to discover the truth and simultaneously use it to maintain order. What makes things worse is that these two processes are often contradictory, because it is frequently easier to maintain order through fictions. (Location 975)
That’s a major reason why the history of human information networks isn’t a triumphant march of progress. While over the generations human networks have grown increasingly powerful, they have not necessarily grown increasingly wise. If a network privileges order over truth, it can become very powerful but use that power unwisely. (Location 994)
Instead of a march of progress, the history of human information networks is a tightrope walk trying to balance truth with order. In the twenty-first century we aren’t much better at finding the right balance than our ancestors were in the Stone Age. (Location 997)
necessarily make the world a better place. It only makes the need to balance truth and order more urgent. (Location 999)
today. It is also mandatory reading for anyone wishing to understand how after two millennia of being one of the most pacifist groups in history, Jews built one of the most formidable armies in the world. Not for nothing was Bialik named Israel’s national poet.[2] (Location 1020)
When all is said and done, the essence of patriotism isn’t reciting stirring poems about the beauty of the motherland, and it certainly isn’t making hate-filled speeches against foreigners and minorities. Rather, patriotism means paying your taxes so that people on the other side of the country also enjoy the benefit of a sewage system, as well as security, education, and health care. (Location 1049)
Evolution has adapted our brains to be good at absorbing, retaining, and processing even very large quantities of information when they are shaped into a story. (Location 1068)
computers. The ability of computers to create intersubjective realities is an extension of the power of clay tablets and pieces of paper. (Location 1111)
it? Written documents were much better than human brains in recording certain types of information. But they created a new and very thorny problem: retrieval.[17] (Location 1145)
The brain is remarkably efficient in retrieving whatever information is stored in its network of tens of billions of neurons and trillions of synapses. Though our brain archives countless complex stories about our personal life, our national history, and our religious mythology, healthy people can retrieve information about any of them in less than a second. (Location 1147)
New highlights added October 9, 2024 at 12:32 PM
At the heart of the bureaucratic order, then, is the drawer. Bureaucracy seeks to solve the retrieval problem by dividing the world into drawers, and knowing which document goes into which drawer. (Location 1173)
Sewage isn’t the stuff of epic poems, but it is a test of a well-functioning state. (Location 1280)
It takes a minute to tweet allegations of bias, fraud, or corruption, and many weeks of arduous work to prove or disprove them. (Location 1290)
Authority belongs to the people who control the junctions that link the various chains. (Location 1295)
In bureaucratic societies, the lives of ordinary people are often upended by unidentified officials of an unfathomable agency for incomprehensible reasons. (Location 1381)
benevolence. All powerful information networks can do both good and ill, depending on how they are designed and used. Merely increasing the quantity of information in a network doesn’t guarantee its benevolence, or make it any easier to find the right balance between truth and order. (Location 1473)
Bureaucracy and mythology are both essential for maintaining order, and both are happy to sacrifice truth for the sake of order. (Location 1483)
New highlights added October 11, 2024 at 1:25 PM
“To err is human; to persist in error is diabolical.” (Location 1493)
In order to function, self-correcting mechanisms need legitimacy. (Location 1500)
It took centuries of hairsplitting debates among learned Jewish sages—known as rabbis—to streamline the canonical database and to decide which of the many texts in circulation would get into the Bible as the official word of Jehovah and which would be excluded. (Location 1607)
Whereas the archives of Egyptian pharaohs and Assyrian kings empowered the unfathomable kingly bureaucracy at the expense of the masses, the Jewish holy book seemed to give power to the masses, who could now hold even the most brazen leader accountable to God’s laws. (Location 1636)
Even when people agree on the sanctity of a book and on its exact wording, they can still interpret the same words in different ways. (Location 1655)
The dream of bypassing fallible human institutions through the technology of the holy book never materialized. With each iteration, the power of the rabbinical institution only increased. “Trust the infallible book” turned into “trust the humans who interpret the book.” (Location 1690)
While churches made decisions about texts, the texts themselves shaped the churches. As a key example, consider the role of women in the church. (Location 1795)
These views were reflected in texts like the First Epistle to Timothy. In one of its passages, this text, attributed to Saint Paul, says, “A woman should learn in quietness and full submission. I do not permit a woman to teach or to assume authority over a man; she must be quiet. For Adam was formed first, then Eve. And Adam was not the one deceived; it was the woman who was deceived and became a sinner. But women will be saved through childbearing—if (Location 1797)
But modern scholars as well as some ancient Christian leaders like Marcion have considered this letter a second-century forgery, ascribed to Saint Paul but actually written by someone else.[54] (Location 1801)
In opposition to 1 Timothy, during the second, third, and fourth centuries CE there were important Christian texts that saw women as equal to men, and even authorized women to occupy leadership roles, like the Gospel of Mary[55] or the Acts of Paul and Thecla. (Location 1803)
For centuries, Thecla was one of the most revered Christian saints and was seen as evidence that women could baptize, preach, and lead Christian communities.[57] (Location 1809)
Just as most Jews forgot that rabbis curated the Old Testament, so most Christians forgot that church councils curated the New Testament, and came to view it simply as the infallible word of God. (Location 1816)
New highlights added October 14, 2024 at 11:04 PM
The attempt to bypass human fallibility by investing authority in an infallible text never succeeded. (Location 1862)
By 1600, all kinds of fringe people—heretics, revolutionaries, proto-scientists—could disseminate their writings much more rapidly, widely, and easily than ever before. (Location 1878)
In fact, print allowed the rapid spread not only of scientific facts but also of religious fantasies, fake news, and conspiracy theories. (Location 1885)
The Hammer of the Witches became one of the biggest bestsellers of early modern Europe. It catered to people’s deepest fears, as well as to their lurid interest in hearing about orgies, cannibalism, child murders, and satanic conspiracies. The book had gone through eight editions by 1500, another five by 1520, and sixteen more by 1670, with many vernacular translations.[75] (Location 1939)
Such claims fueled mass hysteria, which in the sixteenth and seventeenth centuries led to the torture and execution of between 40,000 and 50,000 innocent people who were accused of witchcraft.[78] (Location 1953)
if the witch-hunters really wanted to find diabolical evil, they just had to look in the mirror. (Location 1972)
But witches became an intersubjective reality. Like money, witches were made real by exchanging information about witches. (Location 1986)
Witch hunts were a catastrophe caused by the spread of toxic information. They are a prime example of a problem that was created by information, and was made worse by more information. (Location 2032)
The history of the early modern European witch craze demonstrates that releasing barriers to the flow of information doesn’t necessarily lead to the discovery and spread of truth. (Location 2040)
It can just as easily lead to the spread of lies and fantasies and to the creation of toxic information spheres. (Location 2041)
a completely free market of ideas may incentivize the dissemination of outrage and sensationalism at the expense of truth. It is not difficult to understand why. Printers and booksellers made a lot more money from the lurid tales of The Hammer of the Witches than they did from the dull mathematics of Copernicus’s On the Revolutions of the Heavenly Spheres. (Location 2042)
The history of print and witch-hunting indicates that an unregulated information market doesn’t necessarily lead people to identify and correct their errors, because it may well prioritize outrage over truth. For truth to win, it is necessary to establish curation institutions that have the power to tilt the balance in favor of the facts. (Location 2051)
Scientific institutions do reach a broad consensus about the accuracy of certain theories—such as quantum mechanics or the theory of evolution—but only because these theories have managed to survive intense efforts to disprove them, launched not only by outsiders but by members of the institution itself. (Location 2090)
Self-correcting mechanisms are ubiquitous in nature. (Location 2099)
New highlights added October 26, 2024 at 6:47 AM
The most celebrated moments in the history of science are precisely those moments when accepted wisdom is overturned and new theories are born. (Location 2190)
While such mechanisms are vital for the pursuit of truth, they are costly in terms of maintaining order. (Location 2292)
Just as sacrificing truth for the sake of order comes with a cost, so does sacrificing order for truth. (Location 2298)
democracies believe that it is possible to maintain strong self-correcting mechanisms even in politics. Dictatorships disavow such mechanisms. (Location 2305)
a dictatorship is a centralized information network, lacking strong self-correcting mechanisms. A democracy, in contrast, is a distributed information network, possessing strong self-correcting mechanisms. (Location 2339)
In a democracy, if 99 percent of people want to dress in a particular way and worship a particular god, the remaining 1 percent should still be free to dress and worship differently. (Location 2351)
while a dictatorship is about one central information hub dictating everything, a democracy is an ongoing conversation between diverse information nodes. (Location 2362)
A democracy is not a system in which a majority of any size can decide to exterminate unpopular minorities; it is a system in which there are clear limits on the power of the center. (Location 2380)
Supporters of strongmen often don’t see this process as antidemocratic. They are genuinely baffled when told that electoral victory doesn’t grant them unlimited power. Instead, they see any check on the power of an elected government as undemocratic. However, democracy doesn’t mean majority rule; rather, it means freedom and equality for all. Democracy is a system that guarantees everyone certain liberties, which even the majority cannot take away. (Location 2404)
Biology professors claim that humans evolved from apes because the evidence supports this, (Location 2545)
Poor people have many insights about poverty that escape economics professors, and ethnic minorities understand racism in a much more profound way than people who never suffered from it, for example. (Location 2710)
The only way to have a large-scale political conversation among diverse groups of people is if people can gain some understanding of issues that they have never experienced firsthand. (Location 2715)
In a large polity, it is a crucial role of the education system and the media to inform people about things they have never faced themselves. (Location 2716)
prior to the development of modern information technology, there are no examples of large-scale democracies anywhere. It should be stressed that in many large-scale autocracies local affairs were often managed democratically. (Location 2745)
democracy and autocracy aren’t absolutes; they are part of a continuum. In the early nineteenth century, out of all large-scale human societies, the United States was probably the closest to the democratic end of the continuum. (Location 2867)
Mass media made large-scale democracy possible, rather than inevitable. (Location 2908)
And it also made possible other types of regimes. In particular, the new information technologies of the modern age opened the door for large-scale totalitarian regimes. (Location 2909)
Being killed by a rebellious subordinate was the biggest occupational hazard not just for Roman emperors but for almost all premodern autocrats. (Location 2945)
Legalism posited that humans were naturally greedy, cruel, and egotistical. It emphasized the need for strict control, argued that punishments and rewards were the most effective means of control, and insisted that state power not be curtailed by any moral consideration. Might was right, and the good of the state was the supreme good. (Location 2993)
The Bolsheviks claimed they knew how to finally end all oppression and create a perfectly just society on earth. (Location 3030)
a key tenet of totalitarian regimes is that wherever people meet and exchange information, the regime should be there too, to keep an eye on them. (Location 3084)
Altogether, some five million kulaks would be expelled from their homes by 1933. As many as thirty thousand heads of households were shot. (Location 3191)
The most important lesson Soviet parents taught their children wasn’t loyalty to the party or to Stalin. It was “keep your mouth shut.”[103] Few things in the Soviet Union were as dangerous as holding an open conversation. (Location 3233)
We tend to think of popes as medieval relics, but actually they are masters of modern technology. (Location 3283)
totalitarianism wants all information to pass through the central hub and doesn’t want any independent institutions making decisions on their own. (Location 3294)
“Americans grow up with the idea that questions lead to answers,” he said. “But Soviet citizens grew up with the idea that questions lead to trouble.” (Location 3326)
Despite—or perhaps because of—its utter lack of compassion and its callous attitude to truth, Stalinism was singularly efficient at maintaining order on a gigantic scale. (Location 3435)
Information systems can reach far with just a little truth and a lot of order. (Location 3448)
When more and more information flows to only one place, will it result in efficient control or in blocked arteries and, finally, a heart attack? (Location 3456)
The Soviets tried to close the gap by stealing and copying Western technology—which only guaranteed that they always remained several years behind. (Location 3494)
the first Soviet personal computer appeared only in 1984, at a time when in the United States people already had eleven million PCs.[126] (Location 3496)
Instead of dividing democracies from totalitarian regimes, a new Silicon Curtain may separate all humans from our unfathomable algorithmic overlords. (Location 3523)
Humans are more likely to be engaged by a hate-filled conspiracy theory than by a sermon on compassion. So in pursuit of user engagement, the algorithms made the fateful decision to spread outrage.[21] (Location 3644)
We are in danger of losing control of our future. A completely new kind of information network is emerging, controlled by the decisions and goals of an alien intelligence. (Location 3665)
Intelligence is the ability to attain goals, such as maximizing user engagement on a social media platform. Consciousness is the ability to experience subjective feelings like pain, pleasure, love, and hate. In humans and other mammals, intelligence often goes hand in hand with consciousness. (Location 3675)
99 percent of the processes in our body, from respiration to digestion, happen without any conscious decision making. (Location 3682)
Computers could potentially become more powerful members than humans. (Location 3757)
If power depends on how many members cooperate with you, how well you understand law and finance, and how capable you are of inventing new laws and new kinds of financial devices, then computers are poised to amass far more power than humans. (Location 3760)
Religions throughout history claimed a nonhuman source for their holy books; soon that might be a reality. Attractive and powerful religions might emerge whose scriptures are composed by AI. (Location 3805)
When we engage in a political debate with a computer impersonating a human, we lose twice. First, it is pointless for us to waste time in trying to change the opinions of a propaganda bot, which is just not open to persuasion. Second, the more we talk with the computer, the more we disclose about ourselves, thereby making it easier for the bot to hone its arguments and sway our views. (Location 3814)
For millennia human beings have lived inside the dreams of other humans. In the coming decades we might find ourselves living inside the dreams of an alien intelligence. (Location 3860)
For thousands of years prophets, poets, and politicians have used language to manipulate and reshape society. Now computers are learning how to do it. (Location 3866)
The computer revolution is bringing us face-to-face with Plato’s cave, with maya, with Descartes’s demon. (Location 3876)
In computer evolution, the distance from amoeba to T. rex could be covered in a decade. (Location 3926)
Organic evolution took four billion years to get from organic soup to apes on the moon. Computers may require just a few centuries to develop superintelligence, expand to planetary sizes, contract to a subatomic level, or sprawl over galactic space and time. (Location 3927)
When we write computer code, we aren’t just designing a product. We are redesigning politics, society, and culture, and so we had better have a good grasp of politics, society, and culture. (Location 3965)
A person or corporation with little money in the bank but a huge data bank of information could be the wealthiest, or most powerful, entity in the country. (Location 4033)
Taxes aim to redistribute wealth. They take a cut from the wealthiest individuals and corporations, in order to provide for everyone. However, a tax system that knows how to tax only money will soon become outdated as many transactions no longer involve money. (Location 4036)
Some people—like the engineers and executives of high-tech corporations—are way ahead of politicians and voters and are better informed than most of us about the development of AI, cryptocurrencies, social credits, and the like. Unfortunately, most of them don’t use their knowledge to help regulate the explosive potential of the new technologies. (Location 4056)
in history, time and place are crucial, and no two moments are the same. It matters a great deal that America was colonized by the Spaniards in the 1490s rather than by the Ottomans in the 1520s, (Location 4102)
Politics involves a delicate balance between truth and order. As computers become important members of our information network, they are increasingly tasked with discovering truth and maintaining order. (Location 4123)
New highlights added November 1, 2024 at 3:49 PM
Another way to understand the difference between computers and all previous technologies is that computers are fully fledged members of the information network, whereas clay tablets, printing presses, and radio sets are merely connections between members. (Location 3746)
In tax literature, “nexus” means an entity’s connection to a given jurisdiction. Traditionally, whether a corporation had nexus in a specific country depended on whether it had physical presence there, in the form of offices, research centers, shops, and so forth. (Location 4010)
States have thousands of years of experience in taxing money. They don’t know how to tax information—at least, not yet. (Location 4041)
The real power of the Securitate and the KGB was not an ability to constantly watch everyone, but rather their ability to inspire the fear that they might be watching, which made everyone extremely careful about what they said and did.[7] (Location 4201)
In a world where humans monitored humans, privacy was the default. But in a world where computers monitor humans, it may become possible for the first time in history to completely annihilate privacy. (Location 4316)
“Before Tripadvisor, the customer was only nominally king. After, he became a veritable tyrant, with the power to make or break lives.”[46] (Location 4477)
a comprehensive social credit system will annihilate privacy and effectively turn life into a never-ending job interview. (Location 4525)
Computers are consequently pushing humans toward a new kind of existence in which we are always connected and always monitored. (Location 4554)
Information isn’t truth. A total surveillance system may form a very distorted understanding of the world and of human beings. (Location 4563)
Humans are very complex beings, and benign social orders seek ways to cultivate our virtues while curtailing our negative tendencies. But social media algorithms see us, simply, as an attention mine. The algorithms reduced the multifaceted range of human emotions—hate, love, outrage, joy, confusion—into a single catchall category: engagement. (Location 4685)
war is not an emotional outbreak, a heroic adventure, or a divine punishment. War is not even a military phenomenon. Rather, war is a political tool. According to Clausewitz, military actions are utterly irrational unless they are aligned with some overarching political goal. (Location 4768)
by dividing reality into separate drawers, bureaucracy encourages the pursuit of narrow goals even when this harms the greater good. (Location 4814)
the problem with computers isn’t that they are particularly evil but that they are particularly powerful. (Location 4835)
since computers aren’t humans, we cannot rely on them to notice and flag possible misalignments. (Location 4865)
The danger of utilitarianism is that if you have a strong enough belief in a future utopia, it can become an open license to inflict terrible suffering in the present. (Location 5040)
The problem we face is not how to deprive computers of all creative agency, but rather how to steer their creativity in the right direction. (Location 5120)
Computers could be frighteningly efficient in imposing false labels on people and making sure that the labels stick. (Location 5153)
like the human network before it, the computer network might fail to find the right balance between truth and order. (Location 5289)
As Socrates taught, being able to say “I don’t know” is an essential step on the path to wisdom. (Location 5303)
Civilizations are born from the marriage of bureaucracy and mythology. (Location 5330)
On the way to creating the perfect industrial society, Stalinists and Nazis learned how to industrially murder millions of people. (Location 5365)
The great advantage of liberal democracy is that it possesses strong self-correcting mechanisms, which limit the excesses of fanaticism and preserve the ability to recognize our errors and try different courses of action. (Location 5391)
New technology doesn’t have to be a morality tale in which every golden apple contains the seeds of doom. (Location 5411)
When a computer network collects information on me, that information should be used to help me rather than manipulate me. (Location 5421)
A democratic society should never allow all its information to be concentrated in one place, no matter whether that hub is the government or a private corporation. (Location 5441)
For the survival of democracy, some inefficiency is a feature, not a bug. (Location 5446)
If democracies increase surveillance of individuals, they must simultaneously increase surveillance of governments and corporations too. (Location 5453)
In human history, oppression can take the form of either denying humans the ability to change or denying them the opportunity to rest. (Location 5468)
Just reading this paragraph, for example, is changing your brain structure a little, encouraging neurons to make new connections or abandon old links. You are already a little different from what you were when you began reading it. (Location 5489)
History is full of rigid caste systems that denied humans the ability to change, but it is also full of dictators who tried to mold humans like clay. (Location 5502)
intellectuals tend to appreciate intellectual skills more than motor and social skills. But actually, it is easier to automate chess playing than, say, dish washing. (Location 5525)
computers may outperform humans in recognizing human emotions, precisely because they have no emotions of their own. (Location 5552)
New highlights added November 1, 2024 at 11:27 PM
when we consider the likelihood that various social roles and jobs will be automated, a crucial question is what do people really want: Do they only want to solve a problem, or are they looking to establish a relationship with another conscious being? (Location 5580)
Technically, it is far easier for a robot to conduct a wedding ceremony than to drive a car. Yet many assume that human drivers should be worried about their job, while the work of human priests is safe, (Location 5590)
We regard entities as conscious not because we have proof of it but because we develop intimate relationships with them and become attached to them.[17] (Location 5600)
Progressives tend to downplay the importance of traditions and existing institutions and to believe that they know how to engineer better social structures from scratch. Conservatives tend to be more cautious. (Location 5620)
An ancient tradition may seem ridiculous and irrelevant, but abolishing it could cause unanticipated problems. (Location 5625)
The most important human skill for surviving the twenty-first century is likely to be flexibility, and democracies are more flexible than totalitarian regimes. (Location 5670)
While racism has borrowed its basic plotline from evolution, the concrete details are pure mythology. There is no biological basis for separating humanity into distinct races, and there is absolutely no biological reason to believe that one race is “pure” while another is “impure.” (Location 5700)
algorithms make decisions by relying on numerous data points, whereas humans find it very difficult to consciously reflect on a large number of data points and weigh them against each other. (Location 5809)
To function, a democracy needs to meet two conditions: it needs to enable a free public conversation on key issues, and it needs to maintain a minimum of social order and institutional trust. (Location 5910)
New highlights added November 4, 2024 at 8:28 AM
the financial system managed to protect itself for thousands of years by enacting laws against counterfeiting money. (Location 5964)
whereas flooding people with data tends to overwhelm them and therefore leads to errors, flooding AI with data tends to make it more efficient. (Location 6043)
Dissident bots will present a far bigger challenge to totalitarian regimes that have entire cemeteries in their closets and zero tolerance of criticism. (Location 6126)
As long as humanity stands united, we can build institutions that will control AI and will identify and correct algorithmic errors. Unfortunately, humanity has never been united. (Location 6224)
With only 300,000 citizens, Qatar is nevertheless pursuing ambitious foreign policy aims in the Middle East, is playing an outsized rule in the global economy, and is home to Al Jazeera, the Arab world’s most influential TV network. (Location 6254)
New highlights added November 5, 2024 at 10:12 PM
unlike industrial power, the world’s algorithmic power can be concentrated in a single hub. (Location 6425)
For centuries, new information technologies fueled the process of globalization and brought people all over the world into closer contact. Paradoxically, information technology today is so powerful it can potentially split humanity by enclosing different people in separate information cocoons, ending the idea of a single shared human reality. (Location 6488)
thinking he could win his own salvation by torturing his body was the sin of pride. (Location 6546)
A society that understands identities in terms of biological bodies should also care more about material infrastructure like sewage pipes and about the ecosystem that sustains our bodies. (Location 6561)
Game theory posits that the most dangerous situation in an arms race is when one side feels it has an advantage but that this advantage is slipping away.[40] (Location 6605)
Among humans, the precondition for cooperation isn’t similarity; it is the ability to exchange information. (Location 6613)
An out-of-control AI, just like an out-of-control virus, puts in danger humans in every nation. No human collective—whether a tribe, a nation, or the entire species—stands to benefit from letting power shift from humans to algorithms. (Location 6638)
states are not the fundamental particles of human reality, but rather the product of arduous processes of building trust and cooperation. (Location 6690)
New highlights added November 7, 2024 at 8:39 PM
The decline of war didn’t result from a divine miracle or from a metamorphosis in the laws of nature. It resulted from humans changing their own laws, myths, and institutions and making better decisions. (Location 6740)
just as overly optimistic views of history could be dangerous illusions, overly pessimistic views could become destructive self-fulfilling prophecies. (Location 6755)
New highlights added November 9, 2024 at 7:19 AM
One of the chief lessons of history is that many of the things that we consider natural and eternal are, in fact, man-made and mutable. (Location 6770)
Every old thing was once new. The only constant of history is change. (Location 6774)