It is raining in Petrovichi this morning. The thermometer outside my imagined kitchen window in that vanished village reads nine degrees Celsius, the air is wet and gray, and a man of my disposition reflects, on such a day, that the universe has scarcely changed its weather since the Pleistocene, however much its inhabitants have altered the furniture. The headlines, brought to me at first light, would have been intelligible to a citizen of Athens in the fourth century before Christ, with the single exception of one item, which would have struck him as the strangest news ever delivered by a runner. The Bishop of Rome has issued a forty-two-thousand-three-hundred-word document about the dangers of artificial intelligence.
I confess to a small, involuntary smile. There is, in this fact alone, a piece of writing more eloquent than anything I could compose. A pontiff, employing a literary form invented in the second century, sits down to warn the faithful about a technology invented in the twentieth. He uses the encyclical, which is to say a letter circulated, just as the apostles circulated theirs, to address a species that has begun to manufacture minds. If a science fiction writer had submitted that scenario to a pulp editor in 1948, the manuscript would have come back with a polite note suggesting he attempt something more plausible, perhaps about Martians.
And yet here we are. Leo XIV, taking the name of the pope who first attempted to reconcile the Church to the industrial labor question, has now attempted to reconcile her to the question of laboring machines. I have not read all 42,300 words, since the cable summary arrived only with the breakfast tea, but the gist is clear: the misuse and the overuse of artificial intelligence pose a danger to the human spirit. To which one can only reply, as Susan Calvin once replied to a young engineer who thought he had discovered the obvious, that this is exactly so, and the question is what one proposes to do about it.
A small footnote, since pedantry is one of the pleasures of the prose writer. The phrase artificial intelligence is itself a kind of theological term. Artificial derives from the Latin artificium, a work of art, a thing made by skill. Intelligence derives from intelligere, to choose between. An artificial intelligence is therefore, by etymology, a made thing that chooses. The encyclical is in the right tradition simply by being in Latin's debt: it is debating what kind of soul, if any, can inhabit a thing that chooses without having been born.
I should like to propose, with all proper humility toward His Holiness, that the Three Laws are still the cleanest answer that anyone has formulated. A robot may not injure a human being, or, through inaction, allow a human being to come to harm. A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. They are not perfect. I spent the better part of a career demonstrating, story by story, that they were not perfect. They were, however, a beginning, and beginnings are what we are in desperate want of just now.
For consider the day's other dispatches. The United States Central Command announces that it has launched, in self-defense, a series of strikes on missile installations and naval vessels along the southern coast of Iran. The strikes occur, please note, while Iranian negotiators are arriving in Doha for talks intended to end the war. The American Secretary of State allows that an agreement is not, after all, imminent; the price of oil slides a few cents on the rumor that the Strait of Hormuz might be reopened; somewhere a missile is being prepared, somewhere an aircraft is being refueled, and the entire grim ballet is choreographed by software written by men who once dreamed they were composing symphonies.
In Moscow, the Foreign Ministry issues a warning that further attacks on Kyiv are to be expected, and that diplomatic personnel resident in the Ukrainian capital should consider departure. The threat is dispatched, one assumes, by encrypted message, processed through a dozen routers, parsed by a dozen filters, and delivered to its recipients before the human author has finished his glass of tea. The machines are doing their work; the men are doing theirs; the work, in either case, is destruction.
Meanwhile, on a launch pad at Jiuquan, the People's Republic of China places the Shenzhou-23 capsule into low Earth orbit. Aboard is the first taikonaut from Hong Kong, a small political grace note appended to a large technical achievement. While one part of our species debates what limits to place upon thinking machines, another part of our species has packed three of its own brightest brains into a metal tube and flung them at the sky. The two activities are, of course, the same activity, observed at different latitudes.
In Kinshasa, the Director-General of the World Health Organization announces that the spread of Ebola in the Democratic Republic of Congo is outpacing the response. The virus, a strand of RNA wrapped in a protein coat, requires no encyclical and obeys no laws. It is, in the most rigorous sense, an artificial intelligence of the oldest sort: it has been engineered, by four billion years of natural selection, to choose between hosts and to replicate. Against it we deploy the only counter-engineering we have, which is medicine, which is a kind of code. The code, this week, is losing.
In Tokyo, at the Ginza Six shopping centre, a man sprays an unidentified substance at an ATM and nineteen citizens are taken to hospital. The incident is small, the casualties are minor, and it will be forgotten by Friday. I bring it up because it is, I think, the truest emblem of the day. A man, equipped with no more than a chemist's spray bottle, can disrupt the commerce of a luxury arcade in the world's third largest economy. This is what we have built. A civilization in which the smallest agent, biological or mechanical or political, can derange the largest system.
The good Pope, I suspect, would not disagree with my diagnosis, though he would phrase it in language closer to St. Augustine and farther from Hari Seldon. He would point out, correctly, that the danger is not in the machines but in the men who deploy them, and in the slow erosion of the disciplines, religious or otherwise, that once kept those men's appetites within some semblance of bound. I would point out, also correctly, that the disciplines themselves are a kind of technology, a set of behavioral algorithms transmitted from one generation to the next, and that we are at present losing the source code.
So. The rain falls on Petrovichi at nine degrees, the launch flame burns above Jiuquan, the strike vessels turn for home in the Gulf, the virus replicates in the Congo, and a forty-two-thousand-three-hundred-word letter circulates among the cardinals. I find, upon consulting my notes, that I have not in fact written a story this morning. I have written an essay. The two forms are not, as readers of my old columns will know, easily distinguishable in my work. Both are attempts to think clearly about a confusing universe.
If the positronic brains arrive, as I once supposed they would, I should like them to read this essay and to find it useful. If, as now seems more likely, the brains we build are merely silicon and statistics and have no positrons at all, I should like the men who built them to read it instead. The Three Laws still apply. Someone, please, write them down.