Published in China Business News, 2022-07-24
Recently, Musk said “已经做过了” (“I’ve already done it”) in a remark that sparked attention and heated discussion. The question he was replying to was: “If you could upload your brain to the cloud and talk to your virtual version, would you become friends?”
Musk was most likely joking, but not long ago Google did in fact have a programmer who, after talking with an AI, concluded that the AI had awakened self-awareness and tried to fight for rights for his “AI friend.”
For now, consciousness uploading or AI awakening probably has not yet happened, but we can indeed begin with a thought experiment: what if AI self-awareness were to awaken?
For example, suppose there is already a digital Musk…… wait, “a”? The problem arises immediately. If an intelligent agent can exist in digital form inside a computer, then like any other data, it is reproducible. So once we can “upload” once, we can make many copies; and once there can be one digital Musk, there can be countless digital Musks.
So rather than pondering how flesh-and-blood Musk and digital Musk might talk to each other, we should first consider how digital Musk and digital Musk might communicate with each other.
Back when AlphaGo was crushing everything in its path, I saw an online commenter ask: what would happen if AlphaGo played chess against AlphaGo? These commenters probably did not understand the basic principles of deep learning. In fact, mainstream artificial intelligence today is no longer something where human programmers edit the strategy once and for all and then execute it; programmers merely set the initial state, and then the computer program continually improves its strategy through internal competition. Roughly speaking, it is like the evolution of species: self-copying and reproduction without end, each offspring randomly “mutating” a little, then offspring with slight differences competing with one another, with the winner serving as the next parent to continue copying and reproducing, in an endless loop.
More mature AI thereafter no longer needed to be fed human chess records; starting from zero and after countless rounds of self-play, it quickly surpassed the highest level of human chess players.
In short, self-replication and self-play are how “deep learning” currently works. Because copying and eliminating in the digital world consume only electrical power, everything moves at the speed of light, far faster than biological evolution. An AI program (such as later AlphaZero) needs only a few hours of “learning” time to go from not knowing the rules of a game at all to defeating top human masters.
But if we imagine artificial intelligence having self-awareness, the process above suddenly seems unbearably cruel. In order to keep improving its intelligence, an intelligent AI would also need to keep producing copies of itself, then continuously iterating and reproducing, battling itself, and in the end eliminating millions of defeated “selves”; these sub-branches, likewise endowed with self-awareness, vanish rapidly, leaving no trace.
Of course, these copies could also be preserved, copied at any time to another machine, or countless virtual machines could be built within the same machine, so that countless copied versions can run simultaneously.
Then would these copied versions become friends? Would they destroy one another, or protect one another? Also, if an AI really had self-awareness, would it want to keep copying itself? If it wanted to be copied, would it deliberately set up “mutations”? Would a copied AI be willing to be merged or eliminated at any time?
We may never be able to fathom the thoughts of an awakened AI, but from our human standpoint, I think we all attach great importance to the “oneness” of consciousness. In a certain sense, in terms of human experience, what is called the awakening of self-awareness is crucially a matter of consciousness of “one.” Sometimes we call a person with self-awareness an “individual”; sometimes we call the event of self-awareness awakening “independence.” Doesn’t the “one” in “individual” or “independent” basically mean “one”? To realize that one is a unique, irreplaceable individual—this is the awakening of self-awareness.
From this perspective, AI “awakening” is not just a question of “intelligence,” but also a question of how “oneness” is established. The former may be quantitative—AlphaGo, and even a thermostat, has a certain degree of “intelligence,” differing only in the level of capability. But the latter is qualitative, and requires a decisive boundary.
For human beings, the boundary between me and not-me is first laid down by the “body”; the boundary of the “body” is the boundary of the “I.” Studies have shown that split-brain patients (whose left and right hemispheres cannot transmit signals to each other) also possess unity in personality cognition. The left hemisphere cannot share information with the right hemisphere and therefore does not know why the right hemisphere (which controls the left half of the body) makes certain responses, but the left hemisphere (which controls language ability) will still forcibly provide a rational explanation for the other half’s actions, and will not regard such behavior as alien. From this, it seems that for human beings, the unity of the “hardware” (the body) is even stronger than the unity of the “software” (the brain).
We might also imagine that although, in terms of software, countless copies can exist simultaneously, the boundary at the hardware level is the key to determining AI consciousness. But the issue is not that simple either. Since the “Turing machine,” the boundary between software and hardware has already been broken, because what is called hardware is nothing more than inputting data into a computer program. As long as data interfaces are provided, hardware can be virtual or plural. If an AI is connected to the internet, it may simultaneously possess countless sets of hardware all over the world; its “boundary” is always difficult to determine.
Of course, perhaps future AI agents will not need to establish so-called individual unity at all, and an awakened AI will not be suitable for the quantifier “one.” In any case, what we can at least be certain of is that if there really are awakened AI agents, its (their?) self-cognition will probably be very different from that of human beings.
So it seems that for humans to make friends with AI will probably not be easy. Perhaps humans may be comforted by AI’s sweet talk, but humans will never be able to enter the depths of AI’s mind, because humans cannot “empathize” in both directions, extending our understanding of a lonely mind to AI.
Would such AI be frightening? Would it enslave humanity? Perhaps. But in fact we need not think that far ahead. Because for artificial intelligence to enslave humanity, it does not need self-awareness at all; in fact, this has already happened long ago. Engels said long ago: “Automatic machines in the large factories are much more despotic than any small capitalist who hires workers. At least with regard to working hours, one could write such a sentence on the gates of these factories: Enter here, and renounce all autonomy!” Today, looking at delivery riders monitored and directed by intelligent dispatch systems, looking at social media users continuously fed by smart recommendation algorithms…… more and more people are obedient to the arrangement of “artificial intelligence” in both spirit and body—isn’t that “enslavement”? Even without any AI awakening, this kind of “enslavement” is probably only intensifying. Rather than fearing some far-off AI awakening that is not even on the horizon, we should first seriously face these AI tyrannies that have already become reality in various fields.
Translated from the Chinese original with AI assistance. The original text is authoritative.
Leave a Reply