The Threat of Artificial Intelligence by Ned Desmond | Articles | First Things
As is often the case in the Digital Age, public- and private-sector applications of AI are racing ahead of anyone s ability to determine their consequences. The UC Berkeley computer scientist Stuart Russell, author of Human Compatible: Artificial Intelligence and the Problem of Control, says that the results of one such system, social media engagement algorithms, have produced a civilization-level AI catastrophe that nobody expected: the damaging polarization of society. The point of these algorithms is to maintain users attention, especially to promote click-throughs. This sounds like a customary situation in which a purveyor of goods adapts to the needs and tastes of the customer. But that s not what really happens, Russell explains:
The solution is simply to present items that the user likes to click on, right? Wrong. The solution is to change the user s preferences so that they become more predictable. A more predictable user can be fed items that they are likely to click on, thereby generating more revenue. People with more extreme political views tend to be more predictable in which items they will click on. . . . Like any rational entity, the algorithm learns how to modify the state of its environment in this case, the user s mind in order to maximize its own reward.
In other words, instead of shaping the product to the customer s needs, social media algorithms manipulate their subjects. As the process continues, small encouragements have large effects, altering tastes and interests in a slow and cunning brainwashing that reinforces extreme dispositions.
Russell warns, The consequences include . . . the dissolution of the social contract. If people can be so slyly altered, if a computer system can change their behavior without their even realizing it, the rational-choice assumptions that underlie the modern social order collapse. The very idea that we are self-aware, rational players participating in a democratic system comes into question, and with it the basic tenets of liberal political order.
This might be overstated but I don’t think so.