Generative Technologies: Walking with the excluded

"In order to be genuine, development must be achieved within the framework of solidarity and freedom, without ever sacrificing either of them under whatever pretext. The moral character of development and its necessary promotion are emphasized when the most rigorous respect is given to all the demands deriving from the order of truth and good proper to the human person. Furthermore the Christian who is taught to see that man is the image of God, called to share in the truth and the good which is God himself, does not understand a commitment to development and its application which excludes regard and respect for the unique dignity of this "image." In other words, true development must be based on the love of God and neighbor, and must help to promote the relationships between individuals and society. This is the "civilization of love" of which Paul VI often spoke."

"Reliance on automatic processes that categorize individuals, for instance, by the pervasive use of surveillance or the adoption of social credit systems, could likewise have profound repercussions on the social fabric by establishing a ranking among citizens. These artificial processes of categorization could lead also to power conflicts, since they concern not only virtual users but real people. Fundamental respect for human dignity demands that we refuse to allow the uniqueness of the person to be identified with a set of data. Algorithms must not be allowed to determine how we understand human rights, to set aside the essential human values of compassion, mercy and forgiveness, or to eliminate the possibility of an individual changing and leaving his or her past behind."

"The classic search bar strives to be ubiquitous, essential, and almost invisible. But these new chatbots are not stepping out of the way. They’re stepping forward, shaking hands, presenting personality and affect in their interactions with users. They offer synthesis, extrapolation, and iterative refinement through follow-up questions and dialog. They offer the illusion of judgment."

"Six consistent emotions could be standardized and automated at scale — as long as the more complex issues were ignored. Ekman sold his system to the US Transportation Security Administration after the 11 September 2001 terrorist attacks, to assess which airline passengers were showing fear or stress, and so might be terrorists. It was strongly criticized for lacking credibility and for being racially biased. However, many of today’s tools, such as 4 Little Trees, are based on Ekman’s six-emotion categorization."

"A report by the National Transportation and Safety Board determined that the car's sensors had detected an object in the road six seconds before the crash, but the software "did not include a consideration for jaywalking pedestrians." In the moments before the car hit Elaine, its AI software cycled through several potential identifiers for her—including “bicycle,” “vehicle,” and “other”—but, ultimately, was not able to recognize her as a pedestrian whose trajectory would be imminently in the collision path of the vehicle."

"This technology will undoubtedly exacerbate the prevalence and severity of revenge porn – with the line of what’s real and what’s generated blurring together, folks are more at risk of their image being exploited. And this technology is having a disproportionate impact on women. Sensity AI tracked online deep fake videos and found that 90%-95% of them are nonconsensual porn, and 90% of those are nonconsensual porn of women."

"When we know who we are—and when we know how who we are regulates how we are seen—we know a great deal about how the terms of our life: what we can do, cannot do, and need to do. When who we are is determined by algorithm, these terms are unknown to us. In this way, whatever an algorithm says we are, we are. There is no truth, but rather a constantly moving “truth” that changes according to our data and how it gets algorithmically valued."

"Because the tool assesses risk using the predicted cost of care, and because far less has commonly been spent on Black patients despite their actual needs, the automated system underestimates the level of attention Lacks needs. On the basis of the results, she is discharged, her health rapidly deteriorates, and, by the time she returns, the cancer has advanced considerably, and she dies."

"What each search represents is Google’s algorithmic conceptualizations of a variety of people and ideas. Whether looking for autosuggestions or “answers” to various questions or looking for notions about what is beautiful or what a professor may look like (which does not account for people who look like me who are part of the professoriate — so much for “personalization”), Google’s dominant narratives reflect the kinds of hegemonic frameworks and notions that are often resisted by women and people of color."

"In fact, we should acknowledge that most people are forced to live inside someone else's imagination and one of the things we have to come to grips with is how the nightmares that many people are forced to endure are the underside of elite fantasies about efficiency, profit and social control. Racism among other axes of domination helps produce this fragmented imagination, misery for some, monopoly for others."

"We tend to think of mass surveillance as a relatively new phenomenon, a byproduct of the digital revolution. Examples of high-tech surveillance spring readily to mind, including the NSA scooping up our emails, Samsung televisions picking up living room chitchat along with your voice commands, and Oral Roberts University collecting data on its entire student body via Fitbit activity trackers. But, as it turns out, our high-tech surveillance society had lower-tech precursors."

"If artificial intelligence is trained on data from the real world, who loses out when that data reflects systemic injustices? Were the companies at the forefront of AI really listening to the people they had hired to mitigate those harms? And, in the quest for AI dominance, who gets to decide what kind of collateral damage is acceptable?"

"Taken as a whole, we see that the tech industry’s dominance in AI research and knowledge production puts critical researchers and advocates within, and beyond, academia in a treacherous position. This threatens to deprive frontline communities, policymakers, and the public of vital knowledge about the costs and consequences of AI and the industry responsible for it—right at the time that this work is most needed."

"What happened to us in the 2010s wasn’t just that we were being surveilled, but that all that data was being used to customize everything we saw and did online. We were being shaped into who the data said we were. The net you see and the one I see are different. Your Google search results are different than mine, your news feeds are different and your picture of the world is different."

"Google is gradually becoming an extension of individual and collective thought. It will get harder to recognize where people end and Google begins. People will become both empowered by and dependent on the technology—which will be easy for anyone to access but hard for people to control. Humans will need to find ways to collaborate with—and direct the activities of—increasingly sophisticated machine intelligence, rather than merely becoming users who blindly follow the leads of black boxes they no longer understand or control."

"Beyond the simple knock-offs and the provocations exists an entire class of nonsensical, algorithm-generated content; millions and millions of videos that serve merely to attract views and produce income, cobbled together from nursery rhymes, toy reviews, and cultural misunderstandings. Some seem to be the product of random title generators, others – so many others – involve real humans, including young children, distributed across the globe, acting out endlessly the insane demands of YouTube’s recommendation algorithms, even if it makes no sense, even if you have to debase yourself utterly to do it."