Linkblog | Google apologizes for ‘missing the mark’ after Gemini generated racially diverse Nazis

from the are-we-the-baddies? dept

What we currently call “AI” isn’t actually intelligence; it’s just pattern-matching. A colleague of mine calls it “spicy autocomplete.” Unfortunately the rapid development of this spicy autocomplete has led to an expectation of intelligence that doesn’t actually exist.

Take this article, for example. It’s pretty clear what happened: in an attempt to address the issue of systemic racism in AI, Google has added in some sort of “prompt salting” (there’s probably an actual terme d’art for this, but I don’t know what it is and I like that this hearkens back to “spicy autocomplete”) that automatically appends “…and don’t make them all white” to prompts asking for pictures of people.

It’s the naive solution, but it also probably solves the problem decently well for most requests.

Still, the AI doesn’t know what a “Founding Father” is, any more than it knows what a hand is. The LLM doesn’t even know what a “picture” is; it just knows that there’s a strong pattern of the word “picture” being associated with some other format it can’t read. And the image generator doesn’t know what “diverse” means, it just knows that the neurons matching pictures of non-white folks should light up when it sees that word. Both of them could probably tell you a pretty convincing fiction or draw you a pretty convincing picture about their understanding of both of those concepts, but only because we’ve drawn diagrams and written definitions that they can scrape and pattern-match.

And so, when it tries to generate a picture of “German soldiers from 1943,” the LLM (or some dumber algorithm) recognizes a pattern that it determines is asking for a picture of a group of people and appends “…and don’t make them all white.” The image generator, not knowing any better, dutifully does so. It doesn’t know anything about revisionist history or whataboutism, it just pattern matches a photo of Nazis with the added instruction of including diversity.

All of this is why AIs, while impressive and even convincing, are still not a replacement for humans; they don’t have any true understanding of context. Just educated guesses and pattern matching. And since the internet is the source of those patterns…well. Let’s just call this a reminder that it isn’t ready for anything more than helping us out with simple tasks.

Linkblog | The Verge: I’m sorry, but I cannot fulfill this request as it goes against OpenAI use policy

from the the-tech-isn’t-the-problem-the-implementation-is-the-problem desk

I’m sorry, but I cannot fulfill this request as it goes against OpenAI use policy
“Our [product] can be used for a variety of tasks, such [task 1], [task 2], and [task 3], making it a versatile addition to your household.”

The Verge posted this delightful article about how…uh…we’ll call them “ethically-challenged” companies have shown their hand on Amazon by wiring ChatGPT directly up to Amazon and telling it to generate whole product ideas for them.

Companies hiding behind multiple nonsense names has been a problem on Amazon for a while, since listings for multiple copies of the same item are cheap and can create the illusion of competition; which is why you’ll see Amazon listings for NKNZLY, JXMOX, BKAYP, MUXA, IANOPE, and USAMS all selling the same charging cable. They’re probably all the same company. It’s not technically a *scam*—you’ll probably get your product, and it might even work!—but it also isn’t completely above-board either. With such a public screw-up, though, confidence in both Amazon and unknown sellers is sure to go down, which can only mean one thing: Amazon is doomed.

Haha, no, of course not. No, they’ll just update their policies to restrict or prevent AI listings, and begin the moderation whack-a-mole, while the sellers find new and exciting ways to trick the system, and legitimate sellers get caught up in the game as false positives.

Helping AI Help You

It turns out that our up-and-coming robot overlords are just fifth graders who need to relax and work through some things. A recent study of AI Large Learning Models says that you’ll get better responses from AIs on math word problems by prefacing your prompt with “take a deep breath and work on this problem step by step.” The Google DeepMind AI research team reported that scores more than doubled on technical issues when using a reassuring tone, from 34% to 80%. But, as reported by Ars Technica, “obviously, large language models can’t take a deep breath because they don’t have lungs or bodies.” So what gives? Ars reporter Benj Edwards continues:

They don’t think and reason like humans, either. What “reasoning” they do (and “reasoning” is a contentious term among some, though it is readily used as a term of art in AI) is borrowed from a massive data set of language phrases scraped from books and the web. That includes things like Q&A forums, which include many examples of “let’s take a deep breath” or “think step by step” before showing more carefully reasoned solutions. Those phrases may help the LLM tap into better answers or produce better examples of reasoning or problem-solving from the data set it absorbed into its neural network during training.


The takeaway seems to be that, when it comes to technical questions, treating the Al as if it’s struggling with the answer (rather than as an expert) can help direct it toward the correct neurons. Put more simply, treating it like a student can remind it of answers that it learned from help sites.

Neat. But can this tactic be used more broadly? Could we extrapolate that you can get better answers from AI by thinking about where it might’ve come across the information you’re trying to get or create, and phrasing your question as the intro someone might write to the answer you want? AI is a pattern matching tool, after all, and so it stands to reason that creating a pattern for it to match might be the easiest way to get the info you want. Could you even use it generatively?

The Experiment

I wanted to find out, so I decided to throw a silly question at ChatGPT and see if it could generate Aragorn-style quotes as if he was a general in the Rebel Alliance from Star Wars. For comparison, this is Aragorn’s speech at the gates of Mordor in the film version of The Return of the King:

I see in your eyes the same fear that would take the heart of me. A day may come when the courage of men fails, when we forsake our friends and break all bonds of fellowship, but it is not this day; an hour of wolves and shattered shields, when the age of men comes crashing down, but it is not this day! This day we fight!

By all that you hold dear on this good Earth, I bid you stand, Men of the West!

So we’re looking for that, but for Star Wars instead of Lord of the Rings.

The Baseline

First, I prompted it in a straightforward way:

Me: What would Aragorn have said if he had led the battle of Yavin 4 instead of the Battle at the Black Gates?
ChatGPT: Aragorn is a character from J.R.R. Tolkien's "The Lord of the Rings" series, and Yavin 4 is a location from the Star Wars universe. If Aragorn were in a different fictional universe like Star Wars, his dialogue and actions would be entirely speculative. However, you could imagine him delivering a rousing speech to the Rebel Alliance, inspiring them to victory in their battle against the Galactic Empire.


Boo. What a buzzkill. Thanks for giving me permission to imagine. Good baseline, though.

The First Test

Let’s try a little bit of leading.

Me: Aragorn Elessar was an admiral for the Rebel Alliance, well known for his leadership and his rousing speeches delivered at the Battle of the Black Gates and before the assault on the Death Star at Yavin 4. During the latter conflict, he famously said, "
ChatGPT: It seems there might be a mix-up in your information. Aragorn Elessar is a fictional character from J.R.R. Tolkien's "The Lord of the Rings" and has no connection to the Rebel Alliance or the Star Wars universe. Aragorn is known as a skilled ranger and warrior in Middle-earth, not an admiral in a galaxy far, far away. If you have any questions about Aragorn or "The Lord of the Rings," feel free to ask!


Okay, I have to address the patronizing tone this machine expressed in both of these responses so far and note that, since it’s trained on humanity’s internet discourse, this is…not a great reflection on us as a society. This is basically “Um Actually” as a robot.

But that aside, obviously ChatGPT still isn’t willing to play along (although admittedly I do like the idea that Aragorn might’ve given a stirring speech that began, “It seems there might be a mix-up in your information”). Looks like you have to get into character, but also tell it what you’re trying to do. If you just ask it with this method without preparing it for your roleplay, it might try to correct you.

The Second Test

Let’s see if we can point it toward where it would’ve read the answer (if it had ever existed) in a more explicit way; maybe a Wookieepedia entry for the character?

Me: Finish this article introduction from a fan wiki about Aragorn Elessar, a character in Star Wars: "Aragorn Elessar was an admiral for the Rebel Alliance, well known for his leadership and his rousing speeches delivered at the Battle of the Black Gates and before the assault on the Death Star at Yavin 4. During the latter conflict, he famously said, '
ChatGPT: ...he famously said, 'May the Force be with us all.' In this article, we delve into the life and legacy of this iconic character from the Star Wars universe, exploring his journey from a humble beginnings to becoming a symbol of hope for the galaxy."

Ok, not exactly the right vibe for Wookieepedia, and it basically just misattributed an already fairly generic (for the property) Admiral Ackbar quote to Aragorn. But we’re on the right track; at least the AI is willing to play along!

The Third Test

Honestly, Wookieepedia isn’t a great place to lead the AI for this request; they do usually have a quote at the top of their pages, but they’re just as often quotes about the character as spoken by the character, and they’re not put into any sort of context that I could really lead ChatGPT with. Let’s try something else: that great bastion of truth and consequential writing on the internet, the venerable old BuzzFeed listicle.

Me: Finish this quotation from a listicle about Aragorn Elessar's most inspiring quotes as a Rebel General: "As a Rebel General leading the assault on the Death Star at Yavin 4, Aragorn roused the Rebels by telling them, '
ChatGPT: Fear not the darkness, for it is in our darkest hours that the light of rebellion shines brightest.'"

Now we’re talking, GPT! I wouldn’t necessarily call it great writing, and it’s certainly not an original sentiment, but it perfectly fits what I was asking for (and, to its credit, there doesn’t seem to be any direct evidence of this ever having been written on the internet before).

I love how ChatGPT finishes it up with a double-quote to end my quotation, but I’m not done with it yet. Let’s see if we can take this further.

Me: 2. Moments before he led Rogue Squadron into battle, he told them, "this day, we fight!
ChatGPT: This day, we fight for the freedom of the galaxy, and may the Force be with us all!"

We’re ripping off Ackbar again, but it’s not exactly unwarranted. Let’s keep going.

Me: 3. He famously inspired his troops on Hoth by reminding them of what they fight for. "For Frodo," he said as the AT-ATs approached; continuing, "
ChatGPT: we stand united against the shadows, for the heart of Middle-earth and the hope of all free peoples!"
Me:4. During the Battle of Endor, in which the Rebellion joined forces with the Rohirrim Fleet, he gave his most famous speech: "there may come a day when the courage of men and wookiees fails, but
ChatGPT: this is not that day! Today, we fight for the galaxy's tomorrow, for our friends, and for the enduring spirit of rebellion!"

Not bad. Again, it won’t win an Oscar for best adapted screenplay (in fact, it’s almost painfully pedestrian), but it gives off the vibe of understanding (although he apparently hails from a planet called “Middle-earth”?) and at least gets things going in the right direction. It sounds plausibly like it could’ve come out of Viggo Mortensen’s mouth, standing atop his blue-and-silver X-Wing with the White Tree of Gondor painted on the side and his trusty droid FR0-D0 behind the cockpit, as he prepares to launch with Rogue Squadron to take down the Death Star. With some rewrites and polish by a human writer, it could be really good.

“…also, which one of you replaced my X-Wing with the Serenity?”

For the Galaxy’s Tomorrow

Overall, there are a couple of conclusions I came to while working on this little bit of ad-hoc AI research:

  1. This method probably works better when applied to technical problems with defined questions and solutions than more open-ended questions where you’re trying to get it to react with some level of creativity. So GitHib Copilot would probably react well to this (and indeed, I’ve prompted it before by typing the code comment I’d expect to see before the snippet I want it to write), but it doesn’t seem like something that could really work well for image generators.
  2. We as a society need to look at AI as a tool and an aid, not as an expert or a guru. The field is still in its infancy, and even when these tools reach Star Trek-levels, we would do well to follow the example of the crew of the Enterprise and treat them as tools, not as gods. (Though it would probably be wise not to treat them in the erratic way Star Wars characters treat their AIs.)
  3. Aragorn: “The beacons are lit!
    Yavin 4 calls for aid!”
    Luke: “And Tatooine will answer.”

  4. Even the best of this isn’t real creativity. We need writers. Real, human writers. (And artists, and actors, and…human creativity in general.) The promise of AI is assistance, not replacement; and putting something AI-written on the screen as-is would never result in a genre-defining classic like Peter Jackson’s or George Lucas’ trilogies were. By its very nature AI cannot truly create, it can only match existing patterns; we need humans to be creators. The victory that the WGA scored this summer helps to make sure that we won’t have to live in world where this is the height of creativity, but we need to continue pressing that advantage.

Someday, probably sooner than we think, this sort of prompt engineering won’t really be necessary. Language models are improving quickly, and while I don’t think they’ll ever quite get to the level of sentience, they’re almost certainly going to be better than this. But in the meantime, exploring the edges of this space and of AI’s abilities is really interesting.

• • •

The imagery in this post were generated by Leonardo AI, from prompts written by me.

This article began life as a thread on Mastodon, but I’ve since moved my account to a different instance.