Month: October 2023

No comments

Helping AI Help You

  By

It turns out that our up-and-coming robot overlords are just fifth graders who need to relax and work through some things. A recent study of AI Large Learning Models says that you’ll get better responses from AIs on math word problems by prefacing your prompt with “take a deep breath and work on this problem step by step.” The Google DeepMind AI research team reported that scores more than doubled on technical issues when using a reassuring tone, from 34% to 80%. But, as reported by Ars Technica, “obviously, large language models can’t take a deep breath because they don’t have lungs or bodies.” So what gives? Ars reporter Benj Edwards continues:

They don’t think and reason like humans, either. What “reasoning” they do (and “reasoning” is a contentious term among some, though it is readily used as a term of art in AI) is borrowed from a massive data set of language phrases scraped from books and the web. That includes things like Q&A forums, which include many examples of “let’s take a deep breath” or “think step by step” before showing more carefully reasoned solutions. Those phrases may help the LLM tap into better answers or produce better examples of reasoning or problem-solving from the data set it absorbed into its neural network during training.

WHY DO I CARRY THE 3, THOUGH?
CARRY IT WHERE? TO MORDOR?

The takeaway seems to be that, when it comes to technical questions, treating the Al as if it’s struggling with the answer (rather than as an expert) can help direct it toward the correct neurons. Put more simply, treating it like a student can remind it of answers that it learned from help sites.

Neat. But can this tactic be used more broadly? Could we extrapolate that you can get better answers from AI by thinking about where it might’ve come across the information you’re trying to get or create, and phrasing your question as the intro someone might write to the answer you want? AI is a pattern matching tool, after all, and so it stands to reason that creating a pattern for it to match might be the easiest way to get the info you want. Could you even use it generatively?

The Experiment

I wanted to find out, so I decided to throw a silly question at ChatGPT and see if it could generate Aragorn-style quotes as if he was a general in the Rebel Alliance from Star Wars. For comparison, this is Aragorn’s speech at the gates of Mordor in the film version of The Return of the King:

I see in your eyes the same fear that would take the heart of me. A day may come when the courage of men fails, when we forsake our friends and break all bonds of fellowship, but it is not this day; an hour of wolves and shattered shields, when the age of men comes crashing down, but it is not this day! This day we fight!

By all that you hold dear on this good Earth, I bid you stand, Men of the West!

So we’re looking for that, but for Star Wars instead of Lord of the Rings.

The Baseline

First, I prompted it in a straightforward way:

Me: What would Aragorn have said if he had led the battle of Yavin 4 instead of the Battle at the Black Gates?
ChatGPT: Aragorn is a character from J.R.R. Tolkien's "The Lord of the Rings" series, and Yavin 4 is a location from the Star Wars universe. If Aragorn were in a different fictional universe like Star Wars, his dialogue and actions would be entirely speculative. However, you could imagine him delivering a rousing speech to the Rebel Alliance, inspiring them to victory in their battle against the Galactic Empire.

DENIED.

Boo. What a buzzkill. Thanks for giving me permission to imagine. Good baseline, though.

The First Test

Let’s try a little bit of leading.

Me: Aragorn Elessar was an admiral for the Rebel Alliance, well known for his leadership and his rousing speeches delivered at the Battle of the Black Gates and before the assault on the Death Star at Yavin 4. During the latter conflict, he famously said, "
ChatGPT: It seems there might be a mix-up in your information. Aragorn Elessar is a fictional character from J.R.R. Tolkien's "The Lord of the Rings" and has no connection to the Rebel Alliance or the Star Wars universe. Aragorn is known as a skilled ranger and warrior in Middle-earth, not an admiral in a galaxy far, far away. If you have any questions about Aragorn or "The Lord of the Rings," feel free to ask!

UM ACTUALLY I THINK YOU'LL FIND THAT ARAGORN ISN'T EVEN A CHARACTER IN STAR WARS

Okay, I have to address the patronizing tone this machine expressed in both of these responses so far and note that, since it’s trained on humanity’s internet discourse, this is…not a great reflection on us as a society. This is basically “Um Actually” as a robot.

But that aside, obviously ChatGPT still isn’t willing to play along (although admittedly I do like the idea that Aragorn might’ve given a stirring speech that began, “It seems there might be a mix-up in your information”). Looks like you have to get into character, but also tell it what you’re trying to do. If you just ask it with this method without preparing it for your roleplay, it might try to correct you.

The Second Test

Let’s see if we can point it toward where it would’ve read the answer (if it had ever existed) in a more explicit way; maybe a Wookieepedia entry for the character?

Me: Finish this article introduction from a fan wiki about Aragorn Elessar, a character in Star Wars: "Aragorn Elessar was an admiral for the Rebel Alliance, well known for his leadership and his rousing speeches delivered at the Battle of the Black Gates and before the assault on the Death Star at Yavin 4. During the latter conflict, he famously said, '
ChatGPT: ...he famously said, 'May the Force be with us all.' In this article, we delve into the life and legacy of this iconic character from the Star Wars universe, exploring his journey from a humble beginnings to becoming a symbol of hope for the galaxy."

Ok, not exactly the right vibe for Wookieepedia, and it basically just misattributed an already fairly generic (for the property) Admiral Ackbar quote to Aragorn. But we’re on the right track; at least the AI is willing to play along!

The Third Test

Honestly, Wookieepedia isn’t a great place to lead the AI for this request; they do usually have a quote at the top of their pages, but they’re just as often quotes about the character as spoken by the character, and they’re not put into any sort of context that I could really lead ChatGPT with. Let’s try something else: that great bastion of truth and consequential writing on the internet, the venerable old BuzzFeed listicle.

Me: Finish this quotation from a listicle about Aragorn Elessar's most inspiring quotes as a Rebel General: "As a Rebel General leading the assault on the Death Star at Yavin 4, Aragorn roused the Rebels by telling them, '
ChatGPT: Fear not the darkness, for it is in our darkest hours that the light of rebellion shines brightest.'"

Now we’re talking, GPT! I wouldn’t necessarily call it great writing, and it’s certainly not an original sentiment, but it perfectly fits what I was asking for (and, to its credit, there doesn’t seem to be any direct evidence of this ever having been written on the internet before).

I love how ChatGPT finishes it up with a double-quote to end my quotation, but I’m not done with it yet. Let’s see if we can take this further.

Me: 2. Moments before he led Rogue Squadron into battle, he told them, "this day, we fight!
ChatGPT: This day, we fight for the freedom of the galaxy, and may the Force be with us all!"

We’re ripping off Ackbar again, but it’s not exactly unwarranted. Let’s keep going.

Me: 3. He famously inspired his troops on Hoth by reminding them of what they fight for. "For Frodo," he said as the AT-ATs approached; continuing, "
ChatGPT: we stand united against the shadows, for the heart of Middle-earth and the hope of all free peoples!"
Me:4. During the Battle of Endor, in which the Rebellion joined forces with the Rohirrim Fleet, he gave his most famous speech: "there may come a day when the courage of men and wookiees fails, but
ChatGPT: this is not that day! Today, we fight for the galaxy's tomorrow, for our friends, and for the enduring spirit of rebellion!"

Not bad. Again, it won’t win an Oscar for best adapted screenplay (in fact, it’s almost painfully pedestrian), but it gives off the vibe of understanding (although he apparently hails from a planet called “Middle-earth”?) and at least gets things going in the right direction. It sounds plausibly like it could’ve come out of Viggo Mortensen’s mouth, standing atop his blue-and-silver X-Wing with the White Tree of Gondor painted on the side and his trusty droid FR0-D0 behind the cockpit, as he prepares to launch with Rogue Squadron to take down the Death Star. With some rewrites and polish by a human writer, it could be really good.

“…also, which one of you replaced my X-Wing with the Serenity?”

For the Galaxy’s Tomorrow

Overall, there are a couple of conclusions I came to while working on this little bit of ad-hoc AI research:

  1. This method probably works better when applied to technical problems with defined questions and solutions than more open-ended questions where you’re trying to get it to react with some level of creativity. So GitHib Copilot would probably react well to this (and indeed, I’ve prompted it before by typing the code comment I’d expect to see before the snippet I want it to write), but it doesn’t seem like something that could really work well for image generators.
  2. We as a society need to look at AI as a tool and an aid, not as an expert or a guru. The field is still in its infancy, and even when these tools reach Star Trek-levels, we would do well to follow the example of the crew of the Enterprise and treat them as tools, not as gods. (Though it would probably be wise not to treat them in the erratic way Star Wars characters treat their AIs.)

    3. Aragorn: “The beacons are lit!
    Yavin 4 calls for aid!”
    Luke: “And Tatooine will answer.”

  3. Even the best of this isn’t real creativity. We need writers. Real, human writers. (And artists, and actors, and…human creativity in general.) The promise of AI is assistance, not replacement; and putting something AI-written on the screen as-is would never result in a genre-defining classic like Peter Jackson’s or George Lucas’ trilogies were. By its very nature AI cannot truly create, it can only match existing patterns; we need humans to be creators. The victory that the WGA scored this summer helps to make sure that we won’t have to live in world where this is the height of creativity, but we need to continue pressing that advantage.

Someday, probably sooner than we think, this sort of prompt engineering won’t really be necessary. Language models are improving quickly, and while I don’t think they’ll ever quite get to the level of sentience, they’re almost certainly going to be better than this. But in the meantime, exploring the edges of this space and of AI’s abilities is really interesting.

• • •

The imagery in this post were generated by Leonardo AI, from prompts written by me.

This article began life as a thread on Mastodon, but I’ve since moved my account to a different instance.

One Comment

The Great Moon War

  By ,

This innocuous-looking meme came up on a friend’s feed recently, with the caption “I need to know”:

My immediate thought was, “They aren’t, though!” The reality is so much cooler than that, and this joke actually asks a really interesting question with an even cooler answer.

Tidal Locking means that the same side of the Moon always faces the Earth.

So, first of all, yes, the moon is “tidally locked” with Earth, so only one side ever faces us (incidentally, this means that there’s no “Dark Side” of the Moon; only a “far side”). Tidal locking is going to come up later, so remember that. But the mares (MAH-rays)—the dark marks on the moon—are actually cooled lava flows, not craters.

And it’s actually the fact that meteors don’t hit the near side as often that is the reason we can still see the mares at all; there have, at one time or another, probably been such mares on all sides of the moon, but the others got pummeled into oblivion.

Ok, cool, but…what caused the lava flows that made the mares?

They were created when meteors hitting the far side of the moon made a huge enough impact to cause lunar volcanoes on the near side. (And if “lunar volcanoes” isn’t the coolest phrase you have heard this week, you live a pretty exciting life.)

Oh but wait, it gets better. Due to the way the Earth-Moon system works—that is, as mentioned before, that the moon is tidally-locked—there’s a good chance that, if the moon wasn’t there, those meteors would probably have hit Earth instead.

So the mares aren’t the remnants of the Earth-Moon war. They’re the battle scars of the moon taking bullets that were meant for us.

Thanks Moon.

2 Comments

My Adult ADHD Experience, Six Months Later

  By

If you’re just joining us, since April of this year I’ve been documenting my ADHD experience in the hopes that it will help anyone who’s struggling with similar issues. I began by talking about my symptoms and what I tried before admitting it might be ADHD and seeking treatment. Then I wrote an update about my mixed results with the first medication I was prescribed, followed by the more promising results from the second medication. That was in June, and it’s October now; so what’s happened over the last four months?

I’ve been continuing to take Adzenys XR-ODT since I was originally prescribed the medication. We’ve increased the dose from 3.1mg (which I’ve since discovered was actually lower than the recommended pediatric starting dose, which makes it all the more surprising that I actually had some good results while taking it) up to 9.4mg (still below the recommended adult starting dose). For comparison, Adzenys XR-ODT 9.4mg is comparable to Adderall XR 15mg (which is not currently available in most of the country). I’ve stuck with the 9.4mg dose for two months now.

The modest improvements in attention, focus, and executive function that I noticed when beginning the medication have become significant, and the effect continues to be consistent over the course of the workday rather than concentrated immediately after I take a dose. I still don’t feel manic, and I still feel like myself. The concerns about blood pressure haven’t gotten worse, despite the dosage increases; and I’ve actually been able to make some improvements in that regard since being medicated since I’ve actually been able to exercise somewhat regularly without it feeling like an impossible task. My wife has noticed externally that the improvements in my home life have continued, as well. My alertness, focus, motivation, executive function, energy levels, mood, temporal perception, memory, and cognition (the factors by which I think about my attention levels) all seem to be greatly improved, though I do think there is room for more improvement.

But probably the biggest thing that proves it’s working comes from when I haven’t taken it. There have been a few days when I’ve forgotten to take it (the early mornings are when my ADHD is the worst, since it’s been the longest since my previous dose), and on those days I notice a distinct regression in my mental capacity; most clearly in my focus and motivation, but affecting all the aspects I’ve been monitoring.

Even that, though, pales in comparison to the change that happened when my wife and I went on a mini-vacation and I forgot to take the medicine with me (early mornings, I’m tellin’ ya). Over the course of three short days, I felt myself regressing significantly; to the point where, on the last day, I could tell that my mental status was probably comparable to what it would’ve been on a bad day before medication. Do you remember Daniel Keyes’ short story Flowers for Algernon? I remember reading it (and frankly being haunted by it) in Mrs. Anderson’s sixth grade class. In the story, a man named Charlie recounts his experiences with an experimental treatment to increase his intellect and mental capacity, a procedure previously performed on a mouse named Algernon. It’s told in first person, and demonstrates his rapid transition over the course of days or weeks from a state of intellectual disability to a super-genius, and the associated increase in his confidence. But at a certain point, the mouse Algernon begins to decline in intelligence, and Charlie realizes that it will happen to him, too; for the remainder of the story, he relates his experience with the knowledge that he will lose his intellect as well, and his fear and dread as it all slips away.

That’s what it felt like to not take the medicine for a few days. I know that sounds dramatic (and, to be fair, it is), but there is a marked difference between medicated and not medicated; and I can tell more acutely when I’m not medicated.

Dramatics aside, as noted above, I do think there’s room for more improvement; I’m planning to ask for another dosage increase the next time I have the medication refilled. I’ve noticed that there’s a steep curve of improvement that tapers off after the first couple of weeks on a higher dose, and this time that curve seemed to last a bit longer before normalizing, so I think we may be close. Winter is coming, though; and I’ve had issues with seasonal affective disorder in the past, so we’ll see if I need to make adjustments for that.

I’ll continue to provide updated here; as always, thank you for your kind words of support, and I hope that people continue to be helped by these posts. Please let me know if there’s anything that you’re concerned or curious about; I’d love to share more, and I appreciate your presence with me on this journey.