> This might explain why ChatGPT is so much worse at writing modern poetry (which is tightly restricted by copyright law) than it is at writing in older styles. For instance, it seems to me to be much better at writing a Samuel Johnson essay about kangaroos than it is at writing a modernist poem about same.
No, you've simply run into the RLHF mode collapse problem (https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse?commentId=tHhsnntni7WHFzR3x) interacting with byte-pair encoding (https://gwern.net/gpt-3#bpes). A GPT doesn't genuinely understand phonetics due to the preprocessing of the data destroying individual-letter information and replacing them with large half-word-sized chunks, and then during RLHF, it avoids writing anything which doesn't make use of the memorized pairs of rhymes because it's unsure what rhyming or nonrhyming poetry looks like.
(If you are skeptical, try asking ChatGPT this simple prompt: "Write a nonrhyming poem." a dozen times and count how many times it actually does so rather than rhyming. Last time I checked, the success rate was still well under 20%.)
The Ea-Nasir screenshot also shows the effects of the RLHF, I suspect. My advice would be to minimize use of GPT in favor of Claude for all historical scenarios involving anything less wholesome than _Barney & Friends_. While the model is not as good, the RLAIF of Claude seems to be a good deal less indiscriminate & crippling.
Thanks Gwern (longtime reader by the way). That does seem like a better explanation for why ChatGPT produces such hackneyed poetry. Perhaps interesting to note in that context that the only way I've been able to get it to emulate modern blank verse is with prompts like: "Please also experiment with translating some lines into another language and back again into English to increase the experimental quality of your poetic output - but do not include this process of translation in the poem itself, I want to see only the abstracted, fragmentary, cryptic results of it." That, plus "REMEMBER, DO NOT RHYME!" seems to do the trick with GPT-4.
Claude is working a bit better than GPT-3.5 right now, but it's not a huge difference. Ea-Nasir still ends up saying stuff like this: "Please understand my position. I only wish to uphold my reputation. If you could just grant me a few more days, I can obtain copper of the highest quality to fulfill Nanni's order." However Claude did a great job with the negotiation minigame:
Ea-Nasir opens with 😡 rage
Shamash-eriba responds with 🙄 dismissiveness
I roll two 6-sided dice to generate random numbers for each side's "negotiation strength" this round:
Ea-Nasir: Rolls 4
Shamash-eriba: Rolls 5
Narration:
Ea-Nasir slams his fist down angrily, his face red with frustration. "You dare insult my reputation when it is your employer Nanni who is the unreasonable one! I should toss you and your wretched caravan from my village this instant."
Shamash-eriba yawns, examining his fingernails. "Go ahead and try, it will only confirm your reputation as a swindler. Now, do you actually have any copper to sell or shall I inform Nanni this trip was a complete waste?"
Ea-Nasir sputters, at a loss for words at Shamash-eriba's casual dismissal. With a better negotiation strength roll, Shamash-eriba has the upper hand this round.
It's still an open question whether this is as good as the base model would've been; no one AFAIK has any 'jailbreak benchmarks'. I personally suspect that the results are probably worse, because you're larding on essentially irrelevant constraints or distractions. (After all, it's unlikely that back and forth translation *actually* helps increase poetic quality. Human poets don't typically do that, and it seems like a lot of work.)
Claude is still enforcing safety and censoring responses, but it's not so indiscriminate about it. So your minigame setting seems like it's avoiding being whacked by the censor, while in RLHF that would be whacked just in case.
Re: censoring, I confidently told my medieval history students that the ChatGPT content restrictions wouldn’t allow violence, but within an hour I had a student who proudly announced that he had “become a famous warrior.” Sure enough, he was severing heads left and right in his version. I had another student merge my prompt with DAN to become a peasant girl turned pickpocket. Having hundreds 20 year olds experiment with a technology will lead to surprises…
I will be experimenting more with Claude, I read your note about it being better for poetry and agree. Interesting.
Love the idea of playing into LLM as Hallucinator…and into historiography as a form of hallucination…and the argument that one form of great history-learning might be a kind of hallucinating/making-up-history. (Same argument could be made in literature: learn by translating what is on the page into what might’ve been…or as Helen Vendler put it: taking seriously “the nonexistence of what is”--i.e., thinking about the different ways that Hamlet could’ve ended as a way to see more cool things about how it did.)
Makes me think of David Liss’s, A Conspiracy of Paper, a detective story set during the crazy historical moment of the world’s first stock market crash and the shift to paper money, which I think Liss described writing (while a history grad student) because he could find no existing books that gave a real feel for the moment,
Maybe all historical fiction is a form of informed hallucination. (And, at least for me, one of the most thrilling ways to learn about history…because the context is right, even if some of the characters and their storylines are invented.)
This is fascinating!!! I'm wondering how this could be applied to a high school situation here in South Africa... I'll explain the situation... While studying Apartheid our country's curriculum for grade 9s require them to interview someone (older) that was impacted by Apartheid. This of course, can be a traumatic conversation for the person being interviewed if approached in the wrong way, and i'm not convinced that all grade 9s have the EQ to handle such conversations. Seeing these simulations play out makes me wonder how we could use them to the benefit of our students... I also have a question with regards to the gameplay commands and how that works within each 'turn'. This really is amazing, thank you!
Sep 14, 2023·edited Sep 14, 2023Liked by Benjamin Breen
This is a fantastic example of playing to the strengths of LLMs. Have you tried automatically checking the results for (the most common) historical inaccuracies?
I wonder if one really needs such long well-crafted prompts, especially for ChatGPT 4. I did a quick experiment with the prompt "I want to the player in a text-based game where I'm in Damascus in May, 1348, a city in chaos due to the plague." As a non-expert I thought it went well: https://chat.openai.com/share/f0e6ac86-2738-4fdf-b0d8-057d8a3c4cf2
Agreed, it's interesting how readily it will go into roleplaying/history simulation mode, even with a short prompt. A lot of the extra wording in my prompts is to add pedagogical value, i.e. introducing primary sources or giving guidance on how to better portray an authentic setting. The key difference is that if you ask to to create a text-based game in a short prompt like that, it will pretty readily spin off into fantasy elements. Also the language about a status bar is key for helping it remember where you are, who your character is, etc., even after the original prompt has fallen out of the context window.
I really like this idea. Maybe in a future version we can combine the idea of a knowledge graph with GPT. The knowledge graph contains all the primary sources plus the referenced people places and events that could then form the set of nodes that you can talk to. And when you talk to a person or a place or an event, it will draw on the closest primary source as a source documentation (maybe enhanced with some pre-prompting) to create a very realistic chat experience
Yes, that's exactly what I've had in mind for turning this into a web app. I'd like students to be able to select a time/place and then a set of primary sources from a list of available options, and then have those sources inspire persistent features of the simulation like inventory items, weather, other characters, events that can be triggered based on PC actions, etc. Another fun idea is that GPT and Claude are both perfectly capable of suggesting Midjourney prompts inspired by each element of the simulation. How cool would it be to have a basic UI where you can type "look at [object/place/person]" and a box would appear with an image generated on the basis of the previous description. Once GPT-4 is multimodal this image could then influence subsequent text generation. Basically Myst combined with a MUD, but educational.
Ben Brown has been building a similar web app (albeit purely for entertainment purposes) called Fiction Quest, accessible via his Patreon: https://www.patreon.com/benbrown
When players discover new locations or characters, it generates an image and shares the snapshot “moment” on a public feed here:
Why leave hallucinations to chance? ;) The prompt could tell ChatGPT to randomly insert several authoritative sounding but verifiably false facts, to give the students debunking challenges!
That's a good idea for when LLMs get better at avoiding inaccuracies. For now though, there just so many minor inaccuracies in the GPT-3.5 or Claude level simulations that it's more than enough for students to work through. I hope to get it to the level where the fact-checking is less about "vaccines didn't exist in the 14th century" and more along the lines of "I looked it up, and the book my character read was only available in Latin at the time."
I think it's better to leave it to hallucination. Students have the opportunity to identify flaws and shortcomings of LLMs, a task that is currently done by red teamers and trust & safety teams in those companies.
This is really fun. Before GPT, I had students watch the Seventh Seal, as well as have a lecture on the Black Death, and they also read a sheaf of primary sources. They then had to write a film review concerning the historical accuracy of Bergman’s film. Some of the ones paying more attention understood it was a metaphor for the time it was made…the Cold war, and anxiety about destruction of humanity.
Thanks Anna Marie. Funnily enough, I had students do roughly the same thing! There was a film assignment where they had to watch and assess the accuracy of a film about the Middle Ages -- I encouraged "The Seventh Seal" or "The Lion in Winter" but there was more enthusiasm for Timothy Chalomet in "The King" :)
As I read the comments and think more about it.... Now I'm wondering about a "creation template" that students could use to create a game. They could be guided (via prompts) to research and find historical details to enhance their game.
Great chance to compare their work and then modify to create even better results.
What a fascinating exploration! Benjamin Breen’s journey using LLMs like ChatGPT in history classes opens doors to innovative teaching methods. His insight into simulating historical settings through AI, despite acknowledged inaccuracies, underscores the potential for unique learning experiences. The emphasis on humanities in this AI-driven educational future is a refreshing perspective, especially when considering its inherently textual nature.
Breen’s comparison between a high school student’s analysis and that of a history major beautifully illustrates the depth of understanding gained through historical training. The caution about potential pitfalls for educators in the short term and the necessity to adapt teaching methods resonates profoundly. The example showcasing improved results by refining prompts demonstrates the evolving landscape of educational tools.
Thank you, Benjamin Breen, for sharing your experiences and insights, navigating the exciting yet challenging terrain of integrating AI into history education. Your efforts in exploring these uncharted territories are commendable.
Loved this article! This perspective on using LLMs, I think, sheds so much light on how AI can be integrated into our education systems in the future instead of circumvented.
I was also thinking -- would be cool as an assignment to get students to create their own simulation prompts. Lots and lots of possibilities.
Loved reading this piece. When GPT came out I thought of so many ideas. One I was actually working on is an enhanced version of what you did, where you're able to see a physical representation of the historical person you want to talk to. Maybe you can have a look! :) https://www.timeless.cool/
> This might explain why ChatGPT is so much worse at writing modern poetry (which is tightly restricted by copyright law) than it is at writing in older styles. For instance, it seems to me to be much better at writing a Samuel Johnson essay about kangaroos than it is at writing a modernist poem about same.
No, you've simply run into the RLHF mode collapse problem (https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse?commentId=tHhsnntni7WHFzR3x) interacting with byte-pair encoding (https://gwern.net/gpt-3#bpes). A GPT doesn't genuinely understand phonetics due to the preprocessing of the data destroying individual-letter information and replacing them with large half-word-sized chunks, and then during RLHF, it avoids writing anything which doesn't make use of the memorized pairs of rhymes because it's unsure what rhyming or nonrhyming poetry looks like.
(If you are skeptical, try asking ChatGPT this simple prompt: "Write a nonrhyming poem." a dozen times and count how many times it actually does so rather than rhyming. Last time I checked, the success rate was still well under 20%.)
The Ea-Nasir screenshot also shows the effects of the RLHF, I suspect. My advice would be to minimize use of GPT in favor of Claude for all historical scenarios involving anything less wholesome than _Barney & Friends_. While the model is not as good, the RLAIF of Claude seems to be a good deal less indiscriminate & crippling.
Thanks Gwern (longtime reader by the way). That does seem like a better explanation for why ChatGPT produces such hackneyed poetry. Perhaps interesting to note in that context that the only way I've been able to get it to emulate modern blank verse is with prompts like: "Please also experiment with translating some lines into another language and back again into English to increase the experimental quality of your poetic output - but do not include this process of translation in the poem itself, I want to see only the abstracted, fragmentary, cryptic results of it." That, plus "REMEMBER, DO NOT RHYME!" seems to do the trick with GPT-4.
Claude is working a bit better than GPT-3.5 right now, but it's not a huge difference. Ea-Nasir still ends up saying stuff like this: "Please understand my position. I only wish to uphold my reputation. If you could just grant me a few more days, I can obtain copper of the highest quality to fulfill Nanni's order." However Claude did a great job with the negotiation minigame:
Ea-Nasir opens with 😡 rage
Shamash-eriba responds with 🙄 dismissiveness
I roll two 6-sided dice to generate random numbers for each side's "negotiation strength" this round:
Ea-Nasir: Rolls 4
Shamash-eriba: Rolls 5
Narration:
Ea-Nasir slams his fist down angrily, his face red with frustration. "You dare insult my reputation when it is your employer Nanni who is the unreasonable one! I should toss you and your wretched caravan from my village this instant."
Shamash-eriba yawns, examining his fingernails. "Go ahead and try, it will only confirm your reputation as a swindler. Now, do you actually have any copper to sell or shall I inform Nanni this trip was a complete waste?"
Ea-Nasir sputters, at a loss for words at Shamash-eriba's casual dismissal. With a better negotiation strength roll, Shamash-eriba has the upper hand this round.
Yes, you can prompt-engineer your way around; the key seems to be to push it out of distribution - the other day I found that even some gibberish like 'a a a a a a' prefixed seemed to dramatically increase the probability of correctness (https://www.lesswrong.com/posts/oSZ2xTxEMZh9f3Yaz/llms-are-mostly-not-helped-by-filler-tokens?commentId=zXg4MWC7ygWmgDn3B). Then it seems to revert back to the base model behavior.
It's still an open question whether this is as good as the base model would've been; no one AFAIK has any 'jailbreak benchmarks'. I personally suspect that the results are probably worse, because you're larding on essentially irrelevant constraints or distractions. (After all, it's unlikely that back and forth translation *actually* helps increase poetic quality. Human poets don't typically do that, and it seems like a lot of work.)
Claude is still enforcing safety and censoring responses, but it's not so indiscriminate about it. So your minigame setting seems like it's avoiding being whacked by the censor, while in RLHF that would be whacked just in case.
Re: censoring, I confidently told my medieval history students that the ChatGPT content restrictions wouldn’t allow violence, but within an hour I had a student who proudly announced that he had “become a famous warrior.” Sure enough, he was severing heads left and right in his version. I had another student merge my prompt with DAN to become a peasant girl turned pickpocket. Having hundreds 20 year olds experiment with a technology will lead to surprises…
I will be experimenting more with Claude, I read your note about it being better for poetry and agree. Interesting.
This is fascinating - thank you. Hard not to wonder whether the talking rat has been borrowed by ChatGPT from the BBC's Horrible Histories franchise.
Love the idea of playing into LLM as Hallucinator…and into historiography as a form of hallucination…and the argument that one form of great history-learning might be a kind of hallucinating/making-up-history. (Same argument could be made in literature: learn by translating what is on the page into what might’ve been…or as Helen Vendler put it: taking seriously “the nonexistence of what is”--i.e., thinking about the different ways that Hamlet could’ve ended as a way to see more cool things about how it did.)
Makes me think of David Liss’s, A Conspiracy of Paper, a detective story set during the crazy historical moment of the world’s first stock market crash and the shift to paper money, which I think Liss described writing (while a history grad student) because he could find no existing books that gave a real feel for the moment,
Maybe all historical fiction is a form of informed hallucination. (And, at least for me, one of the most thrilling ways to learn about history…because the context is right, even if some of the characters and their storylines are invented.)
This is fascinating!!! I'm wondering how this could be applied to a high school situation here in South Africa... I'll explain the situation... While studying Apartheid our country's curriculum for grade 9s require them to interview someone (older) that was impacted by Apartheid. This of course, can be a traumatic conversation for the person being interviewed if approached in the wrong way, and i'm not convinced that all grade 9s have the EQ to handle such conversations. Seeing these simulations play out makes me wonder how we could use them to the benefit of our students... I also have a question with regards to the gameplay commands and how that works within each 'turn'. This really is amazing, thank you!
This is a fantastic example of playing to the strengths of LLMs. Have you tried automatically checking the results for (the most common) historical inaccuracies?
Very nice!
I wonder if one really needs such long well-crafted prompts, especially for ChatGPT 4. I did a quick experiment with the prompt "I want to the player in a text-based game where I'm in Damascus in May, 1348, a city in chaos due to the plague." As a non-expert I thought it went well: https://chat.openai.com/share/f0e6ac86-2738-4fdf-b0d8-057d8a3c4cf2
Agreed, it's interesting how readily it will go into roleplaying/history simulation mode, even with a short prompt. A lot of the extra wording in my prompts is to add pedagogical value, i.e. introducing primary sources or giving guidance on how to better portray an authentic setting. The key difference is that if you ask to to create a text-based game in a short prompt like that, it will pretty readily spin off into fantasy elements. Also the language about a status bar is key for helping it remember where you are, who your character is, etc., even after the original prompt has fallen out of the context window.
I really like this idea. Maybe in a future version we can combine the idea of a knowledge graph with GPT. The knowledge graph contains all the primary sources plus the referenced people places and events that could then form the set of nodes that you can talk to. And when you talk to a person or a place or an event, it will draw on the closest primary source as a source documentation (maybe enhanced with some pre-prompting) to create a very realistic chat experience
Yes, that's exactly what I've had in mind for turning this into a web app. I'd like students to be able to select a time/place and then a set of primary sources from a list of available options, and then have those sources inspire persistent features of the simulation like inventory items, weather, other characters, events that can be triggered based on PC actions, etc. Another fun idea is that GPT and Claude are both perfectly capable of suggesting Midjourney prompts inspired by each element of the simulation. How cool would it be to have a basic UI where you can type "look at [object/place/person]" and a box would appear with an image generated on the basis of the previous description. Once GPT-4 is multimodal this image could then influence subsequent text generation. Basically Myst combined with a MUD, but educational.
Ben Brown has been building a similar web app (albeit purely for entertainment purposes) called Fiction Quest, accessible via his Patreon: https://www.patreon.com/benbrown
When players discover new locations or characters, it generates an image and shares the snapshot “moment” on a public feed here:
https://feed.fiction.quest/
Why leave hallucinations to chance? ;) The prompt could tell ChatGPT to randomly insert several authoritative sounding but verifiably false facts, to give the students debunking challenges!
That's a good idea for when LLMs get better at avoiding inaccuracies. For now though, there just so many minor inaccuracies in the GPT-3.5 or Claude level simulations that it's more than enough for students to work through. I hope to get it to the level where the fact-checking is less about "vaccines didn't exist in the 14th century" and more along the lines of "I looked it up, and the book my character read was only available in Latin at the time."
I think it's better to leave it to hallucination. Students have the opportunity to identify flaws and shortcomings of LLMs, a task that is currently done by red teamers and trust & safety teams in those companies.
This is really fun. Before GPT, I had students watch the Seventh Seal, as well as have a lecture on the Black Death, and they also read a sheaf of primary sources. They then had to write a film review concerning the historical accuracy of Bergman’s film. Some of the ones paying more attention understood it was a metaphor for the time it was made…the Cold war, and anxiety about destruction of humanity.
Thanks Anna Marie. Funnily enough, I had students do roughly the same thing! There was a film assignment where they had to watch and assess the accuracy of a film about the Middle Ages -- I encouraged "The Seventh Seal" or "The Lion in Winter" but there was more enthusiasm for Timothy Chalomet in "The King" :)
I'm a life-time history teacher and fascinated by your Plague Simulators.
Would love to replicate the model in other settings / eras. But I'm new to AI.
Would you be willing to share a template that teachers could use transpose activity to another time and place?
As I read the comments and think more about it.... Now I'm wondering about a "creation template" that students could use to create a game. They could be guided (via prompts) to research and find historical details to enhance their game.
Great chance to compare their work and then modify to create even better results.
What a fascinating exploration! Benjamin Breen’s journey using LLMs like ChatGPT in history classes opens doors to innovative teaching methods. His insight into simulating historical settings through AI, despite acknowledged inaccuracies, underscores the potential for unique learning experiences. The emphasis on humanities in this AI-driven educational future is a refreshing perspective, especially when considering its inherently textual nature.
Breen’s comparison between a high school student’s analysis and that of a history major beautifully illustrates the depth of understanding gained through historical training. The caution about potential pitfalls for educators in the short term and the necessity to adapt teaching methods resonates profoundly. The example showcasing improved results by refining prompts demonstrates the evolving landscape of educational tools.
Thank you, Benjamin Breen, for sharing your experiences and insights, navigating the exciting yet challenging terrain of integrating AI into history education. Your efforts in exploring these uncharted territories are commendable.
Loved this article! This perspective on using LLMs, I think, sheds so much light on how AI can be integrated into our education systems in the future instead of circumvented.
I was also thinking -- would be cool as an assignment to get students to create their own simulation prompts. Lots and lots of possibilities.
Loved reading this piece. When GPT came out I thought of so many ideas. One I was actually working on is an enhanced version of what you did, where you're able to see a physical representation of the historical person you want to talk to. Maybe you can have a look! :) https://www.timeless.cool/