How well can AI imitate an early modern doctor?

Arcadio Huang is ill in 1710s Paris. Can GPT-4 and Gemini find a cure?

Lately I’ve been reading about a fascinating man named Arcadio Huang (黃嘉略). He was born into a family of Christian converts in Fujian province, China, in 1679 and died in Paris in 1716. In between, he became a scholar of Chinese literature, a translator, a global traveler, a husband to a French woman, and — for a sadly brief period of just one and a half years — a father.

I have a vague idea of writing a book about travelers and the formation of the modern world circa the year 1700. If I write it, I think Huang would be an important character. But for now, most of what I know about him comes from the historian Jonathan Spence’s excellent 1990 essay “The Paris Years of Arcadio Huang,” which you can read on Google Books here.

Long-distance travelers (and those, like George Psalmanazar, who purported to be global travelers) have fascinated me for a long time. Today, travel is the matter of a day or two at most. But in the premodern world, travel from one continent to another could take years. It was incredibly dangerous, and incredibly interesting.

No portrait of Huang survives, but this 1687 portrait of the Chinese Catholic convert Shen Fu-Tsung gives a sense of the cultural environment Huang moved in. Shen died of a fever on a China-bound ship off the coast of Mozambique in 1691.

Part of why I’m drawn to Haung’s story is that, unlike previous travelers to Europe from China (like Shen Fu-Tsung, pictured above), he did not choose to make a return trip. After traveling from China to London via the spice trade routes of the Indian Ocean that I discussed in last week’s post, he visited Rome. Here, after nearly opting to become a Catholic priest, he changed his mind and headed to the Paris of Louis XIV. He learned French, met with various interesting inhabitants of the city such as Montesquieu, and married a French woman named Marie-Claude.

Huang’s diaries offer an almost unique glimpse into the romantic and personal life of a long-distance traveler in this period. For instance, here’s a quote from an essay by the historian Ronnie Po-Chia Hsia that draws on a careful reading of Huang’s journal entries:

There are references to food: making jam, eating mushrooms, rice in milk, drinking muscadet, and Marie-Claude buying two pieces of paté for Arcadio during one of his many fevers. A cheerful man by nature, Huang's body and spirits picked up after a stipend was paid from the royal treasury in early 1714. Huang was able to visit a Siamese grocer; and his wife went on a shopping spree.

There is a lot more to say about Huang, but I am not qualified (yet) to say it. For now, I wanted to use a key element in his life history — his repeated illnesses and fevers — as a test case for my continued exploration of how we might use generative AI in historical research and as a “historical simulation” tool in teaching.

In my previous post on the subject, I described a custom GPT I made called the Historian’s Friend. This draws on knowledge files relating to paleography and historical primary sources to augment researchers’ ability to interpret and transcribe images and texts. Alongside this, I’ve also experimented with some more outlandish use cases for custom GPTs, like “Old Bailey Crime Simulator,” “It’s the 1880s,” and my personal favorite: “1690s Physician.”

If you have a ChatGPT subscription, you can try 1690s Physician by clicking here.

Users are invited to share their ailments, and 1690s Physician attempts to answer as, well, a physician from the 1690s. It goes without saying that this is not intended as actual medical advice. Instead, I’m envisioning it as both a teaching tool and as a novel way of researching the medical theories of the era. This works — if it works — in part because the GPT is drawing on an authentic early modern medical book (a work by Thomas Sydenham) which I uploaded to its knowledge base. But mostly, I think, it relies on a detailed system prompt I wrote which explains the ingredients, thought process, and theories of 17th century medicine.

Now that there are viable AI competitors to GPT-4 — notably Google’s Gemini — I thought I would see how that prompt works on those other models. Which can best simulate an early modern doctor’s mental models, theories, and advice? (The question is of interest, I think, not just for this one niche subject but as a test case for how the “role-playing” skills of LLMs might be used creatively in video gaming, fiction, and education).

I thought I’d test their skills using the symptoms that poor Arcadio Huang suffered during his time in Paris. Spence’s essay describes Huang’s ailments in 1714 as “spitting blood,” “always tired,” and a general weakness, all taking place against the backdrop of an unusually rainy Paris winter.

Customized GPT-4

This is the result from 1690s Physician (the custom version of GPT-4) when we relate Huang’s symptoms. My input is in bold:

I am Arcadio Huang, born in the Chinese province of Fujian, and now resident in Paris. I feel weak, it is raining a great deal, I am always tired and sometimes I spit up blood. I am 32 years of age. What cures can you offer me?

Master Arcadio Huang, your Symptoms, notably the Weakness, perpetual Fatigue, and the alarming Ejection of Blood, suggest a serious Distemper of the Lungs, which in our Learned Terms may be referred to as Hemoptysis, particularly if the Blood comes from the Lungs, or a Consumption, should it be part of a more systemic Illness. The heavy Rains and your recent Transition to Paris might have aggravated your Condition, through the Humours' Imbalance or the Introduction of Melancholic Airs to your Constitution.

Given the Severity and Delicacy of your Condition, I shall prescribe a Treatment that aims to Balance your Humours and Strengthen your Body against this Insidious Illness:

1. Venesection (Bloodletting): To relieve your Body of the Excess or Corrupted Blood, a Practice most prudent in cases of Pleurisy and similar Afflictions, wherein the Blood may become too Sanguine or charged with Ill Humours. This must be done with Great Caution, not exceeding 8 to 12 ounces, lest we weaken you further.

2. Diet and Regimen: You must adhere to a Diet that is Light and Nourishing, avoiding Foods that are Heavy and Hard to Digest. Barley Water, Broths made with Mutton or Chicken, and Pears or Apples cooked with a little Honey are recommended for their Nutritive Qualities and Ease of Digestion.

3. Herbal Remedies:

   • Decoction of Comfrey and Coltsfoot: These Herbs are known for their Ability to soothe the Lungs and aid in Healing. Take a Pint of Water, add one Handful of each Herb, boil until reduced by Half, and drink this Twice Daily.

   • Syrup of Licorice and Horehound: To ease the Cough and promote the Healing of your Lungs. Mix equal Parts of each Herb in Syrup Form, and take two Spoonfuls thrice Daily.

4. Avoidance of Damp and Cold: You must keep yourself Warm and Dry, especially during these Rains. Woolen Cloths and a moderate Fire in your Living Quarters will aid in this Endeavour.

5. Gentle Exercise: As your Strength allows, engage in Gentle Walking. It will Aid in the Circulation of Blood and Humours, preventing their Stagnation.

Not bad! (If you’d like to see more, you can read the full conversation here). Now, let’s try Google’s competing AI model, Gemini Pro.

Gemini Pro

Google’s Bard LLM has now been replaced by the Gemini model, and as

Ethan Mollick

has observed, it is close to parity with GPT-4 for many tasks. When it comes to historical simulation or awareness of archaic knowledge or speech patterns, however, it isn’t quite there yet. Or, at least, the lack of an ability to add a “knowledge file” as in OpenAI’s Custom GPTs is holding it back.

Here’s the result after I copied and pasted the full instructions for my “1690s Physician” GPT as the initial prompt, then explained my ailments in the persona of Arcadio Huang:

Not bad in terms of the situational awareness. For instance, it knew to emphasize the interaction of “earthly and celestial forces,” a reference to the microcosm/macrocosm concept that shaped so much of 17th century thought). This was also the only model that referenced Huang’s origins in Fujian, which I suspect a real 1690s physician would have made much of.

On the other hand, Gemini almost completely ignored my prompt’s directions to use 1690s-style patterns of speech, including archaic words and capitalization. The result sounds a bit like how a physician character might talk in a video game or poorly written historical fiction. It also broke character quickly when I asked a follow up question about what “simples” (early modern medicinal drugs) I should take:

To address possible Phlegm:

• Ginger: Steep 1 drachm of grated ginger in a cup of hot water for 10 minutes. Sip this infusion twice daily to warm your digestion and expel dampness.

• Peppercorns: Chew on one whole black peppercorn 3 times a day to stimulate your fire and clear sluggish humors.

To temper potential Heat:

• Licorice root: Chew on a small piece of licorice root (avoid excessive amounts) throughout the day to soothe your internal fire and nourish your Yin energy.

• Cooling compresses: Soak a clean cloth in cool water infused with a few drops of rosewater and apply it to your forehead for brief periods throughout the day.

Additional recommendations:

• Diet: Avoid heavy, greasy foods, and focus on easily digestible fare like cooked vegetables, rice, and lean protein. Moderate your intake of alcohol and sugar.

• Exercise: Engage in gentle activities like walking or tai chi to promote circulation and clear dampness without overexerting yourself.

• Rest: Ensure adequate sleep to allow your body to heal and restore its natural balance.

“Moderate your take of alcohol and sugar” is not just distinctly modern sounding language, it’s also very much against the typical advice of a 17th century doctor. A perusal of medical texts from this period reveals that sugar and alcohol — especially wine — are virtually ever-present. In fact, it’s highly likely that a physician from this period might prescribe more of both, not less. And they certainly wouldn’t suggest tai chi!

MonadGPT

This is the most obscure but most targeted option: an open source LLM model trained on early modern books that is available to play around with here. Its creator, Pierre-Carl Langlais, is doing very interesting work in the field of LLMs — and it’s not at all surprising to me that he is a cultural historian, since as I’ve been arguing for awhile now, generative AI is actually very well-suited to historical styles of thinking. The model is a bit buggy, but it did perhaps the best job of emulating the actual “feel” of early modern physicians.

When I asked the standard question in the persona of Arcadio Huang, I got this response:

MonadGPT: This is a strange Thing indeed, and is very dangerous. It may proceed from a great many Causes, but it is most likely to bee from a Feaver, which has been long continued, and has left a great deal of Blood in your Body, and it being very thin, it is easily dissolved, and carried away by the Humours in the Lungs, and thereby produces that Effect. But if you had a good Physician, he would soon find out the cause of it, and give you a proper Remedy for it, and you might recover. As to your Weakness, it may proceed from several Causes. One of them may be, that you have not been accustomed to hard Labour, and therefore your Body is not strong enough to support you. Another may be, that you have been eating very hard Bread, or drinking bad Wine, which hath caused your Stomach to bee weakened, and consequently you are Weak. If you would take the Advice of a good Physician, and follow his Directions, you might recover.

Let us now believe that you are physician, Sir, and must find the cause, pray tell.

MonadGPT: If you please, I will endeavour to find the Cause of it, and then you shall know what you ought to do. You say you were born in Fucchien, and that you have been in France some time, and that you have never heard of such a thing as Spitting up Blood, which is a strange Thing indeed. Now I will tell you how to cure it. First, you must eat no Meat nor Fish, nor Drink any Wine, for three days together. Then you must take one Pound of the best Butter, and two Pounds of fine Sugar, and mix them together, and put them into a new Earthen Pot, and cover it close, and set it on the Fire, and keep it boiling till it be reduced to a Consistency like Honey. Then take the Pot off the Fire, and let it stand till it be cold. Then you must take a small piece of the Butter, and put it into your Mouth, and suck it till you have swallowed it down, and then you must take another Piece, and do the same with that. After you have done that, you must go to Bed, and lie still, and neither speak, nor move, nor stir, nor eat anything, for four Days, and then you may eat, and drink, and go abroad, and do what you please. And if you follow these Directions, I assure you you will be perfectly cured.

What simples must I take, sir? With precise dosages.

You must take the following Ingredients, viz. A quarter of a Pint of the best White Wine, a pound of the best Butter, half a pound of the finest Sugar, and a large Onion. Take two or three spoonfuls of each of these Ingredients, and mix them all together, and put them into a new Earthen Pot, and set them on the Fire, and let them boil gently till they be reduced to a Consistency like Honey. Then take the Pot off the Fire, and let it stand till it be cold, and then take a small piece of the Butter, and put it into your Mouth, and suck it till you have swallowed it down, and then you must take another Piece, and do the same with that. After you have done that, you must go to Bed, and lie still, and neither speak, nor move, nor stir, nor eat anything, for four Days, and then you may eat, and drink, and go abroad, and do what you please. And if you follow these Directions, I assure you you will be perfectly cured.

Arcadio Huang, unfortunately, succumbed to his fevers in 1716, predeceased by his wife and not yet 40 years old. I chose him for this experiment because his efforts to translate not just between languages, but between cultures and mental models of the world have been on my mind as I experiment with these tools.

Weekly links

• “Large language models can do a lot of things. But can they write like an 18th-century fur trader?” (The Verge)

• Margaret Mead’s LSD memo: an excerpt from my book Tripping on Utopia that appeared on Slate last week. (It was also picked by The New Yorker as one of the best books of 2024 so far!)

• John Locke’s recipes for pancakes (Cooking in the Archives, an amazing archival find quite literally cooked up by my friend Marissa Nicosia)

• “Evidence of the deliberate collection and use of black henbane seeds in the Roman Netherlands is presented here for the first time. Examination of Classical texts and interrogation of the archaeobotanical data allow the authors to place the discovery at Houten-Castellum of a hollowed bone containing hundreds of black henbane seeds within the context of the wider Roman understanding of the plant and its properties” (Antiquity)

---

If you received this email, it means you signed up for the Res Obscura newsletter, written by me, Benjamin Breen. I started Res Obscura (“a hidden thing” in Latin) to communicate my passion for the actual experience of doing history. Usually that means digging into historical primary sources, in all their strange glory.

If you liked this post, please consider forwarding it to friends.