When technology follows art

From optics to machine learning, artists have played an important, if underrated, role in the history of technology

The newest issue of the New Yorker has a great profile of Nvidia founder and CEO Jensen Huang. At the end, there’s this exchange regarding the impact of the generative AI systems that are powered by Nvidia’s graphics chips:

I wondered if it might eliminate art.

“It will make art better!” Diercks said.

Participants in debates like this tend to assume that disruptive new technologies are acting upon the world of art. But what’s fascinating about this history is that the Nvidia GPUs which have made contemporary generative AI possible were originally inspired by artistic goals. To be more precise, they were inspired by the needs of computer graphics artists in the gaming industry. Here’s how the article explains Nvidia’s origins:

Huang liked video games and thought that there was a market for better graphics chips. Instead of drawing pixels by hand, artists were starting to assemble three-dimensional polygons out of shapes known as “primitives,” saving time and effort but requiring new chips.

Without that initial demand from artists, there would have been no Nvidia. And without Nvidia, there would be no ChatGPT as we know it today.

This is far from the only time that an artistic goal has inspired technological and scientific innovation. The rest of this article is a bit meandering (I wrote it in spare moments while potty training our toddler) but I hope it’s an interesting tour of some times when artistic innovation helped inspire changes in science and technology — and not the other way around.

Opticians and painters

Many Renaissance artists employed various forms of optical technologies, like the camera obscura, in their work. These practices, in turn, helped influence the science of optics itself.

In 2005, the historian of science Antoni Malet wrote an interesting article about the early history of the telescope in the first two decades of the seventeenth century. In it, he surfaces some early historical references to lenses used in magnification. Around the time the telescope was invented in the first decade of the 17th century, the word “art” could describe the work of both natural philosophers (scientists) and of painters. Malet cites the Spanish author of one 1620s treatise on optics, for instance, who called it an “art” which centered on imitating the function of the eye.

This treatise makes it clear that one technology in particular, the camera obscura, was an inspiration for the tinkering which led to the telescope and microscope. Here’s an excerpt:

find a room that has a door to a courtyard or corridor where the sun shines, and then close and seal all its joints tightly so that no light enters. And in this state, make a hole in the door, of a size that you can cover with a convex lens of two degrees or two and a half, which should be made of glass for the best effect. After covering the hole with the lens, place a thin sheet of lead or tin with another very round and smaller hole, like a coin. And with everything fixed on the door, have some people stand in the courtyard or corridor in such a way that the sun shines on them, because if not, they won't be visible. On the inside of the room, place a white paper in front of the hole, at a distance of about half a yard, more or less, and you will see all the figures from outside represented on the paper, albeit small, but with colors and features so distinct that they seem to be in vibrant illumination.

As far back as the 11th century, the Muslim natural philosopher Ibn al-Haytham (Alhazen) described the camera obscura phenomenon. The Latin translation of his treatise on optics was widely influential and helped inspire similar research by both Roger Bacon and Leonardo da Vinci.

Illustration of a “portable” camera obscura in Athanasius Kircher's Ars Magna Lucis Et Umbra (1646), which summarized accounts of the technique from the fifteenth and sixteenth centuries.

By the late medieval period, variations on the camera obscura were being employed by traveling magicians and showmen who utilized it for what we might now call “multimedia experiences.” It’s unclear when painters first began adapting this and similar optical aids into their artistic practice. The earliest textual evidence seems to be from the 1550s, when Giambatista della Porta described the use of the camera obscura as an aid for drawing in his influential book Magia Naturalis (1558).

In short: artists seem to have been using optical technologies in their work by the mid-sixteenth century at the latest. And, as suggested by the multiple meanings of “art” (ars) at this time, many Renaissance artists were also active participants in the emerging science of optics in the period that immediately preceded the still-mysterious invention of the telescope by a group of Dutch opticians in 1608-09.

The earliest known depiction of a telescope appears in Brueghel the Elder’s Extensive Landscape with View of the Castle of Mariemont” (ca. 1608-1612).

Light in the age of Romantic science

While I strongly suspect that artistic applications of lenses helped inspire the development of the science of optics, I don’t really buy the argument — known as the Hockney-Falco thesis — that optical technologies caused the increasing realism of Renaissance painting. Realism has been an element in painting and sculpture for thousands of years; it’s a choice that an individual artist makes, not a new ability unlocked by upgrading technology.

I do, however, think there’s something interesting here about the relationship between artistic and technological goals. Here’s another, less well-studied, case of art and technology mutually influencing one another in the decades bookending the year 1800. Today, somewhat misleadingly, we label this period “the Romantic era.” Historians of science even speak of Romantic science, as typified by the likes of Sir Humphry Davy, with his intense interest in Romantic poetry, his self-experimentation with laughing gas, and his overall persona as an intrepid adventurer into the unknown worlds of natural philosophy.

But although Romantic art tends to be associated with fanciful, idealized figures and a turn away from realism — think William Blake and Henry Fuseli — it also reveals an intense interest in the science of natural and artificial light. To my eyes, many paintings from final third of the eighteenth century look cinematic or photographic in a new and distinctive way.

As an example of this, here are two of my favorite paintings ever, both by Joseph Wright of Derby: A Philosopher Lecturing on the Orrery (1766) and An Experiment on a Bird in an Air Pump (1768).

Art historians classify both in the older Renaissance genre of “candlelight paintings.” But look carefully at the light in both. It’s far brighter than what a single candle could possibly cast.

To me, it looks much more like the eery white glow of phosphorous.

A subsequent painting by Wright — The Alchemist Discovering Phosphorus (1771) — seems to me to be pretty strong circumstantial evidence that Wright was not just depicting, but actively experimenting with the chemistry of light, including phosphorus, in this period.

A detail of Wright’s 1771 painting.

What makes this artistic interest in artificial light of real importance for the history of technology is the fact that Joseph Wright was an informal member of the Lunar Society of Birmingham. This was a key social group of the early Industrial Revolution whose members experimented extensively with the chemistry of artificial light and light-sensitive materials.

In fact, Lunar Society member Thomas Wedgwood has even been labelled “the first photographer.”

Here’s Wikipedia on his contribution:

Wedgwood is the first person reliably documented to have used light-sensitive chemicals to capture silhouette images on durable media such as paper, and the first known to have attempted to photograph the image formed in a camera obscura… In his many experiments, possibly with advice on chemistry from his tutor Alexander Chisholm and members of the Lunar Society, Wedgwood used paper and white leather coated with silver nitrate.

Interestingly, this work included experiments with phosphorus.

There’s more I wanted to say here about the history of film and, in particular, about the ways that the slicing of reality into ever tinier fragments of time in early cinema may have influenced late 19th and early 20th century physics.

But I will leave that for a later post. In the meantime, here’s a final example that brings things back to the current era of AI art. The below painting is by Frederic Edwin Church. It was created in 1861 as a patriotic symbol of support for the Union cause in the Civil War:

Our Banner In The Sky (1861) by Frederic Edwin Church.

Look familiar? It’s still not clear what the lasting impact of the first wave of generative AI art will be, but if AI art does end up being a major part of the aesthetics of the 2020s — and I think it will — then the “spiral town” images which went viral earlier this year may well be remembered as a key inflection point.

Viral images created with Stable Diffusion by X user @MrUgleh in September, 2023 (source).

It’s not that the creators of these images were directly copying Church. But it’s intriguing to notice the ways that aesthetic innovations have foreshadowed subsequent technologies. In various ways, and to varying degrees, sometimes technology really does follow art. It may be no solace to contemporary artists that the demand for advanced GPUs came first from creators in an emerging artistic field (computer graphics). But it is definitely interesting to me as a historian of science and technology.

It makes me wonder what the artistic styles of the 2020s might be able to tell us about where technology is heading. If you have any thoughts about any of this, please let me know in the comments.

Weekly Links

• Those looking for more on the intersections between art, technology, and science should check out Jimena Canales’ book A Tenth of a Second: a History (University of Chicago Press, 2009). (Canales is one of the more original and interesting historians of science working today. Her more recent book Bedeviled: A Shadow History of Demons in Science is also great.) And for more recent coverage of a similar theme in the Cold War, see Patrick McCray’s Making Art Work (MIT Press, 2020).

• A shout-out, as well, to Paul Graham’s 2003 essay “Painters and Hackers.”

• A complete scan of Egyptian Magic by E. A. Wallis Budge, 1901.

• “A windowless telecommunications hub, 33 Thomas Street in New York City embodies an architecture of surveillance and paranoia. That has made it an ideal set for conspiracy thrillers.” (Places Journal)

• A very deep dive into whether it was a rope or a camel that was originally supposed to be going through the needle’s eye.

If you’d like to support my work, please pre-order my book Tripping on Utopia: Margaret Mead, the Cold War, and the Troubled Birth of Psychedelic Science or share this newsletter with friends you think might be interested.

And a reminder: if you are in a position to spread the word about the book, review it, or (best of all!) write an article about it, please email me and I’ll personally mail you an advance copy.

Many thanks for reading.