People tend to rejoice in the disclosure of a secret. Or, at the very least, media outlets have come to realize that news of “mysteries solved” and “hidden treasures revealed” generate traffic and clicks. So I’m never surprised when I see AI-assisted revelations about famous masters’ works of art go viral.
Over the past year alone articles have been published, highlighting how artificial intelligence recovered a “secret” painting of a “lost lover” of Italian painter Modigliani, “brought to life” a “hidden Picasso nude”, “resurrected” Austrian painter Gustav Klimt’s destroyed works and “restored” portions of Rembrandt’s 1642 painting “The Night Watch.” The list goes on.
Art historians have become increasingly concerned about the coverage and circulation of these projects. They have not, in actuality, revealed one secret or solved a single mystery. What they have done is generate feel-good stories about AI.
Are we actually learning anything new?
Take the reports about the Modigliani and Picasso paintings. These were projects executed by the same company, Oxia Palus, which was founded not by art historians but by doctoral students in machine learning. In both cases, Oxia Palus relied upon traditional X-rays, X-ray fluorescence and infrared imaging that had already been carried out and published years prior – work that had revealed preliminary paintings beneath the visible layer on the artists’ canvases.
The company edited these X-rays and reconstituted them as new works of art by applying a technique called “neural style transfer.” This is a sophisticated-sounding term for a program that breaks works of art down into extremely small units, extrapolates a style from them and then promises to recreate images of other content in that same style. Essentially, Oxia Palus stitches new works out of what the machine can learn from the existing X-ray images and other paintings by the same artist.
But outside of flexing the prowess of AI, is there any value – artistically, historically – to what the company is doing? These recreations don’t teach us anything we didn’t know about the artists and their methods.
Artists paint over their works all the time. It’s so common that art historians and conservators have a word for it: pentimento. None of these earlier compositions was an Easter egg deposited in the painting for later researchers to discover. The original X-ray images were certainly valuable in that they offered insights into artists’ working methods. But to me, what these programs are doing isn’t exactly newsworthy from the perspective of art history.
The humanities on life support
So when I do see these reproductions attracting media attention, it strikes me as soft diplomacy for AI, showcasing a “cultured” application of the technology at a time when skepticism of its deceptions, biases and abuses is on the rise. When AI gets attention for recovering lost works of art, it makes the technology sound a lot less scary than when it garners headlines for creating deep fakes that falsify politicians’ speech or for using facial recognition for authoritarian surveillance.
These studies and projects also seem to promote the idea that computer scientists are more adept at historical research than art historians. For years, university humanities departments have been gradually squeezed of funding, with more money funneled into the sciences. With their claims to objectivity and empirically provable results, the sciences tend to command greater respect from funding bodies and the public, which offers an incentive to scholars in the humanities to adopt computational methods.
Art historian Claire Bishop criticized this development, noting that when computer science becomes integrated in the humanities, “[t]heoretical problems are steamrollered flat by the weight of data,” which generates deeply simplistic results. At their core, art historians study the ways in which art can offer insights into how people once saw the world. They explore how works of art shaped the worlds in which they were made and would go on to influence future generations. A computer algorithm cannot perform these functions.
How absurd to think that black-and-white photographs from 100 years ago would produce colors in the same way that digital photographs do now. And yet, this is exactly what AI-assisted colorization does. That particular example might sound like a small qualm, sure. But this effort to “bring events back to life” routinely mistakes representations for reality. Adding color does not show things as they were but recreates what is already a recreation – a photograph – in our own image, now with computer science’s seal of approval.
Art as a toy in the sandbox of scientists
Rather than lionising these studies as heroic achievements, those responsible for conveying their results to the public should see them as opportunities to question what the computational sciences are doing when they appropriate the study of art. And they should ask whether any of this is for the good of anyone or anything but AI, its most zealous proponents and those who profit from it.
