by Drew Thomas, University College Dublin
The Protestant Reformation was Europe’s first mass media event, marked by the rapid spread of printed materials. Protestants and Catholics used images to convey powerful messages, often sparking controversy. For instance, Martin Luther’s famous 1522 translation of the New Testament into German included woodcut illustrations of the Apocalypse featuring the papal tiara, which enraged Catholics. In response, Catholics placed the papal tiara on the head of God the Father in their depictions. This clash underscores the significance of images in religious and polemical discourse during the Reformation.
Fast forward to the digital age, where historians are now exploring ways to study these types of images at scale. The “Visualizing Faith: Print, Piety and Propaganda” project at University College Dublin uses AI to investigate how visual media was used in the Reformation.[1] This blog post looks into using large language models (LLMs) to generate descriptions of early modern Bible illustrations. We will examine a case study involving Luther’s first German translation of the complete Bible, which contains 117 woodcut illustrations, and explore how AI can help classify these images efficiently and accurately.
The Foundation: Ornamento Project
The foundation of this research lies in the Ornamento project, a collaboration with myself and Professor Sandy Wilkinson at University College Dublin.[2] In this project, we used a convolutional neural network (CNN) to identify pages of early modern books with illustrations or ornamentation. We then employed the image-matching software VISE from the Oxford Visual Geometry Group to group repeat instances.[3] This effort resulted in a corpus of 5.7 million items, providing a substantial dataset for further analysis.
The Challenge: Generating Descriptions
As a preliminary case study, I focused on the 1534 Luther Bible, which features 117 unique woodcut illustrations. A digital copy of this book is available from the Berlin State Library.[4] To assign classifications to these illustrations, I needed accurate descriptions. I gathered ground truth data from Carl C. Christensen’s study, “Luther and the Woodcuts to the 1534 Bible,” published in the Lutheran Quarterly.[5] This study provided detailed captions of the illustrations, serving as a benchmark for evaluating the AI-generated descriptions.
Leveraging GPT-4o
Using the OpenAI API, I employed GPT-4o to generate 1-2 sentence descriptions of the illustrations. I conducted this process twice: once with just the illustrations and once with the full pages from the Bible. The results were telling. When using the full pages, the descriptions were significantly more accurate compared to using just the cropped illustrations.
Here is an example output for the illustration depicting the story of Jonah: “The woodcut illustration depicts the biblical scene of Jonah being cast into the sea before being swallowed by the great fish, as described in the Book of Jonah. The image shows Jonah being thrown overboard from a ship amidst a stormy sea, with the large fish waiting to swallow him below.”
The LLM’s ability to read the old German page headings, chapter numbers, and some surrounding text — all in 16th-century gothic type — allowed it to make more informed decisions. For example, when generating descriptions from cropped images alone, 50 out of 117 descriptions were incorrect. In contrast, when using the full pages, only 4 descriptions were inaccurate. This stark difference highlights the importance of context in accurately interpreting and describing historical illustrations.
Overcoming Challenges
While the use of GPT-4o proved beneficial, several challenges remain. Multi-illustration pages pose a problem when using the full pages as inputs. Separate descriptions must be generated for each image. Printers also reused some illustrations to represent different scenes. In these cases, the context provided by the surrounding text was crucial for accurate identification. Additionally, running titles across the headers of two pages sometimes led to confusion, as the LLM might not accurately identify the book. Lastly, it was common at this time for illustrations to depict more than one scene, with one in the foreground and others in the background, complicating the classification process.
Looking Forward: Studying Images at Scale
This project demonstrates the potential of AI to study historical images at scale, offering a promising avenue for historians. By leveraging advanced language models and contextual information, we can enhance our understanding of early modern illustrations and their roles in shaping historical narratives.
The implications of this research extend beyond the Luther Bible. It opens up possibilities for analysing vast collections of historical images, identifying patterns, and uncovering new insights. As we continue to refine these methods, we move closer to a future where AI can assist historians in tackling large-scale projects that were previously unimaginable.
Biography
Drew Thomas is a Science Foundation Ireland – Irish Research Council Pathway Fellow in the School of History at University College Dublin. Specialising in early modern material culture, he investigates the history of communication during the Protestant Reformation. Currently, he leads the digital humanities project ‘Visualizing Faith: Print, Piety and Propaganda’, using artificial intelligence to investigate the use of visual media by Catholics and Protestants during the Reformation. He received his Bachelor’s degree from Saint Louis University (2010), Master’s degree from Harvard University (2012) and a PhD in History from the University of St Andrews (2018). He is the author of The Industry of Evangelism: Printing for the Reformation in Martin Luther’s Wittenberg (Brill, 2022).
[1] Funded by the Science Foundation Ireland – Irish Research Council Pathway Programme
[2] Alexander S. Wilkinson, ‘Ornamento Europe: Towards an Atlas of the Visual Geography of the Renaissance Book’, in Arthur der Weduwen and Malcolm Walsby (eds.) The Book World of Early Modern Europe (Leiden: Brill, 2022), pp. 547-562.
[3] Abhishek Dutta, Relja Arandjelović, and Andrew Zisserman. 2021. VGG Image Search Engine. Retrieved 15 July 2024 from https://www.robots.ox.ac.uk/~vgg/software/vise/
[4] Biblia das ist die gantze Heilige Schrifft Deudsch (Wittenberg: Hans Lufft, 1524). Staatsbibliothek zu Berlin, 4″ Bu 9402.
[5] Carl C. Christensen. “Luther and the Woodcuts to the 1534 Bible.” Lutheran Quarterly 19, no. 4 (Winter 2005): 392–413.