by N. Ashokan, Y. Habsi, A. Marella, H. Zhang
Talkie is a language model developed by Nick Levine, David Duvenaud, and Alec Radford. Radford was the lead author of OpenAI’s original 2018 GPT paper. Most AI systems learn from the modern internet, but Talkie was trained on English text that was published before 1931. This includes books, newspapers, scientific journals, patents, and court decisions. The cutoff comes from copyright law, since U.S. works published before 1931 are now in the public domain. As a result, the model’s knowledge ends where copyright protection begins. With 13 billion parameters, Talkie is the largest “vintage” language model built so far, according to its developers. When asked about the future, it responds how a writer from 1930s might. It even predicts that a second world war is unlikely.
On Thursday, October 8, Levine and Radford will lead a free hands-on session in Room 224 of Memorial Library from 10 a.m. to 12 p.m., with lunch to follow. The session is designed to connect technical computing with humanistic research, and participants will move between small-group discussions and hands-on demonstrations. Participants will work on optical character recognition (OCR), the process that turns scanned pages into machine-readable text. They will also practice running models locally and handling archives of pre-1931 text. No experience is needed, although bringing a laptop would be helpful. Anybody interested in attending would have to register through a Qualtrics form posted on the Digital Scholarship Hub.
This session is worth attending for anyone curious about where training data comes from. According to Talkie’s creators, part of the gap between their model and modern ones comes from OCR errors, which means that the quality of a scanned 1920s newspaper page affects what the model learns. Overall, the session gives students a chance to see the human decisions that shape a model before training begins. Since the model is openly available, attendees can also keep experimenting with it afterward through Talkie’s online chat demo.
For anyone who cannot attend on Thursday morning, Levine and Radford will also give a talk on Talkie on Wednesday, October 7, from 4:00 to 5:30 p.m. in the same room. Drawing on how models reason without modern internet data, the talk will address questions about historical simulation, data stewardship, and the ethics of digital archives.
