Interview with Prof. Edoro and students in English 178 Class

We were in the middle of tackling Jorge Luis Borges’s “The Library of Babel” when Professor Karu Sankaralingam came to visit the Writing Machines course. Borges was helping us grapple with combinatorial mathematics and the possibility of total knowledge. Who better to help us think about the implications of these questions than a computer scientist working at the forefront of AI?

Professor Sankaralingam is a professor of computer architecture at UW-Madison’s school of AI and Computing, as well as a research director at NVIDIA. We discussed mathematics, computer science, literature, and science fiction. He shared insights on the possibilities and limits of machine-generated knowledge. Along the way, we talked about mathematical proofs, the pleasures of programming, the rights of content creators, in addition to a few book recommendations.

Ainehi Edoro: Let’s begin with a little introduction. Could you tell us about your research and some of the work you’re currently doing?

Karu Sankaralingam: I’m a professor in the Computer Science Department here at UW-Madison. I came here in 2007, so I’ve been here for a while. My research area originally is computer architecture, which is the part of building computer chips.

More recently, I’ve been spending some of my time at Nvidia. I’m a research director there in their research lab, working on how to build better computer chips, more specifically focused on building chips for AI and deep learning, and more recently, looking at LLM inference, and how we can make these more energy efficient, make them faster and such. So that’s kind of my technical background.

For the past two years or so, I’ve had my own journey with using LLMs. I have a little bit of a love-hate relationship based on the content creator rights, which we can come to later.

But setting that aside, my big thing with LLMs, maybe we’ll have time to get into it, is reasoning LLMs. About a year ago, they had reached a level where, for the kinds of things I do in my professional work, like designing better chips and asking questions about whether I should use this technique or that technique, they had reached a level of sophistication that was either matching or exceeding mine.

And that led me to rethink aspects of my professional work and how I could leverage these models. That’s something I’m spending a lot of time on these days.

Ainehi Edoro: In the conversations you and I have had in the past, one thing that has come up is the distinction you make between LLMs and reasoning models. I’d like to begin there. How do you understand LLMs, and what distinguishes reasoning models from them? Why do you think this distinction is important for how we understand these technologies?

Karu Sankaralingam: First, I’ll describe how I think of what we might call text LLMs. You go into a chatbot, ask a question, and it gives you a response. My view may be kind of simplistic compared to all the things you’ve been thinking about in the course, but you can think of it as two pieces. One is that you’ve taken every piece of text that’s ever been generated, and then you’ve built a machine that can do instant recall and instant sentence completion. So that gives you about 20% of what a modern LLM can do.

In addition, you’ve also somehow captured all the meaningful patterns that are semantically, syntactically, and even arguably aesthetically meaningful for people. You can add that on top of the recall. Think of it as an approximate lookup engine. That’s how I view text LLMs. That’s why they’re able to write short paragraphs and then build on that to write longer paragraphs and stories and such.

Now, reasoning LLMs, for those of you who don’t know what they are, are the things that people use today where, with one-shot prompts, they can generate an entire program or solve these really open millennium math problems. So when you think about it, what do they have to do with sentence completion? It makes no sense, right?

Let’s take, for example, the basics of sentence completion. Actually, I saw that you started today’s class with a writing exercise. For me, writing is very hard. I first do notes and then I write. Without the first step, I can’t do the second step.

That’s how I think of reasoning LLMs. They are trained to do something and provide the scratchwork of how they got to where they got to, doing that in small steps. And each step is then verified: did it make meaningful progress toward solving that problem?

Think of it as teaching someone to solve a problem. I teach you how to do addition, then multiplication, then division, breaking it up into small algebraic properties. I teach you all of these steps. And then when I give you a new problem, I train you with a reward function that teaches you good ways to compose these subproblems. That’s what reasoning LLMs are. Just like how, for text, we can do approximate lookup of meaningful sentences. It’s that on steroids, where they are rewarded for meaningful composition of arbitrarily unrelated ideas, if you will.

And the power they have, particularly for math and precise problems, is that because there is Lean – a mathematical language that allows you to check things – every step can be proven to be correct or not. So there is no notion of ambiguity about whether this was a bad thing to do. It’s right or it’s wrong, and then you put these things together.

That’s kind of my distinction of how I view reasoning LLMs. Because they have this verification step, they can search gigantic spaces and gigantic combinations of things. They are able to prove these math problems that have eluded human mathematicians for centuries or decades.

Ainehi Edoro: So what’s the value in that? When we hear about machines generating language, as a typical chatbot can do, it really messes with people’s heads. We have this thing that can write and create media in ways that we thought were reserved for humans.

But I sense that with reasoning models, there’s a different kind of response. They don’t seem to provoke the same level of anxiety. Take the Navier–Stokes situation, for example. The response seems to be more one of fascination than concern. Why do you think that is? What’s the difference?

Karu Sankaralingam: There are at least two, maybe three camps in how people view the value of these reasoning LLMs.

Some people in the math community have questioned whether a machine producing a proof and solving a problem is how math should work. Math involves community building and contextualizing things, and so on. And there’s a little bit of genuine debate about whether we should be rethinking all of this.

So that’s one view. A simpler, safer route to take, where there’s not that much controversy, is coding. Because I think the entire computer programming community went through all our stages of grief, and we’ve accepted it. I love programming. There are lots of computer science students here. Any of you freshmen? Maybe not. Okay, so maybe you relate or not, but I love programming, and I belong to this group of people.

We felt that programming was a genuine artistic expression. We really felt that. People care so much about code quality, and nobody can define what code quality actually means. People try to define it, but we just love programming. And that was really how we expressed ourselves, right?

And when Codex and all these things came out, the idea that anyone should care about how your code looks—we were all like, whatever, man. This was a debate about a year ago. I think these things have advanced so much.

I don’t know if there’s anybody who’s still holding on to this ideal, this belief that coding is a genuine form of artistic expression, if you will.

Ainehi Edoro: Do you think there’s still space in programming for that kind of aesthetic experience?

Karu Sankaralingam: I would like there to be.

Ainehi Edoro: Large language models have automated writing, right? And there’s been a lot of grief around that in English departments as well. What’s funny is that mathematicians and writing people are finding themselves in a similar position, having to rethink their relationship to practices that they’ve cared about.

But I want to push back a little on the idea of loss. Even though writing has been automated, it hasn’t lost its aesthetic, personal, or embodied dimensions. People still write for the pleasure of writing, for the experience of making something through language. So I’m wondering whether there is a similar sense in programming. Even if AI can produce the code, is there still something about the experience of writing code that is valuable in itself?

P. T. (Student): Do you think there are people working in mathematics today whose ways of thinking are so different from the norm that they could produce major breakthroughs, but whose abilities aren’t necessarily recognized or rewarded by the current system?

Karu Sankaralingam: I don’t know. But I can talk about an example, a metaphor. I don’t know if any of you follow chess. Gukesh Dommaraju is the current world chess champion. I think he’s the only current grandmaster who actually did not train with an engine until he reached an Elo rating of 2500.

And then there’s Magnus Carlsen, who many people believe is the greatest chess player ever. His Elo rating has never been reached. But Magnus is in awe of some of Gukesh’s moves. Magnus is like, “I would not have thought of it.” But Magnus’s rating is higher, right? If there’s a world championship, Gukesh would probably lose to Magnus, partly because Magnus’s endgame is very, very good.

But to answer your question, I think the system is set up so that it’s going to take a few more years. I think in the next three to four years, this group of people will just show up. Right now, the system doesn’t reward them for the way they think and what they do.

And there’s this great book called 10 Great Theorems in Math. It’s an amazing book. It makes all the great theorems easy to read. There’s a chapter about Gauss. You read it and think, God, how did this guy do this? It’s just insane, right? Gauss did what he did at that age. It’s just crazy.

H. Z. (Student): You said that reasoning LLMs use Lean for verification, but I think there are many areas of mathematics that haven’t been formalized in Lean yet.

There’s also another problem with Lean. With LLMs, there’s no guarantee that the Lean formulation actually matches the original mathematical statement. I think this happened with the Navier–Stokes problem last week or the week before. It was discovered that there was a minor problem that could be fixed.

So my question is, as mathematical arguments get more and more complicated, how do we verify that LLMs are actually generating the correct mathematical statements and that Lean is verifying what we intend it to verify?

Karu Sankaralingam: Yeah, it’s a fascinating point. To be honest, I’m not a Lean ninja, so I can’t answer your question at the technical depth you’re asking.

I think so far, some of the problems that have come up could be caught by a human just kind of plowing through the proof. It didn’t require a major ninja-level analysis of these proofs, right?

And I remember this quote, maybe I’m just imagining someone said it, about Andrew Wiles, who proved Fermat’s Last Theorem. I think his original proof had a few small errors, and someone said something like, “A proof is considered correct when the broader math community accepts it.” There’s no notion of a correct proof. So that’s what I think is going to happen.

H. Z. (Student): Yeah, it’s actually a minor mistake, which we all believe can be fixed. But my question is that LLMs are solving problems in ways that are harder for us to understand than human proofs. And that makes it more difficult to review the results.

Karu Sankaralingam: Yes, I agree. The complexity of these systems is getting huge. And I think there’s a related problem: even when AI produces a solution that appears to be correct, that doesn’t necessarily mean people will be willing to use it.

Let’s say you’re building a computer chip. Today, if a Gen AI system came up with the design of a chip, even if it was better on every figure of merit you could think of, very few companies would go build that.

And I think the reason is that no one has a full understanding of what that thing is. My theory is that even if you came up with that full understanding, you would still have a weird sense of discomfort because you didn’t come up with it. You’re always going to feel, “I’m missing something here.” Unless you get this complete understanding of the thing, you’re always going to have that discomfort.

So I don’t know what it means for math. I think what’s probably going to happen is that some entire subdiscipline of math is going to emerge whose whole goal is to try to explain, to connect these dots in a way that matches human intuition.

Because that doesn’t exist right now, right? By definition, even if a mathematician used Lean for their proof, their intuition came first. Now the solution is already there. And once there’s a solution, it’s hard for people to say what the intuition is, right?

Ainehi Edoro: So, between LLMs, reasoning models, Lean, and other related technologies, can we imagine a world in which every possible problem has an answer? Even problems that haven’t yet been formulated? A world in which we could theoretically assume that there is no problem that cannot be solved?

I’m thinking here of Borges’s “The Library of Babel,” where every possible book already exists, every possible combination of letters, and therefore every possible answer. If we imagine something similar with these technologies, a world in which every possible problem and every possible answer already exists, would meaning still be possible?

In other words, what understanding of knowledge is embodied in these technologies?

Karu Sankaralingam: Wow. I have this view. I don’t know if all the information theory people would even agree with it.

So let’s do this thought experiment. Suppose we deleted all the literature about quantum computing, scrubbed it all from the world, and then retrained Astra or Opus, or whatever the later version of Opus is, with the exact same training process.

The thought experiment is: would these LLMs come up with the idea of quantum computing and develop all the theories that humans have developed for quantum computing?

That’s a good question. I’m inclined, maybe because I don’t want to say yes to your question, that these things can invent everything that humans can invent, I’m inclined to say the answer is no.

Ainehi Edoro: I would be inclined to say the answer is no as well. But why?

Karu Sankaralingam: It’s hard for me to think that, knowing everything before it and even knowing every adjacent field, calculus could be thought of as a composition of known ideas. That’s my belief. Some people may disagree with me.

What LLMs can do, I believe, is take everything that’s out there, do the complete cross-product composition of every combination of every meaningful thing, figure out what’s meaningful, what’s not meaningful, and figure out the frontier knowledge. They can do that.

Yeah, that I completely believe. The only reason that the labs are not doing it is because there’s not any economic value to it. And the reason they’re proving all these hard math problems is because that generates hype. Some people think that it helps them move the frontier of knowledge toward superintelligence.

But let’s return to the cross-product of ideas. Let’s take my subfield, building chips. I can take some completely crazy idea in information theory on compressed sampling, about which I know a little bit. And now an LLM is an expert on everything there that I will never be able to be an expert in. And it is also an expert on every aspect of chip design.

It is conceivable that an LLM would be able to take something from there and apply it to how chips should be designed. So given enough time, enough resources, and the proper harness, LLMs could do this cross-product of everything that exists, right?

But that’s different from the kind of invention I’m talking about with calculus or quantum computing. There are some ideas that don’t feel like borrowing from another space. Quantum computing, to me, seems like that. It’s not simply borrowing ideas from one space and putting them in another field.

Calculus is the easiest example I can think of. That’s probably because I’m not a mathematician. Maybe mathematicians don’t feel that way. Quantum computing seems to be another such thing. It doesn’t feel like simply applying the ideas of computing to quantum states. It feels like something more.

Ainehi Edoro: Okay. And we are at 11:42am. Anybody have one last burning question they want to ask?

H. Z. (Student): Regarding what you said about jumping from one space to another, referencing experiences from one space to another, is that similar to category theory in mathematics?

Karu Sankaralingam: Yeah, I mean, it’s a good question. Part of it, I can also tell you why I think that way, partly because the field I work in is computer hardware.

Many of our seminal ideas, at some level, involve someone finding something in a different area and then going, “Oh my God,” and putting it into one of the things we do.

Some of you may know about this. One of the big ideas in computer hardware is—this may be news to some of you, but maybe not—it’s kind of really crazy.

When you write a program, a program gets broken up into primitive instructions. A modern computer doesn’t actually run it in the order in which you told it to. Because if you do that, it’ll be very slow. It runs them in whatever order it wants, kind of, and then predicts things.

This thing is called branch prediction, and that’s one of the big ideas in how computers are built today. You could kind of think of it as borrowing something from some other field and putting it here.

Ainehi Edoro: So going back to all this stuff you have been saying about cross-product composition and borrowing ideas across domains, are you saying that even though LLMs can draw on knowledge from different domains and combine ideas in new ways, there’s still a kind of creative leap involved in producing something truly novel that LLMs may not be capable of making?

Karu Sankaralingam: Yes. I think they can do the borrowing from one to the other. If nothing has already been invented in all the fields we already know, I’m kind of thinking they cannot do that.

Student: What do you make of compute as an asset class, like trading futures on the rental price of a video chip? Do you think that’s a realistic thing that markets should pursue? What do you make of that idea?

Karu Sankaralingam: Oh, I don’t know. I mean, it’s not something I have honestly thought about a lot. And I have my own views on value creation and all of those things. There’s a related book, if you’re not familiar with it, one of my favorite books, The Ministry for the Future. It’s maybe one of the best books I’ve read on science fiction and climate change. I was reminded of it because, when everyone was going through the cryptocurrency craze, there’s actually a climate-like token that’s created, a carbon token that’s used to incentivize people to control climate change.

Ainehi Edoro: So this is the second book you’re suggesting for us today. Any others you want to add to the list?

Karu Sankaralingam: I’m tempted to suggest one. I love it. It’ll make you think. Have you guys covered the labor impact of LLMs?

Ainehi Edoro: Next week.

Karu Sankaralingam: Okay, then this book could be awesome.

Ainehi Edoro: Empire of AI?

Karu Sankaralingam: No, no. A Spectre, Haunting. For those of you who don’t know, those are the first three words of The Communist Manifesto.

And this book is absolutely wild. It’s by my second favorite science fiction author, China Miéville. He’s British, but this isn’t science fiction. It’s a modern reinterpretation of The Communist Manifesto. It’s an unputdownable nonfiction book. I listened to the audiobook in a single session. It’s not very long.

Ainehi Edoro: You know how Google, in its mission statement, talks about organizing all the world’s information? There’s this idea that we can have access to, generate, or manipulate all the knowledge in the world. Reminds me of Borges’ story and the fantasy of knowledge at the scale of the universe. Do you think large language models are animated by a similar vision, this totalizing vision of knowledge?

Karu Sankaralingam: Yeah, so I’m going to answer the first part through a particular lens, because I think one of the conversations missing, and it’s a bit of a lost opportunity for the math community in this math apocalypse that’s happened, is to actually take a step back and talk about content creator rights.

These technologies were built out of content with creators never giving consent for their artistic products to be used in this way. And yes, everyone has their political views. This is being litigated in the Supreme Court. Let’s see how it lands, right? Regardless of how it lands, I think now is the time to start a conversation.

What does content created by humans mean in this world? Does human-created content allow these machines to get better? My current belief is yes. That goes back to the point you made about organizing information. There’s organizing all the information that exists. There’s then synthesizing new information, and then new content that only a human could come up with. Which is almost kind of the converse of the attention economy, right? I think this is going to be super, super valuable in terms of what moves the frontier of knowledge.

I think that the idea of organizing information comes from this perspective that information is a commodity. That’s where that kind of view comes from, on some level. Maybe it’s an extreme way of articulating it.

Ainehi Edoro: As a computer scientist who loves to read fiction, what place do you think fiction has in helping us grapple with the “baffling complexities” of these technologies?

Karu Sankaralingam: Science fiction inspires me and makes me think about things that I would not otherwise have thought of. That’s why I love science fiction. I think that its role will hopefully be even bigger. What’s happening is, in a good way, the creativity bar of what humans have to do to move the frontier, or to be noticed, or get credit for, or even get joy from, has either gone to a higher level or moved into another dimension. Fiction will be even more inspirational and valuable in this world where machines are doing what we thought was ideation and creation that only we, as humans, could do.

Ainehi Edoro: Thank you so much.

Karu Sankaralingam: Thank you very much. It was fun.

Image: by Reyansh Sawant