The Quest to ‘Solve All Diseases’ with AI: Isomorphic Labs’ Max Jaderberg

[00:00] We have set up the company from day one to really go after this big ambition. This isn't about [00:06] developing therapeutics for a particular indication or particular targets [00:12] is really thinking about how do we create a very general drug design engine with AI, something that we can apply to not just a single target or even a single modality, but we can apply this again and again across any different disease area. And that's what we're stepping towards at the moment. [00:31] Bye. [00:48] Today we're excited to welcome Max Yoderbergh to the show. [00:51] Chief AI Officer of Isomorphic Labs, which launched Out of DeepMind with a goal of revolutionizing drug discovery using AI. [00:59] Last summer, they released AlphaFold 3, [01:02] a stunning breakthrough that allows us to model not just the structure of proteins, [01:07] but of all molecules. [01:08] and their interactions with each other. [01:11] That led to Demis Asabas winning the Nobel Prize in Chemistry last year. [01:15] Max describes their vision for what a holy grail model for drug design. [01:19] and what Agents for Science look like. [01:21] He draws parallels to his experiences building AlphaStar and Capture the Flag, [01:26] and the research directions of building agents and games more broadly.

[01:30] Specifically, [01:31] With 10 to the power of 60 possible drug molecule structures, [01:34] We need to build both generative models and agents that can learn how to explore and search through the whole potential design space. [01:43] Max also describes his vision for what a GPT-3 moment for the field might look like. [01:47] describing it more akin to AlphaGo's famous Move 37, [01:51] when we start to see things that exhibit superhuman levels of creativity in AI drug design. [01:57] and that stun even humans ourselves. [02:00] This is one of my favorite episodes yet. [02:02] Enjoy the show. [02:05] Max, thank you so much for joining us today here in London. No, it's a pleasure to be with you here. Yeah, it's fantastic. [02:11] Awesome timing, too, with the launch of AlphaFold 3 and with Demis winning the Nobel Prize in Chemistry, which is a true testament to everything that you and your team have done over the last couple of years. Yeah, 2024 was definitely a busy year for us. Lots of big breakthroughs. The Nobel Prize was just incredible to see. I think amazing recognition for this seminal piece of work. [02:41] authoring many seminal papers while at DeepMind, including for Capture the Flag and Alpha Star, um, [02:49] breakthroughs in the world of deep learning. Can you walk us through some of the key questions that you had in your field of research around deep reinforcement learning at the time? [02:59] Yes. So at DeepMind, I worked on a whole host of stuff, early days of computer vision and deep generative models. But it was really reinforcement learning that ended up hooking me there. DeepMind was the place in the world to be working on reinforcement learning at that time. And

[03:19] Really, the question in our minds was, [03:22] How can we actually get to a point where we could get an AI? [03:27] that could go off and do any task you wanted it to do. [03:31] And, you know, the dominant paradigm at that point in time was supervised learning. Yeah. And... [03:39] Supervised learning is very different from reinforcement learning. They're both learning techniques. But supervised learning, you need to know what the answer to your question is. And that's how you train the model. So in supervised learning, you give an example, and then you supply the model with the answer to that question. [03:56] home. [03:57] Now, that can be great if you already know everything about the problem that you're training this AI to do. [04:03] There's no network to do. But most times you don't. Yeah. I mean, there's just so many problems in the world where we don't know what the answer is. We don't know what the solution is. And if you think about, you know, I think about how I want AI to be applied to the world. [04:19] Yes, it's going to be great to be able to apply things where we're already good as humans here. But really, you know, the big frontier is can we start applying AI to places where humans don't know how to do this stuff? Or, you know, there's a limit to human. [04:34] performance there. [04:36] And, you know, that's where reinforcement learning is one of the key tools and has real promise here, because in reinforcement learning, you don't need to know what the answer to the question is, you just need to be able to

[04:49] say whether the answer that the model gave you was good or not good. Maybe even how good. [04:56] I'm not good. And, um, [04:58] So this opens up a completely new, you know, [05:01] field of problems to train these models against. And so reinforcement learning and really starting from [05:10] what was one of the big breakthroughs of DeepMind in the early days was working on games like Atari. Yes. The question was, okay, so how can we scale this up from the world of Pong [05:21] and space invaders [05:23] to things that really start to look like real problems in the world. Yeah. And so there was an amazing track of research as we scaled up these methods. Yeah. Did you know that Sequoia was the first investor in Atari back in the day? Oh, really? I didn't know that. That's incredible. Yeah. Yeah. No, those Atari games were, you know, great fun actually to – [05:46] to sort of go back and play in the context of, hey, we've got an agent and, you know, I'm just going to have a game of Pong on the side as well. There's a wonderful wall at Sequoia in our office where we have all these names of legendary IPOs and M&As that have happened. And there's one. [06:03] I think it's called the pizza company. And I love asking folks if they know what that is. And it's actually from Chuck E. Cheese's, which was an original Sequoia investment at the time. Amazing. Amazing. So capture the flag and Alpha Star were incredible breakthroughs at the time. Can you share a little bit about what exactly those breakthroughs were and maybe why you chose those specific games?

[06:28] Yeah, so... [06:29] If you think about the history of AI using video games, why do we use video games at all? Video games are these sort of malleable... [06:40] perfectly encapsulated worlds that as researchers and scientists, you know, we can manipulate them, we can test out different algorithms in them, we can set up different situations. So the perfect test ground for us to develop new algorithms. [06:55] And then you can imagine as a RL researcher, as someone who's thinking about how can we get AI to be as general as possible, [07:04] You're always thinking, okay, we've cracked Atari. How do we get a more complex game? [07:10] The thing that I was personally obsessed with is [07:13] I want these agents... [07:15] to be able to zero-shot be able to do any task. [07:21] And this is a slightly different paradigm from what people were doing at the time with training on Atari, where... [07:28] Normally in reinforcement learning you think about, here's a game, now you get to train on it and get good at it. And then you apply that same algorithm. [07:36] from scratch training on different games. Yes. [07:39] I'd love a different scenario where instead we train an agent and then we can lift it and put it on any new task. Yeah. And that agent will be able to perform well in that task without any more training. Yeah. [07:51] And so to do that, what you're really asking for is generalization over task space. Yes. And that means you need lots and lots of training tasks.

[08:02] So the training data in this RL for agents becomes tasks. Yeah. Not images, not pieces of text, but tasks. And so you can imagine you could go and sit and... [08:13] take a whole game studio and try and hand author [08:17] hundreds of different tasks, lots of little mini games in these virtual worlds. [08:22] um, [08:23] And we did that. We were doing lots of that. And then you can think, yeah, we can actually go further than hand authoring. We can procedurally generate these tasks and games, generating worlds and maps and different objectives and... [08:40] We did that. [08:42] but you keep running into this complexity ceiling, [08:45] that there's only so much complexity that you can hand author or you can design humanly. [08:52] But that's where multiplayer games come in. Yeah. Because as soon as you go from single player to multiplayer, it's not just the agent playing. You've got another player in this game. [09:02] and [09:04] That other player or other players can take on many different characteristics and many different behaviors. So every different player, every different strategy that you're up against, [09:14] fundamentally the game and what the agent is trying to do. You know, [09:18] I go back and think, why are people still obsessed with playing chess? Why does a professional chess player still keep playing chess? It's the same game. But it's actually not because you're playing completely different opponents day after day and new people into the world. Yeah.

[09:34] So the game is continually changing. [09:36] some multiplayer games [09:38] and multi-agent games really encapsulates that huge diversity of tasks that you might encounter [09:44] just from other players being there. And so Capture the Flag was actually one of our first forays into how can we use multiplayer games to really stretch what our reinforcement learning algorithms can do to... [10:00] really forced us to think strongly about how we can generalize to new tasks, how we deal with these multi-agent dynamics. [10:07] So Capture the Flag was a fantastic breakthrough, really showed that we could get to human level performance. [10:14] for these multiplayer first person games yeah and then of course starcraft added on a huge amount of complexity um and was sort of the next frontier that we had to um go after for this [10:25] You were so early in this that so many of these concepts are very, very relevant today in the world of language. How does it feel to see some of this work continue to be played out? [10:35] Yes, brilliant. It's just fantastic, actually. There were so many things that we were talking about. [10:41] Seven years ago. Yeah, yeah, yeah. You know, 2015, 16, 17, 18, 18. [10:47] And [10:48] to see all of these core fundamental concepts be really useful and really applicable today in the world of large language models. You know, the, [10:58] And resulting in performance that we could only really dream about at the time, that's incredibly satisfying. So then in your own words, you said that you moved from building toys to then finding real applications. When did you know that you found the right recipe?

[11:14] I [11:15] So, [11:16] You know, I just love deep learning. I've been obsessed with deep learning [11:21] for you know 10-15 years now um [11:24] And... [11:26] The thing that I love about it is that you have these underlying core concepts these fundamental building blocks and [11:33] that are [11:35] somehow incredibly transferable between different application spaces. Yes. [11:40] So, you know, it's the same building blocks that we were using in computer vision in 2012 as we were using in, you know, generative models, generative models in language, you know, then reinforcement, et cetera, et cetera. Yeah. [11:54] So... [11:55] What I was seeing just again and again was [11:58] This... [11:59] ability to take these core concepts, these same core concepts, [12:04] Take incredible people who understand how they're almost like master chefs of putting these concepts together and these different building blocks together. [12:15] a team of incredible people and go after... [12:19] Really, really challenging problems, problems that you go to conferences at a time and you talk to leading researchers in the field. They say, no, no, no, this is 10 years away. And in the back of your mind, you know, OK, we actually we basically cracked it. Wow. And I saw that happen again and again and again. [12:37] Um, [12:38] You take amazing people, amazing algorithms, amazing computes on really challenging problems, and we can find recipes now.

[12:45] to crack so many problems. And [12:50] So it just got to the point where [12:52] And I've always been quite obsessed with the application of these methods. I want to see this technology have... [12:59] you know, real transformative, positive impacts in the world. And so, you know, we need to start actually going after that. And, you know, the time has been right for, I think, a few years now. [13:11] Well, so you've now had a decade-long relationship working together with one of the greatest scientists, technologists, and founders of our lifetime, Demis. He called you while you were still at Oxford, and then your company, Vision Factory, and DeepMind were both acquired by Google back in 2014, around the same time. And that's when the two of you started to work together now for over 10 years. What was it like, or what has it been like to work with Demis? [13:41] Thank you. [13:42] Yeah, I mean, Demis is an incredible person. [13:47] you know a real character and a real visionary yeah um [13:52] And... [13:53] you know, also amazingly human and relatable. And I think that that really inspires people. So [13:59] It only takes a five-minute conversation to... [14:04] for him to sort of [14:06] really bleed out the depth of ambition that he thinks about. [14:10] and [14:12] just the immediacy of the potential to get, you know, to step towards these ambitions. So,

[14:19] I think... [14:22] He has this great ability to inject a lot of energy. [14:26] into you know a group of very smart people um [14:31] get people to see beyond what's right in front of them. You know, I remember moments sitting, well, standing in the lobby of one of the early DeepMind officers. I think this was the [14:42] It was a toast... [14:44] we were a celebration we were having for the first nature paper from deep mind. Wow. And, um, [14:50] Demis was saying, you know, this is actually just going to be the first of dozens of nature papers. And at the time, this was the first, basically the first machine learning paper in nature. This was the Atari DQN paper. And the prospect of dozens of nature papers was... [15:07] you know, it seems a bit far-fetched. And actually he went further and said, "And we're going to be winning Nobel Prizes." [15:14] And that was 10 years ago. Yeah, that's incredible. The forethought that he has. He's got what I call like one of these rollout minds. Maybe it comes from all of his experience playing chess, but it's he's always rolling out into the future. What are the steps now that are going to lead? [15:31] you know, to this big ambition. [15:34] So yeah, it's been fantastic. I've been working with him for about 10 years now. [15:40] still work really closely together on isomorphic labs and the ambition is as big as ever. [15:45] It's so interesting to hear that you had this ambition and that he had this ambition from the very start. And it's incredible that it's played out that way. Well, I'd love to talk a little bit about isomorphic. You're now embarking on one of the most ambitious missions of our generation to reimagine drug discovery and drug development with AI.

[16:06] Everything goes right, and you realize your vision for isomorphic. What does the world look like? [16:12] Thank you. [16:13] Yeah, you know, we think really big isomorphic. We want to be solving all diseases. [16:19] here and genuinely that scale and the point is that [16:25] this technology. [16:26] that we're building, and AI as a whole field, [16:30] is going to be completely transformative. [16:33] in how we understand biology, in our ability to manipulate and craft chemistry to modulate that biology. [16:41] So we really think about a future where we are solving all diseases, where [16:48] AI is not just helping us [16:50] discover... [16:51] and create and design new therapeutics, but also just understand so much more about our biological world, about how our... [16:59] you know, cells are working, what are the root causes of disease, and therefore opening up new pathways that we can... [17:07] think about modulating. So we [17:12] We have set up the company from day one to really go after this big ambition. This isn't about [17:19] developing therapeutics for a particular indication or particular target of [17:25] It's really thinking about how do we create a very general drug design engine? [17:30] Yeah. [17:30] with AI, something that we can apply to not just a single target or even a single modality,

[17:37] where we can apply this again and again. [17:39] across any different disease area. And that's what we're stepping towards in the moment. [17:44] How does... [17:46] setting out with this ambition of being general, change... [17:51] how you built in practice from day one. [17:54] Yes, a good question. [17:57] When I think about some of the status quo of AI in drug design, there's a lot of [18:03] There's been a lot of use of machine learning models in chemistry and biology, but I would call them [18:10] a lot of the first generation of this sort of application to be more local models, where you might have some data about a particular target or about how particular class of molecules is behaving, and you'll fit a small model. [18:25] you know, [18:25] multi-layer MLP against this data. Yeah. Um, [18:30] to help you generate some predictions that lead to your next round of design. Yeah. Um, [18:35] this is the complete opposite approach of what we were trying to do. So from day one, we were setting out to create models that, [18:42] that generalize across [18:45] chemistry and across target space. So, you know, and a key example of this is something like Alpha Fold and Alpha Fold 3, where [18:55] This is a model that you can apply to [18:57] a whole different host of targets you can apply to [19:00] any protein in the proteome, in the universe of proteins, you can apply it to any small molecule,

[19:08] that you can think of designing, [19:10] without needing to fine-tune it, without needing to fit any local data. And so you can imagine that it completely changes the way that chemists can use these models if you don't need to be adapting this model to every single application. Yeah. So... [19:23] every single one of our internal research projects. And by the way, when I think about what we're going to need to get this breakthrough drug design engine that we've been building, [19:34] uh, [19:35] we need like half a dozen alpha folds. Wow. Alpha fold is just part of the story. Wow. So from day one, we've been, [19:42] setting up these internal research programs, going after these half-dozen [19:47] problems [19:48] We've had significant breakthroughs, obviously, in alpha folds and structure prediction, but also in other key areas. [19:56] And, [19:58] In all of these, these models are general. They can be applied to any target. And then what we're finding actually, they can be applied to [20:05] any modality or lots of different modalities. Yeah. So that's the first time I've heard you say half a dozen alpha folds. Can you share a little bit more about what that means? Yeah. So yeah, alpha fold was obviously a massive breakthrough in understanding, uh, [20:20] biomolecular structure. So what is the structure of proteins and now with our FF3 structure of proteins with small molecules and things like DNA and RNA? [20:30] That's a fundamental step change. It allows us to get experimental level accuracy [20:35] of a really core concept of biochemistry that unlocks a whole bunch of thinking and design work for chemists. Yeah.

[20:42] But... [20:43] You know, my comment here is actually we're probably going to need something like half a dozen more of these sort of breakthroughs. They're sort of getting to experimental level accuracy of different core concepts of biology and chemistry. [20:56] To be able to put this together into... [20:59] something that's really transformative for drug design. Drug design is really, really hard. It's not just a single problem. It's not just about understanding the structure of a protein. Yes. [21:10] It's not even just about designing a molecule that will modulate that protein in the way that you want. You want this molecule to be able to... [21:19] you know, ideally be taken as a pill and go through the body and be absorbed in the right way and reach the right, you know, cell type and actually, you know, go into the cell and not be broken down by the liver in a certain way. So there's just so much complexity. Yeah. [21:33] to hold on to as a drug designer. And each one of those is like, you know, alpha fold level style breakthrough that we've been creating. So interesting. Well, I've also heard you use the words a holy grail model for drug design and agents for science. Can you explain a little bit more about what you mean? [21:51] Yes, so some of these research areas that we've been going after, predicting structure and properties of these molecules and how all of these biomolecules interact and play out over time. [22:05] these really are sort of holy grail [22:08] predictive problems for drug design. And we've made some incredible breakthroughs there, which have really stunned our chemists and step-changed how we do drug design internally at ISO.

[22:21] But what's, I think, a really interesting thing to think about is that [22:26] You could create the best possible predictive model of the world, like an experimental level. [22:32] even better than experimental level model, [22:35] to predict a particular property about a molecule for example [22:38] to be able to predict the outcome of a real experiment. [22:41] So we could have a whole suite of those, but that still wouldn't solve drug design. [22:47] Um... [22:48] And the way to think about this is [22:51] you know, there's this number, 10 to the power of 60, which is perhaps all of the possible drug-like molecules that you could... [22:59] that could exist. [23:02] That's maybe, you know, a bit, you know, takes into account a lot of things. So we could even reduce that by 20 orders of magnitude and get to 10 to the 40. Yeah. [23:13] That's still a lot of things. [23:16] even if you had the best predictive models in the world, [23:19] So let's say you could screen a billion different molecules, you could go and test a billion different molecules, that's 10 to the 9. So, you know, now we're still like 10 to the 31 molecules left on the table. So even with the best predictive models, you're still not even scratching the surface of molecular space that you should be. [23:37] Exploring. [23:38] And this is why we need to go beyond just... [23:41] predictive models of experiment, but also models like generative models, like agents, [23:47] that can actually navigate that whole 10 to the 40, 10 to the 60 space. That's so interesting. Using our predictive models, obviously, to understand how to navigate that space.

[23:57] But so we don't have to exhaustively search because we can never exhaustively search the whole universe of molecules anymore. [24:03] If that makes sense. Just in the same way that AlphaGo couldn't exhaustively search all of the possible Go moves. Right. [24:10] Unlike chess, where you could exhaustively search all possible chess moves. Yeah, yeah, yeah. But, yeah, Molecule Design is much more like Go than it is like chess. [24:19] So that's where generative models come into play. [24:23] agents that utilize generative models, that utilize search techniques, as well as these amazing predictive capabilities to really open up the entirety of molecular space. [24:34] Now, to me, it's actually still amazing that even without AI, we managed to find drugs in this 10 to 60 space, 10 to the 40 space. [24:44] It just says that actually there's probably a lot of redundancy. There's a lot of potential designs. Yeah. You know, if you think about a particular disease indication, a particular target. [24:54] There should be many designs that exist that would be good for that and would be the right sort of product profile for this therapeutic. [25:03] And I think the real potential here is for these [25:07] generative models, these agents as well, to be able to search through this space and really uncover that whole potential design space. That's so interesting. I think in very simplistic Lehman terms, you're both... [25:20] modeling, learning and modeling the game and [25:24] Trying to build the best player to solve different types of games. Yeah. Yeah.

[25:28] So, I mean, you know, I'm incredibly biased by games. I've been playing video games since I was a kid. Grew up in that world. But, you know, that's exactly how I think about it. We've got to be creating our world models, our models of the biochemical world, our biological world. [25:49] And then we don't stop there. We actually then need to be creating agents and generative models that can work out how to explore, how to traverse that, and to basically uncover these [25:59] you know amazing needles in the haystack in chemical space which could be [26:03] life-changing therapeutics for so many millions of people. [26:06] I love that. That is our punchline today. So alpha-pull three is truly groundbreaking. You've taken us from being able to model just the structure of a protein to now being able to model the structure of all molecules and their interactions with each other. [26:36] this. Yeah, so... [26:39] Yeah. [26:39] Alpha Fold 2 was [26:41] you know [26:43] the biggest breakthrough, right? To be able to understand the structure of proteins, [26:48] And then there was something called AlphaFold2Multimer, which then allows you to understand not just the structure of proteins by themselves, each individual protein, but the structure of proteins as they come together and what we call complexes, so how these proteins fit together.

[27:02] that opens up and helps us answer a lot of questions [27:06] in biology, but there's still a big hop to designing therapeutics. And one of the big classes of therapeutics is what's called small molecules. [27:16] So these are molecules that are not proteins. These would be things like caffeine or paracetamol, things that more often you can [27:23] take as a pill. [27:26] And the way that these therapeutics work, these small molecules, is that they go through the body, they go into the cell. [27:32] and they actually come and attach themselves [27:34] to these proteins. These proteins [27:38] They're the fundamental building blocks of life. They form these molecular machines by interacting with other proteins. Yeah. And so you can you can imagine that if you have another molecule, your drug that comes in and attaches itself to a protein over here, then it might disrupt the ability for that protein to interact with another protein, part of its normal machine and day to day life. And so you're modulating the function of that protein with this small molecule. [28:04] And that's the essence of drug design. [28:07] and how therapeutics work. And so you can imagine as a chemist, [28:12] your job, or your drug designer, you're trying to design a small molecule [28:16] that's going to fit to this protein over here and disrupt [28:20] how it normally functions, or in some cases, enhance how it normally functions. And so it'd be really helpful to understand... [28:27] how this small molecule interacts with the protein. What's the structure that it might make? What are the interactions these molecules

[28:34] literally physical interactions that are being made. [28:39] And so that really inspired... [28:41] you know, the creation of alpha-sulf-3, where now we have a model that not only predicts the structure of proteins, but how these proteins interact with small molecules. [28:53] molecular machine building blocks, things like DNA and RNA. [28:58] And this basically opens up the ability to structurally understand, which is a core part of drug design, [29:04] small molecules, it opens up new classes of targets. There are things like transcription factors, which are proteins that sit on DNA. [29:15] and read DNA. And you can imagine now trying to design a small molecule to change or disrupt the function of something like that. And so to do that, you'd really want to be able to see literally in 3D. [29:27] how this all looks. Yeah. And if I make changes to my little molecule, how will that change the way it interacts with this protein and this biomolecular system? [29:36] So Alpha Fold 3, [29:37] is now very, very accurate, allows us to answer a lot of these questions purely in silico, or purely on the computer, where before... [29:45] you would have to go to the lab. [29:47] literally crystallize this stuff. This can take six months, it can take years, sometimes it's even impossible. Now at ISO, our drug designers are, you know, literally sitting, you know, with their laptop, browser-based interface, being able to understand, make changes to their designs, and see the impact of that. Incredible.

[30:06] So there are a couple interactions that AlphaVolt 3 has. [30:11] is focused on proteins and nucleic acids, proteins and ligands, and antibody to antigen. Can you give us some good examples of [30:21] of the impact that alpha-3 now has on the interaction of these different types of proteins and molecules. [30:29] Yeah, so... [30:31] Protein and ligands, that's the same as protein and small molecules. Those two terms, ligands and small molecules, are synonymous. That allows us to understand how small molecule drugs interact. Then we can think about... [30:43] protein interactions, [30:47] There's a whole class of therapeutics called biologics. These are things like antibodies. That allows us to understand how they might interact with our targets. [30:57] opens up new modalities. [31:00] And that also... [31:02] encapsulates the antibody-antigen interface. So if you're designing an antibody, [31:08] You want to understand how your antibody design is going to interact with the protein [31:12] surface there. So it's the same model that we can use across all of these different applications. [31:20] What are the nuances of training a model like AlphaFold3, and what are the benefits of using a diffusion-based architecture? [31:26] Yeah, it's a great question. There are a lot of challenges we had to overcome to get our fold three to work. One of the most interesting things was actually just how do we

[31:36] take [31:38] something like AlphaFold, which was only working with proteins, and then input these new modalities, these new data types, [31:44] of RNA, DNA, small molecules. [31:47] So we had to work out how to tokenize not just proteins, which we kind of knew how to do, but how to tokenize then DNA, how to tokenize small molecules. [31:55] For things like [31:56] DNA and RNA, that's a little bit more obvious. We could tokenize in the bases. [32:01] But then for small molecules, we would really go to... [32:05] We tried a whole bunch of different stuff. It really ended up that this atomic resolution tokenization worked super well. And then you have the question of, okay, how do you actually predict the structure of this mixture of different... [32:20] molecule types. You couldn't use the same framework as AlphaFold2. This is where diffusion modeling just really shun. [32:33] here we could... [32:34] actually model every single individual atom and the coordinates of every atom individually, and have a diffusion model be producing those 3D coordinates. And the [32:46] tokenization that we talked about is conditioning [32:50] the inference of that diffusion process. So interesting. And this was a huge breakthrough. So, you know, we're talking about on our leaderboard, just a massive step change, particularly in small molecule protein levels.

[33:06] interaction accuracy. It was a massive step change and something that really unblocked the rest of the project. - Wow. So data, compute and algorithms, [33:15] We know those three are important in all other adjacent fields. But I was surprised to read an interview with Demis where he shared that we're not data constrained in biology. Can you share your point of view on that? [33:28] I think it doesn't matter what field of machine learning you're in, you're going to feel some data constraints. And I think the point here from Demis is that [33:38] It's not a real bottleneck. [33:41] As in, we can make progress with the data that is out there, the data we can generate. [33:47] and real progress can be made. [33:51] you know, we've got to sit and wait 50 years for like the world to generate data before we can actually make impact here. No, we're not seeing that at all. There are modeling spaces that where the data has been sitting around for years. Yeah. [34:05] that we can see that we can make really substantial decisions [34:09] progress beyond anything that people have experienced before. Yeah. Now, [34:14] Does that mean there's no opportunity for data in biology? Absolutely not. Like, it's going to be a fundamental part of how we, you know, how we continue to develop these models and these systems will be, [34:26] what data we go out and generate, [34:29] And there I think there's just a massive opportunity in my mind. [34:34] Um, [34:36] The data for machine learning in biology hasn't actually been created yet. Yes, there's a lot of historical data.

[34:44] But there's a huge, but that historical data hasn't been created for the purposes of machine learning. [34:49] And so when you're going out and thinking, how do I create data to actually train my model? [34:53] you're thinking in a very different way to how people have gone out and generated data in the past. And there's a big opportunity there to explore. [35:00] What kind of data do you think we're missing here right now? And do we think, do you think that we need... [35:06] anything in synthetic data? [35:09] Yes, so... [35:11] I'm a massive fan of synthetic data. Actually, I have been since the very beginning of my career where... [35:18] I was generating synthetic text data [35:21] just to overcome the fact that I was a PhD student. [35:25] with access to a couple of thousand images, and Google had millions and millions of images. And so instead, I just generated... [35:31] tons and tons of synthetic data and that unblocked things. And, you know, we're seeing the same thing in the, especially the chemistry space, where we have... [35:40] good theory. We actually know a lot about physics. We have the theory of quantum chemistry, [35:50] and quantum mechanics, and we can create simulators out of that. We can approximate that and create more scalable molecular dynamic simulations. This gives the basis for, you know, a whole host of synthetic data. [36:02] Then we have the models themselves. [36:04] that [36:04] Especially we have generative models. This can actually generate data that we can use scoring systems to help. [36:11] really enhance the information content of this data. Yeah.

[36:16] But I think one of the big open spaces will be on... [36:20] what's called in vivo data. So data that you would normally measure on a real animal, something like a mouse or a rat, you know, that you... [36:30] There's some historical data on that. [36:32] but you can't generate. [36:33] tons of that you can't really generate any at all right so then there's a big opportunity to look to [36:39] new data generating technologies. There are some incredible people doing things like organoids on a chip. So ways of starting to [36:49] measure things that you would normally measure on a real animal, but [36:53] you know, [36:54] completely on a chip. So interesting. There's going to be a whole host of new breakthroughs in data generating [37:03] technology in biology and chemistry that's that's gonna you know have big impact on how we think about modeling that world as well. [37:09] Are you working on any of that internally, or are you hoping that other players fill in some of that gap? [37:16] So internally, we actually don't have any of our own labs [37:20] in isomorphic labs, but we [37:24] We work with a whole bunch of other companies. [37:28] You know, we generate a lot of data ourselves, a lot of proprietary data. We've seen an amazing impact of that. It makes a lot of sense. [37:36] So there's a point of view that modeling structure of molecules and modeling their function and the modulation function is very important, but not necessarily always the limiting factor in drug development. What's your point of view on that?

[37:51] Yeah. [37:52] As I touched on before, drug design is really, really complex. And that's before you even get to drug development, which is where you take those designs. [38:00] And you start putting them into real people, clinical trials. [38:04] There are so many bottlenecks throughout this whole design and development space. [38:10] drug development [38:11] is... [38:13] How do we... [38:15] start to approach clinical trials [38:17] How should we test these drugs out in people? How can we do this in a really timely manner? [38:22] but still a really safe manner. There's a lot of bottlenecks there that I think... [38:27] the industry as a whole. [38:29] we will need to work out how to innovate in that space, especially as our predictive models of innovation [38:36] how these molecules will interact with people, how toxic they will be, [38:41] As these predictive models get better and better, [38:44] We will have to change the way. [38:46] that we approach clinical trials to really make use of that. Ultimately to get therapeutics [38:51] into the hands of patients who really desperately need them. Yeah. You know, even in the design of molecules themselves, as we talked about before, [39:01] It's not just understanding the structure of these molecules. It's not even just understanding... [39:07] how these molecules change the function of these proteins, but we need to understand how these molecules change the function of [39:13] pretty much every single protein in our body. Because if we take this as a pill, it's gonna go, [39:19] Everywhere. And that's the major cause of toxicity is when...

[39:25] Yes, you've designed this amazing molecule that perfectly modulates your specific target that you know is keyed. [39:32] to your disease. But also affects other things. But it also affects other things. Now, of course, you do a lot of screening to protect against that, but [39:39] the more we can predict [39:42] that, the better. What's really exciting from my perspective is if we're creating these general models that understand how this molecule interacts with [39:50] This target... [39:51] but also any other target, [39:53] then why can't we just use that same model to understand how these molecules interact with the rest of our body? Right. So interesting. So what is now possible with AlphaFold3 for drug designers? How are you using it internally? [40:06] So AlphaFold3 gives our drug designers the ability to understand how [40:13] their molecule designs, [40:15] really interact with this. [40:16] protein target. [40:18] and this is the target of disease. And so our drug designers can make changes to the design, [40:24] and then see instantly... [40:26] how that changes the way that this molecule physically interacts with the protein target. [40:31] That's really, really powerful. [40:33] before AlphaFold3 [40:36] you would be completely blind to this. You wouldn't actually probably know how your molecule is interacting with your protein. You'd be using your best intuition. Maybe somewhere down the line in the drug design project, you would get your structure crystallized with a particular design. That means... [40:52] going out to a real lab six months later, [40:55] if you're lucky.

[40:57] getting a resolved 3D structure. Even then that's just the 3D structure of a single design. [41:03] Not every single change that you make. So our field three completely changes the way chemists work [41:08] can do this design work [41:11] But I would stress that's nowhere near as far as we want to go. [41:15] Because it's not just about what these molecules look like in terms of interacting. We actually want to know how strongly these molecules interact with this protein. Yeah. We want to know... [41:25] other properties of these molecules. We want to understand how [41:30] the way that these molecules interact with this protein and how that changes the fold or the conformation of the protein, how that changes the function of the protein, how it might actually change [41:39] dynamics of the cell. There are so many questions and these are these other alpha fold like breakthroughs that we're working on that also go you know [41:49] we have created incredible models for that our chemists are using in this design process. Interesting. So you're designing some drugs internally. What targets and programs are you focused on? [42:00] So we have a really exciting internal program of drug design. [42:05] projects. These are focused on immunology and oncology. We've been making some incredible progress there and it's been really exciting to see especially how [42:14] these models have transformed the way that we're actually approaching drug design on these programs. [42:18] You're also working with Eli Lilly and Novartis, and recently you announced an expansion with Novartis' partnership. Can you share a little bit about what these partnerships look like? Yes, so we signed these initial partnerships, two partnerships, one with Eli Lilly, one with Novartis. That was fantastic. They brought some really, really challenging problems to us. I think it's no secret that the

[42:45] sort of targets that, for example, Novartis brought to us. These are sort of targets that [42:52] you know, the field and Novartis, for example, have been working on for, you know, 10 years plus. Wow. [42:58] Um, so these aren't sort of, Oh, we'll, we'll try things out problems. These are for real things. Um, [43:08] Last year was an amazing year, both for our internal projects, but also for these partner projects to really see [43:15] how well these models are working. [43:18] It's allowed us to really uncover new chemical matter, working out new ways [43:24] to modulate these targets that people have worked on for a long time. [43:29] You know, it's been amazing to see this new deal, which has expanded our Novartis collaboration, which I think is a real testament to some of the success of the early days of these partnerships. Congratulations. I think it's an incredible milestone, especially just one year in. [43:44] So I'd love to talk a little bit about the team. You've built a truly excellent team composed of the highest caliber talent across many different fields, AI, chemistry, biology. And you've also brought [43:55] outsiders into the field to help question traditional thinking. Can you share a little bit about how you thought about this? [44:02] Yeah, so... [44:04] The space of AI for drug design, [44:07] hasn't really existed for very long. [44:11] So... [44:12] the chances of finding a world expert

[44:16] at drug design, who's also a world expert at machine learning or deep learning, is basically zero. Just because these fields haven't coexisted [44:26] for long enough. I genuinely think about a new [44:30] sort of a field of science that ISO is breeding, because we are, you know, we have these people who really live and breathe the intersection of this. So, you know, [44:41] Because we can't hire these people, I really think about how do we bring [44:47] the world experts at drug design and medicinal chemistry [44:51] and the world experts of machine learning and deep learning. [44:54] and [44:55] get these incredible people sitting side by side. [45:00] 'cause it's not just enough to have these amazing people [45:03] sitting in their isolated teams. Yeah. [45:05] We need people sitting side by side [45:08] Speaking each other's languages. Yeah. [45:10] with a lot of empathy, [45:12] A lot of curiosity. [45:14] Curiosity to understand this new science, to really build [45:18] intuitions in your own language. [45:20] And we've seen [45:22] Just such amazing things come out of this dynamic where you really have... [45:28] you know, a generalist machine learner who doesn't know anything about chemistry or biology. [45:33] Start to come in and understand. [45:36] the problems of a medicinal chemist and a drug designer. [45:40] And when I think about even hiring clients [45:44] machine learners and machine learning scientists and engineers for the research that we're doing.

[45:49] I'd say 60, 70, 80% of the people [45:53] on our team have no prior knowledge of chemistry or biology, maybe, you know, high school or a university level. And, um, that can actually be a real asset because you come in, [46:06] sort of a little bit naive. Yeah. And as long as you're curious, I think one of the key things is asking people [46:13] you know, the curious questions, asking this like stupid questions. [46:17] And then that allows us to come at the problems from first principles. Yeah. Yeah. [46:21] It almost allows us to break through the dogma [46:25] of previous experience and how people traditionally approach these problems. We can think ground up from scratch. And that's a lot of the mentality of how we think about creating these research breakthroughs. A little naive and highly curious and high agency is a very good thing. Yes, exactly. Exactly. So in November last year, you also made a very big move in launching the AlphaFold server, which releases code and model weights for academic use. Can you share a little bit about why? [46:55] Yes, so I mean, AlphaFold has a long, long lineage of being open for this, you know, academic and scientific use. And it was it was really important. [47:06] with this latest breakthrough of AlphaFold3, that we make sure that this scientific community has access to this functionality because... [47:16] Yes, AlphaFol3 is going to be incredibly useful for drug design. It already is.

[47:21] But it's also useful for, you know, many other areas of fundamental biology and just understanding biology and people are using these. [47:30] people are using our 4.3 server and model it in very, very creative ways. Um, [47:35] So it's very important for us to make sure that there is that sort of free use for non-commercial academic work. And it's been incredible to see the take up of that and the use of the server. [47:48] I'd love to talk a little bit about the future. Can you give us a tease of what else is to come with AlphaFold? [47:54] In terms of [47:56] structure prediction as a problem. [48:00] In my mind, I want to completely solve this. [48:04] I think our fold three is a fantastic step on the way of that. There's a significant breakthrough, um, [48:10] But, you know, it's not 100% accuracy. What does even 100% accuracy mean in this space? [48:18] Like with a lot of areas of science, as you... [48:22] start to push the boundaries, you see that the problem opens up into even more problems. That's the addictive part of doing science, right? And I think that, you know, AlphaFold3 is a good example of that, where as you start to get these capabilities, [48:38] you see that actually there are even more deeper problems that we want to be working on and stepping towards. So, yes, understanding structure better and better and more accurately is always going to be interesting for us. But then it's also not just necessarily about static structure. So alpha-3 models these crystal structures, which are almost static crystallized versions of how these molecules interact.

[49:03] But in reality, [49:05] We don't have crystals inside of us. We, you know, these molecules are in solution. They're moving about the dynamic. [49:10] So you can think, okay, well maybe [49:12] understanding the dynamics [49:14] of these systems is actually also going to be really interesting. Yeah. So, yeah. What does a GPT-3 moment look like in AI biology, and when do we get there? So... [49:24] If I think about GPT-3, [49:27] Thank you. [49:27] This is really a generative model. So something that's generating text. [49:33] And... [49:34] The GPT-3 moment for me was... [49:36] crossing over that boundary between, yeah, we've got generative models of text, and they generate some stuff, and it looks like text, but... [49:46] I'm not convinced that it's generated by a human. - Yeah. - And GPT-3 started to be that first point where you're like, [49:53] Oh shit. This kind of looks like a human. And so this generative model [49:59] is actually recreating [50:00] the distribution of data that is trained on. And what is a generative model? Generative model is something that fits the, [50:07] the manifold of data that is trained on it. So when I think about this applied to biology, [50:14] Um, [50:15] You can think about these generative models [50:17] actually starting to recreate [50:20] at that GPT-3 moment, [50:22] recreate what things would actually look like in reality. [50:26] And that's quite exciting because... [50:29] That means that these models are spitting out things that either they actually exist in the world. Yeah.

[50:35] and we can kind of validate that or maybe even discover new things that exist in the world. [50:39] Thank you. [50:40] or they could exist in the world. - Yeah. - Which means that they could be things that we could design or manufacture or create. [50:47] that would actually be stable and work and exist in our physical reality. Yeah. And I think the the cool thing about this in biology is that unlike with language where with language [51:00] When it generates something at human level quality, we can understand that because it is human derived. [51:06] But a lot of problems in chemistry and biology... [51:09] We even struggle to understand ourselves. [51:11] And so when we get to that GPT-3 moment, I think it will look a lot less like GPT-3, but much more, feel a lot more like Move 37 in AlphaGo. Interesting. Where we're starting to see things. [51:23] that are beyond human understanding, but that do exist in the real world, that exist in our physical reality, but are beyond sort of human comprehension. Right. And that's just going to be mind blowing. [51:36] you know, we're starting to see that internally. [51:38] with our generative models. [51:41] that we're creating designs [51:42] that a human drug designer [51:45] I would say... [51:46] I'm not so sure about that. I much prefer this. [51:50] and then you test it out in physical reality [51:52] And the generative model is correct and the human is wrong. That's fascinating. I love the Move 37 analogy. Yeah. When the model starts to see elements of creativity and surpass the human. Move 37 was this amazing move during the AlphaGo games.

[52:07] against Lisa Dole. It was the 37th move of the game, and it stunned the world, stunned the Go world, because it was uninterpretable by a human. It looked like a mistake. No one had ever played this move in the entirety of thousands of years of human history playing Go. [52:24] And it turned out as you unrolled the game that this was the critical move [52:27] that allowed AlphaGo to beat Lisa Dole in that match. Yeah. And we're going to see so much of that sort of behavior. [52:33] coming out of these models, especially when we're applying them to [52:36] things outside of native human understanding like chemistry and biology. Yeah, I love that. [52:42] Also weren't punched a lot today. [52:44] So when will we see our first AI-generated drug in clinic and also in phase one, two, and three trials? [52:52] So we're making amazing progress on our drug design programs. And [52:57] The thing I think about actually is [53:01] as we start to get a whole bunch of these um uh AI designed assets these molecules get into clinical phase how can we actually start to think about engaging in that clinical development um to you know get these molecules to people [53:20] You know. [53:20] as fast and as safely as possible, because there's so much unmet medical need. [53:28] So yeah, here I think about [53:31] you know, what are going to be new ways to engage with regulatory bodies [53:35] What are going to be new ways to incorporate our predictive models for not only how this molecule works for the disease, but how, as we talked about, how it interacts with the rest of the body, you know, the types of toxicity it may induce.

[53:48] I think there'll be a lot of opportunities to think about just... [53:53] and speeding up this process, maybe even completely changing the way we think about [53:58] human clinical trials as we, you know, our AI models become so, we can design these molecules so much quicker. [54:06] in a much more targeted manner with so much more knowledge about how they work. So that'll change the game. But I think we've got a long way to go as an industry to really work out how that changes. Last question. As isomorphic succeeds and potentially as a whole field succeeds, what happens to the traditional world of pharma? [54:24] I think they become, you know, in some sense, pharma will be using AI. I think there's no world where in five years' time you will be designing a drug without AI. That is an inevitability. Yeah. [54:39] It'll be like, you know, trying to do science without using maths. AI will be this fundamental tool for biology and chemistry. It already is. Yeah. [54:49] at least in isomorphic's world. [54:52] that everyone will be using. So it's not going to be, oh, is it pharma or is it AI? It's going to be one and the same in the sense that the whole industry will adapt to that. Yeah. Amazing. Max, thank you so much for joining us today. This was a fascinating conversation. Yeah, it's been a pleasure. [55:08] Thank you.

[55:38] you