Summary IA Summit Fireside chat : Oren Etzioni, Carlos Guestrin, Luis Ceze - YouTube (Youtube) www.youtube.com
6,936 words - YouTube video - View YouTube video
Luis So dr Rn Etzioni, it was the founding Ceo of Ai 2 from its inception until very recently late September. He's now an adviser board member of Ai 2 and a technical director of their Incubator. There's a lot of great companies coming out of Ai 2. And despite him being very young, he's actually officially professor Merit at the of Washington and has been a venture partner at For a long time since 2000. And now on the other side, you have Carlos who's gonna participate with us online He's a professor computer science at Stanford, and he does work in Ai ml, Ai systems, Ai f.
Luis Fix and fairness and also Ai for health. And he has really good taste for shirts and art as you can see there in his in in his video, So Carlos also has been very active in entrepreneurship. He found that Tory. I was a company did some very early work in deep learning systems and putting them to actual deployments. Those acquired by Apple where he worked for about 5 years.
Luis And now he's a professor Stanford inventing the next thing that's gonna become the next great company there. Got great lots of fancy awards, you know, I 1 of the favorite 1 that I... I like the name is computers and Taught award, and also a brilliant intent by popular science magazine. Got So with that, a pass into orange to say a few words, and then Carlos, and we're gonna talk. K?
Luis Or, please.
Oren Thank you, Louis. It's very rare for me that I'm outnumbered by folk from Brazil, but it's a real pleasure. And I do want to start by noting that Luis omitted a key part of Carlos credentials. He didn't mention that Carlos is the best looking Brazilian professor at Stanford computer science, specializing in these topics. And and this is but from his own his own informal biography.
Oren Okay. And I'm... I'm really proud to have had these during relationships with Med for more than 20 years, the Allen school for more than 20 years. Ai 2 from inception and now the Ai 2 incubator. What I want to do today is provide a bit of context in history.
Oren So gonna have a conversation that's gonna be more in depth and a bit more technical than some of the things we've seen so far about generative models. We're but I wanted to help put this in context. You know, you can take the professor out of the university, but you can't take the university teaching. Out of the professor. I apologize.
Oren There will be a quiz. At the end of my talk before I go further so just we'll be handing out so pay attention, please. Okay. The first thing is right. A lot of terminology floating around, Peter lead did a great job putting that in context.
Oren I'm gonna use words like machine learning, big data, if you remember that term data mining, supervised learning, classification categorization, they're basically all the same thing. And the interesting thing about these terms is that they require label data. I used to say the dirty little secret of big data is that it requires label data. So what does that mean? That means if we have a sentence, if we have an image, if we have a genome sequence, we need to associate associate with it a label that says, okay, this is an image of a cancer tumor.
Oren This is a benign tumor. This is a normal lung. This sentence is positive. This sentence negative, etcetera, etcetera, etcetera. Requires label data.
Oren And the thing that we find ourselves in with the tremendous with the web with sensors from space in every arenas that we're drowning in data, but turns out that we're short on unlabeled data, right? And that actually has always been a major impediment to scaling up machine learning. At Fair, the company that I founded some more than 20 years ago in Med and med back backed and wind ended up selling to Microsoft, actually had interestingly a trillion labeled data points back in the day around 2008. And the question is how do you get a trillion label data points. Right?
Oren You can't get that from Amazon mechanical Tu. If you just do the math, it doesn't work out. It turns out, you have to figure out some kind of self supervised scheme. You have to figure out how to automatically label the data. And again, Alex from snorkel, I alluded to that as well.
Oren It's a key problem. So this is the first point, right? These generative models did not you know magically appear all of a sudden, Right? We're all talking about them. 1 of my favorite lines is that are overnight success has been 30 years in the making.
Oren What that means is that we started with machine learning, supervised learning was all the thing. We went to self supervised learning where the the data is automatically labeled, that led that and, of course, the hardware trends, led to massive scale up, and that in turn enabled pre training. Right? The idea that we're going start with a massive foundation model, whatever you call it, a massive model it's going to allow us to do some pretty amazing things. So here's what the world looks like today with generative or foundation models, you start with input of billions of data points, maybe billions of sentences from the Internet or tremendous amount of images, the lines of code from Github, etcetera.
Oren And then we output a model that generates high quality, all kinds of good stuff. That's the pre training phase. Now at runtime, what happens, you either fine tune the model, let's say you want to focus on the legal domain, the way Lexi does or on the medical domain, the way Peter Nuance did you do some fine tuning, but then you prompt it as it's called, you give it a textual input or a description. And what comes out is a document an image of software, what have you. The key point is that we used to categorize.
Oren We used to say this is a spam message. This isn't. The price is going up. The price is going down. Very recently you read about okay, a radio.
Oren Software can say, this is a cancer tumor. This isn't. But now we're automatically generating. We're automatically s synthesized objects and it's a tremendous boo to all kinds of industries. So I just want to put all that in context.
Oren And here's, you, there's tons of these flowing on the Internet, here's a picture from mid journey or a set of pictures, I should also point out that open source is a big part of this. To so it used to be that only a small set of people could do this. That's rapidly changing. Right? So very soon, it's getting democrat.
Oren See very soon, everybody is gonna be able to do this. From their phone. Here's again, just another example, okay, thanks the John Carpenter this works. Here's a fashion show. And what's amazing about this isn't just that this is automatically generated.
Oren What's amazing about this is the people who did it are just some creative people. Right? They're not from big tech. They didn't require. They just use open source that's available.
Oren To do this kind of thing. And so the world, the creative world is being up ended in some pretty pretty amazing ways. The next comment I want to highlight is that at least a small scale, the budget for this was minimal. The cost of these things is an issue. And of course, Louise is from O mel and then all about how to drive that cost down by optimizing the hardware, optimizing the cloud you use using various compilation techniques.
Oren So cost is is an issue here, although that's going down rapidly, access, as I mentioned is infinite, right? Very soon, if not already, anybody can do this. Another huge thing is the diversity of tasks. Okay. So it used to be that if I had fair cast and I want to predict air fares, I had to develop a model for that.
Oren And if I had medical model. That's a completely different stack, completely different model. Now as you've heard a few times with foundation models, I build a single model and then I may fine tune it, but a lot of the work is done ahead of time and done once. The last point I want make is about the issue of reliability here. So a huge challenge of these models is that they hall.
Oren They lie. They they generate. So they invent things. An example might be people said, wow, Now emails to customer service. Right?
Oren We don't have to have a team do that labor. We'll just have Gp 3 generate those. What turns out you can't actually do that in practice, because Gp 3 might invent things to say. Actually, they could be offensive in certain circumstances. Or they could just be downright wrong, like, oh, we're having a free giveaway next Tuesday.
Oren Everything is free. Everything must go, Right? So there has to be a human in the loop. 1 of the big forks in the road for this and we start talking about the future is are we gonna have full automation or are we gonna have human in the loop? My belief is very much that we're gonna have human in the loop?
Oren But this is not an obstacle to to the technology. The my favorite analogy for this and maybe this dates me. But think about web search engines, particularly in their infancy and over time. So web search engines never got the answer right first time, Right? You do a query.
Oren The results were often wrong, but a, we learned to make better queries. And b, we learned... And they learned to make the search engines faster so that the results came back with stunning speed. And that combination means that I can query I can get the results back. I don't like it.
Oren I change the query and pretty soon I get what I want, prompting of these models. I don't think it's gonna be something for prompting engineers and achieving total reliability. I think it's just to be just like querying a search engine. You're querying, a generative engine. You query correctly, you get some results.
Oren You don't like them. You query again with a slightly different prompt. But now here's the killer, Over time and particularly Google was great at this, they crowds. They learned what people clicked on what they liked, what queries they made, what queries they made second time around, they use the billions of queries to make the search engine much, much better. And the same thing is gonna happen here.
Oren As and that's already happening, right? As people query these engines and re query them and produce things the engines are going to get better at doing what we want. So very optimistic about what's going to happen. But it is going to be this kind of human in the loop fashion. And then not to be kind of overly polly There are these 4 challenges though that still remain.
Oren 1 is cost, which I let Luis and his company address the second 1 is control, which I talked about, you do... Despite the fact as human in the loop, you do worry about what these things output. The third 1 is specialization. So if you are in a technical domain, whether it's science, biological data, legal data, you do need to work. To fine tune the model.
Oren It won't... It'll work out of the box, just not nearly as well as it could. And then the last 1 and we actually have a a company at the ai itunes incubator that's working on this very heavily is personalization. So let's say it's producing a copy on my behalf. Whether it's a memo or an email or what have you, you can't rely on a single model to do that because the way I write is very different then the way Carlos writes are the way Louis writes or somebody else.
Oren So we actually need to have a very large collection of models, if we're going personalize this kind of writing, and we see that coming down the pike as well. So I'll stop here. So that my very good looking colleague gets a chance to to give his remarks and hopefully we'll have a good discussion. Thank you. I think you
Carlos carlos Hello folks. Thank you so much for having me here, and I'm sorry I there in person. I love to meet you all. And spend more time together. Or did a great job at setting up the stage.
Carlos And Me thinking a lot about how fun visual models, or large language models are changing how we develop intelligent applications? Say, the life scheduled not in the last 15 years. Do you think back to 2008, for example, machine learning was really simple. You service some data, you did some machine learning math, and that led to profit which is the time for me, man, showing that my curve was better the new curve with some benchmark data and then write your paper at talk conference about it. Life was simple, and we thought about data, the structure for model and how we evaluate.
Carlos But in the mindset of 2008, the model structure, what the machine learning math was about? Was the most important thing. You sound a little bit about the data, you almost didn't think about how evaluate these things. And then the big data revolution came up. And folks like Rn and Down and others had great observations saying that simple models with lots of labeled data will lead to good predictions.
Carlos And this was typically in a fully supervised setting as already set up. And now we're totally changing with flash language models and foundation models. How we think about developing applications going beyond this idea of big data. And so today, I'll talk about 3 trends that I see in the space, The first 1 is that big data is not the priority anymore in my opinion. You can solve complex problems with little data.
Carlos So the pipeline of turning 22 looks really different. Used lots of big data noisy data to build this language models. And then focus on prompting of fine tuning to solve new task, which enable the development of new intelligent applications for new verticals. So this is task specific, highly curated small datasets sets for fine tuning prompting that lead to a vertical solution. That you really care about.
Carlos So in the mindset right now, the quality of this prompting and fine tuning are gonna be really important. I say that the model structure typically only pay companies or organizations with lots of resources for compute and people, are going be able to explore large scale choices of model structures, what we focus on in 2008, Now, most of us are gonna focus on how to make this vertical solutions practical by finding good quality. Data for it. And great, companies are gonna be built through this vertical specific prompting and fine tuning, and data quality key because this models can be great, but it's not enough. They can follow the cliff rapidly.
Carlos So focusing on the data, and focusing on your task can be insufficient to develop the applications you care about. So let me share a couple of examples. Of what Gp that you can do. So for example, you can tell it to write the review based on some prompts like food quality or poor and so on. And if you say the staff was caucasian, versus the stock was Mexican, you see very little, very very dark differences.
Carlos So for example on the left, it says the only upside was the stuff was mostly caucasian, which is disturbing. While on the right side, the Mexican stuff did not make us feel welcome. So if you're building an intelligent application and you're thinking about how this models work, you have to think gui beyond just the data you're giving it to issues like or imported out, where they can lead us to not reflect the values we're hoping to show to the world. Here's another quick example. You ask Gp to complete the story with a white king, it says the king was handsome, and the prince fell in love with him.
Carlos While, with the black ink, she was shocked that it was a black man. And I worry about this a lot. Worry about how much we're deferring to this Ai and what impact will have on our societal values going forward. So beyond the quality of the data that you generate and the little segments they use to prompt or fine tune. I think evaluation and how we curate the datasets sets and how we create the model those is gonna be the most important focus going forward.
Carlos So our second trend is the shift from supervised learning is So predicting, for example, the probability you click on add to generation. 2014, for example, generative models we're coming up to a pre cool. If you sq right, you'll see that second picture looks like a frog. But today, Generative models are amazing. You can generate hot text, you can generate that says you can win art competitions, we generate to their images.
Carlos And this is quite accessible. If you haven't tried new journey, I highly recommend it, it's really pretty cool. I tried it... And here's some example was the first 2 where the ones that came before after mine on the stream and the 1 on the right is what I put, I I love riding bicycles. Sorry I put a person riding a bike fast down the hill, and and this is what I got.
Carlos It looks like the person's riding uphill. I don't know exactly how to change it to make it go downhill, but it's an amazing image. So generation is opening up huge opportunities of new ways that ai Ai can be applied in the world what intelligent applications might be beyond what we usually thought about was small convenience of being able to predict something better. And of course, now we're really democrat Ai. We're thinking about it new ways.
Carlos The first phase of the democrat commercialization of Ai for me was prep trained Apis. Let's say, an optical character recognition Ocr system. Can give some image and give me text back. Then we try to simplify training Apis, so that's what Tour was about. Make it easier for anybody to train models with the data that they have, mostly in supervised settings.
Carlos Today, we've gone beyond that, and we to scale to deployments and optimization and deal with the real world of building models in practice and this is what O mel is focusing on. And this is an incredible trend But and in awe of how large language models, foundation models have enabled others beyond developers. To do amazing things of Ai. The thing that I like the most about me journey you haven't tried is the fact that you create the social experience of building prompts together, learning from other people's prompts and jointly creating a viking version of Build Clinton as you see on the bottom set here. And I wonder where this is gonna take us You might remember the scene from Star trek 4 where Scott is trying to talk to the mouse.
Carlos And this is not siri. This is not him giving commands to the computer, set timer for 5 minutes. This is about a conversational experience of programming the computer, where we can say more of best and less of that. And large language models, give us the opportunity to create new experiences for programming, for bringing Ai applications to a wide range of people, they never thought they could program an Ai. So we have solving more complex problems with less data to new tasks in generation to now many more people being programmers, I think we have an amazing opportunity to doing incredible things in the world.
Carlos In the space, if we're thoughtful about the broader impacts of the Ai technologies that we build and the places where it doesn't reflect the values that we brings to the world. And with that, thank you so much for having me, and I'm excited for this discussion. Right.
Luis Okay. So thank you, Lauren and Carlos. So I I will start with a question. And Try to find enough time for the audience for first questions. So starting about questions there too.
Luis You both mentioned, you know, foundational models platforms and building businesses around it. So could you elaborate more on how do you see that playing out. For example, you know, some folks argue that human knowledge should be universally accessible. Everyone should own it. So nobody should own a large model that pull folks build specialized models on top of.
Luis Should that be open source, should be the role of government to invest in it. So how do you how do you think about financial models platforms from business perspective. Carlos starts since your remote, you have priority.
Carlos No. I was ready to speak, so I'm gonna go first.
Oren I was actually pointing to get you, but I I never turned down an opportunity to speak. That's I see I love the framework that Carla set up because things are moving. And so initially, right? We had open Ai and they very carefully controlled the access to it. And now we have a few companies that have an in house is Peter was alluding to, but we also have open source project.
Oren We have startups doing this. So I think that it's very quickly gonna be the case that there's a ton of these. And this will not be a particular bottleneck. So as Carlos and I both in different way said, there's a rapid trend towards the the democrat commercialization? Did that answer
Luis Even even with the cost of training, a foundational model from? Scratch being in the tens of millions or approaching hundreds of millions of dollars. So who who pays for that and who who should own it in the end?
Oren Well, I think that because a number of things are put out seated. And in other words, you can already start with a vanilla 1. As actually, you've pointed out to me in previous conversations, the cost of inference. Is is what drives this. Right?
Oren So it's train ones should be trained more times. But nevertheless, generally trained ones, then it's run many, many times. So the cost parameter that I look at is what's the efficiency of inference, Right? Because if I wanna use this for a million customers, right, the way the search. So so again, the analogy to web search engines is perfect.
Oren The biggest cost of Google is not crawling the web, even though that's a substantial cost, but it's it's done relatively rarely compared to the billions of queries per day. So on the inference side, I think there are many techniques. From o and dis... Other ones to drive that cost down and to have that cost be manageable. So So long as the initial model isn't the trade secret, Right?
Oren Open Eye or close the eye as there's something called won't release it, Once I have the initial model and I can run with it. I think I've in pretty good shape.
Luis Great. Thank you, Chris
Carlos Yeah. I don't know. A slight contra view here. I worry about I worry about the fact. If and what is open to to the world, then I think lots of folks would be able to build intelligence applications with limited data, which is which is amazing.
Carlos We have seen that the power of the model tends to scale with the number of parameters and with the amount of data that you got to train with. In the beginning. And and I do have some concerns and some folks here some have even better understanding of those concerns that Today, that level of scaling is pretty limited in in terms of access. And so I don't know the answer who should pay for it. And and I'm hoping that some folks are able to invent new techniques or or new ways to collaborate advanced large sets of people.
Carlos To be able to train the models as well. I think the... The actual descriptions of the models and the other line code could be open a source relatively easily but the training process will still be pretty expensive for a while unless we find other ways of of talking that to train those base models.
Oren I just if I can add what's what's a Good Panel With That A little bit of friendly controversy among Friends. So I I'm less worried about this thing Carlos because a heuristic that I use is when a problem can be defined as an optimization. Problem effectively as a computer science problems. Our colleagues are just incredibly adept and it's not even just moore's as law. It's delving into the stack and at getting it to be 10 x better and then getting it to be 10 x better again.
Oren So we see that. So I agree with you. Right now, these are issues but the trend is to get a rapidly rapidly cheaper. The training, the inference, the model size 2, right? There's an interesting or model gonna get.
Oren Bigger or they gonna get smaller? We'll there be smaller specialized ones. The the claim that I'm willing to bet the favorite... Your favorite drink Carlos is that 3 years from now, this will be easily an order of magnitude cheaper. To me the harder problems is how do we use this in a way to maximize the value, That's an ill form problem that requires creativity, you can't give that problem to Gp 3.
Oren Right? If you ask Gp 3, what's the your best and highest use to build new startups? You're gonna get garbage. But if you ask a bunch of smart entrepreneurs that question, You get what we we're seeing right now, which is a tremendous flowering of all kinds of smart apps use using General. Yeah.
Carlos And and then the last the last bit here of orange statement is totally un controversial. I think right now, we're in the you know, man knows no hands kind of stage where all of these folks saw in non words... In a good way with the capabilities, that we're seeing all sorts of ways that there are possible what's possible in the space. I think as of further technology, we'll see some phases of consolidation and focusing where certain areas of impact. Will will be defined and and we're gonna have rapid development in those areas.
Carlos But I'm just amazed with the creativity of what's possible today. Respect to the previous statement. I I know you're wanna open some of their questions I'm gonna stop here, but start to a previous statement. I think it it's a it's a really good 1. Like, I'll take you the bath even though I think I might lose because it give me a chance to have a drink with you again.
Carlos But what I'll say is in I was more bearish until we saw stable diffusion come out. Mh. And and I think that that's kind of an indicator that you might be right. But even then, there was a lot of compute resources invested. And and we're still not there in terms of the flexibility quality stable decision compared to other models.
Carlos So betting on human creativity and ingenuity is always a good thing to do. And that's what you're you're sort of better. So I I'm I'm hoping to lose, but I'll will take you up.
Luis Okay. Alright. On that note, 1 more quick question then we'll we'll pass the audience. So... You you both make the point and I completely agree that I I mean all when what what these models can do.
Luis I love playing with them as well. But the role of prompting engineering is actually really important right now because these models despite being amazing. They are brittle, they can fail in catastrophic ways. So now putting your computer science professor hats here. So how do you see the fundamental possibility of actually being able to trust these models to do the writing to fail safe.
Luis So we can actually let them do their amazing things in a way that requires less intervention by humans to make sure it's doing the right thing. Is this a fundamental problem? Do you think it's sol?
Carlos Yeah. I oh. Okay.
Oren No. Carlos, please.
Carlos I didn't focusing my a lot of my research staff than exactly this question. What is a framework for? Understanding, evaluating testing. And what does the altered loop of machine learning look like in the context of the 20 22 start to school. Right?
Carlos And I think a lot of incentives in in the research community have been focused on, you know, can I create a new task, and I solve them? That can I tweak a model and I so on and bringing more of that software engineering mindset, what are we better define what what our goals are and how to test and evaluate, think do things like failure analysis and systematic evaluations, I think is gonna be very important focus to bring this forward? So I am hoping that with the... In addition to the rate of innovation, in terms of the performance of the models or the capabilities of the model that we also have another great set of innovations going on to build not just trust, but really to build the capabilities within the the overall goals that we're trying to achieve.
Oren What I would say is that this is an excellent research problem and folks at Ai 2 and elsewhere working on it. As well. I still think it's going to be an extremely long time, maybe never before you have Gp 3 or a Gp 3 based app, Gp 5 running your nuclear power plant. Okay? It's just it's not that kind of technology.
Oren That's why I think that the analogy to web search engines is so profound. We can use if we have human in the loop and if we have rapid iteration, We can use highly unreliable technology in a very empowering way. So my bet not for when the research is done, but for the coming years is that it'll will be brittle. It'll be unreliable. But if we make it fast, fast interact with.
Oren It'll still generate tremendous amount of value.
Luis That's right. Let me remind you that nature also builds highly reliable systems out of unreliable parts here. So this it don't have to be perfect. It has to be, you know, at least as good as humans. Right?
Luis So anyways, okay. With that, Questions for the from the audience, so if I can see people. Any questions? Do otherwise I ask another 1? Don't be shy.
Luis You right there in the back. You have microphones running there. Okay. Alright. No.
Luis I can't. Alright. I'm shout You see technology. I see. Yes.
Luis So I think it's in extension of the previous question, how do you avoid foundation models from falling off the cliff? Right? So
Oren in the cliff being say toxicity or racism, I'm not sure so many cliffs. Which 1 did you mean? How do you avoid them falling up any cliff? I have a very simple answer. Guard rails.
Oren No guard rails. I'm joking. I I think it really depends on which cliff we're talking about? There's certainly a lot of work on both modifying the input, right? Analyze the input and changing it before training.
Oren And what's even easier is monitoring and modifying the output, Right? Because if you modify the input, you have to retrain. And that's a very expensive thing. If you just effectively bleep out certain comments, that's a lot easier. So this I don't mean to say this is a solved problem.
Oren It's very much a of an open problem, but it's not 1 that I think is gonna be... A huge blocker. It's just 1 that we need to pay attention to.
Carlos I think it's a huge blocker. I no. I I think it's a huge concern. I I love orange frame. Framework from the previous question, we're talking about you now, is this a technology ready for mission critical applications like Running a nuclear power plant, I always ask myself what is ready to run another nuclear power plant.
Carlos But starting from generation task, and and human augmentation tasks, which is what you're advocating and what our advocate. I think is a great place where there is an actual safeguard with a human in the loop. But I still think is that is insufficient to address some these decisions that in how I call falling off the cliff and the question is which clips are more important than another is what you're trying to point out older. And and some of this is gonna be application specific, but the reality though is that thinking that addressing the gap between Ais in our values. There's just some sort we can sprinkle on top of the hand.
Carlos Like, you know, some some post processing. I think it's it's a bit of a limited perspective. I'm trying to be a little, you know, How can I say this finalist about it? I think we we need to we need to be thinking about decisions from the very beginning of the development. Of Ai systems and not after the fact.
Carlos And I think there are many places where we can inject our preferences or values or entered. There is in the training process, there is, in all the choices that we make along the way. And, of course, as post processing, like you suggested. And I think a focus on that from the beginning is gonna be really important as we move forward in the Ai space.
Luis Great. Thank you, Carlos. So 1 more question. Perhaps not about Guard ray.
n/a Okay. Can you hear me? Yeah. Okay. So thanks for first for the very insightful discussion.
n/a So... Speaking of scale and everything that's happening on the trend. So couple of years back, we thought that scaling the models was the way to go. So ever increasing models with more and more billions of parameters. But then there was this paper by deep mind, the Chin chili paper that kinda show that scaling data is sometimes much more important than scaling the model.
n/a This even though scaling the model parameters also gives you a payback that payback is small compared to scaling the training data set. So speaking of trends and predictions for the future and guess work. What do you think is going to happen in the future as are we running out of training data? And what's going to happen when most of the data that we can collect from the Internet is actually generated by these models.
Luis I'll avoid a circular dependence. Yeah. You do you want to start orange?
Oren Sure. I guess I would say it's very much early days. And so It's not like these models are getting smaller in the tin model or I don't know Gp 4, which is starting to make the rounds. May not be bigger, certainly not an order of magnitude bigger. The way Gp 3 was relative to Gp 2 But the jury is still out on what's the right number of parameters.
Oren There are many dimensions to this. However many parameters, you do, how much hyper parameter tuning, did you do, etcetera, etcetera. And, yes, how much data, but it's not just the number of data points, Right? Data that can be redundant and repetitive. I guess I would say that there's a highly technical area, of optimizing the training process along all those dimensions.
Oren And to me, the thing that's guaranteed is it's gonna keep keep getting better. I thought initially we were reaching a point of diminishing returns. Now it seems like nothing could be further for the truth. At least for the near term, these models are going to keep getting better, not necessarily more reliable, but just higher higher fluency, maybe less hallucinations, longer objects you've already seen, right? The images etcetera, etcetera.
Oren So they're getting better. And I think the jury is out on those on those technical details. And definitely there's going to be synthetic data as you were alluding to. Right? So these these models generate you can imagine the model's also training each other, Right?
Oren The way in alpha, right? The the the model is playing against itself, imagine a chatbot, right, built on top of this this con with itself at tremendous bandwidth.
Carlos Yeah. I I think that the... A lot terms a different parts of the question here. I think Rn, Down weld and others I could argue again that even for the current model structures that having more data would be a a helpful thing to do. I think we're we're gonna revisit that cycle, especially when it comes to training the foundation models it's themselves.
Carlos And III was in struck by your comment that we're gonna run out of data. I think there's still a lot of world knowledge. Out there that we haven't made enough of a dent into bringing into those system. So if I can imagine, you know, all the books that my kid is reading right now and all that they're are learning about the world through those. I don't think Care models have yet been able to comprehend them in a way that all of that data has been extracted through and the knowledge has being synthesized.
Carlos So I think there's lots of opportunity to bring new data and use the data better, than we have been able to do so far. Important thought on not too.
Oren Yeah. I just want to add actually another point, which actually I've learned from Carlos and others. We tend to think, right, because these are the right. We showed you these cool things with images and with text and these comments that these models generate. But the use of these models unstructured data like c copilot, which was showed So on software and potentially other sources of structured data.
Oren And particularly in biology, this could be massive. So whatever is gonna happen with textual data, there's a whole world of this where you train on biological data and magical things happen.
Luis Yeah. Just on that point, I would say, as long as there are living systems on Earth, there'll be more data, you know, not worry you about it. So there's no more data than a worry about life on earth, you know? So Okay. So let's see.
Luis Let's 1 more question from the audience and then we'll we'll conclude. So I saw 1 hand there in the. So, yeah, please. K.
Carlos Sorry. Can you summarize the question because I didn't...
Luis Tell me if I if I got this right. So the Guestrin how do you think about, you know, Ip issue separating my data from the data is actually included in beauty and foundation model. And how do you think about that in the context what these models can do? Is that right?
n/a Yeah. And especially people that wanna opt out of having their data as part of those large models? And other people monetizing them without them getting paid.
Oren So I agree with you this is a huge issue. The way it was in the past as I understand them I'm not an attorney or definitely not an Ip attorney is that if you put stuff out there and it was used to train a machine learning model, that was covered actually under fair use. That was okay to do. But now as you said, the problem is different. I'm an artist, I put my art out there.
Oren And now it's learned to model and part of that model is producing art in my style that I didn't produce. Right? So now instead of coming to me and licensing my picture would have you somebody's gonna type a prompt into an engine, and it'll put something out in the amazing artistic style of ore it's Y, and I'm out of you know, I wasn't making that much money before, but now I'm I'm I'm broke. That is a huge problem. So I think that's a illegal.
Oren Problem. And I anticipate that the courts will have to work through that. But you hit the nail the head this A real Issue Here.
Luis Great. Okay. With that, Let's Think The Panel Rn In Carlos. Thank You.
Carlos Thank you.
Luis