Episode Categories:
Resources:
Join the discussion: Got something to say? Let us know here
Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineup
Connect with Jordan Wilson: LinkedIn Profile
Try Our Free AI Prompting Course: Register for our free Prime, Prompt, and Polish AI Course!
Revolutionizing Autonomous Driving: How NVIDIA's Latest Tech Could Transform Our Roads
Autonomous driving has been a goal tantalizingly out of reach for many years. However, emerging technologies and recent advancements are pushing this concept towards reality. Business leaders and decision-makers need to stay abreast of these developments to leverage the opportunities and mitigate the risks inherent in this transformation. Here’s a detailed look at the latest from NVIDIA and how it might impact the future of autonomous vehicles.
NVIDIA Halos: A Comprehensive Approach to Safe Driving
NVIDIA's new innovation, Halos, represents a significant leap in autonomous vehicle safety. This full-stack system integrates hardware, software, tools, and safety principles into a unified driving stack. By combining these elements, Halos aims to boost safety not only for NVIDIA’s own autonomous vehicle (AV) programs but also for its partners. Its potential to enhance road safety is especially crucial as AV technology starts to hit mainstream consumer markets.
From Mars to Main Street: The Evolution of Autonomous Technology
The evolution of autonomous vehicles has parallels with the aviation industry in terms of the developmental trajectory. Similar to how flight took a century to achieve today's safety and efficiency, AVs are progressing year on year. Currently, advanced driver assistance systems like Tesla's Autopilot are available on a massive scale and autonomous taxis operate in cities like San Francisco. However, fully scalable deployments remain a significant technological challenge.
Generative Simulation: Creating Diverse and Complex Scenarios with Cosmos
NVIDIA is making strides in simulation technology to address the edge cases in autonomous driving. Their generative simulation platform, Cosmos, allows for the creation of diverse scenarios by generating simulations from textual prompts or images. This assists in stress-testing vehicles under numerous conditions, thereby enabling developers to build more robust systems. While challenges remain, particularly around physics realism, these simulations are poised to become a cornerstone in AV development.
NVIDIA and GM Partnership: A Broader Vision of AI in Automotive and Robotics
The NVIDIA-GM partnership underscores a vision that extends beyond autonomous vehicles to embrace possibilities in manufacturing and AI-driven robotics. This collaboration is aligned with NVIDIA's broader ambitions in advancing ‘physical AI’, where cars are only one embodiment. While specific applications and timelines remain under wraps, this partnership is poised to yield significant advancements across multiple areas within the industry.
Data-Driven Development: Overcoming Localization Challenges
One major challenge in deploying autonomous vehicles is their location-specific nature. The driving culture and infrastructural nuances vary vastly across regions, exemplified by contrasting signaling norms even within the same country. Currently, the industry is shifting towards data-driven solutions, leveraging large datasets—including internet videos—to train AVs more effectively. The goal is to reduce reliance on region-specific data while scaling the operational design domain for economic viability.
Positioning for the Future: Harness the Tech-Driven Transition
NVIDIA’s latest technologies highlight a pivotal moment in the development of autonomous vehicles. As businesses gear up for a future where such technologies become prevalent, understanding and adopting the right tech strategies will be crucial. The convergence of simulation improvements, data-driven learning, and strategic partnerships represents a powerful opportunity for those ready to capitalize on this tech-driven shift.
In summary, NVIDIA's advancements reflect a readiness for substantive changes in the AI and autonomous vehicle landscape. The organization's efforts to drive innovation in safety systems, data utilization, and collaborative infrastructures present a significant opportunity for business leaders to stay ahead in the evolving world of autonomous driving.
Topics Covered in This Episode:
- NVIDIA's GTC Conference and Autonomous Vehicle Announcements
- Marco Pavone's Background and Role at NVIDIA
- Introduction to Halos Full Stack System
- Overview of Current State of Autonomous Vehicles
- NVIDIA's Role in the Auto and Autonomous Vehicle Industry
- NVIDIA and General Motors Partnership
- Changes in Autonomous Driving Technologies
- Generative AI and Simulation in Autonomous Driving
- Challenges in Scaling Autonomous Vehicle Deployments
- Handling Location-Specific Driving Behaviors with Data-Driven Paradigms
- NVIDIA's Use of Foundation Models and Simulation Technologies
Podcast Transcript
Midroll [00:00:00]:
This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life.
Jordan Wilson [00:00:16]:
Could this year be the year that autonomous vehicles become the norm? Right? You know, I know we've been hearing about how AI is going to, you know, improve autonomous driving, and it's gonna bring us, you know, fully autonomous vehicles. We've been hearing this for many years. But I think today's guest is gonna help really open our eyes and our ears, to maybe some of these new technological breakthroughs that have actually changed the research going into this and might make this a possibility. And, you know, we are here at NVIDIA's GTC conference. A lot of new announcements when it comes, to autonomous vehicles, and we're gonna be going over those and, you know, talking about how this might impact all of us. Right? The the the roads we drive, the safety of the future of autonomous vehicles, and I'm very excited for today's conversation. I hope you are too. So, hey.
Jordan Wilson [00:01:14]:
What's going on, y'all? My name is Jordan Wilson, and welcome to Everyday AI. This is your daily livestream podcast and free daily newsletter, helping us all keep up with the ever changing world of AI and how we can actually learn from the experts, you know, helping build it all so we can grow our companies and our careers. That's what you're trying to do. You're definitely in the right place. Make sure if you haven't already, please go to youreverydayai.com. Sign Sign up for the free daily newsletter. We're gonna be recapping all of the important insights from today's interview there as well as everything else that you need to stay ahead. Alright.
Jordan Wilson [00:01:46]:
So enough chitchat. I'm excited for today's guest and talk about everything new that NVIDIA is working on in the autonomous vehicle space. It is a lot. So please help me welcome to the Everyday AI Show, Marco Pavone, the associate profess an associate professor at Stanford University and the lead autonomous vehicle research at NVIDIA. Marco, thank you so much for joining the Everyday AI Show.
Marco Pavone [00:02:08]:
Much for having me.
Jordan Wilson [00:02:09]:
Alright. I'm excited for this conversation. This is, I think, one of those hot topics that people love talking about. But before we dive in, can you just tell everyone a little bit about your your background and what you do, here at NVIDIA? Yeah.
Marco Pavone [00:02:21]:
So I'm a roboticist. So I'm a faculty at the Stanford, and I also did a bit of research at, NVIDIA. My work is in the field of autonomous robotics. So how do we make robotic systems capable of making decisions on, their own, especially in high stake applications, like, for example, self driving vehicles or outer space vehicles. And prior to joining Stanford, I was a a roboticist at another Jet Propulsion Lab. So still working on self driving vehicles, but on Mars
Jordan Wilson [00:02:54]:
instead of Earth. Yeah. Yeah. It's it's it's not every day you get to talk to someone that's, you know, helped the autonomous you know, which which project was that on Mars that it was?
Marco Pavone [00:03:04]:
It was one of the Mars landing missions where NASA sent a rover to Mars, and the mission was a a success.
Jordan Wilson [00:03:11]:
Love that. So, yeah, we're we're we're bringing, you know, some some research from Mars to, you know, your streets here. So, you know, there was a lot that's been announced so far at NVIDIA GTC, when it comes to autonomous vehicles. But, you know, one of the things I wanted to talk about is halos. So can you tell our audience what that is and how it's ultimately, you know, going to impact everyone else on the roads?
Marco Pavone [00:03:36]:
Yeah. Sure. So Halos is a full stack system that comprises hardware, software, tools, and, safety principles to combine all of these elements into a safe driving stack. And, it's exciting because it's basically unifying all the investments that NVIDIA has been made in the past, two years on the topic of automotive safety into, unified product. And I believe this problem is going to boost, safety both with respect to the, you know, own, AV program internal at NVIDIA, but also helping partners in making their program, you know, their programs even safer.
Jordan Wilson [00:04:22]:
So, you know, can you just bring us up to speed? Right? Because, you know, even people, you know, who've listened to this show, we've talked about autonomous vehicles and and AI, you know, in years past, but bring us up to today. What, you know, what has been successful? Because there's obviously, you know, fully self driving cars on the road, right, in certain states where it's allowed. But, you know, where are we at, you know, in the, you know, fully self driving autonomous vehicles? What's working? What's not?
Marco Pavone [00:04:52]:
Well, autonomous vehicles are becoming a reality, and I'm sure that you have heard this sentence many times in about a few years. But they like, you know, if you come to San Francisco, you will see robotaxis, providing, rights to customers without any CPT driver on board. And, very advanced driver assistance systems, like, for example, the Tesla autopilot, are becoming available on a massive scale. So my point is that, the sales driving technology is now graduating into becoming a consumer technology. That's why I feel confident by saying that the autonomous vehicles are becoming a reality. We still have challenges. So we have, you know, solved the problem yet is I'd like to do a parallelism with respect to aviation. It took us, like, a hundred years to get to an industry that is as safe as it is today and as efficient as it is today.
Marco Pavone [00:05:50]:
So this is a marathon in it's not a sprint. What are the key technological challenges? Well, in the context of a full self driving vehicles like, robotaxis systems, their deployments is still relatively limited in few cities in the world. And making, scaling up those deployments still represent significant technological challenges as, requires scaling the algorithms, allow the algorithm algorithm to generalize to new situations in a much more effective way. So we need some innovation, there throughout the development cycle from simulation to training to algorithm design and so on. Same thing for, semiautomated systems. The semiautomated systems are already available worldwide, but, of course, we want to increase the the availability, and that's again and that again requires, technological innovation.
Jordan Wilson [00:06:48]:
And can you maybe just, you know, help our audience better understand, NVIDIA's footprint right now in the you know, kind not even just the autonomous vehicle industry, but just the auto industry in general because, you know, it's something I've found out through the years by getting to talk to, you know, really smart people such as yourself. But, you know, people don't know. You know, you probably have, you know, or might have NVIDIA, you know, hardware in your car. Right? Like Tesla, you know, is probably using, right, NVIDIA's data, you know, their data centers. But can you just bring us up to speed? What is NVIDIA's footprint right now in the auto industry?
Marco Pavone [00:07:22]:
Yeah. Absolutely. So NVIDIA, first of all, is both a product company and an ecosystem company. So NVIDIA has a substantial investment in developing its own, autonomous vehicle solution, which we we call NVIDIA Pride. And it is doing so in collaboration with partners, such as, for example, Mercedes. And it's also helping since it's also an inconsistent company, it's also helping other AV companies to develop their own Mhmm. AV programs in many different ways. There is no, like, a a unique recipe.
Marco Pavone [00:07:58]:
It could be by providing, the NVIDIA or come automotive grade chip. Many of the autonomous vehicle companies out there are using indeed NVIDIA hardware. It may be by providing data centers to train the AI. It could be by providing simulation technologies and so on and so forth. Every company is a bit different, but this is also what from a researcher like myself makes it, exciting because I have an opportunity to really scale up my contributions, even beyond the compliance of NVIDIA to really the entire ecosystem. So, you know, speaking of
Jordan Wilson [00:08:34]:
of different brands or different companies, also some exciting news, NVIDIA and GM. Talk about this partnership a little bit and, you know, when we when we might see, you know, that partnership actually out there on the roads. Right? I'm not gonna hold you to it. Right? But, like, what what's coming in this partnership?
Marco Pavone [00:08:53]:
Well, in terms of timing as you can imagine, there's a little bit of sensitivity. Of course. To comment on that. I would say, though, that this is super exciting partnership. I was also impact involved in the discussion, so really happy to see that coming to fruition. One of the interesting aspects of it is that there's a partnership regarding automotive, but also manufacturing and potentially, email robotics. So it's a very broad partnership, that also plays well with, NVIDIA's, broader ambitions. Of course, as we said before, NVIDIA has a very strong, autonomous vehicle program, but, NVIDIA is also scaling up this program to what we refer to as a physical AI problem, whereby cars are just one, instant session, one embodiment of a broader concept that is that of physical AI.
Marco Pavone [00:09:48]:
In addition, for example, to humanoids or autonomous mobile robots and so on. And so this collaboration with the general motors can also, impose this kind of broader vision. So, like, simply, like, is does this just mean in the future,
Jordan Wilson [00:10:04]:
you know, are we gonna see, you know, different versions of of GM's vehicles have autonomous, capabilities, or is the long term goal with this partnership to have maybe most or all of GM's vehicles? Like, what's that gonna look like? Is it just gonna be kind of, like, certain, certain vehicles in the future are gonna, you know, kind of benefit from this partnership, or is it just kind of all vehicles, like, longer down the line?
Marco Pavone [00:10:29]:
As I said, this is Okay. No. No. That's fine. That's fine. Problem.
Jordan Wilson [00:10:33]:
No. Okay. Alright. Alright. We'll, we'll we'll just have to follow-up on that when the, when the news does come out. Yeah. But, you you know, so I'm curious.
Marco Pavone [00:10:40]:
Is it, like, you know, very fresh out of press?
Jordan Wilson [00:10:42]:
Yeah. It's it's extremely fresh. Extremely fresh. So, yeah, we'll we'll follow-up with that once, once that news is officially, released. But, you know, one thing that I do wanna talk about is, you know, and we even mentioned this. Right? Like, there's been a lot of excitement, right, in this space for three, five, ten years. Yeah. What specifically, do you think has changed over the last couple of years, that that leads you and and and your team to believe that, you know, now is this time, you you know, that, you know, we are kind of hitting that, that moment when this might become much more common, autonomous vehicles on the road.
Marco Pavone [00:11:18]:
So for for all, even more than ten years. Right?
Jordan Wilson [00:11:20]:
So That's true. Yeah.
Marco Pavone [00:11:21]:
Doing my PhD between 02/2006 and 02/2010 at MIT. That was the time where, the autonomous driving technology was, starting with its baby steps. I would say it's almost been twenty years. Okay. There we go. But so the way we build autonomous system today is very different from what we were used to do, twenty years ago. Of course, there have been, of course, a lot of lessons learned, but most importantly, the technology has changed. And there is no, like, this single technology that has really, you know, changed the game completely, but a convergence of technologies from all the way from hardware.
Marco Pavone [00:12:01]:
So, basically, having a dedicated chips and a dedicated sensors, all the way to, as you can imagine, AI becoming pervasive in the design of autonomous systems. And in that context, I think one of the most exciting, opportunities, and this is a little bit debated here in the the community, is the opportunity of, leveraging a so called Internet pretrained models. You might be familiar with TTP, for example. With the idea that, with this type of models, we can, we have an opportunity to bring Internet scale knowledge to the task of, driving. Think about how you learned about driving. It took you a few hours probably to learn how to drive a car simply because you brought a lifetime of experiences Sure. To the task of driving. Well, that's a hypothesis about behind using this kind of interpretive models to bring multiple lifetime of experiences of generalist knowledge to the task of driving.
Marco Pavone [00:13:00]:
So that's another AI in general and potentially Internet between models in particular provide a key opportunities to improve the technology. And another, I will say, big technology that has made amazing process in the past two, three years is the simulation technology. And NVIDIA actually had a number of announcements related to simulation. Simulation is has always been a holy grail in robotics. And and now, finally, we are simulators that we can use throughout the development life cycle from the training of the vehicle, the training of the AI, all the way to the testing of the AI. So the key the challenge with simulation historically has been the so called simulation to realism gap. Mhmm. And this gap is becoming increasingly closer along a a number of dimensions in terms of visual realism, in terms of behavioral realism, how faithfully we replicate the behaviors of humans in the road, and so on and so forth.
Marco Pavone [00:14:01]:
So long story short, I wouldn't say it's a single technology that is really, pushing this industry forward. It's really a convergence of technologies from the chip all the way to the algorithm or to a simulation that all of this you know, now are finally coming together and really pushing this technology forward.
Jordan Wilson [00:14:20]:
Yeah. And and I do wanna talk about that a little bit more. So kind of this this concept of using, you know, NVIDIA's new generative AI technology, Cosmos. Correct? So, like, walk us through that. And, you know, I know this might be difficult, you know, to imagine on the podcast. So in the newsletter, we'll link to some of these videos, you know, and and how Cosmos helps. But walk us through these these simulations and how specifically, right, maybe, you know, since we've hit this generative AI wave, how does that help with, you know, NVIDIA's ability to use more diverse simulations, and how does that make, ultimately the autonomous vehicle, sector safer?
Marco Pavone [00:15:04]:
Yeah. That that's a great question. So, typically, simulation, is restricted to the scenarios that are authored by a human. So a human saying, okay. I want to test the vehicle with a particular intersection, so I'm going to draw a map with respect to which, a vehicle has to drive. It's fine. Of course, it doesn't scale to millions of cases. Right? Or there are new technologies referred to as neuro reconstruction technologies that allow you to reconstruct it in three d scenarios out of drives that are recorded.
Marco Pavone [00:15:40]:
This is all fine, but, for autonomous vehicles, it's really a game of the last, five percent or 1%. It's all about thinking about very complicated corner cases. And that's where generative simulation comes in. So this new technology, and the cosmos is one of the prominent examples, allow you to simulate allow you to generate a simulation out of actual prompts or images. And so the this allows you to create a completely new simulation scenarios to, for example, stress test your vehicle. So it literally allow you to automatically, and in a way that is highly scalable, generate a plethora of corner cases that can allow you to better more robust systems and also test those systems. Now it's still a technology in a development, so there are still challenges. Like, for example, physics realism is a challenge to what extent these generated simulations, for example, obey the law of, physics.
Marco Pavone [00:16:44]:
Mhmm. But there'll be there's been quite a bit of progress in, improving the physics realism. So I'm very hopeful that this technology will be yet another tool that has no time engineering I can leverage in order to build that more capable and safer autonomous vehicles. And it's not just for NVIDIA. I believe that the Cosmos and broadly generate simulate generative simulation is going to have a significant impact in the whole industry, but more broadly in the whole robotics industry.
Jordan Wilson [00:17:15]:
Yeah. And I think that's really important to bring up because, you know, especially if you've been to a city like San Francisco or, you know, I've seen, you know, Waymo's and in Austin, Texas. Right? So, these these vehicles and, this technology has you know, it's been out on the roads. Right? It's out there in the wild, which, you know, allows NVIDIA and other, players in the space to gather that actual real life data. Right? So, you know, I'm wondering, you know, if the simulation side is improving and you're able to, you know, simulate more scenarios with the Cosmos platform, you know, what are still some of those bigger hurdles, right, aside from, you know, just more time and, you know, more data from the real world from the cars. Right? What are some of those other big hurdles that the space is still looking to maybe overcome?
Marco Pavone [00:18:05]:
Well, data is a big one. So simulation is going to help, but you still needed to, have data real data to ground, your, your system. So the the hurdle is to make technologies, AI technologies, that can adapt with, an increasingly lower amount of data to new areas. And it is a problem that maybe it's a bit difficult to appreciate, but it's crucial. Transportation is a very location specific, phenomenon. Like, I don't know if you have Italian followers. I'd like to give example that actually, this is also a tip for you if you go to Italy. I love it.
Marco Pavone [00:18:41]:
So Please. Please.
Jordan Wilson [00:18:42]:
I have to. I have to soon.
Marco Pavone [00:18:43]:
If you go to Italy and someone blinks the lights at you, typically, that is a kind sign. It means that you can cross in front of me. I'm eating at you. If you go to the South Of Italy, that is typically blinking. It's an aggressive sign. It means that don't you dare across in front of me because I'm not going to stop. And if you cross me, we're going to crash. Right.
Marco Pavone [00:19:04]:
So in the same country, few hundreds kilometers apart with two completely different years. So this is what I'm saying. You still need to have some, location specific data and train your AI. So then the game is how to but that is expensive to acquire. Sure. So how through better simulation and the better algorithms, we can decrease the reliance on, real data. There always will be some need for real data. The question is how we can reduce it so that we can really quickly expand to new domains.
Marco Pavone [00:19:38]:
Technically, we refer to those as operational design domains to really make this technology financially viable. We know it's technologically feasible. Waymo is developing a robot accident in San Francisco. But to make it financially viable, we have to be able to scale it up. Scale it up means to be able to kick it out of flywheel that is not too honorous in terms of how much data we we need. Simulation is one tool, but better algorithm design is another tool. Driving down the cost of some key sensors, like, for example, as you might have heard, there is a quite a bit of discussion in the community in terms of to what extent you want to have a sensor system that is very much camera centric Mhmm. Or maybe also relying on ladders and so on and so forth.
Marco Pavone [00:20:24]:
So these are additional discussion. Again, it's not just a single technology. It's a combination of technologies, but I will say the capability of scaling operational design domains more seamlessly is the major challenge, which will be solved through a combination of a number of technologies from redundancy at a sensor level to simulation to algorithms that adapt more quickly to new scenarios with the last day.
Jordan Wilson [00:20:49]:
That's that's a fascinating example. Right? Because I never thought about that that, you know, it's not just a one size fits all approach for autonomous vehicles. Yeah. Yeah. You're like, definitely think about that. So, like, like, as an example. Right? Like, even I'm thinking in The United States, people have different driving styles. Right? From state to state, city, you know, big city, urban areas.
Jordan Wilson [00:21:12]:
Right? So, you know, I'm curious, you know, what are some maybe, successes that you've found in addressing those or how do you even go about knowing those things? Right? Aside from, like, you know, you you know, because, you know, you you have lived there, you know, but for everyone else and for all these other challenges that maybe the industry hasn't thought about, I mean, how do you, you know, start to tackle these issues? Is it just, you know, maybe, oh, there's there was an accident, and we don't know why because, you know, all of our data was right. And, you know, is that kind of how you discover these things?
Marco Pavone [00:21:42]:
Yeah. So that's one of the reason why this, AV industry has been increasingly shifting from a paradigm whereby most of the possible cases were hypothesized by humans and then coded into the brain of the autonomous vehicles to a data driven paradigm where we let the autonomous vehicle learn from experience, basically, from demonstrations because that is more scalable as a technology. So all these new behaviors essentially are the to a large extent that learned from data that is acquired through either test vehicles or even maybe dashcam videos. The good thing is that videos are bound. Yeah. On the Internet. And that is yet another opportunity to more seamlessly, scale up the AI to new operational design domain. So bottom line is that at these days, many of those, behavioral nuances are learned, through, data, which again means that we need to have technologies that allow us to make as much use as much efficient use of this data as possible.
Marco Pavone [00:22:49]:
The good thing is that, data abounds. So we do need data, acquired from a a test fleet for sure. But one thing that a bounce on the Internet is videos and especially driving videos. And that is again a new modality that, as an autonomy engineers, we have at our disposal to allow the schedule this technology in a even more efficient way.
Jordan Wilson [00:23:17]:
You know, and I'm curious because on the data point, I may be wrong in this assumption, but I'm guessing very early on, a lot of the data that you might get, you know, from those driving videos, let's say, if there is a bunch from five years ago, I'm guessing that the other cars, for the most part, were not autonomous. Right? They were driven by humans. So so, you know, I'm curious. Like, what are you all at NVIDIA? And, again, maybe just the broader industry doing to account for that. Right? Because what if in five years, it's 10% autonomous vehicles and, you know, how can you say, oh, in the training data, this was an autonomous vehicle versus this is a human. And, you know, what challenges does that, you know, bring in the future? Will it just be, you know, autonomous vehicles be able to more communicate with each other so they know, okay. You're an autonomous vehicle. You know, you're you have the halo system, so this is what you're gonna do.
Marco Pavone [00:24:06]:
That's a great question, and there are a lot of sub questions embedded there. So first of all, as you alluded to, now that we're moving toward a more data driven paradigm, then the brain of the autonomous vehicles becomes very much dependent on the data that you use to train the brain on. So it becomes imperative, and it's one of the hallmarks of kilos to develop, AI creation workflows that allow you to remove unsafe behaviors or biases from your, training set. And interestingly, this is another domain where we use Internet pretrained models Mhmm. Now not not as drivers, but as a judges that, you know, allow us to judge whether a given demonstration is, demonstration of safe driving or not. And, of course, there are humans in the loop to align the judgment of, these AI models to we, as humans, we think is acceptable driving. But we train these basically AI models to serve as judges at scale to remove all those biases that you were mentioning. Now moving forward, AV is a very weird technology because we are solving the hardest problem first.
Marco Pavone [00:25:22]:
Going back again to aviation, you know, I have this kind of dual, outer space, and earth based background. It's like as if the bad brothers, the first problem they wanted to solve was a supersonic fun. That's the state of the EV industry. We are solving the hardest problem first because we're solving the problem where we there are only few automated vehicles. Everybody else is a human. Like, you have a human driven vehicle. There is no dedicated infrastructure, But is this basically what it is? Now in the future, there will be a higher penetration of autonomous vehicle. So your question is how that will change the technology.
Marco Pavone [00:26:02]:
Definitely, it provides an opportunities to make the technology even safer. But all as with everything, you have an opportunity, but you also have a challenge. What is the challenge? Well, there are multiple. So let's assume that, for example, autonomous vehicles could communicate with each other. In principle, that is great because it allows some level of coordination, which has a clearly an immediate impact on safety. But it exposes your decision making, capabilities to external interference. So for example, cybersecurity becomes much more of a threat than it is now where the system is basically very much confined within the vehicle. You might introduce latencies.
Marco Pavone [00:26:46]:
You know, sometimes when you when you have a call on your phone, you don't care if your, communication drops, for a little bit. That could be fatal in the case of, an automated vehicles. So I'm not to not to say that these are impossible challenges. I'm just saying that it's not as simple as people might think that, be able to be able to be able to infrastructure might simplify the problem. Let alone the challenge of who is going to place that infrastructure, how you're going to standardize that infrastructure. So, yes, when there will be higher penetration autonomous vehicles, there will be opportunities to make this technology even safer. But exact mechanics about how we will do it is still subject to the discussion.
Jordan Wilson [00:27:28]:
So, Marco, we've covered a lot in this conversation. So, you know, as we wrap up, what do you think is maybe the one most important thing, for our, viewers and listeners to know about, you know, specifically, even new advancements that were announced here at GTC and how that is seemingly going to quickly change, the future of autonomous, driving on our roads?
Marco Pavone [00:27:53]:
I think there were two announcements broadly that are going to have a significant impact in the field of vehicle autonomy. First, the months and amounts all the announcements related to simulation. And second, announcements related to, foundation models, Internet retraining models that would be used in the context of physical AI. One of the big announcements actually, these are months that was made at CS, and that has been refined at GDC is that of, Cosmos Mhmm. Which is by now a sort of umbrella term where we cover both, video generation models, particularly useful for simulation, and the reasoning models, particularly useful for, autonomous driving, in the in the real world. So these are definitely technologies that are, worthwhile to keep in mind if you are an autonomy researchers. Many of these technologies are available open source on the admin phase. And at NVIDIA, this one of the reason why I'm excited about being at NVIDIA.
Marco Pavone [00:28:56]:
We we publish a lot, so we share a lot of our knowledge. And, again, this is because NVIDIA is both a product company and an ecosystem company. So we want to make sure that as we grow, the entire ecosystem really grows. Mhmm.
Jordan Wilson [00:29:09]:
I I I love it. I think I just, became so much, more informed, on on everything, you know, autonomous vehicles, what NVIDIA is working on, and I really, hope that our audience did as well. So, Marco, thank you so much for your time and coming on the Everyday AI Show to share with us. We appreciate it.
Marco Pavone [00:29:28]:
Likewise. And if you want to become an autonomy engineer, you're welcome to take some of the courses.
Jordan Wilson [00:29:33]:
There we go. Hey. At least I'm I'm part part of the way there. Right? I went from zero to one, I think. So, hey. That was a lot of fantastic information. NVIDIA is working on a ton of advancements. So if you missed anything in there, if you wanna know more, we're gonna be recapping today's conversation in the newsletter.
Jordan Wilson [00:29:49]:
So thank you for joining us. If you haven't already, go sign up for that newsletter. Read it today. It's gonna be a great one at youreverydayai.com. So thank you so much for tuning in. Hope to see you back tomorrow in everyday for more everyday AI. Thanks, y'all.
Midroll [00:30:01]:
And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers, and we'll see you next time.