UK AV Act 2024 comes to life
The CAV Research Lab just hosted a very interesting webinar that correctly identified the key problems with wider AV deployments, not only in the UK, but in the US, as well. The webinar, titled "Can AI Ever Be a ‘Careful and Competent Driver’? Safety, Ethics, and Evidence in Automated Driving" is summarized below, and included the following guest panelists:
John McDermid, Co-Director, Institute for Safe Autonomy, University of York
Dr. Nick Reed, Founder at Reed Mobility
Jamie Hodsdon, Policy Manager UK and Western Europe, Waymo
And moderated by:
Prof. Saber Fallah, Professor of Safe AI and Autonomy, University of Surrey
Prof. Clare Mutzenich, Professor of Practice in Human-AI Interaction, Loughborough University
The central theme was that both AV companies and public safety regulators do not have adequate evidence today to determine whether or not an AV is being operated on the road safely.
Retrospect’s main mission is to ensure that the obligations and uncertainty limitations of the people developing and operating AVs are embodied in the technology itself, such that the technology cannot drive beyond its limits. There is no better vantage point to assess how the system is operating than from inside the system itself. But I don’t have to say anything about how important it is to holistically measure real-time trajectory risk, and validate that risk continuously in the vehicle. I’ll let the panelists speak for themselves, using their own words in the short summary of excerpts we’ve provided for you below.
Can AI Ever Be a “Careful and Competent Driver”? (YouTube)
3:40
Saber Fallah, “We know that the UK AV Act defined the concept of Careful and Competent Act, and asked for Self Driving Entities to provide evidence for that, but the first question is that: What does Careful and Competent Driver actually mean?”
Nick Reed (Reed Mobility), “I think it’s a great foundation but there is work to do, for companies to be able to demonstrate that they are reaching that standard of careful and competent.”
Saber Fallah, “How do we measure a Careful and Competent Driver? Do we have any metrics or do we have any definitions for measuring the Careful and Competent Driver?”
John McDermid (University of York), “I don’t think there are any very simple definitions, as Nick said. But let me try to make a couple of observations that may help… Of course, we have statistics for accident rates for drivers, so at an absolute minimum you would want to see autonomous vehicles doing better than that, as it’s beginning to happen…
“One thing Nick alluded to is interacting with others. You know, actually driving is a process of social cognition, not just how we maneuver the vehicle. We decide some vehicle’s being aggressive, so we leave them more space, for example. And that’s much, much harder to measure. It’s not recorded in any objective way.
Jamie Hodsdon (Waymo), “I think the point that’s often missed about the AV Act in the UK is that the focus is very much on the organizations that are developing the technology and operating the services… it’s not just about saying is the vehicle, as an object, is safe because the vehicle is just a sort of robot, if you like. The question is, has the organization developed the vehicle and the service in a suitable way?
And I think in terms of, to your specific question, around what evidence to supply, it’s very much - and it’s conversations we’re having with DfT – that it’s about: can you demonstrate that you’re a reputable organization, that you’re doing the right things in the way you develop the technology and can you generate the right artifact, as they’d say.
20:20
Saber Fallah “From your point of view, as a safety assurance expert, what artifacts do the entities, the Automated Self Driving Entities, should they provide to the government and to the public for these purposes?”
John McDermid (University of York), “That’s a very big question. So, let me try to give you a brief answer to that. Some key things: I think it’s very important at the whole vehicle level we understand that the concept of operations and how it will achieve safety under adverse conditions, which might include heavy rain which obscures sensors, failures of traffic lights, but also internal failures; if a brake actuator fails and brake locks on, what does the vehicle do?
“What do the regulators need? I think the answer is quite wide visibility of the processing going through, and samples of evidence to show those processes have actually been implemented. In practice, a lot of the development work and verification work will be done in simulation…. There is no way a regulator could see all of that, realistically assess it, but they need to see enough samples of that that they can have confidence that not only the plan is good, they’ve been implemented well.
“There also needs to be visibility of how that evolves while the systems are in service, and you know, that needs to be done in such a way that protects intellectual property, but I think working with a regulator that could be done.”
25:25
Nick Reed (Reed Mobility), “There’s different levels of evidence.”
“There’s the evidence to show what has been done in preparing the vehicle for deployment… There’s evidence around what you’ve done for the deployment you’re wishing to deliver.
"[And] there's the ongoing evidence thereafter to show that your vehicles are continuing to be safe, and living up to the standards that you reported in that preparatory safety case.
"And I think there's work needed there, in how we prove, on an ongoing basis, and in a way that is practical and acceptable both to the public but also is sort of manageable and viable for the ASDE’s, the developers and operators, to be able to give that on-going evidence of safe operation."
"And it can't be at the level of incidents only. It has to be something more about the level of risk that the vehicles are experiencing or causing and enable a regulator help the operators manage their vehicles and manage the risk to other road users and the communities where the vehicles are deployed.
"We came up with this approach called Digital Commentary Driving. That it would be a standardized mechanism - a standardized format for collecting data which would reflect the information that a vehicle must know if it's to operate safely:
• What things is it seeing,
• What predictions is it making about how those entities might be moving,
• What confidence does it have in those predictions, and then
• How is it going to adapt its driving behavior in response."
"It's not perfect, but I think in terms of giving you that level of assurance and supporting public perceptions of trust about the safety of vehicles, it would enable you to create tools that a regulator can use and apply those tools to the data regardless of which operator was providing them, because it's in that standardized approach whether it was Waymo or any other AV developer."
"It would enable them to give insights into whether those vehicles are operating safely, so that's something been working on for a while.
"Something like it will have to emerge, whether it's DCD exactly, the Digital Commentary Driving as we created it, or something along those lines. It feels like we will get somewhere near what that aimed to achieve."
28:28
Clare Mutzenich (Loughborough University), “There was another webinar this morning, and Rebecca Posner was on that panel and she was making a great point about ‘acceptance’ or acceptability. That it shouldn’t really be about ‘convincing’ the public that it’s safe…
“And that perception of safety is what’s so important. And really, what are we doing as an industry other than having these great conversations and promoting these kinds of outreach and discussion on areas like LinkedIn? Where is the general public involved in this kind of conversation?”
45:01
Saber Fallah, "[T]ell me, what would you or each of you want to see before saying an AI driver had met the Careful and Competent threshold. Again, we've talked about safety, we've talked about AVs, but to me AI and AV are not separated. The driver is AI. What evidence do you want to see that the AI driver is Careful and Competent?"
John McDermid (University of York), "A safety case sufficient to say we can deploy in the real world on a small scale and gain further data to validate that the pre-deployment assessment was accurate."
Jamie Hodsdon (Waymo), "I suppose I'm a little bit biased, but I'd like to see UK government approval of Waymo vehicles."
Nick Reed (Reed Mobility), "At a basic level it's that statistical proof of safety, of a reduction in collisions causing death and serious injury. But then ongoing proof to show that the vehicles are operating safely and seeing and doing everything they need to do in order to be considered as operating safely.
"You know, that statistical measure is of no comfort to the family members of someone who's been killed as the result of a crash, so we need to know that the vehicles are behaving themselves in the way we, as experts, and we as society expect them to."
END
What do you think? Do you think AV companies are doing enough to address potential safety concerns? Or is a government approval all that is needed at this point?
If you would like to see more metrics and tools that objectively and provably measure safe driving, as well as aggressive driving, and unacceptable driving, be sure to subscribe and contact us here, on our website. We’d love to hear from you.
