Self-driving cars are hitting city streets like never before. In August the California Public Utilities Commission (CPUC) granted two companies, Cruise and Waymo, permits to run fleets of driverless robo taxis 24/7 in San Francisco and to charge passengers fares for those rides. This was just the latest in a series of green lights that have allowed progressively more leeway for autonomous vehicles (AVs) in the city in recent years.

Almost immediately, widely publicized accounts emerged of Cruise vehicles behaving erratically. One blocked the road outside a large music festival, another got stuck in wet concrete and another even collided with a fire truck. These incidents have brought renewed attention to the potential difficulties of integrating self-driving cars into the urban environment. Fewer than 10 days after CPUC’s 24/7 approval, the California Department of Motor Vehicles requested that Cruise halve the size of its operating fleet in San Francisco while the regulator investigates the fire-truck collision.

Yet despite these hiccups and some ongoing opposition, self-driving car companies have continued to expand testing and operations to more U.S. cities. Many proponents say these vehicles could offer a safer alternative to human drivers, make transportation accessible to more people, improve traffic flows around cities and decrease the environmental impact of cars. So far, though, most of those promised benefits remain squarely in a possible future. Meanwhile, here in the present, complications have become plainly apparent. Scientific American asked experts in the field what’s important to understand about the current state of the robo-taxi rollout.

How Do Robo Taxis Work?

AVs, such as those operated by Cruise and Waymo, shouldn’t be confused with commercially available cars, such as Teslas, that come with some driver-assistance features. Unlike those vehicles, which require a human driver to oversee control at all times, robo taxis demonstrate what’s often called “Level 4” autonomy: they can navigate a predetermined area without a person monitoring and managing each movement from behind a steering wheel. This is made possible via a very detailed internal map, says Ramanarayan Vasudevan, an engineer who researches autonomous vehicles at the University of Michigan.

That satellite-linked map covers a limited area within which these vehicles are “geofenced,” meaning they can’t operate beyond their designated bounds. “The map is like a cheat code,” Vasudevan explains. With it, the cars can tackle complex environments. They also rely on an extensive network of sensors, including cameras, sound detectors and laser-based lidar, all of which help place a vehicle inside the map’s lanes and provide real-time information on weather, passing pedestrians, construction and other road conditions.

From there, machine-learning algorithms—trained on big sets of both modeled and real-world driving data—translate a vehicle’s map and sensor information into projections about what nearby cars, people or cyclists are likely to do, Vasudevan says. Additional artificial intelligence programming tells the AV how to move, while preset guidelines ensure it follows traffic laws. And if it gets stuck in a difficult situation, says Waymo software engineer Josh Herbach, a human operator can remotely guide it to perform the appropriate action from a desk in a control center.

Are Robo Taxis Safer Than Human Drivers?

Waymo and Cruise maintain that their AVs are safer than human drivers. Both companies have released data that they claim support this, and some outside researchers agree. Yet the truth may not be as cut-and-dried as company stats make it seem, says Steven Shladover, an engineer at the University of California, Berkeley, who has been studying vehicle automation for 50 years.

“I actually think there is not yet enough data available to the public, including researchers like me, to be able to judge” relative safety, Shladover says. AV companies do not share all the data they collect. Often negative incidents only come to light if they’re recorded or posted on social media. Even the data on human drivers are incomplete because they only include major collisions, Shladover adds.

The released data could easily be flawed because in order to accurately compare robo taxis with humans, it’s necessary to account for how many miles are being driven and under what conditions—data that simply do not exist. People have many more accidents than AVs do, but more than 100 million humans are estimated to drive on U.S. roads every day, compared with just hundreds of driverless taxis. Looking at estimates of how many driving hours people log nationwide, “crashes with human drivers are actually extremely rare events” relative to the total amount of time humans spend on the road, Shladover says. Data from the National Highway Traffic Safety Administration seem to back this up.

Though a perfect automated system would undoubtedly be a better driver than a person, Vasudevan notes that such a system simply doesn’t exist. “Our ability as humans to build millions and millions of lines of code in a way that is 100 percent correct is nearly impossible for the same reason that humans are imperfect at driving,” he says. Asked if he thinks AVs are safer drivers than people, Vasudevan generally agrees with Shladover. “I don’t think that story is as clear as self-driving car enthusiasts would like us to believe,” Vasudevan says.

What About Other Benefits?

Beyond safety, one commonly touted AV advantage is that driverless cars will boost transportation accessibility and affordability while also reducing the need for car ownership. Another is that more AVs on the road might lessen traffic congestion and ease city planning by enabling vehicles to communicate with each other and ensuring that they correctly follow the best routes at the right times. Yet neither of these possibilities has become reality yet—and getting there isn’t a guarantee, says Dan Chatman, an urban planning and transit researcher at U.C. Berkeley.

Although he’s optimistic about AVs and excited by their potential in San Francisco, Chatman acknowledges there’s a version of the future in which robo taxis cause more problems than they fix. They could exacerbate urban sprawl, clog up roads instead of relieving traffic and dissuade people from the most sustainable forms of transportation: biking, walking and public transit. “A lot depends on the extent to which they are regulated,” he says. There are policies that could encourage people to ditch cars, such as road pricing, which would ensure that the true environmental and social cost of running a vehicle—whether human- or robot-driven—is paid by its operator. “We haven’t done a great job of regulating automobility in general. And so this is a chance for us to recalibrate how we deal with” cars, Chatman says.

There are two possible roads before us: one with appropriate regulation, caution and transparency and the other without them. The robo taxis might be driving themselves, but it’s still up to us humans to choose which route we’ll take.