How does openpilot control a car?

How does openpilot control a car?

Review of the comma con 2021 talk by Robbe Derks

Controlling the car involves 3 things

  1. How to connect to the CAN Buses

  2. Figure out the messages format and their meaning

  3. Send a better control message

    Components of the modern car

How to connect to the CAN Buses

Your car contains a collection of computer nodes linked along a bus network architecture. The nodes are referred to as Electronic control units(ECU), and the bus topology is called the controller area network (CAN). You can learn about the CAN bus here in the intro video

Of all the messages and nodes, what do we need to control the car?

We need the messages from the ADAS Camera which joins the vision and radar data since it provides adaptive cruise control messages (ACC) which then enable us to control GAS and BRAKES (also called longitudinal control). In addition, it also provides LANE KEEP ASSIST messages which enable us to control the steering control. As an added convenience, this setup can be easily installed near the windshield.

Now we understand where to tap into the car's electronics, we need to intercept the messages generated by the LKAS system so that we can provide the necessary messages to control the car. Below is a simplified architecture of how comma devkit (comma3) interacts with the car's system.

The RADAR messages are read directly. Car's stock ADAS camera is isolated and instead, we use comma3's camera as a proxy and generate a CAN message in the same format as the stock ADAS system would generate. Hence the car's system would be agnostic as to which camera is generating. CAN to USB interface made possible by PANDA convert the messages between openpilot running on comma3 and the car system.

If you are interested to learn more it can be found in this excellent blog post

CAN Messages: Figuring out the format and the meaning

We have to generate the same message as the stock ADAS system to control the car. How do we do that? As a starting point, we have to understand CAN messages right? since they seem to be the language the CAR talk and understand.

CAN messages are unique to each car manufacturer and they do not disclose that information to the public. These files which define the messages are called DBC files.

Since it's not publicly available, one has to reverse engineer the messages and interpret what they mean by trial and error. Fortunately, comma and its community have done this on a wide variety of cars through their open-source project called OpenDBC

DBC Files

DBC files contain the definition for the CAN object. They tell you how to read the message. In the example below the CAN message is referring to engine speed.

Instead of explaining the conventions followed in DBC files, I shall link to the excellent site here. The steps for decoding rules of typical CAN data are given here. You can play around with DBC files here

Send Better Control Messages

At this point, we know how to construct the message, so we have to send it to the car's inbuilt control system to execute the command. controlsd in the openpilot stack is responsible for constructing the message and passing that on to the PANDA which then passes the messages to the car to execute. See below for the visual

Controlsd

This is the service that controls your car.

  1. Planning Stack: It receives output from the machine learning model in the form of path trajectories and velocities/accelerations and converts it to control signals.

  2. Control Loops: The vehicle agnostic signals from the planning stack such as acceleration, steering angle etc. are then converted to vehicle-specific CAN commands that work for that car’s API, through a closed-loop control system running at 100Hz.

Safety

At this point we understand the message to create and how to create them, now the job is to do it safely. There are multiple safety layers between the comma device and the actuators, this includes the panda which runs all the safety-critical safety code written in MISRA C. For example, the panda will block the message if the acceleration is outside the -3.0 to 1.5 range mentioned below for Toyota.

Here is comma.ai's safety document.

References

comma con 2021 talk by Robbe Derks

OpenPilot: An Overview and the Port to the Honda Clarity: Hardware

CAN Bus Explained - A Simple Intro

DBC Editor for CAN Bus Database Files

Did you find this article valuable?

Support Abhishek Sreesaila by becoming a sponsor. Any amount is appreciated!