Driving SMARTS 2022
The Driving SMARTS 2022 is a benchmark used in the NeurIPS 2022 Driving SMARTS competition.
Objective is to develop a single policy capable of controlling single-agent or multi-agent to complete different driving scenarios in the
driving_smarts_2022_env() for environment details.
In each driving scenario, the ego-agents must drive towards their respective mission goal locations. Each agent’s mission goal is given in the observation returned by the environment at each time step.
The mission goal could be accessed as
observation.ego_vehicle_state.mission.goal.position which gives an
(x, y, z) map coordinate of the goal location.
Any method such as reinforcement learning, offline reinforcement learning, behavior cloning, generative models, predictive models, etc, may be used to develop the policy.
The scenario names and their missions are as follows. The desired task execution is illustrated by a trained baseline agent, which uses PPO algorithm from Stable Baselines3 reinforcement learning library.
A single ego agent must make a left turn at an unprotected cross-junction.
A single ego agent must make a left turn at an unprotected T-junction.
One ego agent must merge onto a 3-lane highway from an on-ramp, while another agent must drive on the highway yielding appropriately to traffic from the on-ramp.
One ego agent must merge onto a 3-lane highway from an on-ramp.
Three ego agents must cruise along a 3-lane highway with traffic.
One ego agents must cruise along a 3-lane highway with traffic.
One ego agent must navigate (either slow down or change lane) when its path is cut-in by another traffic vehicle.
One ego agent, while driving along a 3-lane highway, must overtake a column of slow moving traffic vehicles and return to the same lane which it started at.
The underlying environment returns raw observations of type
Observation at each time point for each ego agent.
This benchmark allows ego agents to use any one of the following action spaces.
See the list of available zoo agents which are compatible with this benchmark. A compatible zoo agent can be evaluated in this benchmark as follows.
$ cd <path>/SMARTS $ scl zoo install <agent path> # e.g., scl zoo install zoo/policies/interaction_aware_motion_prediction $ scl benchmark run driving_smarts_2022==0.0 <agent_locator> --auto_install # e.g., scl benchmark run driving_smarts_2022==0.0 zoo.policies:interaction-aware-motion-prediction-agent-v0 --auto-install