Learn to Bet: Using Reinforcement Learning to Improve Vehicle Bids in Auction-Based Smart Intersections

Area of Science:

• Intelligent Transportation Systems
• Artificial Intelligence
• Machine Learning

Background:

• The proliferation of the Internet of Things (IoT) will lead to cities with autonomous vehicles and intelligent management systems.
• These systems need to interact with city infrastructure and vehicles to optimize urban mobility.
• Resource management, particularly cost savings, is crucial for the economic viability of autonomous systems.

Purpose of the Study:

• To propose a reinforcement learning model for autonomous connected vehicles (ACVs).
• To enable ACVs to save resources, specifically budget, within auction-based intersection management systems.
• To evaluate the trade-off between cost savings and trip times under various traffic conditions.

Main Methods:

• Developed a model using Deep Q-learning, a type of reinforcement learning.
• Trained multiple models with variations in traffic conditions to identify optimal performance.
• Compared the proposed model's performance against existing and random strategies.

Main Results:

• The reinforcement learning model demonstrated robustness across different traffic scenarios.
• Significant budget savings were achieved: at least 20% in heavy traffic and up to 74% in light traffic compared to a standard bidder.
• Savings were approximately three times greater than those achieved by a random bidding strategy.
• Minimal increase in waiting times was observed, indicating an effective balance between cost and efficiency.

Conclusions:

• The proposed reinforcement learning model offers a practical solution for resource savings in autonomous vehicle navigation.
• The model's ability to achieve substantial cost reductions without compromising travel time suggests feasibility for real-world deployment.
• This approach is well-suited for future intelligent urban environments utilizing auction-based traffic management.