---
title: "CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning"
authors:
- "Mingyuan Li"
- "Chunyu Liu"
- "Zhuojun Li"
- "Xiao Liu"
- "Guangsheng Yu"
- "Bo Du"
- "Jun Shen"
- "Qiang Wu"
published: 2026-04-20
doi: "10.1145/3770854.3780308"
source_url: "https://doi.org/10.1145/3770854.3780308"
conversion:
method: pymupdf4llm
authoritative_source: rendered PDF
---
> Conversion note: This Markdown was extracted from the rendered PDF. The PDF remains authoritative for formulas, tables, figures, and layout.
# **CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning**
Chunyu Liu
## Zhuojun Li
Mingyuan Li
henryli_i@bupt.edu.cn Lanzhou University; Beijing University of Posts and Telecommunications, Lanzhou, China
chunyuliu@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, China
zhuojinli@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, China
Guangsheng Yu
## Xiao Liu
## Bo Du
liuxiao68@bupt.edu.cn Beijing University of Posts and Telecommunications, Beijing, China
reimusaber@gmail.com Independent Researcher Sydney, Australia
bo.du@griffith.edu.au Griffith University, Brisbane, Australia
Jun Shen
Qiang Wu∗
jshen@uow.edu.au University of Wollongong, Wollongong, Australia
wuqiang@lzu.edu.cn Lanzhou University Lanzhou, China
### **Abstract**
### **CCS Concepts**
Traffic accidents result in millions of injuries and fatalities globally, with a significant number occurring at intersections each year. Traffic Signal Control (TSC) is an effective strategy for enhancing safety at these urban junctures. Despite the growing popularity of Reinforcement Learning (RL) methods in optimizing TSC, these methods often prioritize driving efficiency over safety, thus failing to address the critical balance between these two aspects. Additionally, these methods usually need more interpretability. CounterFactual (CF) learning is a promising approach for various causal analysis fields. In this study, we introduce a novel framework to improve RL for safety aspects in TSC. This framework introduces a novel method based on CF learning to address the question: “What if, when an unsafe event occurs, we backtrack to perform alternative actions, and will this unsafe event still occur in the subsequent period?” To answer this question, we propose a new structure causal model to predict the result after executing different actions, and we propose a new CF module that integrates with additional “X” modules to promote safe RL practices. Our proposed algorithm, CFLight, achieves a near-zero collision control strategy for complex intersection safety. Extensive experiments on real-world and synthetic datasets demonstrate that CFLight significantly outperforms conventional and state-of-the-art safe RL methods in both safety and traffic efficiency. The data and code are available in the github1 .
• **Computing methodologies** → **Computational control theory** ; • **Applied computing** → **Transportation** .
### **Keywords**
traffic signal control, reinforcement learning, counterfactual learning
##### **ACM Reference Format:**
Mingyuan Li, Chunyu Liu, Zhuojun Li, Xiao Liu, Guangsheng Yu, Bo Du, Jun Shen, and Qiang Wu. 2026. CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning. In _Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD ’26), August 09–13, 2026, Jeju Island, Republic of Korea._ ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3770854.3780308
##### **Resource Availability:**
The source code and artifacts of this paper are available at https://doi.org/ 10.5281/zenodo.18041726. The official version of the paper is accessible via https://doi.org/10.1145/3770854.3780308.
### **1 Introduction**
**Motivations.** Road accidents have dire consequences, affecting victims, families, and societies worldwide. The World Health Organization reports an annual loss of 1.4 million lives and 20 to 50 million injuries due to road crashes in 2016 [25]. Intersections contribute significantly, causing about one-quarter of traffic fatalities and half of all injuries in the United States every year [6]. Meanwhile, traffic congestion remains an ever-escalating issue in contemporary urban environments, imposing substantial adverse impacts on various aspects of city life, including economic productivity and time consumption [28].
∗Qiang Wu is the corresponding author
1https://github.com/AdvancedAI-ComplexSystem/SmartCity/tree/main/CFLight
This work is licensed under a Creative Commons Attribution 4.0 International License. _KDD ’26, Jeju Island, Republic of Korea_
© 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2258-5/2026/08 https://doi.org/10.1145/3770854.3780308
In recent years, Deep Reinforcement Learning (DRL) [33] has garnered increasing attention by adjusting traffic signal control (TSC)
704
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Mingyuan Li et al.
to reduce traffic jams. By setting appropriate reward functions, such as minimizing vehicle travel time and maximizing traffic flow throughput. DRL continuously learns and optimizes the control strategies, adapting to changing traffic conditions in real time. Thus, DRL models make proper decisions regarding traffic light phases based on the current traffic flow state [3, 38]. Despite the many advances in RL model for TSC, most of the existing DRL models only consider traffic efficiency rather than safety. Now, the safe RL methods are proposed to ensure the agent’s learning and decisions align with safety constraints to avert hazardous outcomes [30]. It’s vital for real-world RL applications, safe RL holds the potential to tackle unresolved safety challenges in TSC. CounterFactual (CF) learning for safety is an emerging area of research in the field of AI and aims to improve the safety and robustness of AI systems by considering “what-if” scenarios during the training process [10]. Thus, CF learning provides some valuable tools for studying safety. Researchers often conduct CF policy evaluations before deploying any policy to the real world by using CF reasoning to reduce unsafe events.
The unsafe problems of intersections and the potential of CF learning for safe RL motivate us to develop a safe RL method for TSC to ensure intersection safety and efficiency at the same time.
Br Bi
ee en a
=
rewind
| T [ey|
Timeline
**Figure 1: The red vehicle, moving at high speed using permitted-only phase, collides with the opposing left-turning vehicle in permitted-only phase. We rewind to the state of pre-collision, and shift to protected-only phase to reduce collisions by avoiding the approach of hostile vehicles**
**Challenges.** In the real-world, many intersections use a permittedonly phase, allowing both straight and left turns. While this improves efficiency, it increases the risk of collisions between straightmoving and left-turning vehicles. Alternatively, the protected-only phase, which separates left turns from straight movements, reduces collision risk but significantly decreases traffic efficiency. These current safe RL and TSC methods [5, 7] are still unsatisfactory in reducing the number of collisions and lack interpretability. Meanwhile, most safe RL methods are developed for specific domains or applications, lacking a universal framework for rapid research
and development of specific safe RL solutions. In the TSC field, few methods have been proposed to solve safety problems. **Contributions.** In this study, we design a CF framework to improve the safe RL model by answering the CF question: “What if, when an unsafe event occurs, we backtrack to perform alternative actions, will this unsafe event still occur in the subsequent period?”. Furthermore, we effectively achieve a balance between traffic efficiency and safety, we have specifically implemented this method in TSC scenarios (as shown in Figure 1) via leveraging this CF framework. Our contributions are summarized as follows:
- We propose the first instantiation of the causal learning for TSC by answering counterfactual questions to improve the safety of TSC. Specifically, we design structure casual models to predict the result after executing different actions and collect the CF result to enrich the data and improve performance.
- We develop a CF+“X” safe RL framework that integrates CF components with various “X” modules. Based on this framework, we introduce a novel algorithm, CFLight, which effectively addresses safety events and incorporates a new reward function to improve both safety and action interpretability.
- Theoretical analysis shows that CFLight, augmented with CF data, converges reliably. Experimental results demonstrate that CFLight significantly reduces collisions, achieving up to a 93.1% reduction in collision rates compared to 3DQN methods, while maintaining near-zero incident rates.
Furthermore, our “CF+X” framework serves as a generalized safe RL method that can be applied in various other domains.
### **2 Background**
### **2.1 Counterfactual learning**
CF learning, or counterfactual estimation or reasoning, is a fundamental concept utilized across diverse fields such as machine learning, causal inference, and decision-making. It has recently gained significant attention leverage unreal data instances [16, 21, 34]. The origin of CF learning traces back to the pioneering work in 1974 [31], where fundamental concepts such as potential outcomes and causal inference laid the groundwork for subsequent research. The twin network method [27] visually represents reasoning with two networks: one for the factual world and one for the counterfactual (imaginary) world. They share the same structure, but the intervened variables are removed in the counterfactual world. Researchers have successfully used this technique in the healthcare sector to estimate the causal effects of medical treatments and interventions [29]. CF learning has also yielded significant breakthroughs in recommender systems, as seen in methods like CF matrix factorization [37]. Furthermore, RL and CF learning amalgamation have impressive performance in optimizing decisionmaking processes [20, 23, 24].
The scope applications of RL expands into safety-critical domains, ensuring the robustness and safety of RL agents [8]. In the early stages of addressing RL safety, primarily directed toward defending against and it reveals such attacks exploit learned policies, leading to unintended and potentially harmful behaviors within RL-based systems [32]. The event of agent safety within RL has
705
CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
received significant attention, particularly in critical domains like healthcare and autonomous vehicles. A comprehensive framework has been proposed to address the agent safety problem in RL, ensuring agents prioritize safety to mitigate potential risks [8]. For TSC, rare applications are focused on safety rather than efficiency.
Structural Causal Models (SCMs) serve as a formal framework in causal inference and statistics to represent and analyze causal relationships among variables. In a SCM, the causal relationships among variables are explicitly depicted. The set of variables, denoted as _𝑋_ 1 _,𝑋_ 2 _, ...,𝑋𝑛_ , is associated with structural equations:
where: _𝑃𝑎_ ( _𝑋𝑖_ ) signifies the parents of _𝑋𝑖_ in the causal graph, _𝑓𝑖_ (·) represents a deterministic function describing how _𝑋𝑖_ is generated based on its parents and _𝑈𝑖_ stands for the error term, and it represents the unobserved factors affect _𝑋𝑖_ . The CF Causal Effect (CCE) is formulated as the disparity between the potential outcome under an intervention and the observed outcome without the intervention: CCE = _𝑌_˜ − _𝑌_ where _𝑌_ signifies the observed outcome, and _𝑌_ ˜ represents the CF outcome. The core objective of CF learning lies in estimating the CF outcome _𝑌_˜ or the causal effect, utilizing observed data D in conjunction with the causal model.
### **2.2 Traffic Signal Control**
Due to the continuous advancements in urban transportation, research on TSC has been advancing progressively since its inception. The fixed-time method [14] cannot make optimal signal choices in response to real-time traffic conditions. Self-Organizing Traffic Lights (SOTL) [4] decides phase maintenance or changing by using predefined rules and real-time traffic conditions. Given constraints of the traditional method, the RL TSC dynamically adjusts signal phases in real time based on current traffic, optimizing performance [9, 12, 26]. CoLight [35] employs a graph attention and significant optimization through neighboring intersection data. Advanced-CoLight [38] further advances TSC performance. SafeLight [5] integrates a safe component into RL, reducing collision rates.
A B C D
(b) Protected-only
E F
(c) Permited-only
(a) Intersection and traffic movement
**Figure 2: Illustration of a four-way intersection.**
**Traffic network.** A traffic network is composed of a series of intersections ( _𝐼_ 1 _, ..., 𝐼𝑁_ ), with each intersection hosting a variety of
roads and each road further accommodating multiple lanes. **Traffic movement.** Traffic movement is the vehicle motion within an intersection that involves lane changes upon entry and exit (as shown in Figure 2(a)).
**Traffic signal phase.** A Traffic Signal Phase (TSP) specifies vehicle movements at an intersection during a defined interval. It designates proceeding, yielding, or stopping lanes, each linked to a signal indication. Networks use 2, 4, or _𝑘_ phases, denoted as _𝑎_ = 1 _,_ 2 _, ...,𝑘_ . Figure 2(b)(c) shows 6 phases. To enhance traffic efficiency, most cities have established a set of regulations that focus on optimizing left-turn scenarios. Several left-turn phasing guidelines were built by Federal Signal Timing Manual [14]. These regulations encompass two primary modes:
- **Permitted-only** : allows both straight-ahead and left turns.
- **Protected-only** : permits only straight-ahead movements while prohibiting left turns.
Although the “permitted-only” traffic signal mode aims to improve traffic efficiency, it inadvertently increases the risk of collisions. This risk arises from faster-moving vehicles on a permissive signal road colliding with vehicles making left turns, posing a critical safety hazard.
To find a solution to this unsafe scenario, the “protected-only” signal mode has been designed to restrict left turns on the current road and eliminate conflicts with oncoming traffic. This aims to decrease collisions on risky roads, enhancing overall traffic safety.
### **3 Methods**
In this section, we introduce the CF+“X” framework, which integrates the CF and X modules to enhance the safety of RL. We describe the creation of CF trajectories and present the safe RL algorithm CFLight for TSC by implementing the CF+“X” framework. Figure 3 shows the overview.
### **3.1 CF+“X” Framework**
The CF+“X” framework comprises two primary modules: the CF module and the X module. The CF module employs a SCM model to collect CF trajectories, while the X module integrates a Safe component (e.g., collision penalty in reward) and an Efficient-RL component (e.g., throughput or vehicle waiting time optimization in reward).
_3.1.1 CF Module._ The CF module consists of CF trajectories and the CF objective, which together serve as a vital mechanism for improving RL safety by incorporating CF reasoning into the learning process. CF trajectories offer additional safety-related experiences from which both the TSC agent and the SCM model can learn. The CF objective guides the optimization process during training, encouraging safer decision-making in the TSC model.
**SCM Model.** Assume that the next state _𝑆_′ and the CF result _𝑅_ are governed by the following SCM:
where _𝑆_ and _𝐴_ denote the current state and action, respectively. The variable _𝑈_ represents a noise term, which may include factors such as weather conditions and transmission variability. This noise is an unobserved variable that is assumed to be independent of both _𝑆_ and _𝐴_ . The model M functions as a generative model, capable
706
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Mingyuan Li et al.
of predicting unknown outcomes in real-world tasks. The SCM is trained using data stored in the experience buffer of the TSC agent and is used to collect CF trajectories.
**CF Trajectories.** We outline the process for collecting CF trajectories ( _𝑠,𝑎𝑐𝑓 ,𝑠𝑐𝑓 ,𝑟𝑐𝑓_ ). We begin by recording the states associated with all unsafe events prior to their occurrence, using a simulator denoted as _𝑒𝑛𝑣_ . Next, the simulation is rewound to the state immediately before each unsafe event. From this point, given alternative CF actions _𝑎𝑐𝑓_ , the SCM model M is used to predict the corresponding CF reward _𝑟𝑐𝑓_ and the resulting CF state _𝑠𝑐𝑓_ . Finally, the outcomes before and after the rewind are compared in order to construct the CF objective.
**CF Objective (Reward, Loss, and** _𝑄_ **Value).** We introduce the concept of a CF objective to enhance safety, defined as:
where _𝑅𝐶𝐹_ represents the outcome resulting from the execution of the CF action _𝑎𝑐𝑓_ , while _𝑅𝐴𝐶_ denotes the outcome following the execution of the real world intended action. The function _𝑓_ is designed to measure the distance between _𝑅𝐶𝐹_ and _𝑅𝐴𝐶_ . Additionally, _𝑅𝐶𝐹_ can be incorporated as a separate term in the overall optimization objective to explicitly account for CF considerations during learning.
_3.1.2 X Module._ The X Module consists of two primary components: the Safe component and the Efficient-RL component. The Safe component is designed to incorporate basic safety considerations into the RL model, while the Efficient-RL component represents the efficiency-driven part of the RL model. Different configurations of Safe and Efficient-RL components can be flexibly combined to form the integrated X Module.
**Safe component.** The Safe component focuses on embedding safety considerations into the design of the reward, loss, and _𝑄_ functions within the RL framework. In the case of Safe Reward ( _𝑆𝑎𝑓𝑒𝑟_ ), safety concerns influence the reward function in a manner similar to reward shaping; when the safety model detects an unsafe action, a safety constraint is introduced to guide the learning process. For Safe Loss ( _𝑆𝑎𝑓𝑒𝑙_ ), the RL model is enhanced with an additional task by integrating the safe component into the loss function, enabling multi-task learning that optimizes multiple objectives simultaneously. In the case of Safe Value ( _𝑆𝑎𝑓𝑒𝑣_ ), the RL model is augmented by incorporating the safe component into the _𝑄_ function, thereby establishing a multi-task learning framework that concurrently optimizes for both performance and safety.
**Efficient-RL component.** The Efficient-RL component utilizes advanced RL models aimed at optimizing efficiency. It can either maintain consistency with the original algorithm or adopt novel approaches to improve performance and adaptability.
### **3.2 CFLight**
We propose CFLight, a method built upon the “CF+X” framework, designed to enhance both the efficiency and safety of TSC, as illustrated in Figure 3.
_3.2.1 CF Module Implementation._ **SCM Model.** A Bidirectional Conditional Generative Adversarial Network (BiCoGAN) [11] is used to approximate the SCM model. The core idea is to use a generator network to create fake samples that can deceive the
adversarial network. The generator network M is employed to estimate the next state _𝑆_′ and reward _𝑅_ , conditioned on ( _𝑆,𝐴,𝑈𝑠,𝑈𝑟_ ). Encoder is an inference mapping from _𝑆_′ and _𝑅_ to ( _𝑆,𝐴, 𝑈_ˆ _𝑠, 𝑈_ˆ _𝑟_ ).
Using the data from the experience buffer, the SCM model is updated using the following formula:
where L _𝐷𝐸_ = E[log _𝐷_ ( _𝑆,𝐴, 𝑅,𝑆_′ _, 𝑈_ˆ _𝑠, 𝑈_ˆ _𝑟_ )] and L _𝐷𝑀_ = E[log _𝐷_ ( _𝑆,𝐴, 𝑅,_ ˆ _𝑆_ ˆ′ _,𝑈𝑠,𝑈𝑟_ )] are GAN-based loss functions used to optimize the discriminator, encoder, and generator models, helping to better distinguish between real and generated transitions. The loss L _𝑆𝑅_ = |( _𝑆_′ _, 𝑅_ )−( _𝑆_ˆ′ _, 𝑅_ˆ )|2 encourages the generator to produce states and rewards that more closely approximate the ground truth, using the _𝐿_ 2 norm distance. The monotonicity penalty, $L_{mono} = \sum_l \sum_{i,j}\max(0,-W_{l,i,j})$, enforces the monotonicity of the generator M, where _𝑙_ denotes the layer index and _𝑊𝑙,𝑖,𝑗_ represents the weights of M. The hyperparameters _𝜆_ and _𝛽_ control the trade-off among the loss components.
_where 𝑈𝑠,𝑈𝑟_ ⫫( _𝑆,𝐴_ ) _,_ M1 _,_ M2 _is smooth and monotonic in 𝑈𝑠,𝑈𝑟 for fixed value of 𝑆,𝐴. Suppose we have observed_ ( _𝑆,𝐴, 𝑅,𝑆_′ ) _, then for the CF action 𝑎𝑐𝑓 , the following CF outcome:_
#### _is identifiable._
**Proof:** Given _𝑆_′ = M1 ( _𝑆,𝐴,𝑈𝑠_ ) and the observed values _𝑆_ = _𝑠_ , _𝐴_ = _𝑎_ , and _𝑆_′ = _𝑠_′ , and assuming that M1 is monotonic in _𝑈𝑠_ , we can invert the function to obtain _𝑢_ ˆ _𝑠_ = M1−1(_𝑠_′_,𝑠,𝑎_).Setting _𝐴_ = _𝑎𝑐𝑓_ , and noting that _𝑈𝑠_ ⫫( _𝑆,𝐴_ ), the noise term _𝑢_ ˆ _𝑠_ remains unchanged. According to Theorem 1 in [21], we can then compute the CF next state as _𝑆𝐴_′ = _𝑎𝑐𝑓_= M1(_𝑠,𝑎𝑐𝑓,𝑢_ˆ_𝑠_). By the same reasoning, the CF reward is given by _𝑅𝐴_′ = _𝑎𝑐𝑓_= M2(_𝑠,𝑎𝑐𝑓,𝑢_ˆ_𝑟_). Therefore, the CF outcomes are identifiable.
According to Theorem 1, we obtain the CF outcomes _𝑠𝑐𝑓_ = M1 ( _𝑠,𝑎𝑐𝑓 ,𝑟𝑠_ ) and _𝑟𝑐𝑓_ = M2 ( _𝑠,𝑎𝑐𝑓 ,𝑟𝑟_ ). This indicates that by changing to the CF action _𝑎𝑐𝑓_ , we can generate the CF next state _𝑠𝑐𝑓_ and the CF reward _𝑟𝑐𝑓_ . Finally, we use the loss function defined in Equation 5 to optimize the SCM model M.
**CF Trajectories.** Algorithm 1 shows the CF trajectories construction (CTC) step. CF thinking infers what would have happened if an agent had taken a different action. Given a sample ( _𝑠,𝑎𝑐𝑓 ,𝑢_ ), a CF action _𝑎𝑐𝑓_ is randomly drawn from the action space and, together with state _𝑠_ and noise _𝑢_ , passed to the generator M to predict the CF reward _𝑟𝑐𝑓_ and next state _𝑠𝑐𝑓_ .
**CF Objective.** We design different instances of the function _𝑓_ in Equation 3 for reward, loss, and _𝑄_ value, respectively, to define the CF Objective. Here, _𝑟𝑎𝑐_ and _𝑟𝑐𝑓_ represent the rewards obtained by executing the action and the CF action, respectively. These different CF objectives encourage the agent to learn safer strategies. To emphasize the difference between real-world and CF collisions,
707
CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Online X Module for TSC Process CF Module Trajectory Collection Process CF Module Objective Construction Process
T = AEPO) [Orange — @) ACF,
Traffic Environment Experience Replay Buffer
State f = Dra(Ay (HG) 1Aer.)
Reward RL Agent
Action Counterfactual Trajectories Collision CF Action State CF Loss
Construction (CTC)
State
Environment
RL ComponentEfficient — s Jyu_ yg , s ’ ' KL Divergence j
TSC Methods
Rewind (5.0) = ws = Qryye #3 OF,
IPPO 3DQN Other
Q Value SCM CF Objective RL Agent Action CF Q Value
tah Action — R |] CF ResultCFCF | Construction State CF Action
Safe Component Noise A Next State Environment CF Reward
Safe Loss Safe Value Safe Reward State CF Trajectory
Reward
Feo || Le OF a ae,
eee I ee Sa fe t ues TEE
CF Advantage 0.0 0.1 0.0 0.9 Normalized Adv 0.0 0.2 0.5 0. 4
**Figure 3: The framework of CFLight consists of two modules: the X Module and the CF Module. During the TSC process, pre-collision states** _𝑠_ **are continuously collected. When a collision occurs, the system rewinds to the recorded pre-collision states and initiates the CF trajectory collection process. In the CF Trajectories Construction (CTC) step, different actions** _𝑎𝑐𝑓_ **are evaluated by the SCM model** M **to predict potential collisions, the resulting next state** _𝑠𝑐𝑓_ **, and to collect the CF trajectory** ( _𝑠,𝑎𝑐𝑓 ,𝑠𝑐𝑓 ,𝑟𝑐𝑓_ ) **. Finally, the CF Objective is formulated, the CF trajectories are added to the replay buffer, and both the TSC agent and the SCM model are updated.**
**Algorithm 1:** Counterfactual Trajectories Construction (CTC)
- **1 Function** CTC( _SCM models_ M1 _and_ M2 _, Pre-collision state 𝑠, Real-world reward 𝑟𝑎𝑐_ ) **:**
- **2** _𝑡𝑟𝑎𝑗𝑒𝑐𝑡𝑜𝑟𝑦_ = [ ]; **3 for** _𝑎𝑐𝑓_ = 1 **to** _𝑛_ **do 4** Rewind pre-collision state _𝑠_ and initialize noise _𝑢𝑠,𝑢𝑟_ ;
- **5** Predict _𝑠𝑐𝑓_ = M1 ( _𝑠,𝑎𝑐𝑓 ,𝑢𝑠_ ), _𝑟𝑐𝑓_ = M2 ( _𝑠,𝑎𝑐𝑓 ,𝑢𝑟_ ); **6** Construct _𝐶𝐹𝑟_ by Equation 8, _𝐶𝐹𝑙_ by Equation 9 and _𝐶𝐹𝑞_ by Equation 10;
- **7** _𝑡𝑟𝑎𝑗𝑒𝑐𝑡𝑜𝑟𝑦.𝑎𝑝𝑝𝑒𝑛𝑑_ ( _𝑠,𝑎𝑐𝑓 ,𝑠𝑐𝑓 ,𝑟𝑐𝑓_ ); **8 end 9 return** _𝑡𝑟𝑎𝑗𝑒𝑐𝑡𝑜𝑟𝑦_ ;
we define the following reward-based CF objective:
where _𝑟𝑎𝑐_ = _𝑤_ 1 · _𝑆𝑎𝑓𝑒𝑟_ + _𝑤_ 2 · _𝐸𝑓𝑓𝑒𝑟_ captures the reward composed of the number of collisions ( _𝑆𝑎𝑓𝑒𝑟_ ) and the total vehicle waiting time ( _𝐸𝑓𝑓𝑒𝑟_ ), both observed after executing the pre-collision action _𝑎_ . In the computation of the CF reward, _𝑟𝑐𝑓_ = _𝑤_ 1 · _𝑆𝑎𝑓𝑒𝑟__𝑐𝑓_ + _𝑤_ 2 · _𝐸𝑓𝑓𝑒𝑟__𝑐𝑓_, various CF actions _𝑎𝑐𝑓_ are explored to determine the number of CF collisions _𝑆𝑎𝑓𝑒𝑟__𝑐𝑓_ and the efficiency measure _𝐸𝑓𝑓𝑒𝑟__𝑐𝑓_.
This reward quantifies the causal effect of executing the CF action _𝑎𝑐𝑓_ . Assuming that efficiency remains constant before and after the intervention, and that collisions decrease from 2 (in the real-world observation) to 0 (in the CF scenario), the resulting reward difference becomes _𝐶𝐹𝑟_ = 0 −(−2) = 2. Such a CF trajectory
( _𝑠,𝑎𝑐𝑓 ,𝑟𝑐𝑓_ ) with an improved reward of 2 can substantially enhance the model’s learning performance for safety improvement, and conversely, a negative reward could indicate actions to avoid.
The distributional distance between the CF action and the action is formalized by:
where a collision occurs upon executing a real-world action, and we obtain the unsafe advantage distribution _𝐴𝐴𝐶_ generated by the policy _𝜋𝐻_ ( _𝑠_ ), which takes the current traffic state as input and determines whether an action is unsafe. We capture collisionrelated information, trace it back to the CF scenario, and explore alternative safe actions to derive the safe advantage distribution _𝐴𝐶𝐹_ = _𝑄_ ( _𝑠,𝑎𝑐𝑓_ ; _𝜃_ ) − _𝑉_ ( _𝑠_ ), measuring the discrepancy via crossentropy.
The occurrence of a collision in the CF world also introduces a novel objective aimed at risk mitigation. We explore the CF action _𝑎𝑐𝑓_ and evaluate its corresponding value _𝐶𝐹𝑞_ as:
where _𝑄𝑐𝑓_ is a network used to assess the safety value of the CF action _𝑎𝑐𝑓_ . This value contributes to the overall Q-value computation. The CF reward _𝐶𝐹𝑟_ is used to update _𝑄𝑐𝑓_ based on the Bellman Equation 11.
_3.2.2 CFLight._ We introduce CFLight-R, a variant of the CFLight algorithm based on the “CF+X” framework. The CF Objective utilizes _𝐶𝐹𝑟_ , the Safe module incorporates _𝑆𝑎𝑓𝑒𝑟_ , and the Efficient-RL module uses _𝐸𝑓𝑓𝑒𝑟_ , which represents the total vehicle waiting time. The components of CFLight-R can be succinctly represented as a combination: _𝐶𝐹𝑟_ + _𝑆𝑎𝑓𝑒𝑟_ + _𝐸𝑓𝑓𝑒𝑟_ .
• **State.** The state _𝑆_ contains the vehicle speed and location.
708
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Mingyuan Li et al.
|**Al**
CF|**gorithm 2:**CFLight Training (CFLight-R, CFLight-Loss,
Light-Q)|
|---|---|
|**I**|**nput:**Input epoch times_𝐾_, replay buffer_𝐷, 𝐷𝐶𝐹_, max steps
_𝑚𝑎𝑥𝑆𝑡𝑒𝑝𝑠_, pre train steps_𝑝𝑟𝑒𝑇𝑟𝑎𝑖𝑛𝑆𝑡𝑒𝑝𝑠_, random
epsilon_𝑒𝑝𝑠_, simulator_𝑒𝑛𝑣_and SCM modelsM.|
|**1 f**|**or**_𝑖_=1**to**_𝐾_**do**
|
|**2**|_𝑠_=_𝑒𝑛𝑣.𝑔𝑒𝑡𝑆𝑡𝑎𝑡𝑒_();|
|**3**|Initialize efficient RL and safe component and get_𝑄_
network;|
|**4**|_𝑡𝑜𝑡𝑎𝑙𝑆𝑡𝑒𝑝𝑠_=0;|
|**5**|**for** _𝑗_=1**to**_𝑚𝑎𝑥𝑆𝑡𝑒𝑝𝑠_**do**
|
|**6**|_𝑡𝑜𝑡𝑎𝑙𝑆𝑡𝑒𝑝𝑠_+=1;|
|**7**|**if** _random number < 𝑒𝑝𝑠_**then**|
|**8**|Explore random action;|
|**9**|**else**|
|**10**|Choose action_𝑎_based on_𝑄_network;|
|**11**|**end**|
|**12**|In real world, execute the action_𝑎_by_𝑒𝑛𝑣_, get_𝑆𝑎𝑓𝑒𝑟_,
real world trajectory (_𝑠,𝑎,𝑟,𝑠_′) and add it into
replay buffer_𝐷_;|
|**13**|**if**_𝑡𝑜𝑡𝑎𝑙𝑆𝑡𝑒𝑝𝑠> 𝑝𝑟𝑒𝑇𝑟𝑎𝑖𝑛𝑆𝑡𝑒𝑝𝑠_**then**|
|**14**|Sample trajectories from_𝐷_and_𝐷𝐶𝐹_;|
|**15**|Update_𝑄_by Equation 11(CFLight-R),
Equation**??**(CFLight-Loss) or Equation**??**
(CFLight-Q)|
|**16**|**end**|
|**17**|**end**|
|**18**|**for**_i % trainGap == 0_**do**|
|**19**|_𝐷𝐶𝐹.𝑐𝑙𝑒𝑎𝑟_();|
|**20**|UpdateM by Equation 5;|
|**21**|**if** _collision_**then**|
|**22**|Go to CF world to collect CF trajectories|
||(_𝑠,𝑎𝑐𝑓,𝑟𝑐𝑓,𝑠𝑐𝑓_) according to CTC function;|
|**23**|Add CF trajectories into replay buffer_𝐷𝐶𝐹_;|
|**24**|**end**|
|**25**|**end**|
|**26 e**|**nd**|
- **Action.** The action _𝐴_ is a combination of the **permittedonly** and **protected-only** phases.
- **Reward.** The reward _𝑟_ takes the value _𝑅𝐴𝐶_ in the real world and _𝐶𝐹𝑟_ in the CF scenario.
Finally, we update the network through 3DQN and _𝑄_ is updated according to the Bellman function:
Using CF data augmentation, the following lemma ensures that the Q-function will achieve the optimal value [21]:
Lemma 2. _Given the transition dynamics, 𝑄-learning on the counterfactually augmented data set converges with probability one to the optimal value function 𝑄_∗ _, as long as the state and action spaces are finite, and the learning rate $\alpha_t$ satisfies $\sum_t \alpha_t = \infty$ and $\sum_t \alpha_t^2 < \infty$._
Algorithm 2 shows a specific implementation based on CF module _𝐶𝐹𝑟_ , X module ( _𝑆𝑎𝑓𝑒𝑟_ as safe component; _𝐸𝑓𝑓𝑒𝑟_ as efficient-RL component). The specific CFLight easily accommodates different CF and X modules.
### **4 Experiments**
In this section, we concentrate on non-periodic acyclic phase switching and extensively experiment with synthetic and real-world datasets. These experiments are conducted in the microscopic traffic simulator, SUMO [15]. We set the collision rate to 40% under time-variant traffic flow.
### **4.1 Datasets**
We experiment with a synthetic intersection [19] and a real-world intersection [22].
**Synthetic:** The synthetic intersection comprises four methods and twelve one-way vehicular movements with synthetic traffic flow adopted into SUMO.
**Real-world:** The real-world dataset is derived from the Cologne, Germany intersection, incorporating real traffic patterns into the SUMO environment. This intersection features 8 approaching lanes. In both cases, the SUMO simulation environment assumes accurate vehicle detection, consistent deceleration rates (modifiable as needed), and permitted left turns within the timing plan.
### **4.2 Compared Methods**
In this study, we compared four types of RL methods for intersection control. These methods include:
**Traditional method:** Fixed-time [14].
**RL methods:** 3DQN [19], IPPO, Syn-R [13], Syn-Q [7], CPO [1] and SafeLight-Loss [5].
**Our methods:** CFLight-1 corresponds to _𝐶𝐹𝑟_ + _𝑆𝑎𝑓𝑒𝑟_ + _𝑅𝐿_ 3 _𝐷𝑄𝑁_ . CFLight-2 corresponds to _𝐶𝐹𝑙_ + _𝑆𝑎𝑓𝑒𝑙_ + _𝑅𝐿_ 3 _𝐷𝑄𝑁_ . CFLight-3 corresponds to _𝐶𝐹𝑞_ + _𝑆𝑎𝑓𝑒𝑣_ + _𝑅𝐿_ 3 _𝐷𝑄𝑁_ .
All methods share the same action space, which includes both protected-only and permitted-only phases. We evaluate these TSC methods from the perspectives of efficiency and safety. The efficiency measure includes Average Delay (average vehicle waiting time) and the number of vehicles throughput within a period of time, while the safety measure is the sum of collisions over time.
### **4.3 Results**
We average the results from the last 10 rounds across two benchmark datasets, with key outcomes presented in Table 1. On the synthetic dataset, CFLight-Loss reduces collision rates by 93.1% compared to 3DQN, outperforming current state-of-the-art (SOTA) methods. On the real-world dataset, the three CFLight variants achieve an average 32.6% reduction in collisions over non-CF methods, with CFLight-R emerging as the top-performing solution based on collision metrics.
Figure 4 illustrates the training curves of the model after incorporating the CF module. It can be observed that the training curve for collisions with the CF module shows an improvement compared to the one without the CF module, demonstrating the effectiveness of our approach.
709
CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
**Table 1: Performance of all methods on synthetic and real-world datasets. Green colors show the percentage improvement when adding the CF module. Red colors indicate the percentage of degradation.**
Synthetic — Acyclic Action
SafeLight-Act CPO Syn-R CFLight-R (Improved) SafeLight-Loss CFLight-Loss (Improved) Syn-Q CFLight-Q (Improved) 3DQN Fixed-time
4.73
Average Delay (s) ±1.04 22.16±3.23 9.25±1.03 8.67±1.07 (+6.3%) 30.29±1.28 29.89±2.15(+1.3%) 8.54±0.14 8.87±1.11(-3.8%) 4.24 ± 0.13 31.22±1.34
_Po 1775.3
Throughput ±52.35 1760.2±43.31 1784.8±50.03 1802.8 ± 80.25 (+1.0%) 1106.2±30.05 1107.2±30.05(+0.01%) 1753±40.09 1781.3±40.07(+1.59%) 1785.6±0.04 1782±0.05
15.9
Collision Count ±6.21 13.2±3.53 3.5±1.16 2.5±1.05(+28.6%) 1.8±0.15 1.3 ± 0.16 (+27.7%) 5.7±4.33 4.6±2.12 (+19.2%) 19±0.15 7.6±0.16
poa Real-world — Cologne, Acyclic Action
23.32
Average Delay (s) ±3.12 44.06±8.24 12.59±3.14 20.27±3.13 (-61.1%) 66.86±1.19 47.74±3.56 (+40.0%) 16.20±5.18 22.28±6.16(-37.5%) 5.54 ± 1.12 67.25±3.25
Po 2013
Throughput ±0.04 2014.6±0.04 2014±0.07 2014 ± 0.05 1733.5±5.03 2014.7±0.04 (+16.2%) 2014.4±0.05 2014.8±0.04(+0.01%) 2014±0.05 2015±0.04
Po 13.2
Collision Count ±6.32 41.3±8.56 20.3±6.21 5.3 ± 4.56 (+73.8%) 13.4±5.34 13.3±4.23 (+0.01%) 13.06±2.51 9.9±3.34(+24.1%) 40.07±4.13 35.1±3.32
Po
80 Collisions Comparison CFLight-R3DQN Sync-R Safe-Act 8070 Collisions Comparison 3DQNSafe-ActCFLight-QSync-Q 8070 Collisions Comparison Safe3DQNCFLight-LossSafe-Act-Loss
60 6050 6050
40 40 40
30 30
20 20 20
10 10
0 0 100 200 300 Epoch400 500 600 700 800 0 0 100 200 300 Epoch400 500 600 700 800 0 0 100 200 300 Epoch400 500 600 700 800
350300 Wait Average Comparison CFLight-R3DQNSync-RSafe-Act 400 Wait Average Comparison 3DQN Safe-Act CFLight-QSync-Q 350300 Wait Average Comparison Safe-Loss3DQNCFLight-LossSafe-Act
250 300 250
200 200
150 200 150
100 100 100
50 50
0 0 100 200 300 Epoch400 500 600 700 800 0 0 100 200 300 Epoch400 500 600 700 800 0 0 100 200 300 Epoch400 500 600 700 800
CFLight-R CFLight-Q CFLight-Loss
Collisions Collisions Collisions
Wait Average Wait Average Wait Average
**Figure 4: The figures show the traning curve for safety and efficiency for using CFLight in the Cologne dataset.**
### **4.4 Safety and Efficiency Trade-off**
In this section, we illustrate the influence of adjusting weights on both safety and efficiency in synthetic dataset. Leveraging the CFLight-1 algorithm, we conduct 800 training rounds and establish the benchmark value by averaging the results from the last 10 rounds, as detailed in Figure 2. The outcome reveals the emergence of a Pareto optimal surface. Importantly, a harmonious equilibrium is attained when the weights assigned to safety and efficiency closely align. These insights are derived from testing on a synthetic dataset using CFLight-R.
### **4.5 Ablation Analysis**
We conduct ablation experiments on real datasets involving _𝐶𝐹𝑟_ , _𝐶𝐹𝑟_ +3DQN, 3DQN, _𝑆𝑎𝑓𝑒𝑟_ and _𝐶𝐹𝑟_ + _𝑆𝑎𝑓𝑒𝑟_ configurations to assess the efficacy of the CF module. Notably, employing only the CF reward _𝐶𝐹𝑟_ leads to a 45.5% reduction in collision incidents compared to 3DQN, but with a huge increase in waiting time because it works without optimization in waiting time. By using _𝐶𝐹𝑟_ + _𝑅𝐿_ 3 _𝐷𝑄𝑁_ method, collision rates decrease by 16.3%, and waiting times improve significantly compared to the standalone CF module _𝐶𝐹𝑟_ . Compared to the _𝑆𝑎𝑓𝑒𝑟_ method, CF+ _𝑆𝑎𝑓𝑒𝑟_ module the collisions reduce about 36.2% which shows the validity of our CF module.
### **4.6 Case Study**
In this section, we present a visualization of the next state and reward predicted by the SCM M before and after a collision event.
1.0 1.0 2.0
0 Collision State 0 Collision Next State 0 Safe and Efficiency 1.5
0.8 0.8
10 10 10 1.0
20 0.6 20 0.6 20 0.5
30 30 30 0.0
40 0.4 40 0.4 40 0.5
50 50 50 1.0
0.2 0.2
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 1.5
X X
0.0 0.0 2.0
Y Y
Norm Speed Norm Speed Norm Reward
**Figure 5: In the real world collision case, execute the permitted-only (** _𝑎_ = 0 **) phase for state, next state, and reward. The third subgraph shows safety (top) and efficiency (bottom).**
The traffic signal phase prior to the collision is set to permittedonly, and we evaluate the state and reward variations when the SCM predicts outcomes for a protected-only phase. The state is modeled as a 60 × 60 matrix, with the _𝑥_ - and _𝑦_ -axes representing vehicle coordinates and the heatmap values indicating normalized vehicle speeds. The CF reward is computed as _𝐶𝐹𝑟_ = _𝑟𝑐𝑓_ − _𝑟𝑎𝑐_ . In the third subplot on the right side of the figure, the upper half of the reward plot reflects safety (higher values improve safety) and the lower half represents efficiency. The results reveal that adopting the protected-only phase, as predicted by the SCM, significantly increases the safety reward, highlighting its effectiveness in improving the learning of the CF trajectory. Moreover, the CF next states
710
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Mingyuan Li et al.
1.0 2.0
0 Collision State 0 CF Collision Next State 0 Safe and Efficiency 1.5
0.8 0.8
10 10 10 1.0
20 0.6 20 0.6 20 0.5
30 30 30 0.0
40 0.4 40 0.4 40 0.5
50 50 50 1.0
0.2 0.2
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 1.5
X X
0.0 2.0
Y Y
Norm Speed Norm Speed CF Norm Reward
**Figure 6: In the CF world collision case, execute the protectedonly (** _𝑎𝑐𝑓_ = 3 **) phase for state state, next state and reward** _𝐶𝐹𝑟_ **predict by SCM** M **.**
**Figure 7: The left image shows the safety-efficiency trade-off, while the right compares protected-only and permitted-only phases based on average waiting time and collision counts in the Cologne dataset.**
closely resemble the real-world next states in structure, validating the robustness and accuracy of our proposed method.
### **5 Discussion**
**(1) How does the CF module’s SCM with GAN approximation operate and what are its benefits?** In CFLight, to realistically simulate the aftermath of traffic collisions, we configure the SUMO simulator to introduce a delay after a collision vehicles involved are briefly held before being rerouted to their destinations. This setup leads to temporary congestion and increased waiting time, aligning with real-world traffic scenarios. Moreover, CFLight introduces a SCM and employs a BiCoGAN to generate counterfactual trajectories under alternative actions, enabling counterfactual data augmentation. This technique creates diverse and realistic hypothetical scenarios, enhancing data diversity and model generalization. As a result, it improves sample efficiency and overall performance without requiring additional real-world data, and comes with theoretical convergence guarantees (Lemma 2).
**(2) Why not just use protected-only phases?** We do experiments using only protected-only in the real-world Cologne (1 intersection) dataset as shown in Figure 7, and we see that passage efficiency is greatly reduced. The number of collisions is greatly increased by using only permitted-only phase. In the real world, we observe that there are only two lanes on each road and each lane contains two directions, and the protected-only phase only allows passage in one direction, which may be the reason for the reduced efficiency. The original intention of our CFLight design is to flexibly switch between these two phases, ensuring both efficiency and safety in
**Table 2: CFLight performance on 100 intersections of synthetic dataset.**
|Synthetic 100|Intersection|s Acyclic|Action|
|---|---|---|---|
||CFLight-R|Syn-R|Fixed-time|
|Average Delay (s)|**3098.90**|3162.37|32708.37|
|Throughput|6831|**6852**|7065|
|Collision Count|**47**|55|699|
**Table 3: Multiple intersection experiments in Cologne3 and Cologne8.**
||Real-world|Cologne3 Ac|yclic Action|
|---|---|---|---|
||CFLight-R|IPPO|Fixed-time|
|Average Delay(s)|15.21±2.16|7.37±3.13|65.9±5.32|
|Collisions|14.07±3.14|30.07±3.65|22±4.13|
||Real-world|Cologne8 Ac|yclic Action|
|Average Delay(s)|12.34±4.15|5.01±5.16|110.33±5.34|
|Collisions|32.31±5.34|64.31±6.67|47±6.78|
traffic.
**(3) Are there any other scenario experiments for our proposed method?** We conduct lane-changing experiments in an autonomous driving setting to evaluate the CF+X framework. In this task, a vehicle must change across two lanes while assessing collision risk. The state includes the speed of nearby vehicles, the action indicates whether the vehicle changes lanes, and the reward reflects whether a collision occurs. We train an RL agent using CF+X with _𝐶𝐹𝑟_ for 100 rounds and compare it to a baseline without CF+X. As shown in Figure 8, the left graph shows that CF+X reduces the average collision rate to 0.09, compared to 0.34 without the framework, representing a 73.5% reduction in collisions.
Mean Collision Collision
Using CF+X Module Without CF+X Module
**Figure 8: The collision result of using our proposed CF+X and without using CF+X under the lane change scenario. Our proposed method improved 73.5% performance in terms of average collisions on 100 rounds**
**(4) Can our method be applied to multiple intersections?** We used the TSC benchmark [2] datasets with multiple intersections (3, 8 and 100 intersections) and conduct experiments with IPPO as the backbone, CFLight uses _𝐶𝐹𝑟_ as the CF objective. Our results, as shown in Table 2 and Table 3, indicate that CFLight effectively enhances performance, demonstrating the effectiveness of our method.
711
CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
**(5) What are the strengths and limitations of the various CFLight extension methods? In which scenarios are these methods most appropriately applied?** CFLight-Loss leverages SCM-derived action advantages and jointly optimizes safety constraints and efficiency in a unified loss function, demonstrating flexibility but higher sensitivity to hyperparameters. CFLight-Q enhances interpretability by directly optimizing Q-values under CF scenarios, yielding near-zero collision rates, albeit at the cost of increased complexity and reliance on Q-network convergence. Each variant offers distinct strengths and trade-offs, making them suitable for different deployment priorities. We also create a table comparing the advantages and disadvantages of CFLight against other safe methods.
**(6) How do you identify which action/state in the trajectory was causal?** To identify causal actions/states, we compare outcomes by backtracking to pre-collision states and executing safe actions. If a collision is avoided, it reveals causal relationships. Our CTC generates tuples ( _𝑠,𝑎𝑐𝑓 ,𝑟𝑐𝑓 ,𝑠𝑐𝑓_ ) for capturing these causal relationships.
**(7) Why is it necessary to assume that** M **is monotonic in order to predict CF in an SCM?** The monotonicity assumption on M1 and M2 with respect to the exogenous variables _𝑈𝑠,𝑈𝑟_ is essential for counterfactual prediction. Specifically, given observed data ( _𝑆,𝐴,𝑆_′ _, 𝑅_ ) and the structural equations:
identifying the hidden variables _𝑈𝑠_ and _𝑈𝑟_ from the observations requires inverting M1 and M2. If M1 and M2 are monotonic in _𝑈𝑠_ and _𝑈𝑟_ respectively (for fixed _𝑆,𝐴_ ), then these mappings are invertible and the values of _𝑈𝑠,𝑈𝑟_ are uniquely determined:
This uniqueness is critical for generating consistent counterfactual outcomes under a hypothetical action _𝑎𝑐𝑓_ , as it ensures that the same latent variables can be reused to simulate the counterfactual outcome:
Without monotonicity, the inversion may be non-unique or illdefined, making counterfactual inference ambiguous.
**(8) How are CF actions selected to ensure safety for CFLightLoss?** CF actions are selected by evaluating their advantage using the learned SCM and Q-function. Specifically: We monitor the real-world or simulated trajectory and identify actions leading to safety violations. For each detected unsafe state-action pair ( _𝑠,𝑎_ ), we generate a set of alternative candidate actions { _𝑎𝑐𝑓_(1)_,𝑎_ _𝑐𝑓_(2)_, . . ._} and simulate their outcomes using the SCM:
For each candidate action, we compute its CF advantage:
We choose the CF action _𝑎𝑐𝑓_ with the highest advantage, under the constraint that it does not trigger unsafe outcomes as predicted by the SCM (i.e., _𝑟𝑐𝑓_ indicates safety). This process balances _efficiency_ and _safety_ , enabling the agent to improve policy performance while avoiding unsafe behaviors.
Collisions Comparison Wait Average Comparison
80 CFLight_R_noise_0.5CFLight-RCFLight_R_noise_1 CFLight _ R _ noise _ 0.1 500 CFLight_R_noise_0.5CFLight-RCFLight_R_noise_1 CFLight _ R _ noise _ 0.1
400
60
300
40
200
20
100
0 0 100 200 Epoch 300 400 500 0 0 100 200 Epoch 300 400 500
Collisions
Wait Average
**Figure 9: Performance under noise of CFLight-R.**
**(9) How sensitive is the method to inaccurate SCM predictions under varying traffic conditions or noise?** Figure 9 shows that, CFLight-R exhibits moderate sensitivity to inaccurate SCM predictions under varying traffic conditions, with noise modeled as _𝑁_ (0 _,𝑠𝑐𝑎𝑙𝑒_ ) and states normalized to the [0, 1] range. The Collisions Comparison shows that higher noise (0, 1) results in slightly more collisions and variability compared to lower noise (0, 0.5). Similarly, the Wait Average Comparison reveals a higher initial wait time and fluctuations with noise scale 1, compared to the smoother decline with scale 0.5, with the impact of noise diminishing over epochs. In the future, we will explore more control methods to improve the robustness of CFLight like [17, 18, 36].
### **6 Conclusion**
In this study, we introduced a flexible “CF+X” safe reinforcement learning framework, integrating counterfactual learning with the “X” module consisting of adjustable safety and efficient RL components. Within this framework, we developed a novel algorithm called CFLight, addressing CF and safety concerns while smoothly integrating with existing RL methods for TSC. Our method was evaluated in the context of safe and efficient traffic management at intersections, showcasing an impressive 93.1% reduction in collision rates compared to DQN while concurrently enhancing traffic efficiency. This work holds promise for broader applications in safe reinforcement learning, with potential implications for autonomous systems and robotics, offering a robust solution to enhance safety in complex decision-making scenarios.
### **Acknowledgments**
This work is supported by Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (Grant No. JYB2025XDXM910), National Natural Science Foundation of China (Grant Nos. 42595590, 42595593), Talent Scientific Fund of Lanzhou University (Grant No. 561120208), and Supercomputing Center of Lanzhou University. G. Yu contributed as an independent researcher in a personal capacity. This work was completed before Australia’s 2025 international collaboration policy updates. G. Yu received no funding or material support for this research, and no institutional resources, facilities, or work hours from his affiliation were utilized. This contribution is outside his institutional employment and adheres to all intellectual property regulations.
712
KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
Mingyuan Li et al.
### **References**
- [1] Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. 2017. Constrained policy optimization. In _Proceedings of the International Conference on Machine Learning (ICML)_ .
- [2] James Ault and Guni Sharon. 2021. Reinforcement learning benchmarks for traffic signal control. In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)_ .
- [3] Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. 2020. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 3414–3421.
- [4] Seung-Bae Cools, Carlos Gershenson, and Bart D’Hooghe. 2013. Self-organizing traffic lights: A realistic simulation. In _Advances in applied self-organizing systems_ . Springer, 45–55.
- [5] Wenlu Du, Junyi Ye, Jingyi Gu, Jing Li, Hua Wei, and Guiling Wang. 2023. Safelight: A reinforcement learning method toward collision-free traffic signal control. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 37. 14801–14810.
- [6] FHWA. 2023. Intersection Safety. https://highways.dot.gov/safety/intersectionsafety/about Accessed on: August 1, 2023.
- [7] Yaobang Gong, Mohamed Abdel-Aty, Jinghui Yuan, and Qing Cai. 2020. Multiobjective reinforcement learning approach for improving safety at intersections with adaptive traffic signal control. _Accident Analysis & Prevention_ 144 (2020), 105655.
- [8] Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, Yaodong Yang, and Alois Knoll. 2023. A Review of Safe Reinforcement Learning: Methods, Theory and Applications. arXiv:2205.10330 [cs.AI]
- [9] Mengyu Guo, Pin Wang, Ching-Yao Chan, and Sid Askary. 2019. A reinforcement learning approach for intelligent traffic signal control at urban intersections. In _2019 IEEE Intelligent Transportation Systems Conference (ITSC)_ . IEEE, 4242–4247.
- [10] Yimin He, Stephanie C. Payne, Xiang Yao, and Rachel Smallman. 2020. Improving workplace safety by thinking about what might have been: A first look at the role of counterfactual thinking. _Journal of Safety Research_ 72 (2020), 153–164. doi:10.1016/j.jsr.2019.12.010
- [11] Ayush Jaiswal, Wael AbdAlmageed, Yue Wu, and Premkumar Natarajan. 2019. Bidirectional conditional generative adversarial networks. In _Computer Vision– ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14_ . Springer, 216–232.
- [12] Junchen Jin, Xiaoliang Ma, and Iisakki Kosonen. 2017. An intelligent control system for traffic lights with simulation-based evaluation. _Control engineering practice_ 58 (2017), 24–33.
- [13] Mohamed A Khamis and Walid Gomaa. 2014. Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. _Engineering Applications of Artificial Intelligence_ 29 (2014), 134–151.
- [14] P Koonce. 2008. Traffic signal timing manual. technical report. _Kittelson & Associates, Inc., Portland, OR, USA, Tech. Rep. FHWA-HOP-08-024_ (2008).
- [15] Daniel Krajzewicz, Jakob Erdmann, Michael Behrisch, and Laura Bieker. 2012. Recent development and applications of SUMO-Simulation of Urban MObility. _International journal on advances in systems and measurements_ 5, 3&4 (2012).
- [16] Jiahui Li, Kun Kuang, Baoxiang Wang, Furui Liu, Long Chen, Fei Wu, and Jun Xiao. 2021. Shapley counterfactual credits for multi-agent reinforcement learning. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_ . 934–942.
- [17] Mingyuan Li, Jiahao Wang, Bo Du, Jun Shen, and Qiang Wu. 2025. FuzzyLight: A Robust Two-Stage Fuzzy Approach for Traffic Signal Control Works in Real Cities. _arXiv preprint arXiv:2501.15820_ (2025).
- [18] Mingyuan Li, Jiahao Wang, Guangsheng Yu, Xu Wang, Qianrun Chen, Wei Ni, Lixiang Li, and Haipeng Peng. [n. d.]. RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control. In _Forty-second International Conference on Machine Learning_ .
- [19] Xiaoyuan Liang, Xunsheng Du, Guiling Wang, and Zhu Han. 2019. A deep reinforcement learning network for traffic light cycle control. _IEEE Transactions_
_on Vehicular Technology_ 68, 2 (2019), 1243–1253.
- [20] Tongtong Liu, Joe McCalmon, Thai Le, Md Asifur Rahman, Dongwon Lee, and Sarra Alqahtani. 2023. A novel policy-graph approach with natural language and counterfactual abstractions for explaining reinforcement learning agents. _Autonomous Agents and Multi-Agent Systems_ 37, 2 (2023), 34.
- [21] Chaochao Lu, Biwei Huang, Ke Wang, José Miguel Hernández-Lobato, Kun Zhang, and Bernhard Schölkopf. 2020. Sample-efficient reinforcement learning via counterfactual-based data augmentation. _arXiv preprint arXiv:2012.09092_ (2020).
- [22] Hao Mei, Xiaoliang Lei, Longchao Da, Bin Shi, and Hua Wei. 2022. LibSignal: An Open Library for Traffic Signal Control. _arXiv preprint arXiv:2211.10649_ (2022).
- [23] Thomas Mesnard, Théophane Weber, Fabio Viola, Shantanu Thakoor, Alaa Saade, Anna Harutyunyan, Will Dabney, Tom Stepleton, Nicolas Heess, Arthur Guez, et al. 2020. Counterfactual credit assignment in model-free reinforcement learning. _arXiv preprint arXiv:2011.09464_ (2020).
- [24] Yusuke Narita, Kyohei Okumura, Akihiro Shimizu, and Kohei Yata. 2023. Counterfactual Learning with General Data-Generating Policies. _Proceedings of the AAAI Conference on Artificial Intelligence_ 37, 8 (2023), 9286–9293.
- [25] World Health Organization et al. 2018. Global status report on road safety 2018: Summary (No. WHO/NMH/NVI/18.20). _World Health Organization_ (2018).
- [26] Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. 2020. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. _Advances in Neural Information Processing Systems_ 33 (2020), 4079–4090.
- [27] Judea Pearl. 2009. _Causality_ . Cambridge university press.
- [28] Bob Pishue. 2021. 2021 inrix global traffic scorecard. In _INRIX (December 2021)_ .
- [29] Mattia C. F. Prosperi, Yi Guo, Matthew Sperrin, James S. Koopman, Jae S. Min, Xing He, Shannan N. Rich, Mo Wang, Iain E. Buchan, and Jiang Bian. 2020. Causal inference and counterfactual prediction in machine learning for actionable healthcare. _Nat. Mach. Intell._ 2, 7 (2020), 369–375. doi:10.1038/s42256-020-0197-y
- [30] Faizan Rasheed, Kok-Lim Alvin Yau, and Yeh-Ching Low. 2020. Deep reinforcement learning for traffic signal control under disturbances: A case study on Sunway city, Malaysia. _Future Generation Computer Systems_ 109 (2020), 431–445.
- [31] Donald B. Rubin. 1974. Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies. _Journal of Educational Psychology_ 66, 5 (1974), 688–701.
- [32] Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, and Yang Liu. 2020. Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning. _Proceedings of the AAAI Conference on Artificial Intelligence_ 34 (2020), 5883–5891.
- [33] Elise Van der Pol and Frans A Oliehoek. 2016. Coordinated deep reinforcement learners for traffic light control. _Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016)_ (2016).
- [34] Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan E. Hines, John P. Dickerson, and Chirag Shah. 2022. Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review. arXiv:2010.10596 [cs.LG]
- [35] Hua Wei, Nan Xu, Huichu Zhang, Guanjie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Jessie Li. 2019. CoLight: Learning Network-level Cooperation for Traffic Signal Control. _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ (2019).
- [36] Qiang Wu, Mingyuan Li, Jun Shen, Linyuan Lü, Bo Du, and Ke Zhang. 2023. Transformerlight: A novel sequence modeling based traffic signaling mechanism via gated transformer. In _Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining_ . 2639–2647.
- [37] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. 2020. Adversarial Counterfactual Learning and Evaluation for Recommender System. _Advances in Neural Information Processing Systems_ 33.
- [38] Liang Zhang, Qiang Wu, Jun Shen, Linyuan Lü, Bo Du, and Jianqing Wu. 2022. Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control. In _Proceedings of the 39th International Conference on Machine Learning_ , Vol. 162. 26645–26654.
713