His PhD research is on better sequential decision making under uncertainty, approached through deep reinforcement learning and dynamic algorithm configuration — including plasticity preservation and managing high-dimensional, coupled action spaces — e.g. his work on CANDID DAC, which factors large action spaces while keeping interdependent dimensions coordinated.
Publications
2026
Towards Understanding the Impact of Plasticity Loss on Reinforcement Learning in Stochastic Environments
Philipp Bordne and André Biedenkapp
In Nineteenth European Workshop on Reinforcement Learning, 2026
Deep reinforcement learning (RL) is a powerful framework for sequential decision making. Despite many high-profile successes in recent years, deep RL is known to be brittle. An emerging research direction into the causes of this brittleness concerns the loss of plasticity of neural networks. However, plasticity loss has so far been understudied in stochastic environments. Our study addresses this issue and provides a principled analysis of the interplay of noise and plasticity loss. We find that plasticity-preserving methods increase learning robustness in stochastic environments, with their advantage growing under higher observation noise, while combining resets with layer normalization is more often detrimental than beneficial. Further, stochasticity can accelerate the loss of plasticity in harder environments, whereas in easier settings noise can act as a regularizer that sustains plasticity. Lastly, our study shows that plasticity loss is most pronounced in early stages of training, and that in the most challenging stochastic settings TD instability emerges as a more critical failure mode which is more likely to be triggered by the presence of noise.
@inproceedings{bordne-ewrl26a,title={Towards Understanding the Impact of Plasticity Loss on Reinforcement Learning in Stochastic Environments},author={Bordne, Philipp and Biedenkapp, André},year={2026},booktitle={Nineteenth European Workshop on Reinforcement Learning},}
2024
CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC
Philipp Bordne, M. Asif Hasan, Eddie Bergman, Noor Awad, and André Biedenkapp
In Proceedings of the Third International Conference on Automated Machine Learning (AutoML 2024), 2024
High-dimensional action spaces remain a challenge for dynamic algorithm configuration (DAC). Interdependencies and varying importance between action dimensions are further known key characteristics of DAC problems. We argue that these Coupled Action Dimensions with Importance Differences (CANDID) represent aspects of the DAC problem that are not yet fully explored. To address this gap, we introduce a new white-box benchmark within the DACBench suite that simulates the properties of CANDID. Further, we propose sequential policies as an effective strategy for managing these properties. Such policies factorize the action space and mitigate exponential growth by learning a policy per action dimension. At the same time, these policies accommodate the interdependence of action dimensions by fostering implicit coordination. We show this in an experimental study of value-based policies on our new benchmark. This study demonstrates that sequential policies significantly outperform independent learning of factorized policies in CANDID action spaces. In addition, they overcome the scalability limitations associated with learning a single policy across all action dimensions. The code used for our experiments is available under https://github.com/PhilippBordne/candidDAC.
@inproceedings{bordne-automlws24a,title={CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC},author={Bordne, Philipp and Hasan, M. Asif and Bergman, Eddie and Awad, Noor and Biedenkapp, André},booktitle={Proceedings of the Third International Conference on Automated Machine Learning (AutoML 2024)},year={2024},openreview={ZCCZYfstkG},specialtrack={Workshop Track},}
