Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e

Chenwei Xu, Jerry Yao-Chieh Hu, Aakaash Narayanan, Mattson Thieme, Vladimir Nagaslaev +19 more
12/28/2023
cs.LGcs.AIphysics.acc-ph

No confident code implementation found for this paper.

Abstract

We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab). Our primary objective is to regulate the spill process to ensure a consistent intensity profile, with the ultimate goal of creating an automated controller capable of providing real-time feedback and calibration of the Spill Regulation System (SRS) parameters on a millisecond timescale. We treat the Mu2e accelerator system as a Markov Decision Process suitable for Reinforcement Learning (RL), utilizing PPO to reduce bias and enhance training stability. A key innovation in our approach is the integration of a neuralized Proportional-Integral-Derivative (PID) controller into the policy function, resulting in a significant improvement in the Spill Duty Factor (SDF) by 13.6%, surpassing the performance of the current PID controller baseline by an additional 1.6%. This paper presents the preliminary offline results based on a differentiable simulator of the Mu2e accelerator. It paves the groundwork for real-time implementations and applications, representing a crucial step towards automated proton beam intensity control for the Mu2e experiment.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Optimisation of the Accelerator Control by Reinforcement Learning: A Simulation-Based Approach

83% match2025

RL-ABC: Reinforcement Learning for Accelerator Beamline Control

81% match2026

Automated control and optimisation of laser driven ion acceleration

80% match2023

Autonomous Pressure Control in MuVacAS via Deep Reinforcement Learning and Deep Learning Surrogate Models

79% match2025

Cheetah: Bridging the Gap Between Machine Learning and Particle Accelerator Physics with High-Speed, Differentiable Simulations

79% match2 repos722024

Reinforcement Learning Controllers for Fusion Scenario Development

78% match2026

Cite this paper

@article{xu2023beyond,
  title  = {Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e},
  author = {Chenwei Xu and Jerry Yao-Chieh Hu and Aakaash Narayanan and Mattson Thieme and Vladimir Nagaslaev and Mark Austin and Jeremy Arnold and Jose Berlioz and Pierrick Hanlet and Aisha Ibrahim and Dennis Nicklaus and Jovan Mitrevski and Jason Michael St. John and Gauri Pradhan and Andrea Saewert and Kiyomi Seiya and Brian Schupbach and Randy Thurman-Keup and Nhan Tran and Rui Shi and Seda Ogrenci and Alexis Maya-Isabelle Shuping and Kyle Hazelwood and Han Liu},
  year   = {2023},
  eprint = {2312.17372},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.17372v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.17372.svg)](https://codeofpaper.com/paper/2312.17372)

Discussion