Research
My papers, and the work I am doing now. I work with my advisors Prof. Annie Qu and Prof. Rui Miao.
A list of my articles is also available on my Google Scholar profile.
Conference papers
TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
In multi-task PPO, the critic’s per-task gradients differ in scale far more than the actor’s, and the hardest tasks stall. Balancing the critic gives a 717K-parameter shared actor-critic the strongest tail on Meta-World+ MT50: 56.5% success on each seed’s ten worst tasks, against 43.3% for ARS-LN-1024, a baseline with 22.7× the parameters, rerun under the same protocol.
Research since TOPPO
Scaling TOPPO to VLA and LLM RL
Sep 2026
After TOPPO, I tried it at scale. I set up RL fine-tuning for a 3.3B π0 vision-language-action model with RLinf, and built a GRPO/PPO pipeline for LLMs on verl, vLLM and FSDP2, with a Nemotron-3-Super-120B judge self-hosted on vLLM as the reward model. Unlike on small networks, standard gradient surgery on the whole network showed no clear benefit on π0 or on LLMs. That result led to the next project: looking inside the network.
Where tasks compete inside deep networks
Ongoing
This follows from the scaling study above. Gradient-balancing methods such as FairGrad weigh tasks by comparing whole-network gradients. Benchmarking 26 multi-task gradient methods in one PyTorch trainer on NYUv2, CelebA and QM9, I found that a few small early layers carry most of the gradient, so the whole-network comparison mostly reflects them; yet on QM9 the energy targets’ gradients differ only in the final readout layer. I am developing a block kernel that balances each module’s contribution to the task weights.
Runtime-aware HPC job scheduling
Ongoing
Separately from multi-task learning, I wrote slurmsim, a Python and C simulator that reproduces real Slurm’s job start order. Setting each job’s minimum time from a Kaplan–Meier estimate of its runtime, with the user’s limit kept as the hard cap, cut mean turnaround by 8–18% in simulation and by 8% in a replay of a national-lab trace, with almost no extra timeouts.
