Aditya Modi

Aditya Modi


Email: adityamodi94 [at] gmail [dot] com
admodi [at] umich [dot] edu


I'm an Applied Scientist at Microsoft AI. At Microsoft, I work on applied research problems in machine learning aimed at improving their advertising products. My research interests lie in general sequential decision making problems with a focus on the fundamental understanding and applications of interactive learning (reinforcement learning and bandits) and causal inference.

Before joining Microsoft, I finished my PhD at the Computer Science and Engineering department in University of Michigan, Ann Arbor. During my PhD, I worked on theoretical aspects of interactive learning and sequential decision making frameworks like reinforcement learning, bandits and (general) online learning. I was fortunate to be advised by Satinder Singh and Ambuj Tewari. My thesis developed provable sample efficient methods for reinforcement learning under various structural assumptions.

Before my stint in RL theory at UMich, I obtained my undergraduate degree in Computer Science from Indian Institute of Technology, Kanpur in May 2016.

Recent News

  • Our work 'Model-free Representation Learning and Exploration' has been accepted to JMLR with minor revisions.
  • The paper on 'Joint Learning of Linear Time-Invariant Dynamical Systems' has been (provisionally) accepted to Automatica.
  • Our work on reward-free exploration in RL accepted to NeurIPS '22.
  • (Nov '21') Joined Microsoft as an Applied Scientist in the bay area.
  • Defended my PhD thesis in November 2021. A special thanks to my advisors for supporting me through this journey!
  • (Fall '20) Visited the Theory of RL program at Simons Institute for the Theory of Computing as visiting graduate student.
  • Spent summer 2018 with the Adaptive Systems and Interaction group at Microsoft Research Redmond, working with Debadeepta Dey.


  1. Simple Data Sharing for Multi-Tasked Goal-Oriented Problems
    Ying Fan, Jingling Li, Adith Swaminathan, Aditya Modi, Ching-An Cheng
    Generalization in Planning Workshop, NeurIPS '23.

  2. Towards Modular Machine Learning Pipelines
    Aditya Modi, Jivat Kaur, Maggie Makar, Pavan Mallapragada, Amit Sharma, Emre Kiciman, Adith Swaminathan
    Localized Learning Workshop, ICML '23.

  3. On the Statistical Efficiency of Reward-free Exploration in Non-linear Reinforcement Learning
    Jinglin Chen*, Aditya Modi*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
    Conference on Neural Information Processing Systems 2022 (NeurIPS '22 ).
    * Equal contribution.

  4. Multi-task Learning of Linear Control Systems under Instability
    Aditya Modi, Ziping Xu, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari
    ICML Workshop on Complex Feedback in Online Learning 2022 (ICML '22). Coming soon.

  5. Joint Learning-Based Stabilization of Multiple Unknown Linear Systems
    Mohamad Kazem Shirani Faradonbeh, Aditya Modi
    IFAC Workshop on Adaptive Learning and Control Systems 2022 (ALCoS '22 ).

  6. Joint Learning of Linear Time-Invariant Dynamical Systems
    Aditya Modi, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, George Michailidis

  7. Model-free Representation Learning and Exploration in Low-rank MDPs
    Aditya Modi*, Jinglin Chen*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
    Journal of Machine Learning Research (JMLR). *Equal contribution.

  8. Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies
    Shengpu Tang, Aditya Modi, Michael Sjoding, Jenna Wiens
    International Conference on Machine Learning 2020 (ICML '20).

  9. No-regret Exploration in Contextual Reinforcement Learning
    Aditya Modi and Ambuj Tewari
    Conference on Uncertainty in Artificial Intelligence 2020 (UAI '20).
    Abridged version accepted to ICML 2019 workshop on RL for Real Life and RLDM 2019.

  10. Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
    Aditya Modi, Nan Jiang, Ambuj Tewari, Satinder Singh
    International Conference on Artificial Intelligence and Statistics 2020 (AIStats '20).

  11. Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
    Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
    AAAI Conference on Artificial Intelligence 2020 (AAAI '20).
    Invited poster, ICML 2019 Workshop on RL for Real Life.

  12. Markov Decision Processes with Continuous Side Information
    Aditya Modi, Nan Jiang, Satinder Singh, Ambuj Tewari
    Algorithmic Learning Theory 2018 (ALT '18).


  • Microsoft Research, Redmond
    Research Intern, Adaptive Systems and Interaction group.
    July-October 2018.

  • Microsoft Research, Bangalore
    Research Intern, Applied Sciences group.
    May-July 2015.

Professional Service/Activity

  • Program Committee/Reviewing AAAI 2019, AISTATS 2019-23, ICML 2019-24 (Top reviewer '20), NeurIPS 2019-23 (Top reviewer '19,'20,'22), ICLR 2022-24, ICLR Blog Track reviewer 2023, UAI 2022-23, JMLR '24, IEEE Transactions on Information Theory (2022-23).
    European Workshop on Reinforcement Learning (EWRL) 2022-23, RL Conference '24, Conference on Lifelong Learning Agents (CoLLAs) 2022-24, Deep RL workshops (NeurIPS 2020-22), Workshop on RL Theory (ICML '21), Workshop on Theoretical Foundations of RL (ICML '20), NeurIPS workshop on RL for Real Life (2022).
  • Organization: Statistical Machine Learning Reading Group (EECS,2017-18).


  • Introduction to Machine Learning - Winter 2017
    Prof. Jenna Wiens, University of Michigan