Aditya Modi

I'm an Applied Scientist at Microsoft AI. At Microsoft, I work on applied research problems in machine learning aimed at improving their advertising products. My research interests lie in general sequential decision making problems with a focus on the fundamental understanding and applications of interactive learning (reinforcement learning and bandits) and causal inference. My recent work has focused on deployable RL and the combination of RL methods with LLMs.

Before joining Microsoft, I finished my PhD at the Computer Science and Engineering department in University of Michigan, Ann Arbor. During my PhD, I worked on theoretical aspects of interactive learning and sequential decision making frameworks like reinforcement learning, bandits and (general) online learning. I was fortunate to be advised by Satinder Singh and Ambuj Tewari. My thesis developed provable sample efficient methods for reinforcement learning under various structural assumptions.

Before my stint in RL theory at UMich, I obtained my undergraduate degree in Computer Science from Indian Institute of Technology, Kanpur in May 2016.

Recent News

• [2024] Our paper 'How to Solve Contextual Goal-Oriented Problems with Offline Datasets?' has been accepted to NeurIPS '24. Congrats to lead authors Ying and Jingling!
• [2024] Our work 'Model-free Representation Learning and Exploration' has been published in JMLR.
• [2024] The paper on 'Joint Learning of Linear Time-Invariant Dynamical Systems' has been published in Automatica.

Publications/Preprints

1. How to Solve Contextual Goal-Oriented Problems with Offline Datasets?
   Ying Fan, Jingling Li, Adith Swaminathan, Aditya Modi, Ching-An Cheng
   Conference on Neural Information Processing Systems 2024 (NeurIPS '24)
   Preliminary version accepted to Generalization in Planning Workshop, NeurIPS '23 & Workshop on Goal-Conditioned RL, NeurIPS '23.

2. Towards Modular Machine Learning Pipelines
   Aditya Modi, Jivat Kaur, Maggie Makar, Pavan Mallapragada, Amit Sharma, Emre Kiciman, Adith Swaminathan
   Localized Learning Workshop, ICML '23.

3. On the Statistical Efficiency of Reward-free Exploration in Non-linear Reinforcement Learning
   Jinglin Chen*, Aditya Modi*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
   Conference on Neural Information Processing Systems 2022 (NeurIPS '22 ).
   * Equal contribution.

4. Multi-task Learning of Linear Control Systems under Instability
   Aditya Modi, Ziping Xu, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari
   ICML Workshop on Complex Feedback in Online Learning 2022 (ICML '22). Coming soon.

5. Joint Learning-Based Stabilization of Multiple Unknown Linear Systems
   Mohamad Kazem Shirani Faradonbeh, Aditya Modi
   IFAC Workshop on Adaptive Learning and Control Systems 2022 (ALCoS '22 ).

6. Joint Learning of Linear Time-Invariant Dynamical Systems
   Aditya Modi, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, George Michailidis
   Automatica.

7. Model-free Representation Learning and Exploration in Low-rank MDPs
   Aditya Modi*, Jinglin Chen*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
   Journal of Machine Learning Research (JMLR). *Equal contribution.

8. Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies
   Shengpu Tang, Aditya Modi, Michael Sjoding, Jenna Wiens
   International Conference on Machine Learning 2020 (ICML '20).

9. No-regret Exploration in Contextual Reinforcement Learning
   Aditya Modi and Ambuj Tewari
   Conference on Uncertainty in Artificial Intelligence 2020 (UAI '20).
   Abridged version accepted to ICML 2019 workshop on RL for Real Life and RLDM 2019.

10. Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
    Aditya Modi, Nan Jiang, Ambuj Tewari, Satinder Singh
    International Conference on Artificial Intelligence and Statistics 2020 (AIStats '20).

11. Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
    Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
    AAAI Conference on Artificial Intelligence 2020 (AAAI '20).
    Invited poster, ICML 2019 Workshop on RL for Real Life.

12. Markov Decision Processes with Continuous Side Information
    Aditya Modi, Nan Jiang, Satinder Singh, Ambuj Tewari
    Algorithmic Learning Theory 2018 (ALT '18).

Thesis

1. Provably Efficient Reinforcement Learning Under Linear Model Structures: From Tabular to Feature Based Exploration
   Advisors: Satinder Singh and Ambuj Tewari.

Experience

• Microsoft Research, Redmond
  Research Intern, Adaptive Systems and Interaction group.
  July-October 2018.

• Microsoft Research, Bangalore
  Research Intern, Applied Sciences group.
  May-July 2015.

Professional Service/Activity

• Program Committee/Reviewing AAAI 2019, AISTATS 2019-23, ICML 2019-24 (Top reviewer '20), NeurIPS 2019-23 (Top reviewer '19,'20,'22), ICLR 2022-24, ICLR Blog Track reviewer 2023, UAI 2022-23, JMLR '24, IEEE Transactions on Information Theory (2022-23).
  European Workshop on Reinforcement Learning (EWRL) 2022-23, RL Conference '24, Conference on Lifelong Learning Agents (CoLLAs) 2022-24, Deep RL workshops (NeurIPS 2020-22), Workshop on RL Theory (ICML '21), Workshop on Theoretical Foundations of RL (ICML '20), NeurIPS workshop on RL for Real Life (2022).
• Organization: Statistical Machine Learning Reading Group (EECS,2017-18).

Teaching

• Introduction to Machine Learning - Winter 2017
  Prof. Jenna Wiens, University of Michigan