Aditya Modi

I'm an Applied Scientist at Microsoft AI. At Microsoft, I work on applied research problems in machine learning aimed at improving their advertising products. My research interests lie in general sequential decision making problems with a focus on the fundamental understanding and applications of interactive learning (reinforcement learning and bandits) and causal inference.

Before joining Microsoft, I finished my PhD at the Computer Science and Engineering department in University of Michigan, Ann Arbor. During my PhD, I worked on theoretical aspects of interactive learning and sequential decision making frameworks like reinforcement learning, bandits and (general) online learning. I was fortunate to be advised by Satinder Singh and Ambuj Tewari. My thesis developed provable sample efficient methods for reinforcement learning under various structural assumptions.

Before my stint in RL theory at UMich, I obtained my undergraduate degree in Computer Science from Indian Institute of Technology, Kanpur in May 2016.

Recent News

• Our work 'Model-free Representation Learning and Exploration' has been accepted to JMLR with minor revisions.
• The paper on 'Joint Learning of Linear Time-Invariant Dynamical Systems' has been (provisionally) accepted to Automatica.
• Our work on reward-free exploration in RL accepted to NeurIPS '22.
• (Nov '21') Joined Microsoft as an Applied Scientist in the bay area.
• Defended my PhD thesis in November 2021. A special thanks to my advisors for supporting me through this journey!
• (Fall '20) Visited the Theory of RL program at Simons Institute for the Theory of Computing as visiting graduate student.
• Spent summer 2018 with the Adaptive Systems and Interaction group at Microsoft Research Redmond, working with Debadeepta Dey.

Publications/Preprints

1. On the Statistical Efficiency of Reward-free Exploration in Non-linear Reinforcement Learning
   Jinglin Chen*, Aditya Modi*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
   Conference on Neural Information Processing Systems 2022 (NeurIPS '22 ).
   * Equal contribution.

2. Multi-task Learning of Linear Control Systems under Instability
   Aditya Modi, Ziping Xu, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari
   ICML Workshop on Complex Feedback in Online Learning 2022 (ICML '22). Coming soon.

3. Joint Learning-Based Stabilization of Multiple Unknown Linear Systems
   Mohamad Kazem Shirani Faradonbeh, Aditya Modi
   IFAC Workshop on Adaptive Learning and Control Systems 2022 (ALCoS '22 ).

4. Joint Learning of Linear Time-Invariant Dynamical Systems
   Aditya Modi, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, George Michailidis
   Accept provisionally as Regular Paper in Automatica.

5. Model-free Representation Learning and Exploration in Low-rank MDPs
   Aditya Modi*, Jinglin Chen*, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
   Accepted to Journal of Machine Learning Research (JMLR) (under minor revisions). *Equal contribution.

6. Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies
   Shengpu Tang, Aditya Modi, Michael Sjoding, Jenna Wiens
   International Conference on Machine Learning 2020 (ICML '20).

7. No-regret Exploration in Contextual Reinforcement Learning
   Aditya Modi and Ambuj Tewari
   Conference on Uncertainty in Artificial Intelligence 2020 (UAI '20).
   Abridged version accepted to ICML 2019 workshop on RL for Real Life and RLDM 2019.

8. Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
   Aditya Modi, Nan Jiang, Ambuj Tewari, Satinder Singh
   International Conference on Artificial Intelligence and Statistics 2020 (AIStats '20).

9. Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations
   Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz
   AAAI Conference on Artificial Intelligence 2020 (AAAI '20).
   Invited poster, ICML 2019 Workshop on RL for Real Life.

10. Markov Decision Processes with Continuous Side Information
    Aditya Modi, Nan Jiang, Satinder Singh, Ambuj Tewari
    Algorithmic Learning Theory 2018 (ALT '18).

Thesis

1. Provably Efficient Reinforcement Learning Under Linear Model Structures: From Tabular to Feature Based Exploration
   Advisors: Satinder Singh and Ambuj Tewari.

Experience

• Microsoft Research, Redmond
  Research Intern, Adaptive Systems and Interaction group.
  July-October 2018.

• Microsoft Research, Bangalore
  Research Intern, Applied Sciences group.
  May-July 2015.

Professional Service/Activity

• Program Committee/Reviewing AAAI 2019, AISTATS 2019-23, ICML 2019-23 (Top reviewer '20), NeurIPS 2019-23 (Top reviewer '19,'20,'22), ICLR 2022-23, ICLR Blog Track reviewer 2023, UAI 2022-23, IEEE Transactions on Information Theory (2022).
  European Workshop on Reinforcement Learning (EWRL) 2022-23, Conference on Lifelong Learning Agents (CoLLAs) 2022-23, Deep RL workshops (NeurIPS 2020-22), Workshop on RL Theory (ICML '21), Workshop on Theoretical Foundations of RL (ICML '20), NeurIPS workshop on RL for Real Life (2022).
• Organization: Statistical Machine Learning Reading Group (EECS,2017-18).

Teaching

• Introduction to Machine Learning - Winter 2017
  Prof. Jenna Wiens, University of Michigan