Zhiyuan Li (李志远)

Meta MSL
TTIC (on leave)

I am a tenure-track assistant professor at Toyota Technological Institute at Chicago (TTIC). I’m currently on leave at Meta Superintelligence Labs (MSL).

I am also an affiliated faculty member in Computer Science at the University of Chicago. I was also a visiting faculty member at Google Research until April 2026. Before joining TTIC, I was a postdoctoral fellow in Computer Science Department at Stanford University, working with Tengyu Ma. I received my PhD from the Computer Science Department at Princeton University in 2022, where I was advised by Sanjeev Arora. I did my undergraduate study at Yao Class, Tsinghua University.

I am broadly interested in machine learning theory, including optimization in deep learning, reasoning capabilities of Large Language Models (LLMs), modern paradigm of generalization in machine learning (overparameterization, out-of-domain generalization) and its connection to the implicit bias of optimization algorithms.

News

Oct 02, 2026	I’m currently on leave from TTIC at Meta Superintelligence Labs (MSL).
Jul 08, 2026	New preprint: The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought. In the realizable PAC setting, exact-trace learning achieves the standard multiclass sample-complexity rate of the local next-token class, independent of rollout length.
Jun 05, 2026	Gave a talk on recursive models at the Simons Institute Multi-Program AI Reunion.
May 19, 2026	Serving as Area Chair for NeurIPS 2026.
May 13, 2026	Received a $600K NSF CAREER Award for work on architecture-aware optimization theory for deep learning.

Selected and Recent publications

The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought

Zhiyuan Li

arXiv preprint arXiv:2607.07423, arXiv 2026

Abs arXiv Bib

In the realizable PAC setting, full autoregressive Chain-of-Thought traces can be learned under exact-trace loss at the standard multiclass sample-complexity rate of the local next-token class, without dependence on rollout length. The dependence on the local class dimension is worst-case optimal.
@article{li2026optimal, author = {Li, Zhiyuan}, title = {The Optimal Sample Complexity of Learning Autoregressive Chain-of-Thought}, journal = {arXiv preprint arXiv:2607.07423}, year = {2026}, }

Recursive Models for Long-Horizon Reasoning

Chenxiao Yang, Nathan Srebro, and Zhiyuan Li

In Proceedings of the 43rd International Conference on Machine Learning, ICML 2026

arXiv Bib

@inproceedings{yang2026recursive,
  author = {Yang, Chenxiao and Srebro, Nathan and Li, Zhiyuan},
  title = {Recursive Models for Long-Horizon Reasoning},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year = {2026},
}

On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond

Chenxiao Yang, Cai Zhou, David Wipf, and Zhiyuan Li

In The Fourteenth International Conference on Learning Representations, ICLR 2026

arXiv Bib

@inproceedings{yang2026powerful,
  author = {Yang, Chenxiao and Zhou, Cai and Wipf, David and Li, Zhiyuan},
  title = {On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year = {2026},
}

A Tale of Two Geometries: Adaptive Optimizers and Non-Euclidean Descent

Shuo Xie, Tianhao Wang, Beining Wu, and Zhiyuan Li

In The Fourteenth International Conference on Learning Representations, ICLR 2026

arXiv Bib

@inproceedings{xie2026tale,
  author = {Xie, Shuo and Wang, Tianhao and Wu, Beining and Li, Zhiyuan},
  title = {A Tale of Two Geometries: Adaptive Optimizers and Non-Euclidean Descent},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year = {2026},
}

Structured Preconditioners in Adaptive Optimization: A Unified Analysis

Shuo Xie, Tianhao Wang, Sashank Reddi, Sanjiv Kumar, and Zhiyuan Li

In Proceedings of the 42nd International Conference on Machine Learning, ICML 2025

arXiv Bib

@inproceedings{xie2025structured,
  author = {Xie, Shuo and Wang, Tianhao and Reddi, Sashank and Kumar, Sanjiv and Li, Zhiyuan},
  title = {Structured Preconditioners in Adaptive Optimization: A Unified Analysis},
  booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
  year = {2025},
}

PENCIL: Long Thoughts with Short Memory

Chenxiao Yang, Nathan Srebro, David McAllester, and Zhiyuan Li

In Proceedings of the 42nd International Conference on Machine Learning, ICML 2025

arXiv Bib

@inproceedings{yang2025pencil,
  author = {Yang, Chenxiao and Srebro, Nathan and McAllester, David and Li, Zhiyuan},
  title = {PENCIL: Long Thoughts with Short Memory},
  booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
  year = {2025},
}

Non-Asymptotic Length Generalization

Thomas Chen, Tengyu Ma, and Zhiyuan Li

In Proceedings of the 42nd International Conference on Machine Learning, ICML 2025

arXiv Bib

@inproceedings{chen2025non,
  author = {Chen, Thomas and Ma, Tengyu and Li, Zhiyuan},
  title = {Non-Asymptotic Length Generalization},
  booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
  year = {2025},
}

Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity

Shuo Xie, Mohamad Amin Mohamadi, and Zhiyuan Li

In The Thirteenth International Conference on Learning Representations, ICLR 2025

Spotlight arXiv Bib

2025 ICLR Spotlight

@inproceedings{xie2024adam,
  author = {Xie, Shuo and Mohamadi, Mohamad Amin and Li, Zhiyuan},
  title = {Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity},
  booktitle = {The Thirteenth International Conference on Learning Representations},
  year = {2025},
}

Chain of Thought Empowers Transformers to Solve Inherently Serial Problems

Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma

In The Twelfth International Conference on Learning Representations, ICLR 2024

arXiv Bib

@inproceedings{li2024chain,
  author = {Li, Zhiyuan and Liu, Hong and Zhou, Denny and Ma, Tengyu},
  title = {Chain of Thought Empowers Transformers to Solve Inherently Serial Problems},
  booktitle = {The Twelfth International Conference on Learning Representations},
  year = {2024},
}

How Does Sharpness-Aware Minimization Minimize Sharpness?

Kaiyue Wen, Tengyu Ma, and Zhiyuan Li

In The Eleventh International Conference on Learning Representations, ICLR 2023

arXiv Bib

@inproceedings{wen2023does,
  author = {Wen, Kaiyue and Ma, Tengyu and Li, Zhiyuan},
  title = {How Does Sharpness-Aware Minimization Minimize Sharpness?},
  booktitle = {The Eleventh International Conference on Learning Representations},
  year = {2023},
}

What Happens after SGD Reaches Zero Loss?--A Mathematical Framework

Zhiyuan Li, Tianhao Wang, and Sanjeev Arora

In The Tenth International Conference on Learning Representations, ICLR 2022

Spotlight arXiv Bib

2022 ICLR Spotlight

@inproceedings{li2021happens,
  author = {Li, Zhiyuan and Wang, Tianhao and Arora, Sanjeev},
  title = {What Happens after SGD Reaches Zero Loss?--A Mathematical Framework},
  booktitle = {The Tenth International Conference on Learning Representations},
  year = {2022},
}

Why Are Convolutional Nets More Sample-Efficient Than Fully-Connected Nets?

Zhiyuan Li, Yi Zhang, and Sanjeev Arora

In The Ninth International Conference on Learning Representations, ICLR 2021

Oral arXiv Bib

2021 ICLR Oral

@inproceedings{li2020convolutional,
  author = {Li, Zhiyuan and Zhang, Yi and Arora, Sanjeev},
  title = {Why Are Convolutional Nets More Sample-Efficient Than Fully-Connected Nets?},
  booktitle = {The Ninth International Conference on Learning Representations},
  year = {2021},
}