Lawrence Saul

I am a Senior Research Scientist and Group Leader in the Center for Computational Mathematics (CCM) at the Flatiron Institute. I am part of a large and growing research effort in the area of machine learning, both within my own center (ML@CCM) and across all of Flatiron (ML@FI). I work broadly across the areas of high dimensional data analysis, latent variable modeling, variational inference, and representation learning.

Before joining Flatiron, I was a tenured faculty member at UC San Diego and UPenn and a member of the technical staff at AT&T Labs. I also served previously as Editor-in-Chief of the Journal of Machine Learning Research and as Program Chair of the Conference on Neural Information Processing Systems. Before my work in machine learning, I earned a bachelor’s degree in Physics from Harvard and a doctorate in Physics from M.I.T.

News

Recent projects

High dimensional data analysis

Sparse matrices are not generally low rank, and low-rank matrices are not generally sparse. But can one find more subtle connections between these different properties of matrices by looking beyond the canonical decompositions of linear algebra? This paper in SIMODS describes a nonlinear matrix decomposition that can be used to express a sparse nonnegative matrix in terms of a real-valued matrix of significantly lower rank. Arguably the most popular matrix decompositions in machine learning are those—such as principal component analysis, or nonnegative matrix factorization—that have a simple geometric interpretation. This paper in TMLR gives such an interpretation for these nonlinear decompositions, one that arises naturally in the problem of manifold learning.

Graph matching

The problem of graph matching is to find a permutation that maps the nodes of one graph onto the nodes of another. Such a mapping can be discovered by a combinatorial optimization that searches over the space of permutation matrices, by a continuous relaxation that searches over the space of doubly stochastic matrices, or by an interleaving of both these approaches. In March 2025, it was announced that Dan Lee and I won the Flywire VNC Matching Challenge hosted by the Princeton Neuroscience Institute; the name of our team was Old School. The challenge asked participants to construct a method for aligning the connectomes of a male and female fruit fly. The connectomes were represented as large graphs, and by matching these graphs, we determined a correspondence between neurons in the male and female ventral nerve cords. This paper in TMLR describes our method.

Variational inference

Given an intractable distribution p, the problem of variational inference (VI) is to find the best approximation q from some more tractable family. Typically, q is found by minimizing the (reverse) Kullback-Leibler divergence, but in recent papers at ICML and NeurIPS, we have shown how to approximate p by minimizing certain score-based divergences. The first of these papers derives the Batch and Match algorithm for VI with multivariate Gaussian approximations. The second describes an eigenvalue problem (EigenVI) for approximations based on orthogonal function expansions. Finally, the third uses a Feynman identity from quantum field theory to approximate the target density by a product of experts. In related work, this paper in JMLR analyzes the inherent trade-offs in VI with a factorized approximation, and this paper at AISTATS provides some positive guarantees for VI with location-scale families.

Learning with symmetries: weight-balancing flows

Gradient descent is based on discretizing a continuous-time flow, typically one that descends in a regularized loss function. But what if for all but the simplest types of regularizers we have been discretizing the wrong flow? This paper in TMLR makes two contributions to our understanding of deep learning in feedforward networks with homogeneous activations functions (e.g., ReLU) and rescaling symmetries. The first is to describe a simple procedure for balancing the weights in these networks without changing the end-to-end functions that they compute. The second is to derive a continuous-time dynamics that preserves this balance while descending in the network's loss function. These dynamics reduce to an ordinary gradient flow for l2-norm regularization, but not otherwise. Put another way, this analysis suggests a canonical pairing of alternative flows and regularizers.

Recent papers

Google Scholar

Contact