Laboratory for Parallel Numerical Algorithms

Research Topics

Tensor Algebra: Multidimensional data (ubiquitous in scientific computing and machine learning) can be effectively treated via tensor abstractions. Dense and sparse tensor algebra, tensor decompositions, and tensor networks pose challenges in design of efficiency, software abstractions, and numerical methods.

Matrix Computations: Numerical linear algebra underlies most computational approaches in the data sciences. Fast matrix algorithms provide solutions for nonlinear optimization, low-rank approximation, and eigenvalue problems.

Quantum Systems: Tensor representations provide the most natural way to computationally model entanglement (correlation between electrons). We investigate numerical parallel algorithms for tensor computations arising in quantum chemistry (e.g. high-accuracy electronic structure calculations) and quantum computation (e.g. quantum circuit simulation).

Communication-Avoiding Algorithms: Performance and scalability of algorithms and libraries is constrained by data movement in the memory hierarchy and network. We aim to design parallel algorithms that minimize the amount of communication and number of messages. Our group designs such algorithms for problems from a variety of domains, including graph problems, parallel sorting, bioinformatics, and numerical tensor computations.

High Performance Numerical Libraries: Parallel numerical libraries are the glue between fast algorithms and real-world applications. We pursue application-driven research on algorithms by way of developing general and scalable library routines.


(February 2020) Find us at SIAM PP 2020, where there will be three talks and three poster presentations by six participants of our group.

(January 2020) Annika Dugad and Tianyi Hao will be giving poster presentations at QIP 2020, stop by and learn about our work on approximate techniques quantum circuit simulation of quantum algorithms for ground states and dynamics.

(October 2019) We have released new papers on tensor completion and tensor decomposition, as well as a survey on fast convolution algorithms.

(October 2019) Congratulations to Linjian Ma and Yuchen Pang on being awarded Computer Science Gene Golub Fellowships.

(May 2019) See Edward Hutter's presentation on Communication-avoiding Cholesky-QR2, accepted as regular conference paper at IPDPS 2019 in Rio de Janeiro, Brazil.

(May 2019) Welcome (back) to Yuchen Pang and Linjian Ma, who plan to start their PhD work this fall at UIUC as part of LPNA!

(May 2019) LPNA undergraduate researchers are becoming graduates! Zecheng Zhang and Qile Zhi will be pursuing MS programs at Stanford, Naijing Zhang will pursue an MSE at UC Berkeley, Xiaoxiao Wu will pursue an MSCF at CMU, Siyuan Zhang will pursue MS at UIUC, David Zhang will pursue an MS at Cornell, and Hongru Yang will pursue a PhD at UT Austin!

(May 2019) Congratulations to Caleb Ju for winning the Franz Hohn and J.P. Nash Scholarship.

(Dec 2018) Edward Hutter is one of two UIUC PhD students in the DOE CSGF class of 2018 (CS @ Illinois news article).

(May 2018) Congratulations to Pavle Simonovic (completed BS thesis) and Peter Tatkowski (joining ETH Zurich MS program)!

(May 2018) Congratulations to Qile Zhi for winning the Franz Hohn and J.P. Nash Scholarship.

(June 2017) Congratulations to Tobias Wicky for finishing his MS thesis and to Edward Hutter for finishing his BS thesis!


Cyclops Tensor Framework
a distributed-memory library for graph, matrix, and tensor computations

a suite of parallel algorithms for matrix factorization and eigendecomposition



We are always looking for new collaborators and participants. If you are a UIUC student interested in doing research in the area, email Edgar Solomonik (


We are part of the scientific computing group at the University of Illinois at Urbana-Champaign. Below LPNA group photo is from a May 2019 get together to celebrate several graduating students.


Edgar Solomonik
Assistant Professor

Graduate Students

Edward Hutter
PhD Student in Computer Science
Raul Platero
PhD Student in Computer Science
Samah Karim
PhD Student in Computer Science (co-advised with William Gropp)
Linjian Ma
PhD Student in Computer Science
Yuchen Pang
PhD Student in Computer Science
Navjot Singh
MS Student in Applied Mathematics

Visiting Researchers

Annika Dugad (BS Physics)

Undergraduate Students

Wentao Yang (CS)
Caleb Ju (CS)
Zhaoyu Wu (CS)
Yiqing Zhou (Physics & CS)
Tianyi Hao (CS)
Yifan Zhang (Math, Eng Phys, and Stats)
Tim Baer (CS)
Youcef Hadjarab (CS)
Dipro Ray (CS)

Master Theses

Tobias Wicky (2017): A communication-avoiding algorithm for solving linear systems of equations with selective inversion

Bachelor Theses

Zecheng Zhang (2019): Python interface to Cyclops and graph neural networks
Pavle Simonovic (2018): Shared-memory parallel algorithms for sparse matrix operations
Edward Hutter (2017): QR factorization over tunable processor grids

Past Participants / Independent Study Projects

Hung Woei Neoh (Math & CS) (2018-2019) Fast bilinear algorithms for symmetric matrix multiplication
Hongru Yang (CS & Stats) (2018-2019) Optimization algorithms for CP tensor decomposition
Yunxin (David) Zhang (CS) (2018-2019) Algebraic graph algorithms for connectivity
Xiaoxiao Wu (2018-2019) Alternating least squares for tensor completion
Siyuan Zhang (2018-2019) Stochastic gradient descent for tensor completion
Naijing Zhang (2017-2019): Multicontraction scheduling and coordinate descent methods for tensor completion
Fan Huang (2018): Well-conditioned tensors
Eric Song (2018): High-level interface abstractions for parallel tensor decompositions
Ruiqian Yao (2018): Performance modeling, prediction, and training for parallel tensor computation kernels
Eduardo Yap (2018): Generalized tensor contractions by batched matrix multiplication
Peter Tatkowski (2018): Algebraic tensor representaiton of finite element methods; Tensor completion with Cyclops
Qile Zhi (2017): Stability of triangular matrix inversion
Thomas Warther (2017): Parallel neural networks with Cyclops

Video Lectures

web-course Numerical analysis Spring 2018, Fall 2018; CS 450

web-course Parallel numerical algorithms Fall 2017; CS 554

video June 2017; LPNA Lecture; Basics of tensors (Edgar)

video June 2017; LPNA Lecture; Basics of communication complexity (Edgar)

web-course Communication cost analysis of algorithms Fall 2016; CS 598-ES


slides June 2019; Blue Waters Symposium 2019; Redmond, Oregon; Communication-optimal QR factorizations: performance and scalability on varying architectures (Edward)

slides May 2019; IPDPS19; Rio de Janeiro, Brazil; Communication-avoiding Cholesky-QR2 for rectangular matrices (Edward)

February 2019; SIAM CSE19; Spokane, WA; Communication-avoiding Cholesky-QR2 for rectangular matrices (Edward)

March 2018; SIAM PP18; Tokyo, Japan; Communication-avoiding Cholesky-QR2 for rectangular matrices (Edward)

September 2017; BLIS Retreat 2017; Austin TX, USA; Parallel 3D Cholesky-QR2 for rectangular matrices (Edward)

slides July 2017; LPNA Group presentation; Urbana IL, USA; 2D Finite Element Methods and Gather/Scatter (Peter)

slides July 2017; LPNA Group presentation; Urbana IL, USA; Least Squares Updating for Kronecker Products (Raul)

July 2017; LPNA Group presentation; Urbana IL, USA; A new tunable QR factorization algorithm (Edward)

slides May 2017; Illinois Data Science Fundamentals Summit; Urbana IL, USA; Scalable numerical linear algebra for data science (Edgar)

slides May 2017; MolSSI Workshop on Core Software Blocks in Quantum Chemistry: Tensors and Integrals; Monterey Bay CA, USA; An overview of Cyclops Tensor Framework (Edgar)


report Caleb Ju and Edgar Solomonik. Derivation and analysis of fast bilinear algorithms for convolution arXiv:1910.13367 [math.NA], October 2019.
report Navjot Singh, Linjian Ma, Hongru Yang, and Edgar Solomonik. Comparison of accuracy and scalability of Gauss-Newton and alternating least squares for CP decomposition arXiv:1910.12331 [math.NA], October 2019.
report Zecheng Zhang, Xiaoxiao Wu, Naijing Zhang, Siyuan Zhang, and Edgar Solomonik. Enabling distributed-memory tensor completion in Python using new sparse tensor kernels arXiv:1910.02371 [cs.DC], October 2019.
article Edward Hutter and Edgar Solomonik Communication-avoiding Cholesky-QR2 for rectangular matrices IEEE International Parallel and Distributed Processing Symposium (IPDPS), Rio de Jianero, Brazil, May, 2019, to appear.
report Linjian Ma and Edgar Solomonik Accelerating Alternating Least Squares for Tensor Decomposition by Pairwise Perturbation arXiv:1811.10573 [math.NA], November 2018.
article Tobias Wicky, Edgar Solomonik, and Torsten Hoefler Communication-avoiding parallel algorithms for solving triangular systems of linear equations IEEE International Parallel and Distributed Processing Symposium (IPDPS), Orlando, FL, June 2017, pp. 678-687. report