Laboratory for Parallel Numerical Algorithms

Research Topics

Tensor Algebra: Multidimensional data (ubiquitous in scientific computing and machine learning) can be effectively treated via tensor abstractions. Tensors and their decompositions provide numerical tools for hypergraph processing, high-order methods, and neural networks.

Matrix Computations: Numerical linear algebra underlies most computational approaches in data sciences. Fast matrix algorithms provide solutions for nonlinear optimization, low-rank approximation, and eigenvalue problems.

Quantum Systems: Tensor representations provide the most natural way to computationally model entanglement (correlation between electrons). We investigate numerical parallel algorithms for tensor computations arising in quantum chemistry (e.g. high-accuracy electronic structure calculations) and quantum information (e.g. quantum circuit simulation).

Communication Complexity: Performance and scalability of algorithms and libraries is constrained by data movement in the memory hierarchy and network. We aim to design parallel algorithms that minimize the amount of communication and number of messages.

High Performance Numerical Libraries: Parallel numerical libraries are the glue between fast algorithms and real-world applications. We pursue application-driven research on algorithms by way of developing general and scalable library routines.

News

(Oct 2018) Edward Hutter is one of two UIUC PhD students in the DOE CSGF class of 2018 (CS @ Illinois news article).

(May 2018) Congratulations to Pavle Simonovic (completed BS thesis) and Peter Tatkowski (joining ETH Zurich MS program)!

(May 2018) Congratulations to Qile Zhi for winning the Franz Hohn and J.P. Nash Scholarship.

(March 2018) See Edward Hutter's presentation on Cholesky-QR2 at SIAM PP 2018 in Tokyo (slides).

(July 2017) Congratulations to Raul Platero for being a winner of the Outstanding Oral Presentation prize at the 2017 Illinois Summer Research Symposium (ISRS)!

(June 2017) Congratulations to Tobias Wicky for finishing his MS thesis and to Edward Hutter for finishing his BS thesis!

(June 2017) Group webpage is up, welcome!

Software

Cyclops Tensor Framework
a distributed-memory library for graph, matrix, and tensor computations

CANDMC
a suite of parallel algorithms for matrix factorization and eigendecomposition

Sponsors

Openings

We are always looking for new collaborators and participants. If you are a UIUC student interested in doing research in the area, email Edgar Solomonik (solomon2@illinois.edu).

Members

We are part of the scientific computing group at the University of Illinois at Urbana-Champaign.

Faculty

Edgar Solomonik
Assistant Professor

Graduate Students

Edward Hutter
PhD Student
Raul Platero
PhD Student
Samah Karim
PhD Student (co-advised with William Gropp)

Undergraduate Students

Zecheng Zhang (CS)
Naijing Zhang (CS)
Hung Woei Neoh (Math & CS)
Wentao Yang (CS)
Xiaoxiao Wu (CS)
Siyuan Zhang (CS)
Caleb Ju (CS)

Master Theses

Tobias Wicky (2017): A communication-avoiding algorithm for solving linear systems of equations with selective inversion

Bachelor Theses

Pavle Simonovic (2018): Shared-memory parallel algorithms for sparse matrix operations
Edward Hutter (2017): QR factorization over tunable processor grids

Past Participants / Independent Study Projects

Eric Song (2018): High-level interface abstractions for parallel tensor decompositions
Ruiqian Yao (2018): Performance modeling, prediction, and training for parallel tensor computation kernels
Linjian Ma (2018): Pairwise perturbation and multigrid in alternating least squares for CP and Tucker decomposition
Eduardo Yap (2018): Generalized tensor contractions by batched matrix multiplication
Peter Tatkowski (2018): Algebraic tensor representaiton of finite element methods; Tensor completion with Cyclops
Qile Zhi (2017): Stability of triangular matrix inversion
Thomas Warther (2017): Parallel neural networks with Cyclops

Video Lectures

web-course Numerical analysis Spring 2018, Fall 2018; CS 450

web-course Parallel numerical algorithms Fall 2017; CS 554

video June 2017; LPNA Lecture; Basics of tensors (Edgar)

video June 2017; LPNA Lecture; Basics of communication complexity (Edgar)

web-course Communication cost analysis of algorithms Fall 2016; CS 598-ES

Talks

slides March 2018; SIAM PP18; Tokyo, Japan; Communication-avoiding Cholesky-QR2 for rectangular matrices (Edward)

September 2017; BLIS Retreat 2017; Austin TX, USA; Parallel 3D Cholesky-QR2 for rectangular matrices (Edward)

slides July 2017; LPNA Group presentation; Urbana IL, USA; 2D Finite Element Methods and Gather/Scatter (Peter)

slides July 2017; LPNA Group presentation; Urbana IL, USA; Least Squares Updating for Kronecker Products (Raul)

slides July 2017; LPNA Group presentation; Urbana IL, USA; A new tunable QR factorization algorithm (Edward)

slides May 2017; Illinois Data Science Fundamentals Summit; Urbana IL, USA; Scalable numerical linear algebra for data science (Edgar)

slides May 2017; MolSSI Workshop on Core Software Blocks in Quantum Chemistry: Tensors and Integrals; Monterey Bay CA, USA; An overview of Cyclops Tensor Framework (Edgar)

Publications

report Edward Hutter and Edgar Solomonik Communication-avoiding Cholesky-QR2 for rectangular matrices arXiv:1710.08471 [cs.DC], October 2017.
article Tobias Wicky, Edgar Solomonik, and Torsten Hoefler Communication-avoiding parallel algorithms for solving triangular systems of linear equations IEEE International Parallel and Distributed Processing Symposium (IPDPS), Orlando, FL, June 2017, pp. 678-687. report