This talk will review NERSC’s efforts in scaling Deep Learning on the largest CPU and GPU-based HPC systems in the DOE complex. Motivated by challenging scientific problems in high-energy physics, cosmology and climate science, we have developed 2D and 3D convolutional architectures to solve a range of pattern classification, regression and segmentation problems. These projects have resulted in a number of first-time results: scaling Caffe to 9600 Cori/KNL nodes (SC’17) obtaining 15PF performance; scaling TensorFlow to 8192 Cori/KNL nodes obtaining 3.5PF performance, and finally, scaling TensorFlow to 4560 Summit/Volta nodes obtaining 1 ExaOp performance. The talk will review lessons learnt from these projects, and outline future challenges in Deep Learning for Science.

Prabhat leads the Data and Analytics Services Group at Berkeley Lab’s supercomputing center NERSC. In this role, Prabhat is responsible for the Data software stack and services for NERSC’s 7000+ users. Prabhat is a pioneer in the application of Deep Learning, Machine Learning and statistical methods for scientific applications. He has a broad set of interests spanning topics in High Performance Computing, Data Management and Visualization. Prabhat has degrees in computer science from IIT Delhi and Brown University, and is currently pursuing a PhD in Earth and Planetary Sciences from UC Berkeley.