Skip Navigation
Search
Shantenu Jha
Chair, Center for Data Driven Discovery
Brookhaven National Laboratory

Shantenu Jha is the Chair of the Center for Data Driven Discovery (C3D) at Brookhaven National Laboratory and Associate Professor of Computer Engineering at Rutgers University. His research interests are at the intersection of high-performance distributed computing and computational & data science. Shantenu leads the the RADICAL-Cybertools project which are a suite of middleware building blocks used to support large-scale science and engineering applications. He was appointed a Rutgers Chancellor's Scholar (2015) and was the recipient of the inaugural Chancellor's Excellence in Research (2016) for his cyberinfrastructure contributions to computational science. He is a recipient of the NSF CAREER Award (2013), several prizes at SC'xy and ISC’xy as well as the winner of SCALE 2018. More details can be found at http://radical.rutgers.edu/shantenu

Contact Information: shantenu@bnl.gov

Abstract

The Power of Many: The Next Frontier

Current computing trends make the ensemble computational model highly relevant; it has the ability to overcome limitations of single task applications, and to achieve significant performance gains on large-scale parallel machines. Not surprisingly, the concept of running ensembles on large-scale HPC systems is thus gaining in importance. We discuss some of the challenges in executing ensembles at scale. We discuss the abstractions (pilot-systems), software systems (RADICAL-Cybertools) and execution models that we have developed to address many of these challenges. We will discuss how RADICAL-Cybertools, along with advances in statistical and adaptive algorithms have enabled ensemble-based applications to overcome limitations of traditional single task applications. In spite of order(s) of magnitude efficiency gains, much greater improvements are still needed. We will close with a brief mention of novel application architectures that hold promise to provide the needed improvements.