Nikoli Dryden

Most-cited papers

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
T. Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden +1 more
2021Journal of machine learning research1,039 citations
Semantic Scholar
Deep learning for post-processing ensemble weather forecasts
Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden +3 more
2020Philosophical Transactions of the Royal Society A217 citations Open access
Semantic Scholar
Communication Quantization for Data-Parallel Training of Deep Neural Networks
Nikoli Dryden, Tim Moon, S. Jacobs, B. V. Van Essen
2016Workshop on Machine Learning in High Performance Computing Environments215 citations Open access
Semantic Scholar
Data Movement Is All You Need: A Case Study on Optimizing Transformers
A. Ivanov, Nikoli Dryden, Tal Ben-Nun, Shi-Gang Li +1 more
2020Conference on Machine Learning and Systems212 citations
Semantic Scholar
Gluon: a communication-optimizing substrate for distributed heterogeneous graph analytics
Roshan Dathathri, G. Gill, Loc Hoang, Hoang-Vu Dang +4 more
2018ACM-SIGPLAN Symposium on Programming Language Design and Implementation141 citations Open access
Semantic Scholar
Clairvoyant Prefetching for Distributed Machine Learning I/O
Nikoli Dryden, R. Böhringer, Tal Ben-Nun, T. Hoefler
2021International Conference for High Performance Computing, Networking, Storage and Analysis81 citations Open access
Semantic Scholar
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
Nikoli Dryden, N. Maruyama, Tom Benson, Tim Moon +2 more
2019IEEE International Parallel and Distributed Processing Symposium53 citations Open access
Semantic Scholar
Channel and Filter Parallelism for Large-Scale CNN Training
Nikoli Dryden, N. Maruyama, Tim Moon, Tom Benson +2 more
2019International Conference for High Performance Computing, Networking, Storage and Analysis52 citations
Semantic Scholar
Motif Prediction with Graph Neural Networks
Maciej Besta, Raphael Grob, Cesare Miglioli, Nico Bernold +7 more
2021Knowledge Discovery and Data Mining48 citations Open access
Semantic Scholar
Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems
Nikoli Dryden, N. Maruyama, Tim Moon, Tom Benson +3 more
2018Workshop on Machine Learning in High Performance Computing Environments44 citations Open access
Semantic Scholar
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs With Hybrid Parallelism
Yosuke Oyama, N. Maruyama, Nikoli Dryden, Erin McCarthy +5 more
2020IEEE Transactions on Parallel and Distributed Systems43 citations Open access
Semantic Scholar
ENS-10: A Dataset For Post-Processing Ensemble Weather Forecast
Saleh Ashkboos, La-mei Huang, Nikoli Dryden, Tal Ben-Nun +4 more
2022Neural Information Processing Systems25 citations Open access
Semantic Scholar
A Lightweight Communication Runtime for Distributed Graph Analytics
Hoang-Vu Dang, Roshan Dathathri, G. Gill, Alex Brooks +5 more
2018IEEE International Parallel and Distributed Processing Symposium23 citations
Semantic Scholar
Spatial Mixture-of-Experts
Nikoli Dryden, T. Hoefler
2022Neural Information Processing Systems22 citations Open access
Semantic Scholar
A data-centric optimization framework for machine learning
Oliver Rausch, Tal Ben-Nun, Nikoli Dryden, Andrei Ivanov +2 more
2021International Conference on Supercomputing19 citations Open access
Semantic Scholar
Neural Parameter Allocation Search
Bryan A. Plummer, Nikoli Dryden, Julius Frost, T. Hoefler +1 more
2020International Conference on Learning Representations19 citations
Semantic Scholar
Learning Combinatorial Node Labeling Algorithms
Lukas Gianinazzi, M. Fries, Nikoli Dryden, Tal Ben-Nun +2 more
2021arXiv.org17 citations
Semantic Scholar
Neural Graph Databases
Maciej Besta, Patrick Iff, Florian Scheidl, Kazuki Osawa +4 more
2022LOG IN16 citations Open access
Semantic Scholar
Co-design Center for Exascale Machine Learning Technologies (ExaLearn)
Francis J. Alexander, J. Ang, Jenna A. Bilbrey, J. Balewski +43 more
2021The international journal of high performance computing applications16 citations Open access
Semantic Scholar
Breaking (Global) Barriers in Parallel Stochastic Optimization With Wait-Avoiding Group Averaging
Shi-Gang Li, Tal Ben-Nun, Giorgi Nadiradze, Salvatore Di Girolamo +3 more
2020IEEE Transactions on Parallel and Distributed Systems16 citations Open access
Semantic Scholar
Predicting Weather Uncertainty with Deep Convnets
Peter Grönquist, Tal Ben-Nun, Nikoli Dryden, P. Dueben +3 more
2019arXiv.org13 citations
Semantic Scholar
Neural Network Based Silent Error Detector
Chen Wang, Nikoli Dryden, F. Cappello, M. Snir
2018IEEE International Conference on Cluster Computing13 citations
Semantic Scholar
Towards Scalable Parallel Training of Deep Neural Networks
S. Jacobs, Nikoli Dryden, R. Pearce, B. V. Van Essen
2017MLHPC@SC12 citations Open access
Semantic Scholar
Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
Susav Shrestha, Bradley W. Settlemyer, Nikoli Dryden, A. L. Narasimha Reddy
2025Neural Information Processing Systems9 citations
Semantic Scholar
Cached Operator Reordering: A Unified View for Fast GNN Training
Julia Bazinska, Andrei Ivanov, Tal Ben-Nun, Nikoli Dryden +3 more
2023arXiv.org9 citations Open access
Semantic Scholar
Data Movement Is All You Need: A Case Study of Transformer Networks
A. Ivanov, Nikoli Dryden, Tal Ben-Nun, Shi-Gang Li +1 more
20208 citations
Semantic Scholar
STen: Productive and Efficient Sparsity in PyTorch
Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun, Timo Schneider +2 more
2023ACM Transactions on Architecture and Code Optimization (TACO)7 citations Open access
Semantic Scholar
Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning
Bryan A. Plummer, Nikoli Dryden, Julius Frost, T. Hoefler +1 more
2020arXiv.org7 citations
Semantic Scholar
PGDB: A Debugger for MPI Applications
Nikoli Dryden
2014Extreme Science and Engineering Discovery Environment7 citations
Semantic Scholar
PPL: an abstract runtime system for hybrid parallel programming
Alex Brooks, Hoang-Vu Dang, Nikoli Dryden, M. Snir
2015ESPM '155 citations
Semantic Scholar
Learning to Compose SuperWeights for Neural Parameter Allocation Search
Piotr Teterwak, Soren Nelson, Nikoli Dryden, D. Bashkirova +2 more
2023IEEE Workshop/Winter Conference on Applications of Computer Vision4 citations Open access
Semantic Scholar
STen: An Interface for Efficient Sparsity in PyTorch
Andrei Ivanov, Nikoli Dryden
20224 citations
Semantic Scholar
Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
Chen Zhuang, Lingqi Zhang, Du Wu, Peng Chen +7 more
2024International Conference on Supercomputing3 citations Open access
Semantic Scholar
Livermore Big Artificial Neural Network Toolkit
B. V. Van Essen, S. Jacobs, Hyojin Kim, Nikoli Dryden +1 more
20162 citations
Semantic Scholar
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Satoki Ishikawa, Tal Ben-Nun, B. V. Van Essen, Rio Yokota +1 more
2024arXiv.org1 citations Open access
Semantic Scholar
Shapeshifter Networks: Decoupling Layers from Parameters for Scalable and Effective Deep Learning.
Bryan A. Plummer, Nikoli Dryden, Julius Frost, T. Hoefler +1 more
20201 citations
Semantic Scholar
Chasing the Rabbit: A Systematic Exploration of Software-Defined HPC Storage
Hariharan Devarajan, B. Behlendorf, Blake Devcich, Dean Roehrich +14 more
2026IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum0 citations
Semantic Scholar
WADO: A Distributed WORM Storage Service for Asynchronous Data Operations
Karim Youssef, Hariharan Devarajan, Nikoli Dryden, Roger Pearce
2026International Conference on Statistical and Scientific Database Management0 citations
Semantic Scholar
CoDL: A Framework for Studying Cross-Component Interference in Deep Learning Training Pipelines
Druva Dhakshinamoorthy, Ray A. O. Sinurat, Nikoli Dryden, Arnab K. Paul +1 more
2026IEEE International Symposium on High-Performance Parallel Distributed Computing0 citations
Semantic Scholar
CoDL: Studying Cross-Component Interference
Hari Hariharan Devarajan, Ray A. O. Sinurat, Druva Dhakshinamoorthy, Arnab K. Paul +1 more
20260 citations Open access
Semantic Scholar
HORATIO: Bridging Management and Analysis of Traces at Scale
Ray A. O. Sinurat, William Nixon, Haryadi S. Gunawi, Nikoli Dryden +1 more
2026International Conference on Statistical and Scientific Database Management0 citations Open access
Semantic Scholar
A General and Scalable GCN Training Framework on CPU Supercomputers
Chen Zhuang, Peng Chen, Xin Liu, Rio Yokota +5 more
2025ACM SIGPLAN Symposium on Principles & Practice of Parallel Programming0 citations
Semantic Scholar
Large-scale training of deep neural networks
Nikoli Dryden
20190 citations
Semantic Scholar
Comm Engine Sync Thread SCSP Queue Comm Thread Request Container Threading Model Sync Container Worker Worker . . . Worker Shepherd Shepherd
Hoang-Vu Dang, Alex Brooks, Nikoli Dryden, M. Snir
20150 citations
Semantic Scholar
PGAS with Lightweight Threads and the Barnes-Hut Algorithm
Hoang-Vu Dang, Alex Brooks, Nikoli Dryden, M. Snir
20140 citations
Semantic Scholar