In this paper, we optimize single-precision Winograd convolution, a fast algorithm for convolution, on NVIDIA Volta and Turing GPUs. Compared with the state-of-the-art cuDNN 7.6.1’s Winograd convolution, our implementation achieves up to $2.13\times$ speedup on Volta V100 and up to $2.65\times$ speedup on Turing RTX2070. On both devices, our implementation achieves up to $93%$ of device peak.

Apart from analyzing and benchmarking different high-level optimization options, we also build a SASS assembler TuringAs for Volta and Turing to tune the performance at the native assembly level. We find new performance opportunities not only specific to the Winograd convolution but general for the CUDA compiler and native assembly programming. Those opportunities are only observable at SASS level. We make TuringAs publicly available to inspire more works in this area. To the best of our knowledge, this is the first public assembler for Volta and Turing.

#### Mon 24 FebDisplayed time zone: Tijuana, Baja California change

 10:55 - 12:35 Machine Learning/Big Data (Mediterranean Ballroom)Main Conference Chair(s): Shuaiwen Leon Song University of Sydney 10:5525mTalk Optimizing Batched Winograd Convolution on GPUsMain ConferenceDa Yan Hong Kong University of Science and Technology, Wei Wang Hong Kong University of Science and Technology, Xiaowen Chu Hong Kong Baptist University 11:2025mTalk Taming Unbalanced Training Workloads in Deep Learning with Partial Collective OperationsMain ConferenceShigang Li Department of Computer Science, ETH Zurich, Tal Ben-Nun Department of Computer Science, ETH Zurich, Salvatore Di Girolamo Department of Computer Science, ETH Zurich, Dan Alistarh IST Austria, Torsten Hoefler Department of Computer Science, ETH Zurich 11:4525mTalk Scalable Top-K Retrieval with SpartaMain ConferenceGali Sheffi Technion - Israel, Dmitry Basin Yahoo Research, Edward Bortnikov Yahoo Research, David Carmel Amazon, Idit Keidar Technion - Israel institute of technology 12:1025mTalk waveSZ: A Hardware-Algorithm Co-Design of Efficient Lossy Compression for Scientific DataMain ConferenceJiannan Tian University of Alabama, Sheng Di Argonne National Laboratory, Chengming Zhang University of Alabama, Xin Liang , Sian Jin University of Alabama, Dazhao Cheng University of North Carolina at Charlotte, Dingwen Tao University of Alabama, Franck Cappello Argonne National Laboratory