Hung Viet Pham

Hung Viet Pham

Assistant Professor at York University, interested in Software Engineering and Machine Learning

Talks and presentations

Problems and opportunities in training deep learning software systems: an analysis of variance

September 23, 2019

Conference proceedings talk, 35th IEEE/ACM conference on Automated Software Engineering, Virtual Arts Centre Melbourne, Australia

Deep learning (DL) training algorithms utilize nondeterminism to improve models’ accuracy and training efficiency. Hence, multiple identical training runs (e.g., identical training data, algorithm, and network) produce different models with different accuracies and training times. In addition to these algorithmic factors, DL libraries (e.g., TensorFlow and cuDNN) introduce additional variance (referred to as implementation-level variance) due to parallelism, optimization, and floating-point computation.

CRADLE: Cross-Backend Validation to Detect and Localize Bugs in Deep Learning Libraries

May 25, 2019

Conference proceedings talk, 41st ACM/IEEE International Conference on Software Engineering, Montreal, QC, Canada

We propose CRADLE, a new approach that focuses on finding and localizing bugs in DL software libraries. CRADLE (1) performs cross-implementation inconsistency checking to detect bugs in DL libraries, and (2) leverages anomaly propagation tracking and analysis to localize faulty functions in DL libraries that cause the bugs. We evaluate CRADLE on three libraries (TensorFlow, CNTK, and Theano), 11 datasets (including ImageNet, MNIST, and KGS Go game), and 30 pre-trained models. CRADLE detects 12 bugs and 104 unique inconsistencies, and highlights functions relevant to the causes of inconsistencies for all 104 unique inconsistencies.