Deep learning is currently the fastest-growing field in machine learning and is transforming the various segments of our lives. This fast evolving technology was pioneered by GPU-accelerated compute systems and has enabled machines to be trained at a speed, accuracy, and scale that can drive innovation in artificial intelligence. In this talk, I will discuss two of my recent works focused on building accelerator-centric systems for scalable and energy-efficient deep learning: (a) leveraging throughput-optimized GPUs for training, and (b) using latency and energy-optimized ASICs for inference.