A16 winter school on Efficient Deep Learning

Expected Winter 2023. Date not known yet.

Course content

Machine learning has numerous important applications in intelligent systems within many areas, like automotive, avionics, robotics, healthcare, well-being, and security. The recent progress in Machine Learning, and particularly in Deep Learning (DL), has dramatically improved the state-of-the-art in object detection, classification and recognition, and in many other domains. Whether it is superhuman performance in object recognition or beating human players in Go, the astonishing success of DL is achieved by deep neural networks. However, the complexity of DL networks for many practical applications can be huge, and their processing may demand a high computing effort and excessive energy consumption. Their training requires huge data sets, making the training even orders of magnitude more intensive than their already very demanding inference phase. A new development is to move intelligence from the cloud to the IoT edge; this further stresses the need to tame the complexity of DL and Deep Neural Networks.

This joint ASCI-EDL winterschool treats various topics addressing the complexity reduction of DL, including:

• Architectural and Hardware accelerator support for DL, with emphasis on energy reduction, computation efficiency and/or computation flexibility, both for inference and/or for learning;
• Spiking and brain-inspired neural networks and their implementation;
• Efficient mapping of DL applications to target architectures, including many-core, GPGPU, SIMD, FPGA, and HW accelerators;
• Exploiting temporal and spatial data reuse, sparsity, quantization and approximate computing, dynamic neural networks, and other methods, to decrease the complexity and energy demands of DL.
• Efficient learning approaches, including data reduction, online learning, and quality of learning;
• Tools, Frameworks and High-level programming language support for DL;
• NAS: Neural Architecture Search, including Hardware aware NAS;
• Advanced applications exploiting DL.

Above topics will be treated by experts from the Netherlands and abroad.

Required background: Basic knowledge of deep learning and computer architecture.