Portrait de Matthieu Courbariaux n'est pas disponible

Matthieu Courbariaux

Alumni

Publications

BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
Milos Nikolic
Ghouthi Boukli hacene
Ciaran Bannon
Alberto Delmas Lascorz
Matthieu Courbariaux
Omar Mohamed Awad
Yoshua Bengio
Isak Edo Vivancos
Vincent Gripon
Andreas Moshovos
Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time … (voir plus)and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy.
2024-05-18
2024 IEEE International Symposium on Circuits and Systems (ISCAS) (publié)
doi.org
arxiv.org
Attention Based Pruning for Shift Networks
Ghouthi Boukli hacene
Carlos Lassance
Vincent Gripon
Matthieu Courbariaux
Yoshua Bengio
In many application domains such as computer vision, Convolutional Layers (CLs) are key to the accuracy of deep learning methods. However, i… (voir plus)t is often required to assemble a large number of CLs, each containing thousands of parameters, in order to reach state-of-the-art accuracy, thus resulting in complex and demanding systems that are poorly fitted to resource-limited devices. Recently, methods have been proposed to replace the generic convolution operator by the combination of a shift operation and a simpler 1x1 convolution. The resulting block, called Shift Layer (SL), is an efficient alternative to CLs in the sense it allows to reach similar accuracies on various tasks with faster computations and fewer parameters. In this contribution, we introduce Shift Attention Layers (SALs), which extend SLs by using an attention mechanism that learns which shifts are the best at the same time the network function is trained. We demonstrate SALs are able to outperform vanilla SLs (and CLs) on various object recognition benchmarks while significantly reducing the number of float operations and parameters for the inference.
2021-01-09
International Conference on Pattern Recognition (publié)
doi.org
arxiv.org