CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization
Wenzheng Hu
Ning Liu
Zhengping Che
Mingyang Li
Changshui Zhang
Jianqiang Wang
Deep convolutional neural networks are shown to be overkill with high parametric and computational redundancy in many application scenarios,… (see more) and an increasing number of works have explored model pruning to obtain lightweight and efficient networks. However, most existing pruning approaches are driven by empirical heuristics and rarely consider the joint impact of channels, leading to unguaranteed and suboptimal performance. In this article, we propose a novel channel pruning method via class-aware trace ratio optimization (CATRO) to reduce the computational burden and accelerate the model inference. Utilizing class information from a few samples, CATRO measures the joint impact of multiple channels by feature space discriminations and consolidates the layerwise impact of preserved channels. By formulating channel pruning as a submodular set function maximization problem, CATRO solves it efficiently via a two-stage greedy iterative optimization procedure. More importantly, we present theoretical justifications on convergence of CATRO and performance of pruned networks. Experimental results demonstrate that CATRO achieves higher accuracy with similar computation cost or lower computation cost with similar accuracy than other state-of-the-art channel pruning algorithms. In addition, because of its class-aware property, CATRO is suitable to prune efficient networks adaptively for various classification subtasks, enhancing handy deployment and usage of deep networks in real-world applications.
Causal Adversarial Perturbations for Individual Fairness and Robustness in Heterogeneous Data Spaces
Ahmad-reza Ehyaei
Kiarash Mohammadi
Amir-Hossein Karimi
S. Samadi
Caustics: A Python Package for Accelerated Strong Gravitational Lensing Simulations
Connor Stone
Alexandre Adam
Adam Coogan
M. J. Yantovski-Barth
Andreas Filipp
Landung Setiawan
Cordero Core
Ronan Legin
Charles Wilson
Gabriel Missael Barco
ChainBuddy: An AI Agent System for Generating LLM Pipelines
Jingyue Zhang
ChainBuddy: An AI-assisted Agent System for Helping Users Set up LLM Pipelines
Jingyue Zhang
CL-MASR: A Continual Learning Benchmark for Multilingual ASR
Luca Della Libera
Pooneh Mousavi
Salah Zaiem
Common Challenges of Deep Reinforcement Learning Applications Development: An Empirical Study
Mohammad Mehdi Morovati
Florian Tambon
Mina Taraghi
Amin Nikanjam
Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning
Consolidating Separate Degradations Model via Weights Fusion and Distillation
Dinesh Daultani
Real-world images prevalently contain different varieties of degradation, such as motion blur and luminance noise. Computer vision recogniti… (see more)on models trained on clean images perform poorly on degraded images. Previously, several works have explored how to perform image classification of degraded images while training a single model for each degradation. Nevertheless, it becomes challenging to host several degradation models for each degradation on limited hardware applications and to estimate degradation parameters correctly at the run-time. This work proposes a method for effectively combining several models trained separately on different degradations into a single model to classify images with different types of degradations. Our proposed method is four-fold: (1) train a base model on clean images, (2) fine-tune the base model in-dividually for all given image degradations, (3) perform a fusion of weights given the fine-tuned models for individual degradations, (4) perform fine-tuning on given task using distillation and cross-entropy loss. Our proposed method can outperform previous state-of-the-art methods of pretraining in out-of-distribution generalization based on degradations such as JPEG compression, salt-and-pepper noise, Gaussian blur, and additive white Gaussian noise by 2.5% on CIFAR-100 dataset and by 1.3% on CIFAR-10 dataset. Moreover, our proposed method can handle degra-dation used for training without any explicit information about degradation at the inference time. Code will be available at https://github.com/dineshdaultani/FusionDistill.
Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Models
Jerry Huang
Prasanna Parthasarathi
Mehdi Rezagholizadeh
Corticosteroids induce an early but limited decrease in IL-6 dependent pro-inflammatory responses in critically ill COVID-19 patients
Tomas URBINA
Paul GABARRE
Vincent BONNY
Jean-Rémi Lavillegrand
Marc GARNIER
Jérémie JOFFRE
Nathalie MARIO
Geoffroy HARIRI
Matthieu TURPIN
Emmanuel PARDO
Muriel FARTOUKH
Bertrand GUIDET
Eric Maury
Yannick CHANTRAN
Pierre-Yves BOELLE
Guillaume VOIRIOT
Hafid AIT-OUFELLA
Dance of the Neurons: Unraveling Sex from Brain Signals (short paper).
Mohammad-Javad Darvishi Bayazi
Mohammad S. Ghaemi
Jocelyn Faubert