Publications

Words Matter: Leveraging Individual Text Embeddings for Code Generation in CLIP Test-Time Adaptation
Shambhavi Mishra
Julio Silva-Rodríguez
Ismail Ben Ayed
Marco Pedersoli
Jose Dolz
Vision-language foundation models, such as CLIP, have shown unprecedented zero-shot performance across a wide range of tasks. Nevertheless, … (see more)these models may be unreliable under distributional shifts, as their performance is significantly degraded. In this work, we explore how to efficiently leverage class text information to mitigate these distribution drifts encountered by large pre-trained vision-language models (VLMs) during test-time inference. In particular, we propose to generate pseudo-labels for the test-time samples by exploiting generic class text embeddings as fixed centroids of a label assignment problem, which is efficiently solved with Optimal Transport. Furthermore, the proposed adaptation method (CLIP-OT) integrates a multiple template knowledge distillation approach, which replicates multi-view contrastive learning strategies in unsupervised representation learning but without incurring additional computational complexity. Extensive experiments on multiple popular test-time adaptation benchmarks presenting diverse complexity empirically show the superiority of CLIP-OT, achieving performance gains of up to 7% over recent state-of-the-art methods, yet being computationally and memory efficient.
2024-11-01
arXiv (published)
doi.org
arxiv.org
Improving Adversarial Transferability via Model Alignment
Avery Ma
Amir-massoud Farahmand
Yangchen Pan
Philip Torr
Jindong Gu
Neural networks are susceptible to adversarial perturbations that are transferable across different models. In this paper, we introduce a no… (see more)vel model alignment technique aimed at improving a given source model's ability in generating transferable adversarial perturbations. During the alignment process, the parameters of the source model are fine-tuned to minimize an alignment loss. This loss measures the divergence in the predictions between the source model and another, independently trained model, referred to as the witness model. To understand the effect of model alignment, we conduct a geometric analysis of the resulting changes in the loss landscape. Extensive experiments on the ImageNet dataset, using a variety of model architectures, demonstrate that perturbations generated from aligned source models exhibit significantly higher transferability than those from the original source model.
2024-10-31
Lecture Notes in Computer Science (published)
doi.org
arxiv.org
Unsupervised Object Discovery: A Comprehensive Survey and Unified Taxonomy
Jos'e-Fabian Villa-V'asquez
Marco Pedersoli
Unsupervised object discovery is commonly interpreted as the task of localizing and/or categorizing objects in visual data without the need … (see more)for labeled examples. While current object recognition methods have proven highly effective for practical applications, the ongoing demand for annotated data in real-world scenarios drives research into unsupervised approaches. Furthermore, existing literature in object discovery is both extensive and diverse, posing a significant challenge for researchers that aim to navigate and synthesize this knowledge. Motivated by the evidenced interest in this avenue of research, and the lack of comprehensive studies that could facilitate a holistic understanding of unsupervised object discovery, this survey conducts an in-depth exploration of the existing approaches and systematically categorizes this compendium based on the tasks addressed and the families of techniques employed. Additionally, we present an overview of common datasets and metrics, highlighting the challenges of comparing methods due to varying evaluation protocols. This work intends to provide practitioners with an insightful perspective on the domain, with the hope of inspiring new ideas and fostering a deeper understanding of object discovery approaches.
2024-10-30
ArXiv (preprint)
doi.org
arxiv.org
Injury and violence in the context of sustainable development
Kidist Bartolomeos
Ryan Lett
Respicious Boniface
Victoria Munthali
Tarek Razek
Dan Deckelbaum
David Bracco
Ermiyas Belay
Fitsum Kifle
David Ulrich Dalle
Celestin Bilong Mbangtang
Arsene Daniel Nyalundja
Jondre Macaraeg
Irene Dzirasa
Ulrick Sidney Kanmounye
Delanyo Dovlo
Kwadwo Koram
Eugene Nyarko
Desmond T. Jumbam
Emnet Tesfay Shimber … (see 180 more)
Taylor Jaraczewski
Maria Sgro
Ajiel Mae Basmayor
Asegid Ergete
Mary Schroeder
Adam Gyedu
Emmanuel Nakua
Peter Donkor
Charles Mock
Atalel Awedew
Halid Melkamu
Sisay Bekele
Berhanu Hailemariam
Enku Shiferaw
Yishak Shiferaw
Wubetie Yirdaw
Debojit Basak
Deepa Kizhakke Veetil
Nobhojit Roy
Martin Gerdin Wärnberg
Santosh Rath
Mohammed A.S Abdullahi
Kefas Mbaya
Abubakar Kakasanda
Stephanie Danjuma
Hector Olasoji
Alemayehu Bedada
Mpapho Joseph Motsumi
Shimelis Genna Hamda
Demuma Amdisa
Getachew Tilahun
Matthew Boroditsky
Mark Hill
Roy Hilzenrat
Rachel Livergant
Jayd Adams
Catherine Binda
Allison Chhor
Helen Hsiao
Faizal Haji
Esther Chin
Felix Oyania
Caroline Q. Stephens
Sarah Ullrich
Meera Kotagal
Francis Bajunirwe
Doruk Ozgediz
Dionysia Kravarioti
Lye-Yeng Wong
Tsegazeab Laeke Teklemariam
Abenezer Tirsit
Tewodros Liyew
Mark Ferguson
Timothy Plackett
Jaymie Claire Henry
Meseret Abeza
Seye Mesfin Minas
Maryse Bouchard
Dimuthu Tennakoon
Rahul Burra
Fleming Mathew
Annabelle Jones
Sargun Virk
Shlok Patel
Tanaz Vaghaiwalla
James Hudspeth
Tracy Rabin
Virginia Rowthorn
Raymond R. Price
Nakul Raykar
Gilgamesh Eamer
Stephen Mutiso
Yvette Kisaka
Gladwell Gathecha
Ronald Lett
Chibuike Onu
Emmanuel Ameh
Matthias Igoche
Paschal Anyanwu
Eunice Onuh
Oikeh Ojeamen
Edith Terna Yawe
Amina Abubakar
Yakubu Ashoms
Hadiza Suleiman
Naomi Musa
Daniel Kisitu Kyengera
Netsanet Abebe
Richard Gardener
Nebyou Seyoum Abebe
Henok T/Silasie Zeleke
Kacylia Roy Proulx
Shreenik Kundu
Boaz Laor
Riya Sawhney
Taylor Wurdeman
Fabio Botelho
Ayla Gerk
Elena Guadagno
Dan Poenaru
Mengistu Ayele
Azarias Kassahun
Tsegazeab Laeke
Mestet Yibeltal
Bereket Hailu
Ermias Fikru
Shemsedin Ibro
Abdeta Workineh
Fikadu Balcha
Fira Abamecha
Sheka Shemsi
Abdullah Saleh Alruwaili
Gabriel Rodriguez
Anna Jose
Shahd Ebied
Samuel Girma
Abigael Abiy
Hussien Endris Assen
Kalab Tesfaye
Kassaye Demeke
Aklilu Yiheyis
Khalid Jemal
Demeke Yilkal
Ashenafi Amsalu
Lema Derseh
Yophtahe W/Gerima
Tadesse Belayneh
Mekuanint Tiruneh
Almaw Bitew
Sewbesew Yitayih
Tadesse Awoke
Chanyalew Worku
Anissa Mohammed
Mohammed Alemu
Mohammed Yesuf
Fantu Mamo
Kegnie Shitu
Biks Liyew
Ayenew Gucho
Gezahegn Tilahun
Timothy Love
Andrew Chew
Brian Kasagga
Berjo Takoutsing
Obuku Ekwaro
Emmanuel Elobu
Degisew Dersso Mengistu
Alex Zhuang
Bethlehem Shiferew
Gelila Mengistu
Ayalew Zewdie
Nahom Tadelle
Alegnta Gebreyesus
Elise Presser
Katie Iverson
Christopher Dodgion
Thomas G. Weiser
Rachel Koch
Nichole Starr
Davy Lau
Irena Zivkovic
Shahrzad Joharifard
Emilie Joos
Naisan Garraway
Francesca Vituci
Eric O’Flynn
Ines Péric
Léa Simon
Geoffrey Ibbotson
Tsion Seyoum
Aklilu Azazh
Lemlem Beza
Ifeanyichukwu Onah
Chijioke Chukwuma
Dagim Berhanu
Jason Shenoi
Nick Sears
Yoseph Bedore
Richard Caplan
Wongel Tena Shale
invaluable
2024-10-29
Canadian journal of surgery. Journal canadien de chirurgie (published)
doi.org
Injury and violence in the context of sustainable development
Kidist Bartolomeos
Ryan Lett
Respicious Boniface
Victoria Munthali
Tarek Razek
Dan Deckelbaum
David Bracco
Ermiyas Belay
Fitsum Kifle
David Ulrich Dalle
Celestin Bilong Mbangtang
Arsene Daniel Nyalundja
Jondre Macaraeg
Irene Dzirasa
Ulrick Sidney Kanmounye
Delanyo Dovlo
Kwadwo Koram
Eugene Nyarko
Desmond T. Jumbam
Emnet Tesfay Shimber … (see 180 more)
Taylor Jaraczewski
Maria Sgro
Ajiel Mae Basmayor
Asegid Ergete
Mary Schroeder
Adam Gyedu
Emmanuel Nakua
Peter Donkor
Charles Mock
Atalel Awedew
Halid Melkamu
Sisay Bekele
Berhanu Hailemariam
Enku Shiferaw
Yishak Shiferaw
Wubetie Yirdaw
Debojit Basak
Deepa Kizhakke Veetil
Nobhojit Roy
Martin Gerdin Wärnberg
Santosh Rath
Mohammed A.S Abdullahi
Kefas Mbaya
Abubakar Kakasanda
Stephanie Danjuma
Hector Olasoji
Alemayehu Bedada
Mpapho Joseph Motsumi
Shimelis Genna Hamda
Demuma Amdisa
Getachew Tilahun
Matthew Boroditsky
Mark Hill
Roy Hilzenrat
Rachel Livergant
Jayd Adams
Catherine Binda
Allison Chhor
Helen Hsiao
Faizal Haji
Esther Chin
Felix Oyania
Caroline Q. Stephens
Sarah Ullrich
Meera Kotagal
Francis Bajunirwe
Doruk Ozgediz
Dionysia Kravarioti
Lye-Yeng Wong
Tsegazeab Laeke Teklemariam
Abenezer Tirsit
Tewodros Liyew
Mark Ferguson
Timothy Plackett
Jaymie Claire Henry
Meseret Abeza
Seye Mesfin Minas
Maryse Bouchard
Dimuthu Tennakoon
Rahul Burra
Fleming Mathew
Annabelle Jones
Sargun Virk
Shlok Patel
Tanaz Vaghaiwalla
James Hudspeth
Tracy Rabin
Virginia Rowthorn
Raymond R. Price
Nakul Raykar
Gilgamesh Eamer
Stephen Mutiso
Yvette Kisaka
Gladwell Gathecha
Ronald Lett
Chibuike Onu
Emmanuel Ameh
Matthias Igoche
Paschal Anyanwu
Eunice Onuh
Oikeh Ojeamen
Edith Terna Yawe
Amina Abubakar
Yakubu Ashoms
Hadiza Suleiman
Naomi Musa
Daniel Kisitu Kyengera
Netsanet Abebe
Richard Gardener
Nebyou Seyoum Abebe
Henok T/Silasie Zeleke
Kacylia Roy Proulx
Shreenik Kundu
Boaz Laor
Riya Sawhney
Taylor Wurdeman
Fabio Botelho
Ayla Gerk
Elena Guadagno
Dan Poenaru
Mengistu Ayele
Azarias Kassahun
Tsegazeab Laeke
Mestet Yibeltal
Bereket Hailu
Ermias Fikru
Shemsedin Ibro
Abdeta Workineh
Fikadu Balcha
Fira Abamecha
Sheka Shemsi
Abdullah Saleh Alruwaili
Gabriel Rodriguez
Anna Jose
Shahd Ebied
Samuel Girma
Abigael Abiy
Hussien Endris Assen
Kalab Tesfaye
Kassaye Demeke
Aklilu Yiheyis
Khalid Jemal
Demeke Yilkal
Ashenafi Amsalu
Lema Derseh
Yophtahe W/Gerima
Tadesse Belayneh
Mekuanint Tiruneh
Almaw Bitew
Sewbesew Yitayih
Tadesse Awoke
Chanyalew Worku
Anissa Mohammed
Mohammed Alemu
Mohammed Yesuf
Fantu Mamo
Kegnie Shitu
Biks Liyew
Ayenew Gucho
Gezahegn Tilahun
Timothy Love
Andrew Chew
Brian Kasagga
Berjo Takoutsing
Obuku Ekwaro
Emmanuel Elobu
Degisew Dersso Mengistu
Alex Zhuang
Bethlehem Shiferew
Gelila Mengistu
Ayalew Zewdie
Nahom Tadelle
Alegnta Gebreyesus
Elise Presser
Katie Iverson
Christopher Dodgion
Thomas G. Weiser
Rachel Koch
Nichole Starr
Davy Lau
Irena Zivkovic
Shahrzad Joharifard
Emilie Joos
Naisan Garraway
Francesca Vituci
Eric O’Flynn
Ines Péric
Léa Simon
Geoffrey Ibbotson
Tsion Seyoum
Aklilu Azazh
Lemlem Beza
Ifeanyichukwu Onah
Chijioke Chukwuma
Dagim Berhanu
Jason Shenoi
Nick Sears
Yoseph Bedore
Richard Caplan
Wongel Tena Shale
invaluable
2024-10-29
Canadian journal of surgery. Journal canadien de chirurgie (published)
doi.org
Injury and violence in the context of sustainable development
Kidist Bartolomeos
Ryan Lett
Respicious Boniface
Victoria Munthali
Tarek Razek
Dan Deckelbaum
David Bracco
Ermiyas Belay
Fitsum Kifle
David Ulrich Dalle
Celestin Bilong Mbangtang
Arsene Daniel Nyalundja
Jondre Macaraeg
Irene Dzirasa
Ulrick Sidney Kanmounye
Delanyo Dovlo
Kwadwo Koram
Eugene Nyarko
Desmond T. Jumbam
Emnet Tesfay Shimber … (see 180 more)
Taylor Jaraczewski
Maria Sgro
Ajiel Mae Basmayor
Asegid Ergete
Mary Schroeder
Adam Gyedu
Emmanuel Nakua
Peter Donkor
Charles Mock
Atalel Awedew
Halid Melkamu
Sisay Bekele
Berhanu Hailemariam
Enku Shiferaw
Yishak Shiferaw
Wubetie Yirdaw
Debojit Basak
Deepa Kizhakke Veetil
Nobhojit Roy
Martin Gerdin Wärnberg
Santosh Rath
Mohammed A.S Abdullahi
Kefas Mbaya
Abubakar Kakasanda
Stephanie Danjuma
Hector Olasoji
Alemayehu Bedada
Mpapho Joseph Motsumi
Shimelis Genna Hamda
Demuma Amdisa
Getachew Tilahun
Matthew Boroditsky
Mark Hill
Roy Hilzenrat
Rachel Livergant
Jayd Adams
Catherine Binda
Allison Chhor
Helen Hsiao
Faizal Haji
Esther Chin
Felix Oyania
Caroline Q. Stephens
Sarah Ullrich
Meera Kotagal
Francis Bajunirwe
Doruk Ozgediz
Dionysia Kravarioti
Lye-Yeng Wong
Tsegazeab Laeke Teklemariam
Abenezer Tirsit
Tewodros Liyew
Mark Ferguson
Timothy Plackett
Jaymie Claire Henry
Meseret Abeza
Seye Mesfin Minas
Maryse Bouchard
Dimuthu Tennakoon
Rahul Burra
Fleming Mathew
Annabelle Jones
Sargun Virk
Shlok Patel
Tanaz Vaghaiwalla
James Hudspeth
Tracy Rabin
Virginia Rowthorn
Raymond R. Price
Nakul Raykar
Gilgamesh Eamer
Stephen Mutiso
Yvette Kisaka
Gladwell Gathecha
Ronald Lett
Chibuike Onu
Emmanuel Ameh
Matthias Igoche
Paschal Anyanwu
Eunice Onuh
Oikeh Ojeamen
Edith Terna Yawe
Amina Abubakar
Yakubu Ashoms
Hadiza Suleiman
Naomi Musa
Daniel Kisitu Kyengera
Netsanet Abebe
Richard Gardener
Nebyou Seyoum Abebe
Henok T/Silasie Zeleke
Kacylia Roy Proulx
Shreenik Kundu
Boaz Laor
Riya Sawhney
Taylor Wurdeman
Fabio Botelho
Ayla Gerk
Elena Guadagno
Dan Poenaru
Mengistu Ayele
Azarias Kassahun
Tsegazeab Laeke
Mestet Yibeltal
Bereket Hailu
Ermias Fikru
Shemsedin Ibro
Abdeta Workineh
Fikadu Balcha
Fira Abamecha
Sheka Shemsi
Abdullah Saleh Alruwaili
Gabriel Rodriguez
Anna Jose
Shahd Ebied
Samuel Girma
Abigael Abiy
Hussien Endris Assen
Kalab Tesfaye
Kassaye Demeke
Aklilu Yiheyis
Khalid Jemal
Demeke Yilkal
Ashenafi Amsalu
Lema Derseh
Yophtahe W/Gerima
Tadesse Belayneh
Mekuanint Tiruneh
Almaw Bitew
Sewbesew Yitayih
Tadesse Awoke
Chanyalew Worku
Anissa Mohammed
Mohammed Alemu
Mohammed Yesuf
Fantu Mamo
Kegnie Shitu
Biks Liyew
Ayenew Gucho
Gezahegn Tilahun
Timothy Love
Andrew Chew
Brian Kasagga
Berjo Takoutsing
Obuku Ekwaro
Emmanuel Elobu
Degisew Dersso Mengistu
Alex Zhuang
Bethlehem Shiferew
Gelila Mengistu
Ayalew Zewdie
Nahom Tadelle
Alegnta Gebreyesus
Elise Presser
Katie Iverson
Christopher Dodgion
Thomas G. Weiser
Rachel Koch
Nichole Starr
Davy Lau
Irena Zivkovic
Shahrzad Joharifard
Emilie Joos
Naisan Garraway
Francesca Vituci
Eric O’Flynn
Ines Péric
Léa Simon
Geoffrey Ibbotson
Tsion Seyoum
Aklilu Azazh
Lemlem Beza
Ifeanyichukwu Onah
Chijioke Chukwuma
Dagim Berhanu
Jason Shenoi
Nick Sears
Yoseph Bedore
Richard Caplan
Wongel Tena Shale
invaluable
2024-10-29
Canadian journal of surgery. Journal canadien de chirurgie (published)
doi.org
Injury and violence in the context of sustainable development
Kidist Bartolomeos
Ryan Lett
Respicious Boniface
Victoria Munthali
Tarek Razek
Dan Deckelbaum
David Bracco
Ermiyas Belay
Fitsum Kifle
David Ulrich Dalle
Celestin Bilong Mbangtang
Arsene Daniel Nyalundja
Jondre Macaraeg
Irene Dzirasa
Ulrick Sidney Kanmounye
Delanyo Dovlo
Kwadwo Koram
Eugene Nyarko
Desmond T. Jumbam
Emnet Tesfay Shimber … (see 180 more)
Taylor Jaraczewski
Maria Sgro
Ajiel Mae Basmayor
Asegid Ergete
Mary Schroeder
Adam Gyedu
Emmanuel Nakua
Peter Donkor
Charles Mock
Atalel Awedew
Halid Melkamu
Sisay Bekele
Berhanu Hailemariam
Enku Shiferaw
Yishak Shiferaw
Wubetie Yirdaw
Debojit Basak
Deepa Kizhakke Veetil
Nobhojit Roy
Martin Gerdin Wärnberg
Santosh Rath
Mohammed A.S Abdullahi
Kefas Mbaya
Abubakar Kakasanda
Stephanie Danjuma
Hector Olasoji
Alemayehu Bedada
Mpapho Joseph Motsumi
Shimelis Genna Hamda
Demuma Amdisa
Getachew Tilahun
Matthew Boroditsky
Mark Hill
Roy Hilzenrat
Rachel Livergant
Jayd Adams
Catherine Binda
Allison Chhor
Helen Hsiao
Faizal Haji
Esther Chin
Felix Oyania
Caroline Q. Stephens
Sarah Ullrich
Meera Kotagal
Francis Bajunirwe
Doruk Ozgediz
Dionysia Kravarioti
Lye-Yeng Wong
Tsegazeab Laeke Teklemariam
Abenezer Tirsit
Tewodros Liyew
Mark Ferguson
Timothy Plackett
Jaymie Claire Henry
Meseret Abeza
Seye Mesfin Minas
Maryse Bouchard
Dimuthu Tennakoon
Rahul Burra
Fleming Mathew
Annabelle Jones
Sargun Virk
Shlok Patel
Tanaz Vaghaiwalla
James Hudspeth
Tracy Rabin
Virginia Rowthorn
Raymond R. Price
Nakul Raykar
Gilgamesh Eamer
Stephen Mutiso
Yvette Kisaka
Gladwell Gathecha
Ronald Lett
Chibuike Onu
Emmanuel Ameh
Matthias Igoche
Paschal Anyanwu
Eunice Onuh
Oikeh Ojeamen
Edith Terna Yawe
Amina Abubakar
Yakubu Ashoms
Hadiza Suleiman
Naomi Musa
Daniel Kisitu Kyengera
Netsanet Abebe
Richard Gardener
Nebyou Seyoum Abebe
Henok T/Silasie Zeleke
Kacylia Roy Proulx
Shreenik Kundu
Boaz Laor
Riya Sawhney
Taylor Wurdeman
Fabio Botelho
Ayla Gerk
Elena Guadagno
Dan Poenaru
Mengistu Ayele
Azarias Kassahun
Tsegazeab Laeke
Mestet Yibeltal
Bereket Hailu
Ermias Fikru
Shemsedin Ibro
Abdeta Workineh
Fikadu Balcha
Fira Abamecha
Sheka Shemsi
Abdullah Saleh Alruwaili
Gabriel Rodriguez
Anna Jose
Shahd Ebied
Samuel Girma
Abigael Abiy
Hussien Endris Assen
Kalab Tesfaye
Kassaye Demeke
Aklilu Yiheyis
Khalid Jemal
Demeke Yilkal
Ashenafi Amsalu
Lema Derseh
Yophtahe W/Gerima
Tadesse Belayneh
Mekuanint Tiruneh
Almaw Bitew
Sewbesew Yitayih
Tadesse Awoke
Chanyalew Worku
Anissa Mohammed
Mohammed Alemu
Mohammed Yesuf
Fantu Mamo
Kegnie Shitu
Biks Liyew
Ayenew Gucho
Gezahegn Tilahun
Timothy Love
Andrew Chew
Brian Kasagga
Berjo Takoutsing
Obuku Ekwaro
Emmanuel Elobu
Degisew Dersso Mengistu
Alex Zhuang
Bethlehem Shiferew
Gelila Mengistu
Ayalew Zewdie
Nahom Tadelle
Alegnta Gebreyesus
Elise Presser
Katie Iverson
Christopher Dodgion
Thomas G. Weiser
Rachel Koch
Nichole Starr
Davy Lau
Irena Zivkovic
Shahrzad Joharifard
Emilie Joos
Naisan Garraway
Francesca Vituci
Eric O’Flynn
Ines Péric
Léa Simon
Geoffrey Ibbotson
Tsion Seyoum
Aklilu Azazh
Lemlem Beza
Ifeanyichukwu Onah
Chijioke Chukwuma
Dagim Berhanu
Jason Shenoi
Nick Sears
Yoseph Bedore
Richard Caplan
Wongel Tena Shale
invaluable
2024-10-29
Canadian journal of surgery. Journal canadien de chirurgie (published)
doi.org
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks
Thomas Schmied
Thomas Adler
Vihang P. Patil
Maximilian Beck
Korbinian Poppel
Johannes Brandstetter
Günter Klambauer
Razvan Pascanu
Sepp Hochreiter
In recent years, there has been a trend in the field of Reinforcement Learning (RL) towards large action models trained offline on large-sca… (see more)le datasets via sequence modeling. Existing models are primarily based on the Transformer architecture, which result in powerful agents. However, due to slow inference times, Transformer-based approaches are impractical for real-time applications, such as robotics. Recently, modern recurrent architectures, such as xLSTM and Mamba, have been proposed that exhibit parallelization benefits during training similar to the Transformer architecture while offering fast inference. In this work, we study the aptitude of these modern recurrent architectures for large action models. Consequently, we propose a Large Recurrent Action Model (LRAM) with an xLSTM at its core that comes with linear-time inference complexity and natural sequence length extrapolation abilities. Experiments on 432 tasks from 6 domains show that LRAM compares favorably to Transformers in terms of performance and speed.
2024-10-29
ArXiv (preprint)
doi.org
arxiv.org
The State of Data Curation at NeurIPS: An Assessment of Dataset Development Practices in the Datasets and Benchmarks Track
Eshta Bhardwaj
Harshit Gujral
Siyi Wu
Ciara Zogheib
Tegan Maharaj
Christoph Becker
Data curation is a field with origins in librarianship and archives, whose scholarship and thinking on data issues go back centuries, if not… (see more) millennia. The field of machine learning is increasingly observing the importance of data curation to the advancement of both applications and fundamental understanding of machine learning models - evidenced not least by the creation of the Datasets and Benchmarks track itself. This work provides an analysis of dataset development practices at NeurIPS through the lens of data curation. We present an evaluation framework for dataset documentation, consisting of a rubric and toolkit developed through a literature review of data curation principles. We use the framework to assess the strengths and weaknesses in current dataset development practices of 60 datasets published in the NeurIPS Datasets and Benchmarks track from 2021-2023. We summarize key findings and trends. Results indicate greater need for documentation about environmental footprint, ethical considerations, and data management. We suggest targeted strategies and resources to improve documentation in these areas and provide recommendations for the NeurIPS peer-review process that prioritize rigorous data curation in ML. Finally, we provide results in the format of a dataset that showcases aspects of recommended data curation practices. Our rubric and results are of interest for improving data curation practices broadly in the field of ML as well as to data curation and science and technology studies scholars studying practices in ML. Our aim is to support continued improvement in interdisciplinary research on dataset practices, ultimately improving the reusability and reproducibility of new datasets and benchmarks, enabling standardized and informed human oversight, and strengthening the foundation of rigorous and responsible ML research.
2024-10-29
ArXiv (preprint)
doi.org
arxiv.org
Doctoral Symposium Committee
Anthony Cleve
Christian Lange
Silvia Breu
Manar H. Alalfi
Mario Luca Bernardi
Cornelia Boldyreff
Marco D'Ambros
Simon Denier
Natalia Dragan
Ekwa Duala-Ekoko
Fausto Fasano
Adnane Ghannem
Carmine Gravino
Maen Hammad
Imed Hammouda
Salima Hassaine
Yue Jia
Zhen Ming Jiang
Foutse Khomh
Adam Kiezun … (see 11 more)
Jay Kothari
Jonathan Memaitre
Naouel Moha
Rocco Oliveto
Denys Poshyvanyk
Michele Risi
Giuseppe Scanniello
Bonita Sharif
Andrew Sutton
Anis Yousefi
Eugenio Zimeo
Manar H. Alalfi Mario Luca Bernardi Cornelia Boldyreff Anthony Cleve Marco D'Ambros Simon Denier Natalia Dragan Ekwa Duala-Ekoko Fausto Fasa… (see more)no Adnane Ghannem Carmine Gravino Maen Hammad Imed Hammouda Salima Hassaine Yue Jia Zhen Ming Jiang Foutse Khomh Adam Kiezun Jay Kothari Jonathan Memaitre Naouel Moha Rocco Oliveto Denys Poshyvanyk Michele Risi Giuseppe Scanniello Bonita Sharif Andrew Sutton Anis Yousefi Eugenio Zimeo
2024-10-28
2024 IEEE 35th International Symposium on Software Reliability Engineering Workshops (ISSREW) (published)
doi.org
Doctoral Symposium Committee
Anthony Cleve
Christian Lange
Silvia Breu
Manar H. Alalfi
Mario Luca Bernardi
Cornelia Boldyreff
Marco D'Ambros
Simon Denier
Natalia Dragan
Ekwa Duala-Ekoko
Fausto Fasano
Adnane Ghannem
Carmine Gravino
Maen Hammad
Imed Hammouda
Salima Hassaine
Yue Jia
Zhen Ming (Jack) Jiang
Foutse Khomh
Adam Kiezun … (see 11 more)
Jay Kothari
Jonathan Memaitre
Naouel Moha
Rocco Oliveto
Denys Poshyvanyk
Michele Risi
Giuseppe Scanniello
Bonita Sharif
Andrew Sutton
Anis Yousefi
Eugenio Zimeo
Manar H. Alalfi Mario Luca Bernardi Cornelia Boldyreff Anthony Cleve Marco D'Ambros Simon Denier Natalia Dragan Ekwa Duala-Ekoko Fausto Fasa… (see more)no Adnane Ghannem Carmine Gravino Maen Hammad Imed Hammouda Salima Hassaine Yue Jia Zhen Ming Jiang Foutse Khomh Adam Kiezun Jay Kothari Jonathan Memaitre Naouel Moha Rocco Oliveto Denys Poshyvanyk Michele Risi Giuseppe Scanniello Bonita Sharif Andrew Sutton Anis Yousefi Eugenio Zimeo
2024-10-28
2024 IEEE 35th International Symposium on Software Reliability Engineering Workshops (ISSREW) (published)
doi.org
ProtSCAPE: Mapping the landscape of protein conformations in molecular dynamics
Siddharth Viswanath
Dhananjay Bhaskar
David R. Johnson
João F. Rocha
Egbert Castro
Jackson Grady
Alex T. Grigas
Michael Perlmutter
Corey S. O'Hern
Smita Krishnaswamy
Understanding the dynamic nature of protein structures is essential for comprehending their biological functions. While significant progress… (see more) has been made in predicting static folded structures, modeling protein motions on microsecond to millisecond scales remains challenging. To address these challenges, we introduce a novel deep learning architecture, Protein Transformer with Scattering, Attention, and Positional Embedding (ProtSCAPE), which leverages the geometric scattering transform alongside transformer-based attention mechanisms to capture protein dynamics from molecular dynamics (MD) simulations. ProtSCAPE utilizes the multi-scale nature of the geometric scattering transform to extract features from protein structures conceptualized as graphs and integrates these features with dual attention structures that focus on residues and amino acid signals, generating latent representations of protein trajectories. Furthermore, ProtSCAPE incorporates a regression head to enforce temporally coherent latent representations.
2024-10-27
ArXiv (preprint)
doi.org
arxiv.org