Portrait of Kartikeya Badola is unavailable

Kartikeya Badola

Alumni

Publications

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 478 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
X. T. Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Alex Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Liu
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
arxiv.org
arxiv.org
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team Google Rohan Anil
Sebastian Borgeaud
Yonghui Wu
Jean-Baptiste Alayrac
Jiahui Yu
Radu Soricut
J. Schalkwyk
Andrew M. Dai
Anja Hauth
Katie Millican
David Silver
Slav Petrov
Melvin Johnson
Ioannis Antonoglou
Julian Schrittwieser
Amelia Glaese
Jilin Chen
Emily Pitler
Timothy P. Lillicrap
Angeliki Lazaridou … (see 480 more)
Orhan Firat
James L. Molloy
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Benjamin Lee
Fabio Viola
Malcolm Reynolds
Yuanzhong Xu
Ryan Doherty
Eli Collins
Clemens Meyer
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
George Tucker
Enrique Piqueras
M. Krikun
Iain Barr
Nikolay Savinov
Ivo Danihelka
Becca Roelofs
Anais White
Anders Johan Andreassen
Tamara von Glehn
Laksh-man Yagati
Mehran Kazemi
Lucas Gonzalez
Misha Khalman
Jakub Sygnowski
Alexandre Fréchette
Charlotte Smith
Laura Culp
Lev Proleev
Yi Luan
X. T. Chen
James Lottes
Nathan Schucher
Federico Lebron
Alban Rrustemi
Natalie Clay
Phil Crone
Tomas Kocisky
Jeffrey Zhao
Bartek Perz
Dian Yu
Heidi Howard
Adam E. Bloniarz
Jack W. Rae
Han Lu
Laurent Sifre
Marcello Maggioni
Fred Alcober
Dan Garrette
Megan Barnes
Shantanu Thakoor
Jacob Austin
Gabriel Barth-Maron
William Wong
Rishabh Joshi
Rahma Chaabouni
Deeni Fatiha
Arun Ahuja
Ruibo Liu
Yunxuan Li
Sarah Cogan
Jeremy Chen
Chao Jia
Chenjie Gu
Qiao Zhang
Jordan Grimstad
Ale Jakse Hartman
Martin J. Chadwick
Gaurav Singh Tomar
Xavier Garcia
Evan Senter
Emanuel Taropa
Thanumalayan Sankaranarayana Pillai
Jacob Devlin
Michael Laskin
Diego de Las Casas
Dasha Valter
Connie Tao
Lorenzo Blanco
Adrià Puigdomènech Badia
David Reitter
Mianna Chen
Jenny Brennan
Clara E. Rivera
Sergey Brin
Shariq Iqbal
Gabriela Surita
Jane Labanowski
Abhishek Rao
Stephanie Winkler
Emilio Parisotto
Yiming Gu
Kate Olszewska
Yujing Zhang
Ravichandra Addanki
Antoine Miech
Annie Louis
Laurent El Shafey
Denis Teplyashin
Geoff Brown
Elliot Catt
Nithya Attaluri
Jan Balaguer
Jackie Xiang
Pidong Wang
Zoe Ashwood
Anton Briukhov
Alex Webson
Sanjay Ganapathy
Smit Sanghavi
Ajay Kannan
Ming-Wei Chang
Axel Stjerngren
Josip Djolonga
Yuting Sun
Ankur Bapna
Matthew Aitchison
Pedram Pejman
Henryk Michalewski
Tianhe Yu
Cindy Wang
J Christopher Love
Junwhan Ahn
Dawn Bloxwich
Kehang Han
Peter Conway Humphreys
Thibault Sellam
James Bradbury
Varun Godbole
Sina Samangooei
Bogdan Damoc
Alex Kaskasoli
S'ebastien M. R. Arnold
Vijay Vasudevan
Shubham Agrawal
Jason Riesa
Dmitry Lepikhin
Richard Tanburn
Srivatsan Srinivasan
Hyeontaek Lim
Sarah Hodkinson
Pranav Shyam
Johan Ferret
Steven Hand
Ankush Garg
T. Paine
Jian Li
Yujia Li
Minh Giang
Alexander Neitz
Zaheer Abbas
Sarah York
Machel Reid
Elizabeth Cole
Aakanksha Chowdhery
Dipanjan Das
Dominika Rogozi'nska
Vitaly Nikolaev
Pablo G. Sprechmann
Zachary Nado
Lukáš Žilka
Flavien Prost
Luheng He
Marianne Monteiro
Gaurav Mishra
Christoper A. Welty
Joshua Newlan
Dawei Jia
Miltiadis Allamanis
Clara Huiyi Hu
Raoul de Liedekerke
Justin Gilmer
Carl Saroufim
Shruti Rijhwani
Shaobo Hou
Disha Shrivastava
Anirudh Baddepudi
Alex Goldin
Adnan Ozturel
Albin Cassirer
Yunhan Xu
Daniel Sohn
Devendra Singh Sachan
Reinald Kim Amplayo
Craig Swanson
Dessie Petrova
Shashi Narayan
Arthur Guez
Siddhartha Brahma
Jessica Landon
Miteyan Patel
Ruizhe Zhao
Kevin Villela
Luyu Wang
Wenhao Jia
Matthew Rahtz
Mai Gim'enez
Legg Yeung
Hanzhao Lin
James Keeling
Petko Georgiev
Diana Mincu
Boxi Wu
Salem Haykal
Rachel Saputro
Kiran N. Vodrahalli
James Qin
Zeynep Cankara
Abhanshu Sharma
Nicholas Fernando
Will Hawkins
Behnam Neyshabur
Solomon Kim
Adrian Hutter
Priyanka Agrawal
Alex Castro-Ros
George van den Driessche
Tao Wang
Fan Yang
Shuo-yiin Chang
Paul Komarek
Ross McIlroy
Mario Luvci'c
Guodong Zhang
Wael Farhan
Michael Sharman
Paul Natsev
Paul Michel
Yong Cheng
Yamini Bansal
Siyuan Qiao
Kris Cao
Siamak Shakeri
Christina Butterfield
Justin Chung
Paul Kishan Rubenstein
Shivani Agrawal
Arthur Mensch
Kedar Soparkar
Karel Lenc
Timothy Chung
Aedan Pope
Lorenzo Maggiore
Jackie Kay
Priya Jhakra
Shibo Wang
Joshua Maynez
Mary Phuong
Taylor Tobin
Andrea Tacchetti
Maja Trebacz
Kevin Robinson
Yash Katariya
Sebastian Riedel
Paige Bailey
Kefan Xiao
Nimesh Ghelani
Lora Aroyo
Ambrose Slone
Neil Houlsby
Xuehan Xiong
Zhen Yang
Elena Gribovskaya
Jonas Adler
Mateo Wirth
Lisa Lee
Music Li
Thais Kagohara
Jay Pavagadhi
Sophie Bridgers
Anna Bortsova
Sanjay Ghemawat
Zafarali Ahmed
Tianqi Liu
Richard Powell
Vijay Bolina
Mariko Iinuma
Polina Zablotskaia
James Besley
Da-Woon Chung
Timothy Dozat
Ramona Comanescu
Xiance Si
Jeremy Greer
Guolong Su
M. Polacek
Raphael Lopez Kaufman
Simon Tokumine
Hexiang Hu
Elena Buchatskaya
Yingjie Miao
Mohamed Elhawaty
Aditya Siddhant
Nenad Tomasev
Jinwei Xing
Christina Greer
Helen Miller
Shereen Ashraf
Aurko Roy
Zizhao Zhang
Ada Ma
Angelos Filos
Milos Besta
Rory Blevins
Ted Klimenko
Chih-Kuan Yeh
Soravit Changpinyo
Jiaqi Mu
Oscar Chang
Mantas Pajarskas
Carrie Muir
Vered Cohen
Charline Le Lan
Krishna S Haridasan
Amit Marathe
Steven Hansen
Sholto Douglas
Rajkumar Samuel
Mingqiu Wang
Sophia Austin
Chang Lan
Jiepu Jiang
Justin Chiu
Jaime Alonso Lorenzo
Lars Lowe Sjosund
S'ebastien Cevey
Zach Gleicher
Thi Avrahami
Anudhyan Boral
Hansa Srinivasan
Vittorio Selo
Rhys May
Konstantinos Aisopos
L'eonard Hussenot
Livio Baldini Soares
Kate Baumli
Michael B. Chang
Adria Recasens
Benjamin Caine
Alexander Pritzel
Filip Pavetic
Fabio Pardo
Anita Gergely
Justin Frye
Vinay Venkatesh Ramasesh
Dan Horgan
Kartikeya Badola
Nora Kassner
Subhrajit Roy
Ethan Dyer
V'ictor Campos
Alex Tomala
Yunhao Tang
Dalia El Badawy
Elspeth White
Basil Mustafa
Oran Lang
Abhishek Jindal
Sharad Mandyam Vikram
Zhitao Gong
Sergi Caelles
Ross Hemsley
Gregory Thornton
Fangxiaoyu Feng
Wojciech Stokowiec
Ce Zheng
Phoebe Thacker
cCauglar Unlu
Zhishuai Zhang
Mohammad Saleh
James Svensson
Maxwell L. Bileschi
Piyush Patil
Ankesh Anand
Roman Ring
Katerina Tsihlas
Arpi Vezer
Marco Selvi
Toby Shevlane
Mikel Rodriguez
Tom Kwiatkowski
Samira Daruki
Keran Rong
Allan Dafoe
Nicholas Fitzgerald
Keren Gu-Lemberg
Mina Khan
Lisa Anne Hendricks
Marie Pellat
Vladimir Feinberg
James Cobon-Kerr
Tara N. Sainath
Maribeth Rauh
Sayed Hadi Hashemi
Richard Ives
Yana Hasson
YaGuang Li
Eric Noland
Yuan Cao
Nathan Byrd
Le Hou
Qingze Wang
Thibault Sottiaux
Michela Paganini
Jean-Baptiste Lespiau
Alexandre Moufarek
Samer Hassan
Kaushik Shivakumar
Joost Van Amersfoort
Amol Mandhane
Pratik M. Joshi
Anirudh Goyal
Matthew Tung
Andy Brock
Hannah Rachel Sheahan
Vedant Misra
Cheng Li
Nemanja Raki'cevi'c
Mostafa Dehghani
Fangyu Liu
Sid Mittal
Junhyuk Oh
Seb Noury
Eren Sezener
Fantine Huot
Matthew Lamm
Nicola De Cao
Charlie Chen
Gamaleldin Elsayed
Ed Huai-hsin Chi
Mahdis Mahdieh
Ian F. Tenney
Nan Hua
Ivan Petrychenko
Patrick Kane
Dylan Scandinaro
Rishub Jain
Jonathan Uesato
Romina Datta
Adam Sadovsky
Oskar Bunyan
Dominik Rabiej
Shimu Wu
John Zhang
Gautam Vasudevan
Edouard Leurent
Mahmoud Alnahlawi
Ionut-Razvan Georgescu
Nan Wei
Ivy Zheng
Betty Chan
Pam G Rabinovitch
Piotr Stańczyk
Ye Zhang
David Steiner
Subhajit Naskar
Michael Azzam
Matthew Johnson
Adam Paszke
Chung-Cheng Chiu
Jaume Sanchez Elias
Afroz Mohiuddin
Faizan Muhammad
Jin Miao
Andrew Lee
Nino Vieillard
Sahitya Potluri
Jane Park
Elnaz Davoodi
Jiageng Zhang
Jeff Stanway
Drew Garmon
Abhijit Karmarkar
Zhe Dong
2023-12-19
ArXiv (preprint)
doi.org
arxiv.org
Coordination Among Neural Modules Through a Shared Global Workspace
Anirudh Goyal
Aniket Rajiv Didolkar
Alex Lamb
Kartikeya Badola
Nan Rosemary Ke
Nasim Rahaman
Jonathan Binas
Charles Blundell
Michael Curtis Mozer
Yoshua Bengio
Deep learning has seen a movement away from representing examples with a monolithic hidden state towards a richly structured state. For exam… (see more)ple, Transformers segment by position, and object-centric architectures decompose images into entities. In all these architectures, interactions between different elements are modeled via pairwise interactions: Transformers make use of self-attention to incorporate information from other positions and object-centric architectures make use of graph neural networks to model interactions among entities. We consider how to improve on pairwise interactions in terms of global coordination and a coherent, integrated representation that can be used for downstream tasks. In cognitive science, a global workspace architecture has been proposed in which functionally specialized components share information through a common, bandwidth-limited communication channel. We explore the use of such a communication channel in the context of deep learning for modeling the structure of complex environments. The proposed method includes a shared workspace through which communication among different specialist modules takes place but due to limits on the communication bandwidth, specialist modules must compete for access. We show that capacity limitations have a rational basis in that (1) they encourage specialization and compositionality and (2) they facilitate the synchronization of otherwise independent specialists.
2022-01-28
ICLR.cc/2022/Conference (oral)
openreview.net
openreview.net