Portrait of Kartikeya Badola is unavailable

Kartikeya Badola

Alumni

Publications

Generating Complex Question Decompositions in the Face of Distribution Shifts.
Kelvin Han
Claire Gardent
Marah Ihab Abdin
Jyoti Aneja
Hany Hassan Awadalla
Ammar Ahmed Awadallah
Ahmad Awan
Nguyen Bach
Amit Bahree
Arash Bakhtiari
Jianmin Bao
Harkirat Singh Behl
Alon Benhaim
Misha Bilenko
Johan Bjorck
Sébastien Bubeck
Martin Cai
Qin Cai
Vishrav Chaudhary
Dong Chen … (see 342 more)
Weizhu Chen
Yen-Chun Chen 0001
Yi-ling Chen
Hao Cheng
Parul Chopra
Xiyang Dai
Matthew Dixon
Ronen Eldan
Victor Fragoso
Jianfeng Gao
Mei Gao
Min Gao
Amit Garg
Allison Del Giorno
Abhishek Goswami
S. Gunasekar
Emman Haider
Jun-heng Hao
Russell J. Hewett
Wen-Wei Hu
Jamie Huynh
Dan Iter
Sam Ade Jacobs
Mojan Javaheripi
Xin Jin
Nikos Karampatziakis
Piero Kauffmann
Mahoud Khademi
Dongwoo Kim
Young Jin Kim
Lev Kurilenko
James R. Lee
Yin Tat Lee
Yuanzhi Li
Yunsheng Li
Chen Liang
Lars Lidén
Xihui
Zeqi Lin
Ce Lin
Liyuan Liu
Mengchen Liu
Liu Weishung
Xiaodong Liu
Chong Liu
Piyush Luo
Ali Madan
David Mahmoudzadeh
Matt Majercak
Caio Mazzola
César Teodoro
Arindam Mendes
Hardik Mitra
Anh Modi
Brandon Nguyen
Norick Barun
Daniel Patra
Thomas Perez-Becker
Portet Reid
Heyang Pryzant
Marko Qin
Liliang Radmilac
Gustavo Ren
Corby de Rosa
Sambudha Rosset
Roy Olatunji
Olli Ruwase
Amin Saarikivi
Adil Saied
Michael Salim
Shital Santacroce
Ning Shah
Shang Hiteshi
Yelong Sharma
Swadheen Shen
Xia Shukla
Masahiro Song
Andrea Tanaka
Praneetha Tupini
Michael Wu
Bin Wyatt
Can Xiao
Jiahang Xu
Weijiang Xu
Jilong Xu
Sonali Xue
Fan Yadav
Jianwei Yang
Yifan Yang
Ziyi Yang
Donghan Yang
Yu Lu
Chenruidong Yuan
Cyril Zhang
Jianwen Zhang
Zhang
Li Lyna
Yi Zhang
Yue Zhang
Yunan Zhang 0001
Zhang Xiren
Zhou
Phi-3
Priyanka Agrawal
Chris Alberti
Fantine Huot
Joshua Maynez
Ji Ma
Kuzman Ganchev
Viraat Aryabumi
John Dang
Dwarak Talupuru
Saurabh Dash
David Cairuz
Hangyu Lin
Bharat Venkitesh
Madeline Smith
Jon Ander Campos
Yi Chern Tan
Kelly Marchisio
Max Bartolo
Sebastian Ruder
Acyr F. Locatelli
Julia Kreutzer
Nick Frosst
Aidan Gomez
Phil Blunsom
Marzieh Fadaee
Tom B. Brown
Benjamin Mann
Nick Ryder
Melanie Subbiah
Jared Kaplan
Prafulla Dhariwal
Arvind Neelakantan
Pranav Shyam
Girish Sastry
Amanda Askell
Sandhini Agarwal
Ariel Herbert-Voss
Gretchen Krueger
T. Henighan
Rewon Child
Aditya Ramesh
Daniel M. Ziegler
Jeffrey Wu
Clemens Winter
Chris Hesse
Mark Chen
Eric Sigler
Ma-teusz Litwin
Scott Gray
Benjamin Chess
J. Clark
Christopher Berner
Sam McCandlish
Alec Radford
Ilya Sutskever
Dario Amodei Gemma Team
Morgane Rivière
Shreya Pathak Pier
Giuseppe Sessa
Cassidy Hardin
Surya Bhupati-raju
L'eonard Hussenot
Thomas Mesnard
Bobak Shahriari
Alexandre Ramé
Johan Ferret
Peter Liu
Pouya Dehghani Tafti
Abe Friesen
Michelle Casbon
Sabela Ramos
Ravin Kumar
Charline Le Lan
Sammy Jerome
Anton Tsitsulin
Nino Vieillard
Piotr Stańczyk
Sertan Girgin
Nikola Momchev
Matt Hoffman
Shantanu Thakoor
Jean-Bastien Grill
Behnam Neyshabur
Olivier Bachem
Alanna Wal-ton
Aliaksei Severyn
Alicia Parrish
Aliya Ah-mad
Allen Hutchison
Alvin Abdagic
Amanda Carl
Amy Shen
Andy Brock
Andy Coenen
Anthony Laforge
Antonia Paterson
Ben Bastian
Bilal Piot
Boxi Wu
Brandon Royal
Charlie Chen
Chintu Kumar
Chris Perry
Christoper A. Welty
Christopher A. Choquette-Choo
Danila Sinopalnikov
David Wein-berger
Dimple Vijaykumar
Dominika Rogozi´nska
D. Herbison
Elisa Bandy
Emma Wang
Eric Noland
Erica Moreira
Evan Senter
Evgenii Elty-shev
Francesco Visin
Gabriel Rasskin
Gary Wei
Glenn Cameron
Gus Martins
Hadi Hashemi
Hanna Klimczak-Pluci´nska
Harleen Batra
Harsh Dhand
Ivan Nardini
Jacinda Mein
Jack Zhou
James Svens-son
Jeff Stanway
Jetha Chan
J. Zhou
Joana Carrasqueira
Joana Iljazi
Jocelyn Becker
Joe Fer-nandez
Joost Van Amersfoort
Josh Gordon
Josh Lipschultz
Joshua Newlan
Junsong Ji
Kareem Mo-hamed
Kartikeya Badola
Kat Black
Katie Mil-lican
Keelin McDonell
Kelvin Nguyen
Kiranbir Sodhia
Kish Greene
Lars Lowe Sjoesund
Lauren Usui
Laurent Sifre
L. Heuermann
Leti-cia Lago
Lilly McNealus
Livio Baldini
Soares Logan
Lucas Kilpatrick
Luciano Dixon
Martins Machel
Manvinder Reid
Mark Singh
Martin Görner Iverson
Mateo Wirth Mat Velloso
Matt Davi-dow
Matt Miller
Matthew Rahtz
Matthew Watson
Meg Risdal
Mehran Kazemi
Michael Moynihan
Ming Zhang
Minsuk Kahng
Minwoo Park
Mofi Rahman
Mohit Khatwani
Natalie Dao
Nenshad Bardoliwalla
N. Devanathan
Neta Dumai
Nilay Chauhan
O. Wahltinez
Pankil Botarda
Parker Barnes
Paul R. Barham
Paul Michel
Peng-chong Jin
Petko Georgiev
Phil Culliton
Pradeep Kup-pala
Ramona Comanescu
Ramona Merhej
Reena Jana
R. Rokni
Rishabh Agarwal
Ryan Mullins
Samaneh Saadat
S. M. Carthy
Sarah Cogan
Sarah Perrin
S'ebastien M. R. Arnold
Se-bastian Krause
Shengyang Dai
S. Garg
Shruti Sheth
S. Ronstrom
Susan Chan
Timothy Jordan
Bing Yu
Tom Eccles
Tom Hennigan
Tomas Kocisky
Tulsee Doshi
Vihan Jain
Vikas Yadav
Vilobh Meshram
Vishal Dharmadhikari
Warren Barkley
Wei Wei
Wenming Ye
Woohyun Han
Woosuk Kwon
Xiang Xu
Zhe Shen
Zhitao Gong
Zichuan Wei
Victor Cotruta
Phoebe Kirk
Anand Rao
Minh Giang
Ludovic Peran
Tris Brian Warkentin
Eli Collins
Joelle Barral
Zoubin Ghahramani
Raia Hadsell
D. Sculley
Jeanine Banks
Anca Dragan
2025-01-01
NAACL (Long Papers) (published)
doi.org
Plot Twist: Multimodal Models Don’t Comprehend Simple Chart Details
Yasaman Razeghi
Ishita Dasgupta
Fangyu Liu
Vinay Venkatesh Ramasesh
Sameer Singh
Sebastian Mustafa
Ibrahim Goodman
Piotr Alabdul-mohsin
Daniel Padlewski
Xi Salz
Xiong Daniel
Filip Vlasic
Keran Pavetic
Tianli Rong
Yu
Wenliang Dai
Junnan Li
Dongxu Li
Anthony Meng
Huat Tiong … (see 480 more)
Junqi Zhao
Weisheng Wang
Bo Li
Pascale Fung
Chaoyou Fu
Pei-Chun Chen
Yunhang Shen
Yulei Qin
Mengdan Zhang
Xu Lin
Jinrui Yang
Xiawu Zheng
Rohan Anil
Sebastian Borgeaud
Yonghui Wu
Jean-Baptiste Alayrac
Jiahui Yu
Radu Soricut
J. Schalkwyk
Andrew M. Dai
A.E. Hauth
Katie Millican
David Silver
Slav Petrov
Melvin Johnson
Ioannis Antonoglou
Julian Schrit-twieser
Amelia Glaese
Jilin Chen
Emily Pitler
Timothy P Lillicrap
Angeliki Lazaridou
Orhan Fi-rat
James L. Molloy
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Benjamin Lee
Fabio Viola
Malcolm Reynolds
Yuanzhong Xu
Ryan Doherty
Eli Collins
Clemens Meyer
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
George Tucker
Enrique Piqueras
M. Krikun
Iain Barr
Nikolay Savinov
Ivo Danihelka
Becca Roelofs
Anais White
Anders Johan Andreassen
Tamara von Glehn
Laksh-man Yagati
Mehran Kazemi
Lucas Gonzalez
Misha Khalman
Jakub Sygnowski
Alexandre Fréchette
Charlotte Smith
Laura Culp
Lev Proleev
Yi Luan
Xi Chen
James Lottes
Nathan Schucher
Federico Lebron
Alban Rrustemi
Natalie Clay
Phil Crone
Tomas Kocisky
Jeffrey Zhao
Bartek Perz
Dian Yu
Heidi Howard
Adam E. Bloniarz
Jack W. Rae
Han Lu
Laurent Sifre
Marcello Maggioni
Fred Alcober
Dan Garrette
Megan Barnes
Shantanu Thakoor
Jacob Austin
Gabriel Barth-Maron
William Wong
Rishabh Joshi
Rahma Chaabouni
Deeni Fatiha
Arun Ahuja
Ruibo Liu
Yunxuan Li
Sarah Cogan
Jeremy Chen
Chao Jia
Chenjie Gu
Qiao Zhang
Ale Jordan Grimstad
Jakse Hartman
Martin Chad-wick
Gaurav Singh Tomar
Xavier Garcia
Evan Senter
Emanuel Taropa
Thanumalayan Sankaranarayana Pillai
Jacob Devlin
Michael Laskin
Diego de
Las Casas
Dasha Valter
Connie Tao
Lorenzo Blanco
Adrià Puigdomènech Badia
David Reitter
Mianna Chen
Jenny Brennan
Clara E. Rivera
Sergey Brin
Shariq N Iqbal
Gabriela Surita
Jane Labanowski
Abhishek Rao
Stephanie Winkler
Emilio Parisotto
Yiming Gu
Kate Olszewska
Yujing Zhang
Ravi Ad-danki
Antoine Miech
Annie Louis
Laurent El
Denis Teplyashin
Geoff Brown
Elliot Catt
Nithya Attaluri
Jan Balaguer
Jackie Xiang
Pidong Wang
Zoe C. Ashwood
Anton Briukhov
Albert Webson
Sanjay Ganapathy
Smit Sanghavi
Ajay Kannan
Ming-Wei Chang
Axel Stjerngren
Josip Djolonga
Yuting Sun
Ankur Bapna
Matthew Aitchison
Pedram Pejman
Henryk Michalewski
Tianhe Yu
Cindy Wang
J Christopher Love
Junwhan Ahn
Dawn Bloxwich
Kehang Han
Peter Conway Humphreys
Thibault Sellam
James Bradbury
Varun Godbole
Sina Samangooei
Bogdan Damoc
Alex Kaskasoli
S'ebastien M. R. Arnold
Vijay Vasudevan
Shubham Agrawal
Jason Riesa
Dmitry Lepikhin
Richard Tan-burn
Srivatsan Srinivasan
Hyeontaek Lim
Sarah Hodkinson
Pranav Shyam
Johan Ferret
Steven Hand
Ankush Garg
T. Paine
Jian Li
Yu-jia Li
Minh Giang
Alexander Neitz
Zaheer Abbas
Sarah York
Machel Reid
Elizabeth Cole
Aakanksha Chowdhery
Dipanjan Das
Dominika Rogozi´nska
Vitaly Nikolaev
Pablo G. Sprechmann
Zachary Nado
Lukáš Žilka
Flavien Prost
Luheng He
Marianne Monteiro
Gaurav Mishra
Christoper A. Welty
Joshua Newlan
Dawei Jia
Miltiadis Allamanis
Clara Huiyi Hu
Raoul de Liedekerke
Justin Gilmer
Carl Saroufim
Shruti Rijhwani
Shaobo Hou
Disha Shrivastava
Anirudh Baddepudi
Alex Goldin
Adnan Ozturel
Albin Cassirer
Yunhan Xu
Daniel Sohn
Deven-dra Sachan
Reinald Kim Amplayo
Craig Swan-son
Dessie Petrova
Shashi Narayan
Arthur Guez
Siddhartha Brahma
Jessica Landon
Miteyan Patel
Ruizhe Zhao
Kevin Villela
Luyu Wang
Wenhao Jia
Matthew Rahtz
Mai Giménez
Legg Yeung
Hanzhao Lin
James Keeling
Petko Georgiev
Diana Mincu
Boxi Wu
Salem Haykal
Rachel Sapu-tro
Kiran N. Vodrahalli
James Qin
Zeynep Cankara
Abhanshu Sharma
Nicholas Fernando
Will Hawkins
Behnam Neyshabur
Solomon Kim
Adrian Hut-ter
Priyanka Agrawal
Alex Castro-Ros
George van den Driessche
Tao Wang
Fan Yang
Shuo yiin
Paul Chang
Ross Komarek
Mario McIlroy
Luˇci´c Guodong
Wael Zhang
Michael Farhan
Sharman Paul
Paul Natsev
Yong Michel
Yamini Cheng
Siyuan Bansal
Kris Qiao
Siamak Cao
Shakeri Christina
Justin Butterfield
Paul Kishan Chung
Shivani Rubenstein
Arthur Agrawal
Kedar Mensch
Karel Soparkar
Timothy Lenc
Aedan Chung
Pope Loren
Jackie Maggiore
Priya Kay
Shibo Jhakra
Joshua Wang
Mary Maynez
Taylor Phuong
Tobin Andrea
Maja Tacchetti
Kevin Trebacz
Robinson Yash
Sebas-tian Katariya
Paige Riedel
Ke-fan Bailey
Nimesh Xiao
Lora Ghelani
Ambrose Aroyo
Neil Slone
Xuehan Houlsby
Zhen Xiong
Yang Elena
Jonas Gribovskaya
Mateo Adler
Lisa Wirth
M. Lee
Jay Kagohara
So-phie Pavagadhi
Anna Bridgers
Sanjay Bortsova
Ghemawat Zafarali
Tianqi Ahmed
Richard Liu
Vijay Powell
Mariko Bolina
Polina Iinuma
James Zablotskaia
Da-Woon Besley
Timothy Chung
Ramona Dozat
Xi-ance Comanescu
Jeremy Si
Guolong Greer
Su Martin
Raphaël Lopez Polacek
Simon Kaufman
Hex-iang Tokumine
Elena Hu
Yingjie Buchatskaya
Mo-hamed Miao
Aditya Elhawaty
Nenad Siddhant
Jinwei Tomasev
Christina Xing
Helen Greer
Miller Shereen
Aurko Ashraf
Zizhao Roy
Ada Zhang
Mathew Angelos
Milos Filos
Rory Besta
Ted Blevins
Kli-5927 Chih-Kuan
Soravit Yeh
Jiaqi Changpinyo
Oscar Mu
Mantas Chang
Carrie Pajarskas
Muir Vered
Charline Cohen
Krishna Le Lan
Haridasan Amit
Steven Marathe
Sholto Hansen
Ra-jkumar Douglas
Mingqiu Samuel
Sophia Wang
Austin Chang
Jiepu Lan
Justin Jiang
Jaime Alonso Chiu
Lars Lowe Lorenzo
Sébastien Sjösund
Cevey Zach
Thi Gleicher
Anudhyan Avrahami
Boral Hansa
Vittorio Srinivasan
Rhys Selo
Kon-stantinos May
Léonard Aisopos
Hussenot
Livio Baldini
Kate Soares
Michael B Baumli
Adrià Chang
Ben Caine
Alexander Pritzel
Filip Pavetic
Fabio Pardo
Anita Gergely
Justin Frye
Dan Horgan
Kartikeya Badola
Nora Kassner
Subhrajit Roy
Víctor Ethan Dyer
Alex Tomala
Yunhao Tang
Dalia El Badawy
Elspeth White
Basil Mustafa
Oran Lang
Abhishek Jindal
Sharad Mandyam Vikram
Zhitao Gong
Sergi Caelles
Ross Hemsley
Gregory Thornton
Fangxiaoyu Feng
Wojciech Stokowiec
Ce Zheng
Phoebe Thacker
Ça˘glar Ünlü
Zhishuai Zhang
Mohammad Saleh
James Svensson
Maxwell Bileschi
Piyush Patil
Ankesh Anand
Roman Ring
Katerina Tsihlas
Arpi Vezer
Marco Selvi
Toby Shevlane
Mikel Ro-driguez
Tom Kwiatkowski
Samira Daruki
Keran Rong
Allan Dafoe
Nicholas Fitzgerald
Keren Gu-Lemberg
Mina Khan
Lisa Anne Hendricks
Marie Pellat
Vladimir Feinberg
James Cobon-Kerr
Tara N. Sainath
Maribeth Rauh
Sayed Hadi
Richard Hashemi
Yana Ives
YaGuang Hasson
Eric Li
Yuan Noland
Nathan Cao
Le Byrd
Qingze Hou
Thibault Wang
Michela Sottiaux
Paganini Jean-Baptiste
Alexandre Lespiau
Samer Moufarek
Kaushik Hassan
Joost Shivakumar
Amol van Amers-foort
Pratik Mandhane
Anirudh Joshi
Matthew Goyal
Andrew Tung
Hannah Brock
Vedant Shea-han
Cheng Misra
Nemanja Li
Raki´cevi´c Mostafa
Fangyu Dehghani
Sid Liu
Junhyuk Mittal
Seb Oh
Eren Noury
Fantine Sezener
Matthew Huot
Nicola Lamm
Charlie De Cao
Gamaleldin Chen
Ed Elsayed
Mahdis Chi
Ian Mahdieh
Nan Tenney
Ivan Hua
Patrick Petrychenko
Dylan Kane
Rishub Scand-inaro
Jonathan Jain
Romina Uesato
Datta Adam
Oskar Sadovsky
Dominik Bunyan
Rabiej Shimu
,
2024-11-01
Findings of the Association for Computational Linguistics: EMNLP 2024 (published)
doi.org
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team Morgane Riviere
Shreya Pathak
Pier Giuseppe Sessa
Cassidy Hardin
Surya Bhupatiraju
L'eonard Hussenot
Thomas Mesnard
Bobak Shahriari
Alexandre Ram'e
Johan Ferret
Peter Liu
Pouya Dehghani Tafti
Abe Friesen
Michelle Casbon
Sabela Ramos
Ravin Kumar
Charline Le Lan
Sammy Jerome
Anton Tsitsulin
Nino Vieillard … (see 175 more)
Piotr Stańczyk
Sertan Girgin
Nikola Momchev
Matt Hoffman
Shantanu Thakoor
Jean-Bastien Grill
Behnam Neyshabur
Alanna Walton
Aliaksei Severyn
Alicia Parrish
Aliya Ahmad
Allen Hutchison
Alvin Abdagic
Amanda Carl
Amy Shen
Andy Brock
Andy Coenen
Anthony Laforge
Antonia Paterson
Ben Bastian
Bilal Piot
Boxi Wu
Brandon Royal
Charlie Chen
Chintu Kumar
Chris Perry
Christoper A. Welty
Christopher A. Choquette-Choo
Danila Sinopalnikov
David Weinberger
Dimple Vijaykumar
Dominika Rogozi'nska
D. Herbison
Elisa Bandy
Emma Wang
Eric Noland
Erica Moreira
Evan Senter
Evgenii Eltyshev
Francesco Visin
Gabriel Rasskin
Gary Wei
Glenn Cameron
Gus Martins
Hadi Hashemi
Hanna Klimczak-Pluci'nska
Harleen Batra
Harsh Dhand
Ivan Nardini
Jacinda Mein
Jack Zhou
James Svensson
Jeff Stanway
Jetha Chan
Jin Zhou
Joana Carrasqueira
Joana Iljazi
Jocelyn Becker
Joe Fernandez
Joost Van Amersfoort
Josh Gordon
Josh Lipschultz
Joshua Newlan
Junsong Ji
Kareem Mohamed
Kartikeya Badola
Kat Black
Katie Millican
Keelin McDonell
Kelvin Nguyen
Kiranbir Sodhia
Kish Greene
Lars Lowe Sjoesund
Lauren Usui
Laurent Sifre
L. Heuermann
Leti-cia Lago
Lilly McNealus
Livio Baldini Soares
Logan Kilpatrick
Lucas Dixon
Luciano Martins
Machel Reid
Manvinder Singh
Mark Iverson
Martin Gorner
Mat Velloso
Mateo Wirth
Matt Davidow
Matt Miller
Matthew Rahtz
Matthew Watson
Meg Risdal
Mehran Kazemi
Michael Moynihan
Ming Zhang
Minsuk Kahng
Minwoo Park
Mofi Rahman
Mohit Khatwani
Natalie Dao
Nenshad Bardoliwalla
N. Devanathan
Neta Dumai
Nilay Chauhan
O. Wahltinez
Pankil Botarda
Parker Barnes
Paul R. Barham
Paul Michel
Peng-chong Jin
Petko Georgiev
Phil Culliton
Pradeep Kuppala
Ramona Comanescu
Ramona Merhej
Reena Jana
R. Rokni
Rishabh Agarwal
Ryan Mullins
Samaneh Saadat
S. M. Carthy
Sarah Perrin
S'ebastien M. R. Arnold
Se-bastian Krause
Shengyang Dai
S. Garg
Shruti Sheth
S. Ronstrom
Susan Chan
Timothy Jordan
Ting Yu
Tom Eccles
Tom Hennigan
Tomas Kocisky
Tulsee Doshi
Vihan Jain
Vikas Yadav
Vilobh Meshram
Vishal Dharmadhikari
Warren Barkley
Wei Wei
Wenming Ye
Woohyun Han
Woosuk Kwon
Xiang Xu
Zhe Shen
Zhitao Gong
Zichuan Wei
Victor Cotruta
Phoebe Kirk
Anand Rao
Minh Giang
Ludovic Peran
Tris Brian Warkentin
Eli Collins
Joelle Barral
Zoubin Ghahramani
Raia Hadsell
D. Sculley
Jeanine Banks
Anca Dragan
Slav Petrov
Oriol Vinyals
Jeffrey Dean
Demis Hassabis
Koray Kavukcuoglu
Clément Farabet
Elena Buchatskaya
Sebastian Borgeaud
Noah Fiedel
Armand Joulin
Kathleen Kenealy
Robert Dadashi
Alek Andreev
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2… (see more) billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.
2024-07-31
ArXiv (preprint)
doi.org
arxiv.org
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team Morgane Riviere
Shreya Pathak
Pier Giuseppe Sessa
Cassidy Hardin
Surya Bhupatiraju
L'eonard Hussenot
Thomas Mesnard
Bobak Shahriari
Alexandre Ram'e
Johan Ferret
Peter Liu
Pouya Dehghani Tafti
Abe Friesen
Michelle Casbon
Sabela Ramos
Ravin Kumar
Charline Le Lan
Sammy Jerome
Anton Tsitsulin
Nino Vieillard … (see 175 more)
Piotr Stańczyk
Sertan Girgin
Nikola Momchev
Matt Hoffman
Shantanu Thakoor
Jean-Bastien Grill
Behnam Neyshabur
Alanna Walton
Aliaksei Severyn
Alicia Parrish
Aliya Ahmad
Allen Hutchison
Alvin Abdagic
Amanda Carl
Amy Shen
Andy Brock
Andy Coenen
Anthony Laforge
Antonia Paterson
Ben Bastian
Bilal Piot
Boxi Wu
Brandon Royal
Charlie Chen
Chintu Kumar
Chris Perry
Christoper A. Welty
Christopher A. Choquette-Choo
Danila Sinopalnikov
David Weinberger
Dimple Vijaykumar
Dominika Rogozi'nska
D. Herbison
Elisa Bandy
Emma Wang
Eric Noland
Erica Moreira
Evan Senter
Evgenii Eltyshev
Francesco Visin
Gabriel Rasskin
Gary Wei
Glenn Cameron
Gus Martins
Hadi Hashemi
Hanna Klimczak-Pluci'nska
Harleen Batra
Harsh Dhand
Ivan Nardini
Jacinda Mein
Jack Zhou
James Svensson
Jeff Stanway
Jetha Chan
Jin Zhou
Joana Carrasqueira
Joana Iljazi
Jocelyn Becker
Joe Fernandez
Joost Van Amersfoort
Josh Gordon
Josh Lipschultz
Joshua Newlan
Junsong Ji
Kareem Mohamed
Kartikeya Badola
Kat Black
Katie Millican
Keelin McDonell
Kelvin Nguyen
Kiranbir Sodhia
Kish Greene
Lars Lowe Sjoesund
Lauren Usui
Laurent Sifre
L. Heuermann
Leti-cia Lago
Lilly McNealus
Livio Baldini Soares
Logan Kilpatrick
Lucas Dixon
Luciano Martins
Machel Reid
Manvinder Singh
Mark Iverson
Martin Gorner
Mat Velloso
Mateo Wirth
Matt Davidow
Matt Miller
Matthew Rahtz
Matthew Watson
Meg Risdal
Mehran Kazemi
Michael Moynihan
Ming Zhang
Minsuk Kahng
Minwoo Park
Mofi Rahman
Mohit Khatwani
Natalie Dao
Nenshad Bardoliwalla
N. Devanathan
Neta Dumai
Nilay Chauhan
O. Wahltinez
Pankil Botarda
Parker Barnes
Paul R. Barham
Paul Michel
Peng-chong Jin
Petko Georgiev
Phil Culliton
Pradeep Kuppala
Ramona Comanescu
Ramona Merhej
Reena Jana
R. Rokni
Rishabh Agarwal
Ryan Mullins
Samaneh Saadat
S. M. Carthy
Sarah Perrin
S'ebastien M. R. Arnold
Se-bastian Krause
Shengyang Dai
S. Garg
Shruti Sheth
S. Ronstrom
Susan Chan
Timothy Jordan
Bing Yu
Tom Eccles
Tom Hennigan
Tomas Kocisky
Tulsee Doshi
Vihan Jain
Vikas Yadav
Vilobh Meshram
Vishal Dharmadhikari
Warren Barkley
Wei Wei
Wenming Ye
Woohyun Han
Woosuk Kwon
Xiang Xu
Zhe Shen
Zhitao Gong
Zichuan Wei
Victor Cotruta
Phoebe Kirk
Anand Rao
Minh Giang
Ludovic Peran
Tris Brian Warkentin
Eli Collins
Joelle Barral
Zoubin Ghahramani
Raia Hadsell
D. Sculley
Jeanine Banks
Anca Dragan
Slav Petrov
Oriol Vinyals
Jeffrey Dean
Demis Hassabis
Koray Kavukcuoglu
Clément Farabet
Elena Buchatskaya
Sebastian Borgeaud
Noah Fiedel
Armand Joulin
Kathleen Kenealy
Robert Dadashi
Alek Andreev
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2… (see more) billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.
2024-07-31
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 480 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
X. T. Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Alex Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Li Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
Nicholas Fernando
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
arxiv.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 478 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
X. T. Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Alex Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Liu
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
arxiv.org
arxiv.org
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid
Nikolay Savinov
Denis Teplyashin
Dmitry Lepikhin
Timothy P. Lillicrap
Jean-Baptiste Alayrac
Radu Soricut
Angeliki Lazaridou
Orhan Firat
Julian Schrittwieser
Ioannis Antonoglou
Rohan Anil
Sebastian Borgeaud
Andrew M. Dai
Katie Millican
Ethan Dyer
Mia Glaese
Thibault Sottiaux
Benjamin Lee
Fabio Viola … (see 479 more)
Malcolm Reynolds
Yuanzhong Xu
James L. Molloy
Jilin Chen
Michael Acheson Isard
Paul R. Barham
Tom Hennigan
Ross McIlroy
Melvin Johnson
J. Schalkwyk
Eli Collins
Eliza Rutherford
Erica Moreira
Kareem W. Ayoub
Megha Goel
Clemens Meyer
Gregory Thornton
Zhen Yang
Henryk Michalewski
Zaheer Abbas
Nathan Schucher
Ankesh Anand
Richard Ives
James Keeling
Karel Lenc
Salem Haykal
Siamak Shakeri
Pranav Shyam
Aakanksha Chowdhery
Roman Ring
Stephen Spencer
Eren Sezener
Luke Vilnis
Oscar Chang
Nobuyuki Morioka
George Tucker
Ce Zheng
Oliver Woodman
Nithya Attaluri
Tomas Kocisky
Evgenii Eltyshev
Xi Chen
Timothy Chung
Vittorio Selo
Siddhartha Brahma
Petko Georgiev
Ambrose Slone
Zhenkai Zhu
James Lottes
Siyuan Qiao
Ben Caine
Sebastian Riedel
Alex Tomala
Martin J. Chadwick
J Christopher Love
Peter Choy
Sid Mittal
Neil Houlsby
Yunhao Tang
Matthew Lamm
Libin Bai
Qiao Zhang
Luheng He
Yong Cheng
Peter Conway Humphreys
Yujia Li
Sergey Brin
Albin Cassirer
Ying-Qi Miao
Lukáš Žilka
Taylor Tobin
Kelvin Xu
Lev Proleev
Daniel Sohn
Alberto Magni
Lisa Anne Hendricks
Isabel Gao
Santiago Ontan'on
Oskar Bunyan
Nathan Byrd
Abhanshu Sharma
Biao Zhang
Mario Pinto
Rishika Sinha
Harsh Mehta
Dawei Jia
Sergi Caelles
Albert Webson
Alex Morris
Becca Roelofs
Yifan Ding
Robin Strudel
Xuehan Xiong
Marvin Ritter
Mostafa Dehghani
Rahma Chaabouni
Abhijit Karmarkar
Guangda Lai
Fabian Mentzer
Bibo Xu
YaGuang Li
Yujing Zhang
T. Paine
Alex Goldin
Behnam Neyshabur
Kate Baumli
Anselm C. Levskaya
Michael Laskin
Wenhao Jia
Jack W. Rae
Kefan Xiao
Antoine He
Skye Giordano
Laksh-man Yagati
Jean-Baptiste Lespiau
Paul Natsev
Sanjay Ganapathy
Fangyu Liu
Danilo Martins
Nanxin Chen
Yunhan Xu
Megan Barnes
Rhys May
Arpi Vezer
Junhyuk Oh
Ken Franko
Sophie Bridgers
Ruizhe Zhao
Boxi Wu
Basil Mustafa
Sean Sechrist
Emilio Parisotto
Thanumalayan Sankaranarayana Pillai
Chris Larkin
Chenjie Gu
Christina Sorokin
M. Krikun
Alexey Guseynov
Jessica Landon
Romina Datta
Alexander Pritzel
Phoebe Thacker
Fan Yang
Kevin Hui
A.E. Hauth
Chih-Kuan Yeh
David Barker
Justin Mao-jones
Sophia Austin
Hannah Rachel Sheahan
Parker Schuh
James Svensson
Rohan Jain
Vinay Venkatesh Ramasesh
Anton Briukhov
Da-Woon Chung
Tamara von Glehn
Christina Butterfield
Priya Jhakra
Matt Wiethoff
Justin Frye
Jordan Grimstad
Beer Changpinyo
Charline Le Lan
Anna Bortsova
Yonghui Wu
Paul Voigtlaender
Tara N. Sainath
Charlotte Smith
Will Hawkins
Kris Cao
James Besley
Srivatsan Srinivasan
Mark Omernick
Colin Gaffney
Gabriela Surita
Ryan Burnell
Bogdan Damoc
Junwhan Ahn
Andrew Brock
Mantas Pajarskas
Anastasia Petrushkina
Seb Noury
Lorenzo Blanco
Kevin Swersky
Arun Ahuja
Thi Avrahami
Vedant Misra
Raoul de Liedekerke
Mariko Iinuma
Alex Polozov
Sarah York
George van den Driessche
Paul Michel
Justin Chiu
Rory Blevins
Zach Gleicher
Adria Recasens
Alban Rrustemi
Elena Gribovskaya
Aurko Roy
Wiktor Gworek
S'ebastien M. R. Arnold
Lisa Lee
James Lee-Thorp
Marcello Maggioni
Enrique Piqueras
Kartikeya Badola
Sharad Mandyam Vikram
Lucas Gonzalez
Anirudh Baddepudi
Evan Senter
Jacob Devlin
James Qin
Michael Azzam
Maja Trebacz
M. Polacek
Kashyap Krishnakumar
Shuo-yiin Chang
Matthew Tung
Ivo Penchev
Rishabh Joshi
Kate Olszewska
Carrie Muir
Mateo Wirth
Ale Jakse Hartman
Joshua Newlan
Sheleem Kashem
Vijay Bolina
Elahe Dabir
Joost Van Amersfoort
Zafarali Ahmed
James Cobon-Kerr
Aishwarya B Kamath
Arnar Mar Hrafnkelsson
Le Hou
Ian Mackinnon
Alexandre Fréchette
Eric Noland
Xiance Si
Emanuel Taropa
Dong Li
Phil Crone
Anmol Gulati
S'ebastien Cevey
Jonas Adler
Ada Ma
David Silver
Simon Tokumine
Richard Powell
Stephan Lee
Samer Hassan
Diana Mincu
Antoine Yang
Nir Levine
Jenny Brennan
Mingqiu Wang
Sarah Hodkinson
Jeffrey Zhao
Josh Lipschultz
Aedan Pope
Michael B. Chang
Cheng Li
Laurent El Shafey
Michela Paganini
Sholto Douglas
Bernd Bohnet
Fabio Pardo
Seth Odoom
Mihaela Rosca
Cicero Nogueira dos Santos
Kedar Soparkar
Arthur Guez
Tom Hudson
Steven Hansen
Chulayuth Asawaroengchai
Ravichandra Addanki
Tianhe Yu
Wojciech Stokowiec
Mina Khan
Justin Gilmer
Jaehoon Lee
Carrie Grimes Bostock
Keran Rong
Jonathan Caton
Pedram Pejman
Filip Pavetic
Geoff Brown
Vivek Sharma
Mario Luvci'c
Rajkumar Samuel
Josip Djolonga
Amol Mandhane
Lars Lowe Sjosund
Elena Buchatskaya
Elspeth White
Natalie Clay
Jiepu Jiang
Hyeontaek Lim
Ross Hemsley
Jane Labanowski
Nicola De Cao
David Steiner
Sayed Hadi Hashemi
Jacob Austin
Anita Gergely
Tim Blyth
Joe Stanton
Kaushik Shivakumar
Aditya Siddhant
Anders Johan Andreassen
Carlos L. Araya
Nikhil Sethi
Rakesh Shivanna
Steven Hand
Ankur Bapna
A. Khodaei
Antoine Miech
Garrett Tanzer
Andy Swing
Shantanu Thakoor
Zhufeng Pan
Zachary Nado
Stephanie Winkler
Dian Yu
Mohammad Saleh
Lorenzo Maggiore
Iain Barr
Minh Giang
Thais Kagohara
Ivo Danihelka
Amit Marathe
Vladimir Feinberg
Mohamed Elhawaty
Nimesh Ghelani
Dan Horgan
Helen Miller
Lexi Walker
Richard Tanburn
Mukarram Tariq
Disha Shrivastava
Fei Xia
Chung-Cheng Chiu
Zoe Ashwood
Khuslen Baatarsukh
Sina Samangooei
Fred Alcober
Axel Stjerngren
Paul Komarek
Katerina Tsihlas
Anudhyan Boral
Ramona Comanescu
Jeremy Chen
Ruibo Liu
Dawn Bloxwich
Charlie Chen
Yanhua Sun
Fangxiaoyu Feng
Matthew Mauger
Xerxes Dotiwalla
Vincent Hellendoorn
Michael Sharman
Ivy Zheng
Krishna S Haridasan
Gabriel Barth-Maron
Craig Swanson
Dominika Rogozi'nska
Alek Andreev
Paul Kishan Rubenstein
Ruoxin Sang
Dan Hurt
Gamaleldin Elsayed
Renshen Wang
Dave Lacey
Anastasija Ili'c
Yao Zhao
Woohyun Han
Lora Aroyo
Chimezie Iwuanyanwu
Vitaly Nikolaev
Balaji Lakshminarayanan
Sadegh Jazayeri
Raphael Lopez Kaufman
Mani Varadarajan
Chetan Tekur
Doug Fritz
Misha Khalman
David Reitter
Kingshuk Dasgupta
Shourya Sarcar
Tina Ornduff
Javier Snaider
Fantine Huot
Johnson Jia
Rupert Kemp
Nejc Trdin
Anitha Vijayakumar
Lucy Kim
Christof Angermueller
Li Lao
Tianqi Liu
Haibin Zhang
David Engel
Somer Greene
Anais White
Jessica Austin
Lilly Taylor
Shereen Ashraf
Dangyi Liu
Maria Georgaki
Irene Cai
Yana Kulizhskaya
Sonam Goenka
Brennan Saeta
Kiran N. Vodrahalli
Christian Frank
D. Cesare
Brona Robenek
Harry Richardson
Mahmoud Alnahlawi
Christopher Yew
Priya Ponnapalli
Marco Tagliasacchi
Alex Korchemniy
Yelin Kim
Dinghua Li
Bill Rosgen
Kyle Levin
Jeremy Wiesner
Praseem Banzal
Praveen Srinivasan
Hongkun Yu
cCauglar Unlu
David Reid
Zora Tung
Daniel Finchelstein
Ravin Kumar
Andre Elisseeff
Jin Huang
Ming Zhang
Rui Zhu
Ricardo Aguilar
Mai Gim'enez
Jiawei Xia
Olivier Dousse
W. Gierke
S. Yeganeh
Damion Yates
Komal Jalan
Lu Li
Eri Latorre-Chimoto
Duc Dung Nguyen
Ken Durden
Praveen Kallakuri
Yaxin Liu
Matthew Johnson
Tomy Tsai
Alice Talbert
Jasmine Liu
Alexander Neitz
Chen Elkind
Marco Selvi
Mimi Jasarevic
Livio Baldini Soares
Albert Cui
Pidong Wang
Alek Wenjiao Wang
Xinyu Ye
Krystal Kallarackal
Lucia Loher
Hoi Lam
Josef Broder
D. Holtmann-Rice
Nina Martin
Bramandia Ramadhana
Daniel Toyama
Mrinal Shukla
Sujoy Basu
Abhi Mohan
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models… (see more) capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
2024-03-08
ArXiv (preprint)
doi.org
arxiv.org