Florence-2-base-Castollux-v0.4

A microsoft/Florence-2-base fine-tune intended to improve the quality and formatting of image captioning.

Uses <CAPTION> task prompt. Training on <DETAILED_CAPTION> or <MORE_DETAILED_CAPTION> did not seem to make a difference in quality compared to <CAPTION>.

In v0.1 & v0.2 I was only filtering out captions with more than 1000 Florence-2 tokens. v0.3 and up will now be filtering out any captions above 512 T5 tokens.

Model Evaluation

Selected images are not cherry picked. They are from the first batch of images in the randomly chosen eval split.

Source Image	Ground Truth	Original Model Prediction	Fine-tuned Model Prediction
	A two-toned wall is visible on the left, featuring an orange upper section and a blue lower section, with a thin horizontal line separating the two colors. A small, square window with vertical metal bars is embedded in the orange section of the wall. A light orange pipe runs vertically down the orange section of the wall, adjacent to the color dividing line. To the right of this wall is a textured, light grey wall. A grey pipe runs vertically down this wall. A red octagonal stop sign with white lettering that reads "STOP" is mounted on a grey pole, positioned in front of the grey wall. Further to the right, a blue shuttered window is set into the grey wall. A decorative dark blue railing is attached to the wall below the window.	A stop sign sitting on the side of a building.	A red octagonal stop sign with white text "STOP" is positioned on the left side of the image, attached to a light beige stucco wall. To the right of the stop sign is a closed blue door with peeling paint and metal bars, and a black wrought-iron balcony with decorative elements on the right side. The wall is textured and has visible cracks and imperfections. A white pipe runs vertically down the wall between the orange and blue sections of the wall. A small rectangular window with dark metal bars is visible inside the orange section.
	A brown and white dog with floppy ears and an open mouth stands in a dense field of green leafy plants with small white flowers. The dog is positioned in the lower center of the image, facing slightly to the left. The field of plants extends throughout the foreground and midground, creating a thick carpet of green. Behind the dog and the lower field, a steeper bank covered in similar green leafy plants rises up, filling the upper portion of the image. A narrow, dark, dirt path or small stream runs horizontally across the midground, separating the lower field from the higher bank. A fallen log lies along the edge of this path on the right side of the image. The leaves on the higher bank appear slightly smaller and more densely packed than those in the foreground. Some thin, bare branches are visible on the right side of the higher bank.	A dog is sitting in the middle of a lush green field.	A brown and white dog with floppy ears is positioned in the center of the image, facing slightly to the left, with its tongue sticking out. The dog is surrounded by a field of green leafy plants with small white flowers. Behind the dog and to the right is a dense green wall covered in leafy foliage. The ground in the foreground appears to be a dirt path or path. The lighting in the image is soft and natural.
	A very tall waterfall cascades down the center of a dark grey rocky cliff into a dark body of water at the bottom. The cliff is textured and has patches of snow. Sparse trees are at the top of the cliff and on the cliff face. The dark water reflects the cliff and waterfall. The sky is grey.	A large waterfall in the middle of a body of water.	A tall waterfall cascades down a rocky cliff face in the center of the image, with the water appearing dark blue and reflecting the sky above. The cliff face is composed of grey and brown rock, with visible cracks and crevices. At the top of the cliff, a dense forest of dark green trees is visible. The sky is overcast and grey. The water in the foreground is dark blue.
	A brightly colored fish ride with a white head, blue eye, open orange mouth, and multicolored scales of blue, orange, yellow, and white is positioned in the center of the image, mounted on a black base with black handlebars. Behind the fish ride is a cream-colored decorative architectural element with arched openings. To the left of the fish ride are several metal pots and pans, including a large blue enamel pot tipped over with a red handle facing the viewer, and other smaller pots stacked behind it. To the right of the fish ride are several black buckets stacked on top of each other. The fish ride and buckets are situated on a red platform. In the background, there is a weathered blue wooden wall with a white-framed window, and to the left of the wall are aged wooden doors with decorative carvings. There is text visible on a white label attached to the cream-colored architectural element.	A large fish statue sitting on top of a table.	A large metal fish sculpture with orange, white, and blue scales is positioned in the center of the image, facing slightly to the right. The fish has a black eye and a yellow beak. It is mounted on a black metal stand. To the left of the fish is a red bucket with a blue rim and a red handle. Further to the left is a white bucket. In the foreground, there is a stack of black plastic buckets. Behind the fish and bucket are several other metal buckets and containers. The background shows a building with green wooden siding and a white window with multiple panes. A wooden door is visible on the left side, with a metal railing above it. A small white sign with black text is attached to the side of the building.
	A young girl with brown hair and brown eyes is smiling and looking directly at the viewer while standing with her hands on her hips. She is wearing a light blue headband with a flower on the right side of her head. She has on a light blue sleeveless top and a long skirt that is multi-colored with pastel shades of yellow, pink, purple, and blue, and appears to be made of tulle or a similar sheer fabric. She is barefoot and standing on a plain white background.	A little girl wearing a dress with a flower in her hair.	A young girl with fair skin and brown eyes is smiling and looking directly at the viewer, wearing a light blue headband with a large blue flower in the center, a white sleeveless dress with thin straps, and a rainbow-colored tulle skirt. Her hands are placed on her hips, and her feet are bare. The background is a plain white wall.
	A woman with brown wavy hair and fair skin is sitting in a yellow armchair. She is wearing a red sequined dress with thin straps and black high-heeled shoes. Her legs are crossed at the knees, and her right hand is resting on the arm of the chair. She is looking directly at the viewer with brown eyes and wearing makeup including eyeliner and lipstick. To the left of the armchair is a small round white table on which sits an orange vase containing a bouquet of flowers with pink, purple, and yellow blooms. Behind the woman and the table are white vertical blinds and blue curtains. The wall behind the blinds and curtains appears to be a light grey or blue color. The floor is partially visible and appears to be a light color. The bottom of a green bed or couch is visible in the foreground. The lighting in the image is studio-like, with a focus on the woman and the immediate surroundings.	A woman in a red dress sitting on a yellow chair.	A woman with long brown wavy hair is sitting in a yellow armchair. She is wearing a red sleeveless dress with thin straps. Her legs are crossed and she is wearing black high-heeled shoes. The armchair has a curved back and armrests. To the left of the armchair is a small round white table. On the table is a clear glass vase filled with pink and white flowers. Behind the vase is a window with white vertical blinds and a dark blue curtain on the right side. The background is a light blue wall.
	A clear blue sky with a few wispy white clouds in the upper left corner is above a turquoise and clear sea with visible ripples and wave patterns. A small wave with white foam is breaking on the white sandy beach in the foreground. The water is very transparent, allowing the sandy bottom to be seen. The horizon line is straight and separates the sea and sky.	A sandy beach with clear blue water and white sand.	A body of turquoise water fills the majority of the frame, with gentle waves crashing onto a sandy beach in the foreground. The water is a vibrant shade of green, and the waves are white and foamy. The sand is a light beige color. The horizon line is visible in the distance between the water and the beach. The sky above is a clear blue with a few wispy white clouds scattered across it.
	A landscape view shows a waterfall cascading down a rocky cliff face that is illuminated by bright sunlight, with a rainbow visible at the base of the falls, and two small figures standing near the base of the waterfall. The cliff face is mostly brown and grey rock with patches of green vegetation. Behind the sunlit cliff face is a darker, shadowed mountain range with visible rock striations and some sparse vegetation. In the far background to the left is the distinct shape of Half Dome under a cloudy sky. The foreground is filled with a dense, dark green forest of trees, contrasting with the brightly lit waterfall and cliff face. The sky above is mostly cloudy with dark grey clouds, but there are some brighter areas peeking through.	A large waterfall in the middle of a mountain range.	A waterfall cascades down a rocky cliff face on the right side of the image, with the water appearing yellow and white. The cliff face is composed of grey and brown rocks, with patches of green vegetation growing on the top and sides. The waterfall is located in the center of the cliff face and is surrounded by a dense forest of dark green trees. In the background, a large mountain range is visible, covered in dark green coniferous trees. The sky above is cloudy with dark grey and white clouds. The overall scene is dramatic and dramatic, with a focus on the waterfall and the forest below.

Training Settings

Trained with Florence-2ner using this config and ~17K images:

{
    "model_name": "microsoft/Florence-2-base",
    "task_prompt": "<CAPTION>",
    "dataset_path": "./0000_Datasets/Gemini-512lim",
    "wandb_project_name": "Florence-2-base",
    "run_name": "Florence-2-base-Castollux-v0.4-run7",
    "epochs": 2,
    "optimizer": "CAME",
    "learning_rate": 5e-6,
    "lr_scheduler": "REX",
    "gradient_checkpointing": true,
    "freeze_vision": false,
    "freeze_language": false,
    "freeze_other": false,
    "train_batch_size": 8,
    "eval_batch_size": 8,
    "gradient_accumulation_steps": 8,
    "clip_grad_norm": 1,
    "weight_decay": 1e-2,
    "save_total_limit": 3,
    "save_steps": 10,
    "eval_steps": 10,
    "warmup_steps": 10,
    "eval_split_ratio": 0.05,
    "seed": 42,
    "filtering_processes": 128,
    "attn_implementation": "sdpa"
}

Citations

Show Citations

@misc{xiao2023florence2advancingunifiedrepresentation,
      title={Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks}, 
      author={Bin Xiao and Haiping Wu and Weijian Xu and Xiyang Dai and Houdong Hu and Yumao Lu and Michael Zeng and Ce Liu and Lu Yuan},
      year={2023},
      eprint={2311.06242},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2311.06242}, 
}
@misc{wolf2020huggingfacestransformersstateoftheartnatural,
      title={HuggingFace's Transformers: State-of-the-art Natural Language Processing},
      author={Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and Rémi Louf and Morgan Funtowicz and Joe Davison and Sam Shleifer and Patrick von Platen and Clara Ma and Yacine Jernite and Julien Plu and Canwen Xu and Teven Le Scao and Sylvain Gugger and Mariama Drame and Quentin Lhoest and Alexander M. Rush},
      year={2020},
      eprint={1910.03771},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/1910.03771},
}
@misc{dao2023flashattention2fasterattentionbetter,
      title={FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning},
      author={Tri Dao},
      year={2023},
      eprint={2307.08691},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2307.08691},
}
@misc{luo2023cameconfidenceguidedadaptivememory,
      title={CAME: Confidence-guided Adaptive Memory Efficient Optimization}, 
      author={Yang Luo and Xiaozhe Ren and Zangwei Zheng and Zhuo Jiang and Xin Jiang and Yang You},
      year={2023},
      eprint={2307.02047},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2307.02047}, 
}
@misc{chen2021rexrevisitingbudgetedtraining,
      title={REX: Revisiting Budgeted Training with an Improved Schedule}, 
      author={John Chen and Cameron Wolfe and Anastasios Kyrillidis},
      year={2021},
      eprint={2107.04197},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2107.04197}, 
}
@misc{geminiteam2024geminifamilyhighlycapable,
      title={Gemini: A Family of Highly Capable Multimodal Models}, 
      author={Gemini Team and Rohan Anil and Sebastian Borgeaud and Jean-Baptiste Alayrac and Jiahui Yu and Radu Soricut and Johan Schalkwyk and Andrew M. Dai and Anja Hauth and Katie Millican and David Silver and Melvin Johnson and Ioannis Antonoglou and Julian Schrittwieser and Amelia Glaese and Jilin Chen and Emily Pitler and Timothy Lillicrap and Angeliki Lazaridou and Orhan Firat and James Molloy and Michael Isard and Paul R. Barham and Tom Hennigan and Benjamin Lee and Fabio Viola and Malcolm Reynolds and Yuanzhong Xu and Ryan Doherty and Eli Collins and Clemens Meyer and Eliza Rutherford and Erica Moreira and Kareem Ayoub and Megha Goel and Jack Krawczyk and Cosmo Du and Ed Chi and Heng-Tze Cheng and Eric Ni and Purvi Shah and Patrick Kane and Betty Chan and Manaal Faruqui and Aliaksei Severyn and Hanzhao Lin and YaGuang Li and Yong Cheng and Abe Ittycheriah and Mahdis Mahdieh and Mia Chen and Pei Sun and Dustin Tran and Sumit Bagri and Balaji Lakshminarayanan and Jeremiah Liu and Andras Orban and Fabian Güra and Hao Zhou and Xinying Song and Aurelien Boffy and Harish Ganapathy and Steven Zheng and HyunJeong Choe and Ágoston Weisz and Tao Zhu and Yifeng Lu and Siddharth Gopal and Jarrod Kahn and Maciej Kula and Jeff Pitman and Rushin Shah and Emanuel Taropa and Majd Al Merey and Martin Baeuml and Zhifeng Chen and Laurent El Shafey and Yujing Zhang and Olcan Sercinoglu and George Tucker and Enrique Piqueras and Maxim Krikun and Iain Barr and Nikolay Savinov and Ivo Danihelka and Becca Roelofs and Anaïs White and Anders Andreassen and Tamara von Glehn and Lakshman Yagati and Mehran Kazemi and Lucas Gonzalez and Misha Khalman and Jakub Sygnowski and Alexandre Frechette and Charlotte Smith and Laura Culp and Lev Proleev and Yi Luan and Xi Chen and James Lottes and Nathan Schucher and Federico Lebron and Alban Rrustemi and Natalie Clay and Phil Crone and Tomas Kocisky and Jeffrey Zhao and Bartek Perz and Dian Yu and Heidi Howard and Adam Bloniarz and Jack W. Rae and Han Lu and Laurent Sifre and Marcello Maggioni and Fred Alcober and Dan Garrette and Megan Barnes and Shantanu Thakoor and Jacob Austin and Gabriel Barth-Maron and William Wong and Rishabh Joshi and Rahma Chaabouni and Deeni Fatiha and Arun Ahuja and Gaurav Singh Tomar and Evan Senter and Martin Chadwick and Ilya Kornakov and Nithya Attaluri and Iñaki Iturrate and Ruibo Liu and Yunxuan Li and Sarah Cogan and Jeremy Chen and Chao Jia and Chenjie Gu and Qiao Zhang and Jordan Grimstad and Ale Jakse Hartman and Xavier Garcia and Thanumalayan Sankaranarayana Pillai and Jacob Devlin and Michael Laskin and Diego de Las Casas and Dasha Valter and Connie Tao and Lorenzo Blanco and Adrià Puigdomènech Badia and David Reitter and Mianna Chen and Jenny Brennan and Clara Rivera and Sergey Brin and Shariq Iqbal and Gabriela Surita and Jane Labanowski and Abhi Rao and Stephanie Winkler and Emilio Parisotto and Yiming Gu and Kate Olszewska and Ravi Addanki and Antoine Miech and Annie Louis and Denis Teplyashin and Geoff Brown and Elliot Catt and Jan Balaguer and Jackie Xiang and Pidong Wang and Zoe Ashwood and Anton Briukhov and Albert Webson and Sanjay Ganapathy and Smit Sanghavi and Ajay Kannan and Ming-Wei Chang and Axel Stjerngren and Josip Djolonga and Yuting Sun and Ankur Bapna and Matthew Aitchison and Pedram Pejman and Henryk Michalewski and Tianhe Yu and Cindy Wang and Juliette Love and Junwhan Ahn and Dawn Bloxwich and Kehang Han and Peter Humphreys and Thibault Sellam and James Bradbury and Varun Godbole and Sina Samangooei and Bogdan Damoc and Alex Kaskasoli and Sébastien M. R. Arnold and Vijay Vasudevan and Shubham Agrawal and Jason Riesa and Dmitry Lepikhin and Richard Tanburn and Srivatsan Srinivasan and Hyeontaek Lim and Sarah Hodkinson and Pranav Shyam and Johan Ferret and Steven Hand and Ankush Garg and Tom Le Paine and Jian Li and Yujia Li and Minh Giang and Alexander Neitz and Zaheer Abbas and Sarah York and Machel Reid and Elizabeth Cole and Aakanksha Chowdhery and Dipanjan Das and Dominika Rogozińska and Vitaliy Nikolaev and Pablo Sprechmann and Zachary Nado and Lukas Zilka and Flavien Prost and Luheng He and Marianne Monteiro and Gaurav Mishra and Chris Welty and Josh Newlan and Dawei Jia and Miltiadis Allamanis and Clara Huiyi Hu and Raoul de Liedekerke and Justin Gilmer and Carl Saroufim and Shruti Rijhwani and Shaobo Hou and Disha Shrivastava and Anirudh Baddepudi and Alex Goldin and Adnan Ozturel and Albin Cassirer and Yunhan Xu and Daniel Sohn and Devendra Sachan and Reinald Kim Amplayo and Craig Swanson and Dessie Petrova and Shashi Narayan and Arthur Guez and Siddhartha Brahma and Jessica Landon and Miteyan Patel and Ruizhe Zhao and Kevin Villela and Luyu Wang and Wenhao Jia and Matthew Rahtz and Mai Giménez and Legg Yeung and James Keeling and Petko Georgiev and Diana Mincu and Boxi Wu and Salem Haykal and Rachel Saputro and Kiran Vodrahalli and James Qin and Zeynep Cankara and Abhanshu Sharma and Nick Fernando and Will Hawkins and Behnam Neyshabur and Solomon Kim and Adrian Hutter and Priyanka Agrawal and Alex Castro-Ros and George van den Driessche and Tao Wang and Fan Yang and Shuo-yiin Chang and Paul Komarek and Ross McIlroy and Mario Lučić and Guodong Zhang and Wael Farhan and Michael Sharman and Paul Natsev and Paul Michel and Yamini Bansal and Siyuan Qiao and Kris Cao and Siamak Shakeri and Christina Butterfield and Justin Chung and Paul Kishan Rubenstein and Shivani Agrawal and Arthur Mensch and Kedar Soparkar and Karel Lenc and Timothy Chung and Aedan Pope and Loren Maggiore and Jackie Kay and Priya Jhakra and Shibo Wang and Joshua Maynez and Mary Phuong and Taylor Tobin and Andrea Tacchetti and Maja Trebacz and Kevin Robinson and Yash Katariya and Sebastian Riedel and Paige Bailey and Kefan Xiao and Nimesh Ghelani and Lora Aroyo and Ambrose Slone and Neil Houlsby and Xuehan Xiong and Zhen Yang and Elena Gribovskaya and Jonas Adler and Mateo Wirth and Lisa Lee and Music Li and Thais Kagohara and Jay Pavagadhi and Sophie Bridgers and Anna Bortsova and Sanjay Ghemawat and Zafarali Ahmed and Tianqi Liu and Richard Powell and Vijay Bolina and Mariko Iinuma and Polina Zablotskaia and James Besley and Da-Woon Chung and Timothy Dozat and Ramona Comanescu and Xiance Si and Jeremy Greer and Guolong Su and Martin Polacek and Raphaël Lopez Kaufman and Simon Tokumine and Hexiang Hu and Elena Buchatskaya and Yingjie Miao and Mohamed Elhawaty and Aditya Siddhant and Nenad Tomasev and Jinwei Xing and Christina Greer and Helen Miller and Shereen Ashraf and Aurko Roy and Zizhao Zhang and Ada Ma and Angelos Filos and Milos Besta and Rory Blevins and Ted Klimenko and Chih-Kuan Yeh and Soravit Changpinyo and Jiaqi Mu and Oscar Chang and Mantas Pajarskas and Carrie Muir and Vered Cohen and Charline Le Lan and Krishna Haridasan and Amit Marathe and Steven Hansen and Sholto Douglas and Rajkumar Samuel and Mingqiu Wang and Sophia Austin and Chang Lan and Jiepu Jiang and Justin Chiu and Jaime Alonso Lorenzo and Lars Lowe Sjösund and Sébastien Cevey and Zach Gleicher and Thi Avrahami and Anudhyan Boral and Hansa Srinivasan and Vittorio Selo and Rhys May and Konstantinos Aisopos and Léonard Hussenot and Livio Baldini Soares and Kate Baumli and Michael B. Chang and Adrià Recasens and Ben Caine and Alexander Pritzel and Filip Pavetic and Fabio Pardo and Anita Gergely and Justin Frye and Vinay Ramasesh and Dan Horgan and Kartikeya Badola and Nora Kassner and Subhrajit Roy and Ethan Dyer and Víctor Campos Campos and Alex Tomala and Yunhao Tang and Dalia El Badawy and Elspeth White and Basil Mustafa and Oran Lang and Abhishek Jindal and Sharad Vikram and Zhitao Gong and Sergi Caelles and Ross Hemsley and Gregory Thornton and Fangxiaoyu Feng and Wojciech Stokowiec and Ce Zheng and Phoebe Thacker and Çağlar Ünlü and Zhishuai Zhang and Mohammad Saleh and James Svensson and Max Bileschi and Piyush Patil and Ankesh Anand and Roman Ring and Katerina Tsihlas and Arpi Vezer and Marco Selvi and Toby Shevlane and Mikel Rodriguez and Tom Kwiatkowski and Samira Daruki and Keran Rong and Allan Dafoe and Nicholas FitzGerald and Keren Gu-Lemberg and Mina Khan and Lisa Anne Hendricks and Marie Pellat and Vladimir Feinberg and James Cobon-Kerr and Tara Sainath and Maribeth Rauh and Sayed Hadi Hashemi and Richard Ives and Yana Hasson and Eric Noland and Yuan Cao and Nathan Byrd and Le Hou and Qingze Wang and Thibault Sottiaux and Michela Paganini and Jean-Baptiste Lespiau and Alexandre Moufarek and Samer Hassan and Kaushik Shivakumar and Joost van Amersfoort and Amol Mandhane and Pratik Joshi and Anirudh Goyal and Matthew Tung and Andrew Brock and Hannah Sheahan and Vedant Misra and Cheng Li and Nemanja Rakićević and Mostafa Dehghani and Fangyu Liu and Sid Mittal and Junhyuk Oh and Seb Noury and Eren Sezener and Fantine Huot and Matthew Lamm and Nicola De Cao and Charlie Chen and Sidharth Mudgal and Romina Stella and Kevin Brooks and Gautam Vasudevan and Chenxi Liu and Mainak Chain and Nivedita Melinkeri and Aaron Cohen and Venus Wang and Kristie Seymore and Sergey Zubkov and Rahul Goel and Summer Yue and Sai Krishnakumaran and Brian Albert and Nate Hurley and Motoki Sano and Anhad Mohananey and Jonah Joughin and Egor Filonov and Tomasz Kępa and Yomna Eldawy and Jiawern Lim and Rahul Rishi and Shirin Badiezadegan and Taylor Bos and Jerry Chang and Sanil Jain and Sri Gayatri Sundara Padmanabhan and Subha Puttagunta and Kalpesh Krishna and Leslie Baker and Norbert Kalb and Vamsi Bedapudi and Adam Kurzrok and Shuntong Lei and Anthony Yu and Oren Litvin and Xiang Zhou and Zhichun Wu and Sam Sobell and Andrea Siciliano and Alan Papir and Robby Neale and Jonas Bragagnolo and Tej Toor and Tina Chen and Valentin Anklin and Feiran Wang and Richie Feng and Milad Gholami and Kevin Ling and Lijuan Liu and Jules Walter and Hamid Moghaddam and Arun Kishore and Jakub Adamek and Tyler Mercado and Jonathan Mallinson and Siddhinita Wandekar and Stephen Cagle and Eran Ofek and Guillermo Garrido and Clemens Lombriser and Maksim Mukha and Botu Sun and Hafeezul Rahman Mohammad and Josip Matak and Yadi Qian and Vikas Peswani and Pawel Janus and Quan Yuan and Leif Schelin and Oana David and Ankur Garg and Yifan He and Oleksii Duzhyi and Anton Älgmyr and Timothée Lottaz and Qi Li and Vikas Yadav and Luyao Xu and Alex Chinien and Rakesh Shivanna and Aleksandr Chuklin and Josie Li and Carrie Spadine and Travis Wolfe and Kareem Mohamed and Subhabrata Das and Zihang Dai and Kyle He and Daniel von Dincklage and Shyam Upadhyay and Akanksha Maurya and Luyan Chi and Sebastian Krause and Khalid Salama and Pam G Rabinovitch and Pavan Kumar Reddy M and Aarush Selvan and Mikhail Dektiarev and Golnaz Ghiasi and Erdem Guven and Himanshu Gupta and Boyi Liu and Deepak Sharma and Idan Heimlich Shtacher and Shachi Paul and Oscar Akerlund and François-Xavier Aubet and Terry Huang and Chen Zhu and Eric Zhu and Elico Teixeira and Matthew Fritze and Francesco Bertolini and Liana-Eleonora Marinescu and Martin Bölle and Dominik Paulus and Khyatti Gupta and Tejasi Latkar and Max Chang and Jason Sanders and Roopa Wilson and Xuewei Wu and Yi-Xuan Tan and Lam Nguyen Thiet and Tulsee Doshi and Sid Lall and Swaroop Mishra and Wanming Chen and Thang Luong and Seth Benjamin and Jasmine Lee and Ewa Andrejczuk and Dominik Rabiej and Vipul Ranjan and Krzysztof Styrc and Pengcheng Yin and Jon Simon and Malcolm Rose Harriott and Mudit Bansal and Alexei Robsky and Geoff Bacon and David Greene and Daniil Mirylenka and Chen Zhou and Obaid Sarvana and Abhimanyu Goyal and Samuel Andermatt and Patrick Siegler and Ben Horn and Assaf Israel and Francesco Pongetti and Chih-Wei "Louis" Chen and Marco Selvatici and Pedro Silva and Kathie Wang and Jackson Tolins and Kelvin Guu and Roey Yogev and Xiaochen Cai and Alessandro Agostini and Maulik Shah and Hung Nguyen and Noah Ó Donnaile and Sébastien Pereira and Linda Friso and Adam Stambler and Adam Kurzrok and Chenkai Kuang and Yan Romanikhin and Mark Geller and ZJ Yan and Kane Jang and Cheng-Chun Lee and Wojciech Fica and Eric Malmi and Qijun Tan and Dan Banica and Daniel Balle and Ryan Pham and Yanping Huang and Diana Avram and Hongzhi Shi and Jasjot Singh and Chris Hidey and Niharika Ahuja and Pranab Saxena and Dan Dooley and Srividya Pranavi Potharaju and Eileen O'Neill and Anand Gokulchandran and Ryan Foley and Kai Zhao and Mike Dusenberry and Yuan Liu and Pulkit Mehta and Ragha Kotikalapudi and Chalence Safranek-Shrader and Andrew Goodman and Joshua Kessinger and Eran Globen and Prateek Kolhar and Chris Gorgolewski and Ali Ibrahim and Yang Song and Ali Eichenbaum and Thomas Brovelli and Sahitya Potluri and Preethi Lahoti and Cip Baetu and Ali Ghorbani and Charles Chen and Andy Crawford and Shalini Pal and Mukund Sridhar and Petru Gurita and Asier Mujika and Igor Petrovski and Pierre-Louis Cedoz and Chenmei Li and Shiyuan Chen and Niccolò Dal Santo and Siddharth Goyal and Jitesh Punjabi and Karthik Kappaganthu and Chester Kwak and Pallavi LV and Sarmishta Velury and Himadri Choudhury and Jamie Hall and Premal Shah and Ricardo Figueira and Matt Thomas and Minjie Lu and Ting Zhou and Chintu Kumar and Thomas Jurdi and Sharat Chikkerur and Yenai Ma and Adams Yu and Soo Kwak and Victor Ähdel and Sujeevan Rajayogam and Travis Choma and Fei Liu and Aditya Barua and Colin Ji and Ji Ho Park and Vincent Hellendoorn and Alex Bailey and Taylan Bilal and Huanjie Zhou and Mehrdad Khatir and Charles Sutton and Wojciech Rzadkowski and Fiona Macintosh and Konstantin Shagin and Paul Medina and Chen Liang and Jinjing Zhou and Pararth Shah and Yingying Bi and Attila Dankovics and Shipra Banga and Sabine Lehmann and Marissa Bredesen and Zifan Lin and John Eric Hoffmann and Jonathan Lai and Raynald Chung and Kai Yang and Nihal Balani and Arthur Bražinskas and Andrei Sozanschi and Matthew Hayes and Héctor Fernández Alcalde and Peter Makarov and Will Chen and Antonio Stella and Liselotte Snijders and Michael Mandl and Ante Kärrman and Paweł Nowak and Xinyi Wu and Alex Dyck and Krishnan Vaidyanathan and Raghavender R and Jessica Mallet and Mitch Rudominer and Eric Johnston and Sushil Mittal and Akhil Udathu and Janara Christensen and Vishal Verma and Zach Irving and Andreas Santucci and Gamaleldin Elsayed and Elnaz Davoodi and Marin Georgiev and Ian Tenney and Nan Hua and Geoffrey Cideron and Edouard Leurent and Mahmoud Alnahlawi and Ionut Georgescu and Nan Wei and Ivy Zheng and Dylan Scandinaro and Heinrich Jiang and Jasper Snoek and Mukund Sundararajan and Xuezhi Wang and Zack Ontiveros and Itay Karo and Jeremy Cole and Vinu Rajashekhar and Lara Tumeh and Eyal Ben-David and Rishub Jain and Jonathan Uesato and Romina Datta and Oskar Bunyan and Shimu Wu and John Zhang and Piotr Stanczyk and Ye Zhang and David Steiner and Subhajit Naskar and Michael Azzam and Matthew Johnson and Adam Paszke and Chung-Cheng Chiu and Jaume Sanchez Elias and Afroz Mohiuddin and Faizan Muhammad and Jin Miao and Andrew Lee and Nino Vieillard and Jane Park and Jiageng Zhang and Jeff Stanway and Drew Garmon and Abhijit Karmarkar and Zhe Dong and Jong Lee and Aviral Kumar and Luowei Zhou and Jonathan Evens and William Isaac and Geoffrey Irving and Edward Loper and Michael Fink and Isha Arkatkar and Nanxin Chen and Izhak Shafran and Ivan Petrychenko and Zhe Chen and Johnson Jia and Anselm Levskaya and Zhenkai Zhu and Peter Grabowski and Yu Mao and Alberto Magni and Kaisheng Yao and Javier Snaider and Norman Casagrande and Evan Palmer and Paul Suganthan and Alfonso Castaño and Irene Giannoumis and Wooyeol Kim and Mikołaj Rybiński and Ashwin Sreevatsa and Jennifer Prendki and David Soergel and Adrian Goedeckemeyer and Willi Gierke and Mohsen Jafari and Meenu Gaba and Jeremy Wiesner and Diana Gage Wright and Yawen Wei and Harsha Vashisht and Yana Kulizhskaya and Jay Hoover and Maigo Le and Lu Li and Chimezie Iwuanyanwu and Lu Liu and Kevin Ramirez and Andrey Khorlin and Albert Cui and Tian LIN and Marcus Wu and Ricardo Aguilar and Keith Pallo and Abhishek Chakladar and Ginger Perng and Elena Allica Abellan and Mingyang Zhang and Ishita Dasgupta and Nate Kushman and Ivo Penchev and Alena Repina and Xihui Wu and Tom van der Weide and Priya Ponnapalli and Caroline Kaplan and Jiri Simsa and Shuangfeng Li and Olivier Dousse and Fan Yang and Jeff Piper and Nathan Ie and Rama Pasumarthi and Nathan Lintz and Anitha Vijayakumar and Daniel Andor and Pedro Valenzuela and Minnie Lui and Cosmin Paduraru and Daiyi Peng and Katherine Lee and Shuyuan Zhang and Somer Greene and Duc Dung Nguyen and Paula Kurylowicz and Cassidy Hardin and Lucas Dixon and Lili Janzer and Kiam Choo and Ziqiang Feng and Biao Zhang and Achintya Singhal and Dayou Du and Dan McKinnon and Natasha Antropova and Tolga Bolukbasi and Orgad Keller and David Reid and Daniel Finchelstein and Maria Abi Raad and Remi Crocker and Peter Hawkins and Robert Dadashi and Colin Gaffney and Ken Franko and Anna Bulanova and Rémi Leblond and Shirley Chung and Harry Askham and Luis C. Cobo and Kelvin Xu and Felix Fischer and Jun Xu and Christina Sorokin and Chris Alberti and Chu-Cheng Lin and Colin Evans and Alek Dimitriev and Hannah Forbes and Dylan Banarse and Zora Tung and Mark Omernick and Colton Bishop and Rachel Sterneck and Rohan Jain and Jiawei Xia and Ehsan Amid and Francesco Piccinno and Xingyu Wang and Praseem Banzal and Daniel J. Mankowitz and Alex Polozov and Victoria Krakovna and Sasha Brown and MohammadHossein Bateni and Dennis Duan and Vlad Firoiu and Meghana Thotakuri and Tom Natan and Matthieu Geist and Ser tan Girgin and Hui Li and Jiayu Ye and Ofir Roval and Reiko Tojo and Michael Kwong and James Lee-Thorp and Christopher Yew and Danila Sinopalnikov and Sabela Ramos and John Mellor and Abhishek Sharma and Kathy Wu and David Miller and Nicolas Sonnerat and Denis Vnukov and Rory Greig and Jennifer Beattie and Emily Caveness and Libin Bai and Julian Eisenschlos and Alex Korchemniy and Tomy Tsai and Mimi Jasarevic and Weize Kong and Phuong Dao and Zeyu Zheng and Frederick Liu and Fan Yang and Rui Zhu and Tian Huey Teh and Jason Sanmiya and Evgeny Gladchenko and Nejc Trdin and Daniel Toyama and Evan Rosen and Sasan Tavakkol and Linting Xue and Chen Elkind and Oliver Woodman and John Carpenter and George Papamakarios and Rupert Kemp and Sushant Kafle and Tanya Grunina and Rishika Sinha and Alice Talbert and Diane Wu and Denese Owusu-Afriyie and Cosmo Du and Chloe Thornton and Jordi Pont-Tuset and Pradyumna Narayana and Jing Li and Saaber Fatehi and John Wieting and Omar Ajmeri and Benigno Uria and Yeongil Ko and Laura Knight and Amélie Héliou and Ning Niu and Shane Gu and Chenxi Pang and Yeqing Li and Nir Levine and Ariel Stolovich and Rebeca Santamaria-Fernandez and Sonam Goenka and Wenny Yustalim and Robin Strudel and Ali Elqursh and Charlie Deck and Hyo Lee and Zonglin Li and Kyle Levin and Raphael Hoffmann and Dan Holtmann-Rice and Olivier Bachem and Sho Arora and Christy Koh and Soheil Hassas Yeganeh and Siim Põder and Mukarram Tariq and Yanhua Sun and Lucian Ionita and Mojtaba Seyedhosseini and Pouya Tafti and Zhiyu Liu and Anmol Gulati and Jasmine Liu and Xinyu Ye and Bart Chrzaszcz and Lily Wang and Nikhil Sethi and Tianrun Li and Ben Brown and Shreya Singh and Wei Fan and Aaron Parisi and Joe Stanton and Vinod Koverkathu and Christopher A. Choquette-Choo and Yunjie Li and TJ Lu and Abe Ittycheriah and Prakash Shroff and Mani Varadarajan and Sanaz Bahargam and Rob Willoughby and David Gaddy and Guillaume Desjardins and Marco Cornero and Brona Robenek and Bhavishya Mittal and Ben Albrecht and Ashish Shenoy and Fedor Moiseev and Henrik Jacobsson and Alireza Ghaffarkhah and Morgane Rivière and Alanna Walton and Clément Crepy and Alicia Parrish and Zongwei Zhou and Clement Farabet and Carey Radebaugh and Praveen Srinivasan and Claudia van der Salm and Andreas Fidjeland and Salvatore Scellato and Eri Latorre-Chimoto and Hanna Klimczak-Plucińska and David Bridson and Dario de Cesare and Tom Hudson and Piermaria Mendolicchio and Lexi Walker and Alex Morris and Matthew Mauger and Alexey Guseynov and Alison Reid and Seth Odoom and Lucia Loher and Victor Cotruta and Madhavi Yenugula and Dominik Grewe and Anastasia Petrushkina and Tom Duerig and Antonio Sanchez and Steve Yadlowsky and Amy Shen and Amir Globerson and Lynette Webb and Sahil Dua and Dong Li and Surya Bhupatiraju and Dan Hurt and Haroon Qureshi and Ananth Agarwal and Tomer Shani and Matan Eyal and Anuj Khare and Shreyas Rammohan Belle and Lei Wang and Chetan Tekur and Mihir Sanjay Kale and Jinliang Wei and Ruoxin Sang and Brennan Saeta and Tyler Liechty and Yi Sun and Yao Zhao and Stephan Lee and Pandu Nayak and Doug Fritz and Manish Reddy Vuyyuru and John Aslanides and Nidhi Vyas and Martin Wicke and Xiao Ma and Evgenii Eltyshev and Nina Martin and Hardie Cate and James Manyika and Keyvan Amiri and Yelin Kim and Xi Xiong and Kai Kang and Florian Luisier and Nilesh Tripuraneni and David Madras and Mandy Guo and Austin Waters and Oliver Wang and Joshua Ainslie and Jason Baldridge and Han Zhang and Garima Pruthi and Jakob Bauer and Feng Yang and Riham Mansour and Jason Gelman and Yang Xu and George Polovets and Ji Liu and Honglong Cai and Warren Chen and XiangHai Sheng and Emily Xue and Sherjil Ozair and Christof Angermueller and Xiaowei Li and Anoop Sinha and Weiren Wang and Julia Wiesinger and Emmanouil Koukoumidis and Yuan Tian and Anand Iyer and Madhu Gurumurthy and Mark Goldenson and Parashar Shah and MK Blake and Hongkun Yu and Anthony Urbanowicz and Jennimaria Palomaki and Chrisantha Fernando and Ken Durden and Harsh Mehta and Nikola Momchev and Elahe Rahimtoroghi and Maria Georgaki and Amit Raul and Sebastian Ruder and Morgan Redshaw and Jinhyuk Lee and Denny Zhou and Komal Jalan and Dinghua Li and Blake Hechtman and Parker Schuh and Milad Nasr and Kieran Milan and Vladimir Mikulik and Juliana Franco and Tim Green and Nam Nguyen and Joe Kelley and Aroma Mahendru and Andrea Hu and Joshua Howland and Ben Vargas and Jeffrey Hui and Kshitij Bansal and Vikram Rao and Rakesh Ghiya and Emma Wang and Ke Ye and Jean Michel Sarr and Melanie Moranski Preston and Madeleine Elish and Steve Li and Aakash Kaku and Jigar Gupta and Ice Pasupat and Da-Cheng Juan and Milan Someswar and Tejvi M. and Xinyun Chen and Aida Amini and Alex Fabrikant and Eric Chu and Xuanyi Dong and Amruta Muthal and Senaka Buthpitiya and Sarthak Jauhari and Nan Hua and Urvashi Khandelwal and Ayal Hitron and Jie Ren and Larissa Rinaldi and Shahar Drath and Avigail Dabush and Nan-Jiang Jiang and Harshal Godhia and Uli Sachs and Anthony Chen and Yicheng Fan and Hagai Taitelbaum and Hila Noga and Zhuyun Dai and James Wang and Chen Liang and Jenny Hamer and Chun-Sung Ferng and Chenel Elkind and Aviel Atias and Paulina Lee and Vít Listík and Mathias Carlen and Jan van de Kerkhof and Marcin Pikus and Krunoslav Zaher and Paul Müller and Sasha Zykova and Richard Stefanec and Vitaly Gatsko and Christoph Hirnschall and Ashwin Sethi and Xingyu Federico Xu and Chetan Ahuja and Beth Tsai and Anca Stefanoiu and Bo Feng and Keshav Dhandhania and Manish Katyal and Akshay Gupta and Atharva Parulekar and Divya Pitta and Jing Zhao and Vivaan Bhatia and Yashodha Bhavnani and Omar Alhadlaq and Xiaolin Li and Peter Danenberg and Dennis Tu and Alex Pine and Vera Filippova and Abhipso Ghosh and Ben Limonchik and Bhargava Urala and Chaitanya Krishna Lanka and Derik Clive and Yi Sun and Edward Li and Hao Wu and Kevin Hongtongsak and Ianna Li and Kalind Thakkar and Kuanysh Omarov and Kushal Majmundar and Michael Alverson and Michael Kucharski and Mohak Patel and Mudit Jain and Maksim Zabelin and Paolo Pelagatti and Rohan Kohli and Saurabh Kumar and Joseph Kim and Swetha Sankar and Vineet Shah and Lakshmi Ramachandruni and Xiangkai Zeng and Ben Bariach and Laura Weidinger and Tu Vu and Alek Andreev and Antoine He and Kevin Hui and Sheleem Kashem and Amar Subramanya and Sissie Hsiao and Demis Hassabis and Koray Kavukcuoglu and Adam Sadovsky and Quoc Le and Trevor Strohman and Yonghui Wu and Slav Petrov and Jeffrey Dean and Oriol Vinyals},
      year={2024},
      eprint={2312.11805},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2312.11805}, 
}
@misc{geminiteam2024gemini15unlockingmultimodal,
      title={Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context},
      author={Gemini Team and Petko Georgiev and Ving Ian Lei and Ryan Burnell and Libin Bai and Anmol Gulati and Garrett Tanzer and Damien Vincent and Zhufeng Pan and Shibo Wang and Soroosh Mariooryad and Yifan Ding and Xinyang Geng and Fred Alcober and Roy Frostig and Mark Omernick and Lexi Walker and Cosmin Paduraru and Christina Sorokin and Andrea Tacchetti and Colin Gaffney and Samira Daruki and Olcan Sercinoglu and Zach Gleicher and Juliette Love and Paul Voigtlaender and Rohan Jain and Gabriela Surita and Kareem Mohamed and Rory Blevins and Junwhan Ahn and Tao Zhu and Kornraphop Kawintiranon and Orhan Firat and Yiming Gu and Yujing Zhang and Matthew Rahtz and Manaal Faruqui and Natalie Clay and Justin Gilmer and JD Co-Reyes and Ivo Penchev and Rui Zhu and Nobuyuki Morioka and Kevin Hui and Krishna Haridasan and Victor Campos and Mahdis Mahdieh and Mandy Guo and Samer Hassan and Kevin Kilgour and Arpi Vezer and Heng-Tze Cheng and Raoul de Liedekerke and Siddharth Goyal and Paul Barham and DJ Strouse and Seb Noury and Jonas Adler and Mukund Sundararajan and Sharad Vikram and Dmitry Lepikhin and Michela Paganini and Xavier Garcia and Fan Yang and Dasha Valter and Maja Trebacz and Kiran Vodrahalli and Chulayuth Asawaroengchai and Roman Ring and Norbert Kalb and Livio Baldini Soares and Siddhartha Brahma and David Steiner and Tianhe Yu and Fabian Mentzer and Antoine He and Lucas Gonzalez and Bibo Xu and Raphael Lopez Kaufman and Laurent El Shafey and Junhyuk Oh and Tom Hennigan and George van den Driessche and Seth Odoom and Mario Lucic and Becca Roelofs and Sid Lall and Amit Marathe and Betty Chan and Santiago Ontanon and Luheng He and Denis Teplyashin and Jonathan Lai and Phil Crone and Bogdan Damoc and Lewis Ho and Sebastian Riedel and Karel Lenc and Chih-Kuan Yeh and Aakanksha Chowdhery and Yang Xu and Mehran Kazemi and Ehsan Amid and Anastasia Petrushkina and Kevin Swersky and Ali Khodaei and Gowoon Chen and Chris Larkin and Mario Pinto and Geng Yan and Adria Puigdomenech Badia and Piyush Patil and Steven Hansen and Dave Orr and Sebastien M. R. Arnold and Jordan Grimstad and Andrew Dai and Sholto Douglas and Rishika Sinha and Vikas Yadav and Xi Chen and Elena Gribovskaya and Jacob Austin and Jeffrey Zhao and Kaushal Patel and Paul Komarek and Sophia Austin and Sebastian Borgeaud and Linda Friso and Abhimanyu Goyal and Ben Caine and Kris Cao and Da-Woon Chung and Matthew Lamm and Gabe Barth-Maron and Thais Kagohara and Kate Olszewska and Mia Chen and Kaushik Shivakumar and Rishabh Agarwal and Harshal Godhia and Ravi Rajwar and Javier Snaider and Xerxes Dotiwalla and Yuan Liu and Aditya Barua and Victor Ungureanu and Yuan Zhang and Bat-Orgil Batsaikhan and Mateo Wirth and James Qin and Ivo Danihelka and Tulsee Doshi and Martin Chadwick and Jilin Chen and Sanil Jain and Quoc Le and Arjun Kar and Madhu Gurumurthy and Cheng Li and Ruoxin Sang and Fangyu Liu and Lampros Lamprou and Rich Munoz and Nathan Lintz and Harsh Mehta and Heidi Howard and Malcolm Reynolds and Lora Aroyo and Quan Wang and Lorenzo Blanco and Albin Cassirer and Jordan Griffith and Dipanjan Das and Stephan Lee and Jakub Sygnowski and Zach Fisher and James Besley and Richard Powell and Zafarali Ahmed and Dominik Paulus and David Reitter and Zalan Borsos and Rishabh Joshi and Aedan Pope and Steven Hand and Vittorio Selo and Vihan Jain and Nikhil Sethi and Megha Goel and Takaki Makino and Rhys May and Zhen Yang and Johan Schalkwyk and Christina Butterfield and Anja Hauth and Alex Goldin and Will Hawkins and Evan Senter and Sergey Brin and Oliver Woodman and Marvin Ritter and Eric Noland and Minh Giang and Vijay Bolina and Lisa Lee and Tim Blyth and Ian Mackinnon and Machel Reid and Obaid Sarvana and David Silver and Alexander Chen and Lily Wang and Loren Maggiore and Oscar Chang and Nithya Attaluri and Gregory Thornton and Chung-Cheng Chiu and Oskar Bunyan and Nir Levine and Timothy Chung and Evgenii Eltyshev and Xiance Si and Timothy Lillicrap and Demetra Brady and Vaibhav Aggarwal and Boxi Wu and Yuanzhong Xu and Ross McIlroy and Kartikeya Badola and Paramjit Sandhu and Erica Moreira and Wojciech Stokowiec and Ross Hemsley and Dong Li and Alex Tudor and Pranav Shyam and Elahe Rahimtoroghi and Salem Haykal and Pablo Sprechmann and Xiang Zhou and Diana Mincu and Yujia Li and Ravi Addanki and Kalpesh Krishna and Xiao Wu and Alexandre Frechette and Matan Eyal and Allan Dafoe and Dave Lacey and Jay Whang and Thi Avrahami and Ye Zhang and Emanuel Taropa and Hanzhao Lin and Daniel Toyama and Eliza Rutherford and Motoki Sano and HyunJeong Choe and Alex Tomala and Chalence Safranek-Shrader and Nora Kassner and Mantas Pajarskas and Matt Harvey and Sean Sechrist and Meire Fortunato and Christina Lyu and Gamaleldin Elsayed and Chenkai Kuang and James Lottes and Eric Chu and Chao Jia and Chih-Wei Chen and Peter Humphreys and Kate Baumli and Connie Tao and Rajkumar Samuel and Cicero Nogueira dos Santos and Anders Andreassen and Nemanja Rakićević and Dominik Grewe and Aviral Kumar and Stephanie Winkler and Jonathan Caton and Andrew Brock and Sid Dalmia and Hannah Sheahan and Iain Barr and Yingjie Miao and Paul Natsev and Jacob Devlin and Feryal Behbahani and Flavien Prost and Yanhua Sun and Artiom Myaskovsky and Thanumalayan Sankaranarayana Pillai and Dan Hurt and Angeliki Lazaridou and Xi Xiong and Ce Zheng and Fabio Pardo and Xiaowei Li and Dan Horgan and Joe Stanton and Moran Ambar and Fei Xia and Alejandro Lince and Mingqiu Wang and Basil Mustafa and Albert Webson and Hyo Lee and Rohan Anil and Martin Wicke and Timothy Dozat and Abhishek Sinha and Enrique Piqueras and Elahe Dabir and Shyam Upadhyay and Anudhyan Boral and Lisa Anne Hendricks and Corey Fry and Josip Djolonga and Yi Su and Jake Walker and Jane Labanowski and Ronny Huang and Vedant Misra and Jeremy Chen and RJ Skerry-Ryan and Avi Singh and Shruti Rijhwani and Dian Yu and Alex Castro-Ros and Beer Changpinyo and Romina Datta and Sumit Bagri and Arnar Mar Hrafnkelsson and Marcello Maggioni and Daniel Zheng and Yury Sulsky and Shaobo Hou and Tom Le Paine and Antoine Yang and Jason Riesa and Dominika Rogozinska and Dror Marcus and Dalia El Badawy and Qiao Zhang and Luyu Wang and Helen Miller and Jeremy Greer and Lars Lowe Sjos and Azade Nova and Heiga Zen and Rahma Chaabouni and Mihaela Rosca and Jiepu Jiang and Charlie Chen and Ruibo Liu and Tara Sainath and Maxim Krikun and Alex Polozov and Jean-Baptiste Lespiau and Josh Newlan and Zeyncep Cankara and Soo Kwak and Yunhan Xu and Phil Chen and Andy Coenen and Clemens Meyer and Katerina Tsihlas and Ada Ma and Juraj Gottweis and Jinwei Xing and Chenjie Gu and Jin Miao and Christian Frank and Zeynep Cankara and Sanjay Ganapathy and Ishita Dasgupta and Steph Hughes-Fitt and Heng Chen and David Reid and Keran Rong and Hongmin Fan and Joost van Amersfoort and Vincent Zhuang and Aaron Cohen and Shixiang Shane Gu and Anhad Mohananey and Anastasija Ilic and Taylor Tobin and John Wieting and Anna Bortsova and Phoebe Thacker and Emma Wang and Emily Caveness and Justin Chiu and Eren Sezener and Alex Kaskasoli and Steven Baker and Katie Millican and Mohamed Elhawaty and Kostas Aisopos and Carl Lebsack and Nathan Byrd and Hanjun Dai and Wenhao Jia and Matthew Wiethoff and Elnaz Davoodi and Albert Weston and Lakshman Yagati and Arun Ahuja and Isabel Gao and Golan Pundak and Susan Zhang and Michael Azzam and Khe Chai Sim and Sergi Caelles and James Keeling and Abhanshu Sharma and Andy Swing and YaGuang Li and Chenxi Liu and Carrie Grimes Bostock and Yamini Bansal and Zachary Nado and Ankesh Anand and Josh Lipschultz and Abhijit Karmarkar and Lev Proleev and Abe Ittycheriah and Soheil Hassas Yeganeh and George Polovets and Aleksandra Faust and Jiao Sun and Alban Rrustemi and Pen Li and Rakesh Shivanna and Jeremiah Liu and Chris Welty and Federico Lebron and Anirudh Baddepudi and Sebastian Krause and Emilio Parisotto and Radu Soricut and Zheng Xu and Dawn Bloxwich and Melvin Johnson and Behnam Neyshabur and Justin Mao-Jones and Renshen Wang and Vinay Ramasesh and Zaheer Abbas and Arthur Guez and Constant Segal and Duc Dung Nguyen and James Svensson and Le Hou and Sarah York and Kieran Milan and Sophie Bridgers and Wiktor Gworek and Marco Tagliasacchi and James Lee-Thorp and Michael Chang and Alexey Guseynov and Ale Jakse Hartman and Michael Kwong and Ruizhe Zhao and Sheleem Kashem and Elizabeth Cole and Antoine Miech and Richard Tanburn and Mary Phuong and Filip Pavetic and Sebastien Cevey and Ramona Comanescu and Richard Ives and Sherry Yang and Cosmo Du and Bo Li and Zizhao Zhang and Mariko Iinuma and Clara Huiyi Hu and Aurko Roy and Shaan Bijwadia and Zhenkai Zhu and Danilo Martins and Rachel Saputro and Anita Gergely and Steven Zheng and Dawei Jia and Ioannis Antonoglou and Adam Sadovsky and Shane Gu and Yingying Bi and Alek Andreev and Sina Samangooei and Mina Khan and Tomas Kocisky and Angelos Filos and Chintu Kumar and Colton Bishop and Adams Yu and Sarah Hodkinson and Sid Mittal and Premal Shah and Alexandre Moufarek and Yong Cheng and Adam Bloniarz and Jaehoon Lee and Pedram Pejman and Paul Michel and Stephen Spencer and Vladimir Feinberg and Xuehan Xiong and Nikolay Savinov and Charlotte Smith and Siamak Shakeri and Dustin Tran and Mary Chesus and Bernd Bohnet and George Tucker and Tamara von Glehn and Carrie Muir and Yiran Mao and Hideto Kazawa and Ambrose Slone and Kedar Soparkar and Disha Shrivastava and James Cobon-Kerr and Michael Sharman and Jay Pavagadhi and Carlos Araya and Karolis Misiunas and Nimesh Ghelani and Michael Laskin and David Barker and Qiujia Li and Anton Briukhov and Neil Houlsby and Mia Glaese and Balaji Lakshminarayanan and Nathan Schucher and Yunhao Tang and Eli Collins and Hyeontaek Lim and Fangxiaoyu Feng and Adria Recasens and Guangda Lai and Alberto Magni and Nicola De Cao and Aditya Siddhant and Zoe Ashwood and Jordi Orbay and Mostafa Dehghani and Jenny Brennan and Yifan He and Kelvin Xu and Yang Gao and Carl Saroufim and James Molloy and Xinyi Wu and Seb Arnold and Solomon Chang and Julian Schrittwieser and Elena Buchatskaya and Soroush Radpour and Martin Polacek and Skye Giordano and Ankur Bapna and Simon Tokumine and Vincent Hellendoorn and Thibault Sottiaux and Sarah Cogan and Aliaksei Severyn and Mohammad Saleh and Shantanu Thakoor and Laurent Shefey and Siyuan Qiao and Meenu Gaba and Shuo-yiin Chang and Craig Swanson and Biao Zhang and Benjamin Lee and Paul Kishan Rubenstein and Gan Song and Tom Kwiatkowski and Anna Koop and Ajay Kannan and David Kao and Parker Schuh and Axel Stjerngren and Golnaz Ghiasi and Gena Gibson and Luke Vilnis and Ye Yuan and Felipe Tiengo Ferreira and Aishwarya Kamath and Ted Klimenko and Ken Franko and Kefan Xiao and Indro Bhattacharya and Miteyan Patel and Rui Wang and Alex Morris and Robin Strudel and Vivek Sharma and Peter Choy and Sayed Hadi Hashemi and Jessica Landon and Mara Finkelstein and Priya Jhakra and Justin Frye and Megan Barnes and Matthew Mauger and Dennis Daun and Khuslen Baatarsukh and Matthew Tung and Wael Farhan and Henryk Michalewski and Fabio Viola and Felix de Chaumont Quitry and Charline Le Lan and Tom Hudson and Qingze Wang and Felix Fischer and Ivy Zheng and Elspeth White and Anca Dragan and Jean-baptiste Alayrac and Eric Ni and Alexander Pritzel and Adam Iwanicki and Michael Isard and Anna Bulanova and Lukas Zilka and Ethan Dyer and Devendra Sachan and Srivatsan Srinivasan and Hannah Muckenhirn and Honglong Cai and Amol Mandhane and Mukarram Tariq and Jack W. Rae and Gary Wang and Kareem Ayoub and Nicholas FitzGerald and Yao Zhao and Woohyun Han and Chris Alberti and Dan Garrette and Kashyap Krishnakumar and Mai Gimenez and Anselm Levskaya and Daniel Sohn and Josip Matak and Inaki Iturrate and Michael B. Chang and Jackie Xiang and Yuan Cao and Nishant Ranka and Geoff Brown and Adrian Hutter and Vahab Mirrokni and Nanxin Chen and Kaisheng Yao and Zoltan Egyed and Francois Galilee and Tyler Liechty and Praveen Kallakuri and Evan Palmer and Sanjay Ghemawat and Jasmine Liu and David Tao and Chloe Thornton and Tim Green and Mimi Jasarevic and Sharon Lin and Victor Cotruta and Yi-Xuan Tan and Noah Fiedel and Hongkun Yu and Ed Chi and Alexander Neitz and Jens Heitkaemper and Anu Sinha and Denny Zhou and Yi Sun and Charbel Kaed and Brice Hulse and Swaroop Mishra and Maria Georgaki and Sneha Kudugunta and Clement Farabet and Izhak Shafran and Daniel Vlasic and Anton Tsitsulin and Rajagopal Ananthanarayanan and Alen Carin and Guolong Su and Pei Sun and Shashank V and Gabriel Carvajal and Josef Broder and Iulia Comsa and Alena Repina and William Wong and Warren Weilun Chen and Peter Hawkins and Egor Filonov and Lucia Loher and Christoph Hirnschall and Weiyi Wang and Jingchen Ye and Andrea Burns and Hardie Cate and Diana Gage Wright and Federico Piccinini and Lei Zhang and Chu-Cheng Lin and Ionel Gog and Yana Kulizhskaya and Ashwin Sreevatsa and Shuang Song and Luis C. Cobo and Anand Iyer and Chetan Tekur and Guillermo Garrido and Zhuyun Xiao and Rupert Kemp and Huaixiu Steven Zheng and Hui Li and Ananth Agarwal and Christel Ngani and Kati Goshvadi and Rebeca Santamaria-Fernandez and Wojciech Fica and Xinyun Chen and Chris Gorgolewski and Sean Sun and Roopal Garg and Xinyu Ye and S. M. Ali Eslami and Nan Hua and Jon Simon and Pratik Joshi and Yelin Kim and Ian Tenney and Sahitya Potluri and Lam Nguyen Thiet and Quan Yuan and Florian Luisier and Alexandra Chronopoulou and Salvatore Scellato and Praveen Srinivasan and Minmin Chen and Vinod Koverkathu and Valentin Dalibard and Yaming Xu and Brennan Saeta and Keith Anderson and Thibault Sellam and Nick Fernando and Fantine Huot and Junehyuk Jung and Mani Varadarajan and Michael Quinn and Amit Raul and Maigo Le and Ruslan Habalov and Jon Clark and Komal Jalan and Kalesha Bullard and Achintya Singhal and Thang Luong and Boyu Wang and Sujeevan Rajayogam and Julian Eisenschlos and Johnson Jia and Daniel Finchelstein and Alex Yakubovich and Daniel Balle and Michael Fink and Sameer Agarwal and Jing Li and Dj Dvijotham and Shalini Pal and Kai Kang and Jaclyn Konzelmann and Jennifer Beattie and Olivier Dousse and Diane Wu and Remi Crocker and Chen Elkind and Siddhartha Reddy Jonnalagadda and Jong Lee and Dan Holtmann-Rice and Krystal Kallarackal and Rosanne Liu and Denis Vnukov and Neera Vats and Luca Invernizzi and Mohsen Jafari and Huanjie Zhou and Lilly Taylor and Jennifer Prendki and Marcus Wu and Tom Eccles and Tianqi Liu and Kavya Kopparapu and Francoise Beaufays and Christof Angermueller and Andreea Marzoca and Shourya Sarcar and Hilal Dib and Jeff Stanway and Frank Perbet and Nejc Trdin and Rachel Sterneck and Andrey Khorlin and Dinghua Li and Xihui Wu and Sonam Goenka and David Madras and Sasha Goldshtein and Willi Gierke and Tong Zhou and Yaxin Liu and Yannie Liang and Anais White and Yunjie Li and Shreya Singh and Sanaz Bahargam and Mark Epstein and Sujoy Basu and Li Lao and Adnan Ozturel and Carl Crous and Alex Zhai and Han Lu and Zora Tung and Neeraj Gaur and Alanna Walton and Lucas Dixon and Ming Zhang and Amir Globerson and Grant Uy and Andrew Bolt and Olivia Wiles and Milad Nasr and Ilia Shumailov and Marco Selvi and Francesco Piccinno and Ricardo Aguilar and Sara McCarthy and Misha Khalman and Mrinal Shukla and Vlado Galic and John Carpenter and Kevin Villela and Haibin Zhang and Harry Richardson and James Martens and Matko Bosnjak and Shreyas Rammohan Belle and Jeff Seibert and Mahmoud Alnahlawi and Brian McWilliams and Sankalp Singh and Annie Louis and Wen Ding and Dan Popovici and Lenin Simicich and Laura Knight and Pulkit Mehta and Nishesh Gupta and Chongyang Shi and Saaber Fatehi and Jovana Mitrovic and Alex Grills and Joseph Pagadora and Tsendsuren Munkhdalai and Dessie Petrova and Danielle Eisenbud and Zhishuai Zhang and Damion Yates and Bhavishya Mittal and Nilesh Tripuraneni and Yannis Assael and Thomas Brovelli and Prateek Jain and Mihajlo Velimirovic and Canfer Akbulut and Jiaqi Mu and Wolfgang Macherey and Ravin Kumar and Jun Xu and Haroon Qureshi and Gheorghe Comanici and Jeremy Wiesner and Zhitao Gong and Anton Ruddock and Matthias Bauer and Nick Felt and Anirudh GP and Anurag Arnab and Dustin Zelle and Jonas Rothfuss and Bill Rosgen and Ashish Shenoy and Bryan Seybold and Xinjian Li and Jayaram Mudigonda and Goker Erdogan and Jiawei Xia and Jiri Simsa and Andrea Michi and Yi Yao and Christopher Yew and Steven Kan and Isaac Caswell and Carey Radebaugh and Andre Elisseeff and Pedro Valenzuela and Kay McKinney and Kim Paterson and Albert Cui and Eri Latorre-Chimoto and Solomon Kim and William Zeng and Ken Durden and Priya Ponnapalli and Tiberiu Sosea and Christopher A. Choquette-Choo and James Manyika and Brona Robenek and Harsha Vashisht and Sebastien Pereira and Hoi Lam and Marko Velic and Denese Owusu-Afriyie and Katherine Lee and Tolga Bolukbasi and Alicia Parrish and Shawn Lu and Jane Park and Balaji Venkatraman and Alice Talbert and Lambert Rosique and Yuchung Cheng and Andrei Sozanschi and Adam Paszke and Praveen Kumar and Jessica Austin and Lu Li and Khalid Salama and Bartek Perz and Wooyeol Kim and Nandita Dukkipati and Anthony Baryshnikov and Christos Kaplanis and XiangHai Sheng and Yuri Chervonyi and Caglar Unlu and Diego de Las Casas and Harry Askham and Kathryn Tunyasuvunakool and Felix Gimeno and Siim Poder and Chester Kwak and Matt Miecnikowski and Vahab Mirrokni and Alek Dimitriev and Aaron Parisi and Dangyi Liu and Tomy Tsai and Toby Shevlane and Christina Kouridi and Drew Garmon and Adrian Goedeckemeyer and Adam R. Brown and Anitha Vijayakumar and Ali Elqursh and Sadegh Jazayeri and Jin Huang and Sara Mc Carthy and Jay Hoover and Lucy Kim and Sandeep Kumar and Wei Chen and Courtney Biles and Garrett Bingham and Evan Rosen and Lisa Wang and Qijun Tan and David Engel and Francesco Pongetti and Dario de Cesare and Dongseong Hwang and Lily Yu and Jennifer Pullman and Srini Narayanan and Kyle Levin and Siddharth Gopal and Megan Li and Asaf Aharoni and Trieu Trinh and Jessica Lo and Norman Casagrande and Roopali Vij and Loic Matthey and Bramandia Ramadhana and Austin Matthews and CJ Carey and Matthew Johnson and Kremena Goranova and Rohin Shah and Shereen Ashraf and Kingshuk Dasgupta and Rasmus Larsen and Yicheng Wang and Manish Reddy Vuyyuru and Chong Jiang and Joana Ijazi and Kazuki Osawa and Celine Smith and Ramya Sree Boppana and Taylan Bilal and Yuma Koizumi and Ying Xu and Yasemin Altun and Nir Shabat and Ben Bariach and Alex Korchemniy and Kiam Choo and Olaf Ronneberger and Chimezie Iwuanyanwu and Shubin Zhao and David Soergel and Cho-Jui Hsieh and Irene Cai and Shariq Iqbal and Martin Sundermeyer and Zhe Chen and Elie Bursztein and Chaitanya Malaviya and Fadi Biadsy and Prakash Shroff and Inderjit Dhillon and Tejasi Latkar and Chris Dyer and Hannah Forbes and Massimo Nicosia and Vitaly Nikolaev and Somer Greene and Marin Georgiev and Pidong Wang and Nina Martin and Hanie Sedghi and John Zhang and Praseem Banzal and Doug Fritz and Vikram Rao and Xuezhi Wang and Jiageng Zhang and Viorica Patraucean and Dayou Du and Igor Mordatch and Ivan Jurin and Lewis Liu and Ayush Dubey and Abhi Mohan and Janek Nowakowski and Vlad-Doru Ion and Nan Wei and Reiko Tojo and Maria Abi Raad and Drew A. Hudson and Vaishakh Keshava and Shubham Agrawal and Kevin Ramirez and Zhichun Wu and Hoang Nguyen and Ji Liu and Madhavi Sewak and Bryce Petrini and DongHyun Choi and Ivan Philips and Ziyue Wang and Ioana Bica and Ankush Garg and Jarek Wilkiewicz and Priyanka Agrawal and Xiaowei Li and Danhao Guo and Emily Xue and Naseer Shaik and Andrew Leach and Sadh MNM Khan and Julia Wiesinger and Sammy Jerome and Abhishek Chakladar and Alek Wenjiao Wang and Tina Ornduff and Folake Abu and Alireza Ghaffarkhah and Marcus Wainwright and Mario Cortes and Frederick Liu and Joshua Maynez and Andreas Terzis and Pouya Samangouei and Riham Mansour and Tomasz Kępa and François-Xavier Aubet and Anton Algymr and Dan Banica and Agoston Weisz and Andras Orban and Alexandre Senges and Ewa Andrejczuk and Mark Geller and Niccolo Dal Santo and Valentin Anklin and Majd Al Merey and Martin Baeuml and Trevor Strohman and Junwen Bai and Slav Petrov and Yonghui Wu and Demis Hassabis and Koray Kavukcuoglu and Jeff Dean and Oriol Vinyals},
      year={2024},
      eprint={2403.05530},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2403.05530},
}