Location

JetBrains Event Space Christoph-Rapparini-Bogen 23 · München, BY.

About this Event

Cutting Through the Hype: Jina AI’s Scalable Open-Source Solutions for Multimodal AI

In this talk, we will showcase how Jina AI helps organizations cut through the AI hype by providing practical and scalable AI solutions for industrial domains where stability, availability, and business value are crucial. You will learn how our open-source MLOps framework and multimodal products empower a seamless transition from research to industry-level applications across various sectors.

LongForm: Improve instruction tuning with corpus examples and generated instructions.

Instruction tuning enables language models to generalize more effectively and better follow user intent. However, obtaining instruction data can be costly and challenging. Prior works employ methods such as expensive human annotation, crowd-sourced datasets with alignment issues, or generating noisy examples via LLMs. We introduce the LongForm dataset, which is created by leveraging English corpus examples with augmented instructions. We select a diverse set of human-written documents from existing corpora such as C4 and Wikipedia and generate instructions for the given documents via LLMs. This approach provides a cheaper and cleaner instruction-tuning dataset and one suitable for long text generation. We finetune T5, OPT, and LLaMA models on our dataset and show that even smaller LongForm models have good generalization capabilities for text generation. Our models outperform 10x larger language models without instruction tuning on various tasks such as story/recipe generation and long-form question answering. Moreover, LongForm models outperform prior instruction-tuned models such as FLAN-T5 and Alpaca by a large margin. Finally, our models can effectively follow and answer multilingual instructions; we demonstrate this for news generation.

Speakers

Saahil Ognawala is Senior Product Manager at Jina AI, a cutting edge Multimodal AI startup founded in 2020 which has already appeared in Forbes AI30 and CBInsights AI100. Saahil has previously worked as a data scientist and product manager for Munich Re. Prior to that, he finished his M.Sc. and PhD Computer Science at the Technical University of Munich, specializing in deep learning and software engineering.

Abdullatif Köksal is a second-year ELLIS PhD Student at Ludwig Maximilian University of Munich and Cambridge University, co-advised by Hinrich Schütze and Anna Korhonen. His research lies in the fields of unsupervised and few-shot learning and their applications on both large and small language models, such as instruction tuning and prompt-based finetuning. In addition to this, he has conducted research in the areas of bias in PLMs, computational social science, and text generation. Previously, Abdullatif worked as an applied science intern at Amazon Books in Madrid, focusing on structured prediction from long-text documents. He completed a BSc and MSc at Bogazici University, advised by Arzucan Ozgur.