Location

Constructor Academy,

Landsberger Str. 110, 80339 Munich

4th floor

Accessing GPUs From Multiple Providers With dstack Sky

dstack is an open-source tool designed for managing AI infrastructure across various cloud platforms. It’s lighter and more specifically geared towards AI tasks compared to Kubernetes. Due to its support for multiple cloud providers, dstack is frequently used to access on-demand and spot GPUs across multiple clouds. From our users, we’ve learned that managing various cloud accounts, quotas, and billing can be cumbersome. To streamline this process, we introduce dstack Sky, a managed service that enables users to access GPUs from multiple providers through dstack – without needing an account in each cloud provider.

More Labels or Cases? Assessing Label Variation in Natural Language Inference

In this work, we analyze the uncertainty that is inherently present in the labels used for supervised machine learning in natural language inference (NLI). In cases where multiple annotations per instance are available, neither the majority vote nor the frequency of individual class votes is a trustworthy representation of the labeling uncertainty. We propose modeling the votes via a Bayesian mixture model to recover the data-generating process, i.e., the posterior distribution of the ``true’’ latent classes, and thus gain insight into the class variations. This will enable a better understanding of the confusion happening during the annotation process. We also assess the stability of the proposed estimation procedure by systematically varying the numbers of i) instances and ii) labels. Thereby, we observe that few instances with many labels can predict the latent class borders reasonably well, while the estimation fails for many instances with only a few labels. This leads us to conclude that multiple labels are a crucial building block for properly analyzing label uncertainty.

p(doom) - an Open, Entirely Decentralized, Global AI Research Lab

p(doom) is an open, entirely decentralised, global AI research lab and organises the entirety of its research via Discord. With the self-proclaimed goal of conducting high-impact research on ever more intelligent systems, the lab tries to challenge conventional academic incentives. This talk will present two ongoing research projects: Addressing the same-compute-per-token limitation of current-generation LLMs at its root, as well as directly predicting text-based protein function annotations from amino acid sequences. More than presenting research projects, the talk will try to convey a school of thought of choosing research projects for maximum impact.

Agenda

• 18:00 Doors open
• 18:30 - 18:45 Organizers & Host Welcome and Introduction
• 18:45 - 19:30 Andrey Cheptsov, Founder of dstack.ai, Topic: Accessing GPUs From Multiple Providers With dstack Sky
• 19.30 - 20:15 Cornelia Gruber, PhD Candidate in Statistics at LMU, Topic: More Labels or Cases? Assessing Label Variation in Natural Language Inference
• 20:15 - 20.30 Mihir Mahajan and Franz Srambical, p(doom) - an Open, Entirely Decentralized, Global AI Research Lab
• 20.30 - 22.00 Get Together With Food & Drinks

Speakers

Andrey Cheptsov is the creator of dstack, a very lightweight open-source utility that allows to provision ML infra via command-line. He is passionate about open-source and developer tools for AI. Previously, Andrey worked at JetBrains with the PyCharm team.

Cornelia Gruber holds a Bachelor’s degree in Statistics and a Master’s degree in Data Science from LMU Munich. She then started her PhD at the Chair of Applied Statistics in Social Sciences, Economics and Business at LMU to work on the topic of Uncertainty Quantification in Supervised Machine Learning. Since October 2022, Cornelia is a Konrad Zuse School of Excellence in Reliable AI member.

Mihir Mahajan is a CS Student at TUM specializing in Machine Learning and Computer Vision. He has hands-on industry experience due to his time at AWS, where he exclusively focused on topics related to Generative AI and Large Language Models. At p(doom), he focuses on integrating adaptive computation for transformer architecture.

Franz Srambical is a researcher at p(doom) and Helmholtz Munich. His broad and paradigm-agnostic research interest lies in the creation of ever more intelligent systems. As such, he tackles capability limitations of current state-of-the-art neural architectures such as transformers (continual learning; same-compute-per-token limitation; shifting compute to inference time). In his free time, he started p(doom), an open, entirely decentralized, global AI research lab. In a previous life, he worked on distributed systems at Celonis, on EEG classification as a founding member of neuroTUM, on linux kernels, neuroscience and game theory.