About this event

Ekin Akyürek from MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) will explain his recent work including

• The hypothesis that language models learn in-context by learning algorithms running internally.
• Novel way of proving that the Transformer architecture can learn in-context by internal gradient descent and ridge regression without any approximation to the architecture
• The empirical fact that the trained Transformers learn in-context very close to Bayes optimal algorithms, and do not learn in-context by gradient descent.
• Phase shifts between algorithmic behavior and scaling laws based model size vs problem dimension
• The distinction between behavior and computation.

Speaker

Ekin Akyürek is a computer science PhD student at the Massachusetts Institute of Technology (MIT). He studies artificial intelligence through natural language processing and machine learning and is advised by Jacob Andreas.

He works on improving sequence modeling for language processing and understanding.