Linear algebra with Transformers – Paper Explained

Jan 08, 2022

|

39 views

AI Coffee Break with Letitia

Details

Why would one build a transformer to solve linear algebra problems when there is numpy.linalg? Check out the video to find out why this is a cool idea and understand how the transformer works that can solve 9 linear algebra problems (e.g. matrix multiplication, inversion). ► SPONSOR: Weights & Biases 👉 https://wandb.ai ❓ Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community 📺 Symbolic Mathematics with transformers: https://youtu.be/l7ofrfmVsd0 📺 Transformer explained: https://youtube.com/playlist?list=PLpZBeKTZRGPNdymdEsSSSod5YQ3Vu0sKY 📺 GPT-3: https://youtu.be/5fqxPOaaqi0 📺 Foundation models: https://youtu.be/4Cxz4rnnZ7Q 📺 Interpolation vs. Generalization in Deep Learning: https://youtu.be/-GH9vW5FWUs Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 donor, Dres. Trost GbR, Yannik Schneider Paper 📜: Charton, François. "Linear algebra with transformers." arXiv preprint arXiv:2112.01898 (2021). https://arxiv.org/abs/2112.01898 🔗 Openreview discussion between author and reviewrs: https://openreview.net/forum?id=L2a_bcarHcF 🔗 A cat can be an author, too! https://en.wikipedia.org/wiki/F._D._C._Willard 🔗 Gary Marcus: extrapolationhttps://twitter.com/GaryMarcus/status/1411401507610796032 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Outline: 00:00 Linear algebra with transformers 00:41 Weights & Biases (Sponsor) 02:21 Why throwing transformers at linear algebra is cool. 08:08 How do transformers solve linear algebra? 09:50 Encoding matrices for transformers 11:28 Training data and results 12:43 Generalization!? 16:05 Few-shot learning!? 17:36 AI Coffee Break Quiz call to action Music 🎵 : Secret Job – Godmode Cat meowing sound by ignotus : https://freesound.org/people/ignotus/sounds/26104/ ---------------------------------- 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter: https://twitter.com/AICoffeeBreak Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research

00:00 Linear algebra with transformers 00:41 Weights & Biases (Sponsor) 02:21 Why throwing transformers at linear algebra is cool. 08:08 How do transformers solve linear algebra? 09:50 Encoding matrices for transformers 11:28 Training data and results 12:43 Generalization!? 16:05 Few-shot learning!? 17:36 AI Coffee Break Quiz call to action

Category: Research Paper

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

SREcon15 - Architecting and Launching the Halo 4 Services

USENIX | Apr 13, 2015

SREcon15 - Being Afraid—How Paranoia at Dropbox Protects Your Data

USENIX | Apr 13, 2015

ICAPS 2014: Jendrik Seipp on "Diverse and Additive Cartesian Abstraction Heuristics"

ICAPS | Apr 14, 2015

ICAPS 2014: Bram Ridder on "Heuristic Evaluation Based on Lifted Relaxed Planning Graphs"

ICAPS | Apr 14, 2015

ICAPS 2014: Richard Valenzano on "A Comparison of Knowledge-Based GBFS Enhancements and..."

ICAPS | Apr 15, 2015