Size does not matter | The efficiency misnomer | What does the number of parameters mean?

Dec 06, 2021

|

35 views

AI Coffee Break with Letitia

Details

How important is the number of parameters in deep learning models? But what about other measures like FLOPs or speed/throughput? ► Check out our sponsor Aleph Alpha 👉 https://www.aleph-alpha.de/ ! Follow them on Twitter: Aleph__Alpha Paper 📜: Dehghani, Mostafa, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. "The Efficiency Misnomer." arXiv preprint arXiv:2110.12894 (2021). https://arxiv.org/abs/2110.12894 🔗 Megatron-Turing NLG 530B: https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/ Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 donor, Dres. Trost GbR, Yannik Schneider Outline: 00:00 Model efficiency comparison 02:51 FLOPs 03:55 Number of parameters: means what? 06:31 Speed / throughput 09:39 Aleph Alpha (Sponsor) ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter: https://twitter.com/AICoffeeBreak Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research

00:00 Model efficiency comparison 02:51 FLOPs 03:55 Number of parameters: means what? 06:31 Speed / throughput 09:39 Aleph Alpha (Sponsor)

Category: Research Paper

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

ICAPS 2014: Daniel Harabor on "Improving Jump Point Search"

ICAPS | Jul 2, 2014

ICAPS 2014 Invited Talk: Peter Wurman

ICAPS | Jul 2, 2014

ICAPS 2014: Mike Phillips on "PA*SE: Parallel A* for Slow Expansions"

ICAPS | Jul 3, 2014

ICAPS 2014: Vidal Alcázar on "Analyzing the Impact of Partial States..."

ICAPS | Jul 3, 2014

ICAPS 2014: Aijun Bai on "Thompson Sampling Based Monte-Carlo Planning in POMDPs"

ICAPS | Jul 3, 2014