Sign in
Size does not matter | The efficiency misnomer | What does the number of parameters mean?
Dec 06, 2021
|
35 views
AI Coffee Break with Letitia
Follow
Details
πΊ Watch on YouTube π
How important is the number of parameters in deep learning models? But what about other measures like FLOPs or speed/throughput? βΊ Check out our sponsor Aleph Alpha π
https://www.aleph-alpha.de/
! Follow them on Twitter: Aleph__Alpha Paper π: Dehghani, Mostafa, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. "The Efficiency Misnomer." arXiv preprint arXiv:2110.12894 (2021).
https://arxiv.org/abs/2110.12894
π Megatron-Turing NLG 530B:
https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/
Thanks to our Patrons who support us in Tier 2, 3, 4: π donor, Dres. Trost GbR, Yannik Schneider Outline: 00:00 Model efficiency comparison 02:51 FLOPs 03:55 Number of parameters: means what? 06:31 Speed / throughput 09:39 Aleph Alpha (Sponsor) ββββββββββββββββββββββββββ π₯ Optionally, pay us a coffee to help with our Coffee Bean production! β Patreon:
https://www.patreon.com/AICoffeeBreak
Ko-fi:
https://ko-fi.com/aicoffeebreak
ββββββββββββββββββββββββββ π Links: AICoffeeBreakQuiz:
https://www.youtube.com/c/AICoffeeBreak/community
Twitter:
https://twitter.com/AICoffeeBreak
Reddit:
https://www.reddit.com/r/AICoffeeBreak/
YouTube:
https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #researchβ
00:00
Model efficiency comparison
02:51
FLOPs
03:55
Number of parameters: means what?
06:31
Speed / throughput
09:39
Aleph Alpha (Sponsor)
Category: Research Paper
Comments
loading...
Reactions
(0)
| Note
π No reactions yet
Be the first one to share your thoughts!
Reactions
(0)
Note
loading...
Recommended
20:34
ICAPS 2014: Daniel Harabor on "Improving Jump Point Search"
ICAPS
| Jul 2, 2014
1:07:18
ICAPS 2014 Invited Talk: Peter Wurman
ICAPS
| Jul 2, 2014
20:25
ICAPS 2014: Mike Phillips on "PA*SE: Parallel A* for Slow Expansions"
ICAPS
| Jul 3, 2014
14:30
ICAPS 2014: Vidal AlcΓ‘zar on "Analyzing the Impact of Partial States..."
ICAPS
| Jul 3, 2014
13:37
ICAPS 2014: Aijun Bai on "Thompson Sampling Based Monte-Carlo Planning in POMDPs"
ICAPS
| Jul 3, 2014