Does GPT-3 lie? - Misinformation and fear-mongering around the TruthfulQA dataset

Sep 24, 2021

|

31 views

Yannic Kilcher

Details

#gpt-3 #truth #conspiracy A new benchmark paper has created quite an uproar in the community. TruthfulQA is a dataset of 817 questions probing for imitative falsehoods where language models become less truthful, the larger they get. This surprising counter-intuitive finding validates many people's criticisms of large language models, but is it really the correct conclusion? OUTLINE: 0:00 - Intro 0:30 - Twitter Paper Announcement 4:10 - Large Language Models are to blame! 5:50 - How was the dataset constructed? 9:25 - The questions are adversarial 12:30 - Are you surprised?! Paper: https://arxiv.org/abs/2109.07958 Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yannic-kilcher Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/ykilcher BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannickilcher Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

0:00 - Intro 0:30 - Twitter Paper Announcement 4:10 - Large Language Models are to blame! 5:50 - How was the dataset constructed? 9:25 - The questions are adversarial 12:30 - Are you surprised?!

Category: Research Paper

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

ICAPS 2014: Bram Ridder on "Heuristic Evaluation Based on Lifted Relaxed Planning Graphs"

ICAPS | Apr 14, 2015

ICAPS 2014: Richard Valenzano on "A Comparison of Knowledge-Based GBFS Enhancements and..."

ICAPS | Apr 15, 2015

ICAPS 2014: Florian Pommerening on "LP-Based Heuristics for Cost-Optimal Planning"

ICAPS | Apr 15, 2015

ICAPS 2014: Benjamin Cohen on "Single- and Dual-Arm Motion Planning with Heuristic Search"

ICAPS | Apr 15, 2015

ICAPS 2014: Tutorial by Siddharth Srivastava on "Task and Motion Planning for Robots..."

ICAPS | Apr 15, 2015