Channel: StatQuest with Josh Starmer

Category: Education

Tags: josh starmerentropydata sciencemachine learningstatqueststatistics

Description: Entropy is a fundamental concept in Data Science because it shows up all over the place - from Decision Trees, to similarity metrics, to state of the art dimension reduction algorithms. It's also surprisingly simple, but often poorly explained. Traditionally the equation is presented with the expectation that you memorize it without thoroughly understanding what it means and where it came from. This video takes a very different approach by showing you, step-by-step, where this simple equation comes from, making it easy to remember (and derive), understand and explain to your friends at parties. For a complete index of all the StatQuest videos, check out: statquest.org/video-index If you'd like to support StatQuest, please consider... Buying my book, The StatQuest Illustrated Guide to Machine Learning: PDF - statquest.gumroad.com/l/wvtmc Paperback - amazon.com/dp/B09ZCKR4H6 Kindle eBook - amazon.com/dp/B09ZG79HXC Patreon: patreon.com/statquest ...or... YouTube Membership: youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...a cool StatQuest t-shirt or sweatshirt: shop.spreadshirt.com/statquest-with-josh-starmer ...buying one or two of my songs (or go large and get a whole album!) joshuastarmer.bandcamp.com ...or just donating to StatQuest! paypal.me/statquest Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: twitter.com/joshuastarmer 0:00 Awesome song and introduction 1:28 Introduction to surprise 4:34 Equation for surprise 6:09 Calculating surprise for a series of events 9:35 Entropy defined for a coin 10:45 Entropy is the expected value of surprise 11:41 The entropy equation 13:01 Entropy in action!!! #StatQuest #Entropy