The Five Best Things: Nov 22, 2020
A grab bag this week
 |
Sorry about the delay in getting this one out. We picked up a cold around the household, and in our year of 2020, this is an extremely frantic experience. So I needed a day to catch my breath.
I will also be taking a break next weekend due to the Thanksgiving holiday in the U.S.
Today I am kicking off the road-to-GPT-3 series, with an explainer of the attention mechanism - the fundamental component of current era giant Natural Language Processing ML models.
Nvidia announced their Q3 earnings this week; they were as expected. Professional visualization segment revenues dropped 27% YoY, which is puzzling; the 10-Q pins it on weaker demand for desktop workstations vs laptops. The newest product in this segment launched in October, and Nvidia has been pretty supply constrained, so my sense is that Q4 results will round out the story on the graphics segment.
The news of Nvidia partnering with Epic games to launch a Safari-webpage only version of Fortnite broke over the weekend. Earlier this year, Fortnite was kicked out of the App store for offering to charge users a lower price if purchases were made outside of the App store, to get around paying Apple’s 30% tax on any in-app purchases. If the webpage-only version works out, Apple will end up making $0, ; might it finally concede that the 30% take rate is excessive?
The Five Best Things
Gabriel Ilharco & Co: High Performance Natural Language Processing
This was a ~3 hour tutorial on the current state of Natural Language Processing, with accompanying slides here. The first 30 minutes catch you up to the current state of the art in Transformer models. I found the slides to be very well done, though the talk is academic for a layperson audience.
The key mechanism underpinning transformers is the “Attention” block; an intuitive explainer here: when we hear a sentence such as “The dog is a Labrador” we immediately grasp that the two most important words are “dog” and “Labrador”. Attention mechanisms capture the way our minds pick up on this relative importance between words in a sentence. Further, attention blocks can be computed independently, making their processing very parallelizable. This breakthrough has led to order of magnitude improvements in NLP, which I will cover next time.
James Allworth: Intel’s Disruption is Now Complete
James Allworth is the Head of Innovation at Cloudflare and was a student and collaborator of Clayton Christensen, the creator of Disruption theory. In this short article, he first provides a history of Intel being on the winning side of market disruption. Apple announced that its latest Macbook Pro laptops will be powered by the in-house M1 processor, replacing Intel’s processors. Allworth argues that Intel is now on the losing end of low-cost disruption.
When you see the two charts in this article, Allworth’s point is made starkly clear. ARM-based chips have now scaled up from cellphones to laptops, are making inroads in cloud-based server computing and Supercomputing. The server market is at an interesting crossroads - one one hand, ARM-based servers offer customizability and arguably lower operating costs, but need to overcome software compatibility issues, since the vast majority of software has been written to be x86-native. On the other hand, AMD offers a similar cost structure without the customizability, but with software compatibility built-in. There is also the consideration that ARM is getting acquired by Nvidia, which has monopolistic tendencies of its own. I suspect there will be wins for both, but secular losses for Intel in the next few years.
A curated collection of art generated by machine learning, by the Google Arts & Culture team. My favorite is this one where you can add random objects to paintings. I’ll play around with the code one of these days and create my own!
10KDiver: Stock Based Compensation (click below to read the entire thread)
A great thread about how to impute the real cost of stock-based compensation, also known as RSUs, which are handed to executives and employees in lieu of salaries at many companies. It walks through an example of how to infer the cost of SBC, using a real example (Microsoft), and the negative implications for “normal” shareholders of a company.
Texas Monthly: Meet an Intrepid Texan Who Spends Winters at the South Pole
A great write up about Texas native Wayne White, site manager of the Amundsen-Scott South Pole Station in Antarctica, and the only person to hold this post 3 years in a row. A couple of my favorite quotes below -
Nobody at the station wears a mask—with no outsiders since February, there has been no need—and the crew doesn’t social distance. “Some people will say to me, ‘You’re lucky to be down there during this,’” White says. “But I say it’s hellish. We can’t do anything for people back home. We’ve had deaths, unforeseen divorces, hurricanes, and fires back home, and we can’t do a goddamn thing to help … I’d much rather be home than dealing with it here.”
After eating almost exclusively frozen food since February, White is hankering for some Mexican food and a nice fresh salad—and a trip to the grocery store. “You can’t imagine what an H-E-B seems like after being here for a year,” he laughs.
Honorable Mentions
Allure: The Weird World of Olfactory Training How the practice of “physical therapy” for your sense of smell has picked up in the wake of the Covid-19 pandemic.
Rationally Speaking: Are Boomers to blame for Millennials' struggles? This podcast covers a very charged topic. The gist of it is that Boomers cannot be ascribed more responsibility than other generations for the policies that seemingly hurt millennials; more millennials should run for office if we want change to happen.
NYTimes: Dave Grohl, 10-Year-Old Nandi Bushell and One Very Epic Drum Battle Dave Grohl, the drummer for Nirvana and frontman of the Foo Fighters, was challenged to a series of online drumming duels by Nandi Bushell, a 10-year old drumming prodigy from England - over Zoom, of course. The contest adorably concluded on Colbert this week, with Grohl conceding.
NYTimes: That Pig Couch on Craigslist? Not for Sale. (Also, Not a Couch.) A couch that looks like a pig is being grifted around by strangers on Craiglist. Next season’s hottest item at Restoration Hardware
Dunkin Donuts’ Fashion Line No food brand seems to be able to resist a lifestyle business in the age of Instagram influencers, not even ones going through the largest private equity deal of the year
I haven’t read this yet, but I think I will enjoy this deep dive into Operation Warp Speed’s success. Operation Warp Speed is the US plan to make, manufacture and distribute a vaccine for Covid-19, one of the few things the Trump admin did well on.
Beautiful fall pictures from Kashmir. Happy Thanksgiving!
Disclaimer: The views and opinions expressed in this post are my own and do not represent my employer.