Moontower #162
On Diversity
The paradox is that you write to become more fully yourself, but then you find it hard to live with the self you’ve become. I do what I can to remain a possibility instead of a reality. Thus the flow. And, next week, the effort to erase this too.
-Freddie deBoer on why he feels the need to write frequently. I don’t think you need to be a writer to appreciate that feeling.
Friends,
I’ve got this friend and neighbor who I hike walk with after we drop the kids off at school sometimes. It’s like 90 minutes of dorm-room musings befitting of the Moontower scene. Those conversations have influenced a few of these weekly letters.
This recent one was no different. It got me thinking about diversity.
See, this friend has an early-stage startup in the education space. The internal research at his company parses diversity across many dimensions. You are familiar with the capital “D” types of diversity — race, religion, gender, age, sexual orientation, socioeconomic, etc.
Interestingly, he expressed concern that his team might not be diverse enough. Not in a visible way. The team runs the gamut of the capital “D” diversity categories. But he was interested in cognitive diversity. He was concerned that an intellectual echo chamber of fancy-college liberals could lead to blind spots in their collective decision-making.
Now I don’t have much team-building experience. So I flexed some knowledge I recently read. (This is why people read right? To at least have a tennis racket when they find themselves on the court of conversational Wimbledon.) And now you too shall witness the fact that I read a book. Confer prestige heavily and without reservation, my esteemed landsmen.
Nah, really I actually read a book and it said some cool stuff. I’ll get to that in a sec. First, I want to re-surface some points professor Mauboussin made about cognitive diversity. He described it as the training, experience, and personality that make an individual unique.
He writes:
I think one can make the case very seriously and quite rigorously that social category diversity contributes to cognitive diversity, but it is cognitive diversity that we’re after.
He describes a less-talked-about form of diversity as well.
“Values diversity”. You might think about it as a sense of purpose, and on that, you actually want to be low. We want a common mission, even if we are of very different backgrounds, we’re pulling in the same direction.
In other words, my friend’s instincts about diversity are correct. Visible diversity is an imperfect proxy for intellectual diversity.
Back to the book I was reading — Superforecasting by Phil Tetlock and Dan Gardner. The book’s main thesis, which falls out of the lessons from the Good Judgement Project, is that it’s possible to become a well-calibrated forecaster in complex (but not all complex) domains. How good can you become? The best are able to consistently beat prediction markets, something even demonstrably above-average forecasters struggle to do.
This is rightly provocative because markets are effective truth-finding mechanisms. They are an ancient way of coordinating human behavior (democracy is another example of a human-coordination machine…for a contrast between markets and democracy see Dinosaur Markets).
If you read this letter regularly you know I have a lot of respect for the efficiency of markets. It’s not a strong-form academic belief. It’s more of an informal razor: “my null hypothesis is there’s no easy money and the burden of proof is on investors who think otherwise”. The academic compromise is markets are “efficiently inefficient”, reflecting the idea that there’s a cost associated with finding inefficiencies so some amount of inefficiency always exists to justify the hurdle of hunting for it.
To appreciate why markets, under certain conditions, triangulate on the truth, I paraphrase Tetlock's explanation of how the “wisdom of crowds” works:
Bits of useful and useless information are distributed throughout a crowd. The useful information all points to a reasonably accurate consensus while the useless information sometimes overshoots and sometime undershoots but critically…cancels out.
The Role Of Diversity In Truth-Finding
The “under certain conditions” is an important asterisk. The expression “wisdom of crowds” is actually a modern idea that plays off the “madness of crowds”, a term coined nearly 200 years ago by journalist Charles Mackay. For the crowd to generate wisdom, it needs diversity. In other words, the errors in judgment need to be uncorrelated to cancel out.
In Tetlock’s studies, they tested the forecasting abilities of individuals. They were rigorous in their experimental design. They were curious how teams of forecasters would perform against individuals. They further experimented with the composition of those teams.
The eye-opening results underscore the importance of diverse thinking:
Teams are more effective
The results were clear-cut each year. Teams of ordinary forecasters beat the wisdom of the crowd by about 10%. Prediction markets beat ordinary teams by about 20%. And superteams beat prediction markets by 15% to 30%.
“Emergence”: teams are more than the sum of their parts. This cuts both ways…even actively open-minded individuals could surrender to “groupthink”
“Diversity trumps ability”
This provocative claim highlights how the aggregation of different perspectives can improve judgment. The key to diversity was, unsurprisingly, cognitive diversity.
The revealing result: When they constructed the superteams they optimized for ability and those teams happened to be highly diverse because the superforecasters themselves were highly diverse. They did not optimize for diversity first, but it turned out the most diverse teams were the most effective.
The asymmetry of the extremizing algorithm
The “extremizing algorithm” is a technique where you boost a 70% prediction closer to the extreme, perhaps bumping it to 85%. It’s a technique that is employed when the forecasters have diverse perspectives because it leads to better-calibrated forecasts.
You do the opposite (push the forecast probability closer to 50%) to combat “groupthink” if the team is comprised of people who think the same or possess similar knowledge. (The use of the extremizing algo allowed teams of regular forecasters to actually perform better than some superteams!).
My own observation: this is the same logic by which correlated observations “shrink” the sample size, an idea familiar to data analysts.
Example Of Cognitive Diversity
My friend with the start-up gets it. He is concerned that the visible diversity on his team might be a poor proxy for what he really wants — cognitive diversity. If you are an oil company you need geologists, finance people, managers, political connections, real estate expertise. This is a clear example of needing to pull together many object-level competencies.
But cognitive diversity is not just “what do they think about”, but “how do they think”. For an example of what I call meta-cognitive diversity, Tetlock uses the “hedgehog” vs “fox” duality. As a pre-defense of Tetlock, he warns about overstating this dichotomy. (He exemplifies non-binary thinking throughout the book, doing an honest and eloquent job of pointing out tensions and caveats, not unlike the superforecasters themselves).
I’ll take a stab at describing hedgehogs and foxes:
Hedgehogs
Hedgehogs are specialists. The 10,000 hours crowd. The natural endpoint for the logic of economic comparative advantage or simply the rightful throne of the devoted craftsman. The “specialization is for insects” objection is too reductive. We are better off when Eddie Van Halen wants nothing else than to just be Eddie Van Halen.
But there are trade-offs. Hedgehogs often filter observations through the lens of their expertise. (My online friends have a good-natured running joke that I see everything as an option — classic “when all you have is a hammer, everything is a nail” thinking). But a camera lens’ usefulness depends on the context. Sometimes that fisheye or telephoto lens is exactly the wrong tool for the job.
This is inconvenient for the status-aware hedgehog whose incentive to remain consequential leads to motivated reasoning and self-delusion. Academic researchers who become famous for writing about an idea that catches fire have a lot to protect. They become fast friends with Mssr. Confirmation Bias and Madame Overconfidence, the very enemies they used to fight when they were building their reputations of good work. It’s like America fighting against the same Afghans they armed as rebels in the 80s. As DiCaprio’s character in Up In The Air would attest, the warm embrace of fame beats the cold loneliness of cultural anonymity.
Foxes
Foxes are generalists. Businesspeople, investors, politicians, administrators. It’s an imperfect description of course but you know the type. The disadvantages of being an “inch deep and a mile wide” are established. That person is never going to design a bridge or coach a professional sports team. Some are self-aware enough to recognize when they “know enough to be dangerous”. Many are not. Their advantage, however, is their mercenary relationships with lenses. The ego cost of finding the most useful lens is much lower, making foxes at least psychologically fit for reasoning across domains.
Remaining careful not to play into false binaries, the hedghog/fox continuum reminds me of the importance of shifting between diffuse and focused thinking modes. Diffuse thinking (see More Shower Thoughts Please) provides both inspiration and the ability “see over the neighbor’s fence” while focused thinking enables us to synthesize those scattered insights into a useful output. Most of my posts start either in the shower or when I’m taking a walk. (I strongly recommend this old New Yorker piece Why Walking Helps Us Think). “How we think” is a dimension of diversity.
The Complicated Discourse Around Diversity Is Inevitable
When I described the thrust of this piece to my wife, Yinh, she yawned. “So you’re writing a post saying ‘diversity is good’? Who is this news to?”
I immediately got nervous that I was belaboring an obvious point. I asked her why she thought it was so obvious and she started citing studies and initiatives that could easily have 10x’d the length of this post.
Now I can take feedback, but I’m not above quibbling en route to my final destination. So let me get this straight. You read a bunch of stuff arguing that diversity is good, now it’s obvious to you and presumably everyone else who reads, so I shouldn’t spend any time making arguments that diversity is good. I mean, this post is to a Moontower reader as that other research was to you. Next time just put my head in the washing machine woman.
But she still has a point. Diversity, in its many forms, is widely celebrated. In-breeding is taboo. I was just watching the National Parks series on Netflix with the kids and learned that rainforests, the most biologically diverse ecology on land, are the origin of more than 25% of modern medicines. So why do I feel the need to cheerlead an “obvious” point?
The short answer is I don’t think the case is closed on the merit of diversity. I know it’s exhausting to hear me say this, but it depends on contexts. For example, whenever America’s dysfunction is compared to a homogeneous European country, I scratch my head. If we are tribal in nature, our default wiring might simply make governing a melting pot inherently more difficult. I think diversity makes us stronger overall, but some measures of local harmony should expect to suffer. I’m even open to the possibility that tolerance runs counter to our natural instincts. (It just makes a normative approach to overriding our base impulses require extra care. Law-making is always a tug-of-war between collective values and the animals within us.) In other words, I can appreciate how diversity can be a headwind.
But there’s more.
Even if we wave a wand and agree that diversity is an unalloyed good, there remains the harder question. At what cost? The lightning rod version of this question is you have a white student and black student who look the same in all other ways and you need to choose one (the premise is unrealistic, but this is the collapsed version of how these questions get passed around the media and people extrapolate entire political identities on how they’d answer such a fake question). If everything else is the same and there is a non-zero probability that social diversity leads to cognitive diversity, then the optimal (although not necessarily morally fair which is a different criteria battleground) decision gate would say select the black kid (assuming the majority of the student body were white).
Still, even if we agreed on that, a harder question remains. What if the black kid had a slightly worse score on a standardized test? From a strictly efficient-utilitarian point of view (again, moral consideration aside), then we are faced with trade-off on a diversity-competence frontier. In a purely academic sense, I was likely an inferior hire at SIG, but perhaps something about how I thought or acted might have made my “diversity” or “complementariness" worth more than just hiring yet another 1600 Math SAT kid from MIT (or they just exhausted the supply of those, I’m not trying to flatter myself here). The point is that the merit of diversity is fairly intuitive, but doesn’t lend itself to legible number-crunching in the way test scores do.
Pricing “diversity” could very well be a fool’s errand. But I have one final bit of intuition to sprinkle on the problem. In There’s Gold In Them Thar Tails: Part 2, I rehash how nature uses diversity as fuel for evolution.
Diversity is an essential input to progress. Nature’s underlying algorithm of evolution penalizes in-breeding.
In addition to a loss of diversity, signals decay as you get closer to the extremes. This is known as tail divergence. The signal can even flip (ie Berkson’s Paradox).
The point where the signal noise overwhelms the variance in the candidates is an efficient cutoff. Beyond that threshold, selectors should think more creatively than “just raise the bar”.
At some point, incremental diversity is worth more than incremental “signal”. Evolution acts like a basket of options (I really am a hedgehog). It sees a mutation. If it’s useless, discard. If it’s adaptive, exercise it and let it multiply through the population out-competing those without the adaptation. When discernment becomes random, select for diversity. The downside is limited, the upside is massive learning.
Wrapping Up
Diversity is valuable. The word is highly politicized today. There are many arguments, with varying degrees of merit, harping on diversity in the name of fairness. Those are debates that need to be had. But this is not that debate. This has been an argument that diversity is important as a matter of efficiency and flourishing. Perhaps the argument isn’t needed, but I suspect that many reactionary arguments against the equity angles of diversity may dilute the value of diversity as a general concept.
We have seen diverse dimensions all around us: social, cognitive, and values themselves. It’s worth unloading the hangups artificially narrowing the meaning of a beautiful word — “diverse”.
Today’s letter is brought to you by the team at Mutiny Fund:
How can you access a multi-asset strategy concerned with protecting assets and growing long-term wealth?
The Cockroach Strategy seeks to achieve higher long-term, compound growth compared to traditional stock/bond-focused portfolios with more limited drawdowns. It is intended as a total portfolio, a ‘set it and forget it’ approach that strives to give investors peace of mind and meaningful capital appreciation.
The Cockroach strategy consists of a diversified ensemble of assets including stocks, bonds, commodity trend strategies, long volatility strategies, and gold. It is designed to perform across multiple macroeconomic environments: growth, recession, inflation and deflation.
The Cockroach strategy gives investors exposure to asset classes designed to perform in each of those environments including stocks, bonds, commodity trend strategies, long volatility strategies, and gold.
Disclaimer: Investing is risky, and you are reminded that futures, commodity trading, forex, volatility, options, derivatives, and other alternative investments are complex and carry a risk of substantial losses; and that there is no guarantee the strategy will perform as intended.
Money Angle
I’m trying to read books again. I’m like the opposite of a junkie. I start and just give up. I struggle to watch movies for similar reasons. It’s hard for me to sit still and be passively entertained for extended periods of time.
[I recently admit that I haven’t seen Animal House, I saw Trading Places after I had been a trader for a decade, I saw Caddyshack 2 years ago. I’d like to claim ESL but I was born in Brooklyn. Also I’d be doing a disservice to immigrants, many of whom have large movie repertoires because it’s an effective way to learn English. I have no excuse. I could have watched more classic movies instead of fraying the tape of the Dazed And Confused VHS.]
In an era of amazing tv shows, my diet mostly consists of those that come in 30-minute episodes. I’m making my way through all the seasons of Curb Your Enthusiasm again. I’m also watching it backwards. For example, when season 3 ended, I go to Season 2, episode 1. This isn’t the first time I admit to being a serial killer. Pun intended.
Back to books. To get my groove back, I committed to reading 1 chapter a day. It worked. The streak is intact and I got to cross a title off the list that I’ve wanted to read for a long time: Superforecasting — The Art And Science Of Prediction by Phil Tetlock and Dan Gardner.
I really enjoyed this book and would consider it canon for aspiring analysts and traders. You can find my notes below. Here are the high-level takeaways:
The world is complex (a butterfly flaps its wings and small change in initial conditions leads to disproportionate outputs). So our ability to forecast is existentially limited.
But some people are demonstrably better forecasters and it’s not just luck. They are consistent.
Those people are not geniuses. But they have a mix of qualities and procedures that enable them to forecast at a high level.
The key to getting better, same as in sports, is useful feedback and iteration. It takes hard work, “deliberate practice” and a commitment to get better.
Getting better is necessarily a conscious, painstaking process because our natural biases conspire against our judgment in areas where causality is opaque. Which are most questions of interest.
How groups can hinder or improve forecasting. The role of diversity in group decision making.
In many contexts, making accurate predictions is actually undesirable. Regrettably, I want to shout Moloch everywhere I look. The expression to look for in the notes: "kto, kogo?" It’s a disheartening idea.
The leader’s dilemma: balancing decisiveness in the face of uncertainty and intellectual humility
Objections, progress, and goals in the endeavor of forecasting
Continue to…
Notes On Superforecasting (Moontower Book Guides)
Last chance to apply to the free StockSlam workshops I’ll be hosting with Tina and Steiner in NYC the first week of October.
Details and application:
Last Call
A year ago, I talked about seeing comedian Sheng Weng at the Punchline in SF:
His powers of observation on the mundane details of daily life are Seinfeldian. His signature tone and voice deliver self-skewers that you can’t help but turn on yourself.
One of our friends said it was the hardest she can remember laughing for so long. Sheng’s a craftsman and the more you listen to him the better it gets.
He’s actually Yinh’s college friend (they bonded over their rhyming one-syllable Asian first/last name combos). This spring we were supposed to go down to LA to see the taping of his Netflix special (Ali Wong produced it) with my friend Khe and his wife. We had to audible at the last second and missed it, but…it’s finally out!
Check out Sweet & Juicy (Netflix)
Speaking of Khe, he has been a most generous sherpa to so many people trying to raise their productivity game. He’s an alien jedi.
Check out his upcoming bootcamp. Oh yea, it’s totally free.
If you are curious at all as to why I’m always touting Khe and his community, do it.
The $10K Bootcamp is a free 3-day event hosted by Khe Hy from RadReads. During this free event, we'll teach you how to design a system to achieve your goals (while working waaay less). You'll learn how to:
Stay focused on the most important things in life
Think bigger (versus making smaller things better)
Stop putting things off until some imaginary future date
Invest in improving your mind, career and relationships
Event Details:
Cost: Free
Tuesday, September 20th, 11:30 am ET (view in your time zone)
Wednesday, September 21st, 11:30 am ET (view in your time zone)
Thursday, September 22nd, 11:30 am ET (view in your time zone)
Free access to all recordings
From My Actual Life
I saw Nine Inch Nails last Sunday night at the Greek in Berkeley. (Between Leon Bridges last week and the upcoming King Gizzard and The Lizard Wizard show I think I’m asking to be swallowed by the Earth. The Hayward fault runs beneath Cal’s football stadium and the theater. At least I’ll be in my happy place when the worms eat me).
I never owned a NIN album but since I'm alive I know like 6 or 7 songs. Any preoccupation I’ve ever had with Trent Reznor revolved around him renting 10050 Cielo Drive in the Hollywood Hills — the home where the Manson apostles murdered Sharon Tate and her celebrity friends. The recording studio in the house was called “Le Pig”, a reference to a song on the Beatles’ White Album from which Charles Manson inferred a lot of craziness. The killers smeared the word “pig” in blood on the home’s front door. Downward Spiral and Broken were both recorded at Le Pig. The music video for Gave Up features images of the house and studio. (Not to be extra creepy after I already used the word “serial killer” once in this letter, but the house I used to own here in CA looked similar to that infamous abode before we renovated it).
But I must admit. NIN is in the running for best show I’ve ever seen (n of roughly 200). The gulf between the recorded & live experience is indescribable.
The Greek felt like a rave at a temple. In black t-shirts.
Moontower On The Web
📡All Moontower Meta Blog Posts
👽Moontower Volatility Wiki
🧀Moontower Money Wiki
✒️Moontower’s Favorite Posts By Others
🔖Guides To Books I Loved
🎙️Moontower Music
🍸Moontower Cocktails
Becoming a patron
The Moontower letter is and will always be free. My writing is a search “for the others”. The “others” are people like you who are unlearning the mental frames that artificially narrow our choices.
If you are here you already understand that inspiration is a tradable good. It’s not as tangible as a cup of coffee, but it packs 10x the adrenaline with an infinitely longer half-life than caffeine.
If you feel inspired, you can upgrade to becoming a patron.
I watched that Sheng Weng Netflix special and it was GREAT!!! He was so funny, I would love seeing him in person, that would be a blast.