Lessons on creating a $17 billion learning company, going viral with an owl, and ditching short-term thinking with Luis von Ahn (Duolingo)

Luis Von Ahn (00:00:00):
The problem with these short-term things is that the short-term passes, even if it's like two years, two years pass, and then two years later you're like, crap-

Eric Ries (00:00:09):
Oh, yeah.

Luis Von Ahn (00:00:09):
Shouldn't have done that. And so there have been definite cases where we've done things and then a year later we're like, ah, we knew what we were doing was bad and we shouldn't have done that. So I think the biggest thing is just this is where this principle take the long view comes from. It's like we really mean that we want this company to be thriving in many years and whatever many means. I mean, we like to throw around that it's a hundred-year company. I don't know if a hundred is the right timeframe, 50 years, 20 years, I don't know, but it's not two years.

Eric Ries (00:00:43):
Welcome back to The Eric Ries Show. If you've ever seen a video of Duolingo's quirky green wwl and thought to yourself, that is totally unhinged, it's actually by design. Today I'm joined by Luis Von Ahn, the co-founder and CEO of Duolingo. If you've ever tried to learn a new language, there's a good chance you've encountered Duolingo's iconic green owl. With over a hundred million active users and the title of the most downloaded education app in the world, Duolingo has revolutionized language learning and, more recently, expanded into teaching math and music. Today, Duolingo is a public company with a market cap north of $14 billion, all from that little green owl. Behind the success of Duolingo is an incredible story of innovation, resilience, and a company culture that truly embraces long-term vision, with a side helping of viral unhinged fun. In this episode, we talk about how the brand attributes of that green owl are intentionally unhinged but wholesome, as he puts it.

(00:01:43):
And most importantly, we talk about something new, something called the Duolingo Handbook that is actually launching today. This is the manual for how Duolingo teaches its own employees how to stay true to its vision and purpose. An example of in the episode you'll see I call a Leader's Guide after the first one I ever wrote. We'll link to it in the show notes, but this manual is a brilliant how-to. It explains Duolingo's unique approach to balancing monetization with mission, how they resist the temptation of short-term profits in favor of a century-long vision. Plus, you'll see the operational details of how they do it day in and day out, keeping employees aligned and putting customers at the center of everything they do.

(00:02:28):
Plus you'll also hear some early stories. For example, the turning point when Luis and his co-founder realized their product was too boring even for them, and how that challenge sparked their obsession with making learning as engaging as a game and the importance of dog-fooding your own product. If you want to understand how it's possible to build a mission-driven business without sacrificing growth, or if you're just curious about the behind the scenes journey of a company that has touched so many of our lives, this episode is for you. Here's my conversation with Duolingo's, Luis Von Ahn.

(00:03:01):
Hey Luis, thank you so much for doing this.

Luis Von Ahn (00:03:04):
Thank you for having me here, Eric.

Eric Ries (00:03:06):
It's really my pleasure. I want to start with Duo, because you guys have maybe the most viral and crazy social media success certainly of any company in your category. And when you think about ed tech software companies, you don't think of really unhinged owl doing awesome social media stuff. So just like, I love in the Handbook it talks about that Duo's brand attribute is meant to be wholesome but unhinged, which I just thought was such a great phrase, but just tell me a little bit about the origins of that. How did you come to be such a social media powerhouse? That never struck me as your MO, it's like a power social media user. How did it happen?

Luis Von Ahn (00:03:44):
Yeah, I mean, a lot of things came together. So we've always had a mascot and it was a green owl. This when we started Duolingo, we thought it would be good to have a mascot to encourage learners to continue learning. So there was that. We chose an owl because it's tied, at least in Western civilizations, it's tied to knowledge. The owl is green because my co-founder, Severin, hates the color green, and we thought it would be funny to make the owl the color that he hates. So we've had a green owl for a while. At some point, the notifications that Duolingo sends you started being sent from the owl because the icon, the app icon, is the owl's face, and it looks like the notifications come from the owl. At some point, those notifications started being a little passive-aggressive. So for example, we came up with one, we send you a notification every day to remember to use Duolingo, but if you haven't used Duolingo for five days in a row, we stop sending you notifications.

(00:04:49):
At some point, we decided that it was a good idea to say that we're actually stopping. So we started on the fifth day of inactivity, we would send you a notification that said, "Hey, these reminders don't seem to be working. We're going to stop sending them for now." And that's a very passive-aggressive thing to say. And pretty soon all kinds of memes started happening based on that, as if our owl is very passive-aggressive. And then it just escalated. People were saying, "Oh, the owl is going to come and kidnap my family if I don't use Duolingo." Et cetera. So it was the internet came up with these memes of our green owl, Duo, being very insistent and he really wants you to do your lessons and he'll go through great lengths to get you do to your lessons, including kidnapping your family.

(00:05:43):
At first that's just what the internet was doing. Then we started leaning into it ourselves. And so we started some April Fools' campaigns that where the owl would actually... There was one of our first April Fools campaigns was we made one of these cheap local TV lawyer ads, but it was an ad for a lawyer who was saying, "If a loved one or your family has been kidnapped by an owl, I'll get you some compensation and I'll get your people back." So that was our April Fools' campaign, we actually made one of these crappy commercials. And so that had a lot.

(00:06:29):
So that was going. At some point we didn't have a TikTok account, and we had just hired a young woman named Zaria who just come out of college and she was not quite an intern, but it's intern age. And she was like, "Well, we should have a TikTok account." And we're like, "Okay, fine. You can do whatever you want with that TikTok account because we don't have a TikTok account." And then she showed us what she was going to post, and it was these unhinged videos of basically somebody in an owl suit but doing weird stuff. And I personally didn't think this was going to get very much traction, but it turned out that people really liked the owl suit doing weird stuff. And it has grown quite a bit since then. I mean, at this point that owl suit has been invited to movie premieres, all kinds of things, very famous.

(00:07:28):
But yeah, it happened in part because the internet invented it and in part because we had a very junior employee who, I don't know how much she knew what she was doing. In retrospect, she probably will tell you that she knew what she was doing, but she did an amazing job and I was against it.

Eric Ries (00:07:50):
This episode is brought to you by my friends at Wilson Sonsini, the innovation economies law firm. From the earliest days of the semiconductor industry to today's fast advancing AI sector, not to mention healthcare and life sciences, Wilson Sonsini has represented the pioneers behind nearly every technology breakthrough. Based in Silicon Valley, Wilson Sonsini is connected to the top minds and institutions that inspire discovery and innovation, with strong ties to leading venture capital firms and investors in key markets from Silicon Valley to New York and London and beyond. In fact, Wilson Sonsini is consistently top ranked for the number of venture financings that handles each year. For more information, visit ecvc.wsgr.com.

(00:08:32):
Well, I love that. It's just a great lead into talking about the Duolingo Handbook and just the company building philosophy of that, because at every point of that story, there's so many moments when some other company would've definitely gotten off the train. Starting with the very beginning when people have internet memes made about them, most companies' reaction is defensiveness. How dare they do that? That's offensive. They can't use my IP. They can't do this. They have such a tightly controlled message.

(00:08:57):
But then even more so to me, companies talk about hiring top talent all the time, and yet, first of all, they don't really take it that seriously. And second of all, when they do do it, they have this tendency to want to tell them what to do. And I'm always like, if you're hiring the best people in the world, why are you obsessed with telling them what to do? It seems exactly backwards. So talk about just the humility of being willing to take a chance with one of your most precious assets, the corporate brand, and let somebody run with it, even though you yourself thought it was a bad idea.

Luis Von Ahn (00:09:24):
Yeah, the good news, one of our principles is make it fun inside the company. And we don't take ourselves too seriously. I mean, we work seriously, but we don't take ourselves too seriously. So historically, we lean into anything that is funny inside the company or outside the company. And so with the social media, as long as it wasn't offending anybody, we didn't care. And so I think that has been a good principle, just lean into the fun because it makes it so that the workplace is a lot more enjoyable. And in addition to that, our app really stands out in a lot of ways where it many times does things that apps are not supposed to do. Sending you messages, like one of the messages that the owl sends, one of the notifications is like, "Hey, babe, I miss you." Which by the way, we have heard from people that get in trouble because their significant other thinks that they're getting a notification from an affair they're having or something, but it's just our owl. So we do stuff like that because we find it funny.

Eric Ries (00:10:40):
This is such an interesting combination of factors because you have on the one hand, not taking yourself too seriously. But on the other hand, in the Handbook especially, it talks a lot about this commitment to excellence, holding a high bar, the importance of quality. But on the other hand, it also talks about being long-term and not going after short-term metrics, not going after short-term monetization, but still building a sustainable business. So a lot of people, I think, would look at that combination of things and say it's a little bit contradictory or there's a tension between those principles. Talk about how you resolve that tension in yourself and then how you've done that for the company too.

Luis Von Ahn (00:11:16):
Yeah, there's definitely a tension in our principles. I mean, we have these, it's five principles that are outlined in the Handbook. One of them is called Take the Long View, which basically says, "Don't do short-term gimmicks or short-term hacks." A great example of not doing a short-term hack is we could make a lot more money tomorrow if we double the ad load, and we would make a lot more money tomorrow and probably our stock would go up. The problem is that short term thinking, because if we double the ad load, our user growth would slow, and therefore in five years we would probably pay for that. And so we're more interested in what's going to happen in five or 20 years rather than what's going to happen next quarter.

Eric Ries (00:12:01):
Well, you know I love that.

Luis Von Ahn (00:12:04):
And we follow that principle quite a bit. That's not to say we've never done dumb short-term things, we sometimes have, but generally we're pretty good at following that principle. So that's one. You are right. That is a little contradictory with something like having the owl doing crazy things, but we don't see it that way. In the case of the owl doing crazy stuff, we actually think we're building a brand for the long term.

(00:12:32):
And again, we don't do things that are going to hurt us in the short term, but just anything that gets the owl to be more famous, I think that we're good with that. And the other one that you brought up is we have this other principle that is called Raise the Bar, which is basically just talking about a lot of excellence. That one is interesting because in the product, it has to be pixel perfect, and we have these very stickler designers that will not let one pixel off. And it's funny that our TikTok is, I wouldn't say it's pixel perfect, I mean, in fact it does really weird stuff.

Eric Ries (00:13:12):
No, it wouldn't work. It would lose its authenticity if it was [inaudible 00:13:15].

Luis Von Ahn (00:13:15):
Yeah. When we say raise the bar, we don't mean that everything has to be pixel perfect. It just means everything has to be excellent. And there's some stuff that when it's authentic, it's excellent. And I believe our TikTok is excellent, even though the production value, by the way, costs, I've seen them film these videos. I mean, they filmed some of them with me in them. It's just like a 22-year-old with an iPhone filming those videos. So we don't have a fancy camera or anything, but I think the outcome is excellent.

Eric Ries (00:13:46):
I can't tell you how many people sent me the video on the day of the TikTok ban of the owl being thrown in the garbage.

Luis Von Ahn (00:13:53):
Oh, yeah.

Eric Ries (00:13:54):
It's all over. That made such a big impression, and you can feel the authenticity of it. There's no high production value. It's not some stage set with special effects. It was cool.

Luis Von Ahn (00:14:04):
But that was pretty funny. I saw them film. I mean, I was in that video, although I was just of typing stuff.

Eric Ries (00:14:07):
I remember. That's what made me think of it. Yeah.

Luis Von Ahn (00:14:07):
I was just typing stuff. But they were honestly just having fun with it and genuinely sad that TikTok was going away. It turned out TikTok didn't go away, but they were genuinely sad.

Eric Ries (00:14:21):
More like a TikTok kayfabe than a TikTok ban, certainly.

Luis Von Ahn (00:14:24):
Yeah, exactly.

Eric Ries (00:14:25):
Yeah. Okay, so I want to get back to these practices, but one thing I noticed when founders who are as successful as you've been talk about building their company, people sometimes feel like, man, that just seems too hard. How could I ever get to that level? Because of course, you're talking about the green machine at the level that you're at today. So take us back in time, talk about your journey into entrepreneurship. How did you even conceive the idea of becoming an entrepreneur in the first place? And you've done a lot of stuff before Duolingo. Give us a little sampling of your background and what it was like to come here and do this.

Luis Von Ahn (00:15:02):
Well, a couple of things. I mean, first of all, I did not expect to be an entrepreneur. I am a little bit of an accidental entrepreneur. Now, that doesn't mean that I'm not happy that I'm an entrepreneur and things have worked out, but I did not expect that. From very early in my life, I mean, when I was maybe 12, 13, I wanted to be a professor, which is weird, I think most kids want to be like a detective, I wanted to be a professor, which tells you how nerdy I was. So I wanted to be a professor, and I did. I became a professor.

Eric Ries (00:15:31):
What did you want to become a professor of [inaudible 00:15:33]?

Luis Von Ahn (00:15:33):
At first math, but eventually I became a professor of computer science.

Eric Ries (00:15:37):
Sure, yeah.

Luis Von Ahn (00:15:38):
So I became a professor of computer science and I was a professor of computer science, but then I started this project that was called ReCAPTCHA. That was not a company, but it was, I think a lot of people have seen it. It's basically these annoying, by now, what they look like is you have to pick all the pictures that have a traffic light or something like that. Back then it looked like distorted words. So you had to type these distorted words. This was an academic project inside Carnegie Mellon University. But then it turned out that all kinds of companies started using it, even though it was an academic project. Facebook started using it as their signup form. And we were actually making money, but we were still an academic project, and we were getting checks for $50,000 and they were coming to my house.

(00:16:25):
And at some point, Carnegie Mellon University got wind that this was happening, and the lawyers came to my office and they were like, "You can't do this. You're still an academic project, but you're making money off of it. You can't do that." And I was like, "I don't mean, this is just what's happening." And then they told me I had to spin out as a company, and so I did.

(00:16:45):
And that's what started. So that was my first company. It was a company called ReCAPTCHA. I'd eventually got sold to Google, and then after that I started Duolingo. Again, interestingly, it started as an academic project. We were inside Carnegie Mellon University. And then the reason that became a company was because at some point we had maybe about five people working on it inside the university, and we knew we needed to hire more people. And then hiring more people was going to cost us, I don't know, a couple million dollars. Inside the university, that would've required applying for a grant usually from the National Science Foundation. And I started writing that grant and it was going to be like an 80-page document that I needed to write, and we were not going to find out about the funding for nine months or something. And then at around that time, I met these guys, Fred Wilson and Brad Burnham from Union Square Ventures-

Eric Ries (00:17:37):
Sure.

Luis Von Ahn (00:17:38):
Who just on the spot, I mean, it wasn't on the spot randomly, but it took about a week for them to offer me a $3 million check. And I thought that was a lot easier than writing an 80-page grant proposal. And this is why we turned Duolingo to a company. And that's what started as a company.

Eric Ries (00:17:55):
Did CMU get a cut of the equity in the company?

Luis Von Ahn (00:17:58):
CMU got a cut, which by the time we IPO'd was worth more than a building. And I am upset that they don't have a building named Duolingo. I would like that. Or a building with a green owl. But, yes.

Eric Ries (00:18:11):
There should be a statue of the owl at least. Yeah, certainly.

Luis Von Ahn (00:18:13):
At the very least, there should be a statue of the owl, but there's not. So if any of you, Carnegie Mellon head honchos are listening to this, please, I would like a statue.

Eric Ries (00:18:24):
That's really funny. Okay, so you had been, and do I remember right that you were a MacArthur Genius Grant award winner?

Luis Von Ahn (00:18:31):
I did get a MacArthur Genius award.

Eric Ries (00:18:33):
So tell us what that's like. Just talk about that for a second.

Luis Von Ahn (00:18:36):
Yeah, I got that. That was a million years ago. It was in 2006. I had just started being a professor in computer science. It was because of my work on reCAPTCHA. I got a phone call in my office, not even on my cell phone. I never got phone calls in my office. I got a phone call in my office from somebody that just said, "Have you ever heard of the MacArthur Fellowship?" And then I said, "Yeah, yeah, I've heard of it." And they said, "This is not a prank. You've won it." And it's this weird phone call that you get, a complete surprise. And I was very proud of that, but I didn't know what to do with it. But I'm very happy that I won that and it's an amazing program.

Eric Ries (00:19:20):
Talk a little bit about that, because I think one [inaudible 00:19:23] of the things that's really interesting, reading through the Handbook and just I've known you for a while, and Duolingo obviously is well-known. There's this sense of purpose that really just is emanating from the document that although this is a company, yes, you're having fun, yes, there's an economic engine to it, there's something deeper going on. And reCAPTCHA was the same way. I could see a lot of people, maybe a lot of our listeners may not be old enough to quite remember what a disaster the spam wars were before the idea of CAPTCHA was invented. And that was a really important public service that you performed on behalf of the whole internet. And yes, it eventually became a company. Yes, it eventually got sold to Google, but it wasn't really your motivation. What people normally think of as someone who's out to make money, that's not really what they have in mind.

(00:20:08):
So can you talk a little bit about what is the thread of motivation that has guided you through these different projects that have led you having all this success? But I always got the sense from you that it wasn't about, you weren't grasping for the fame and the success. That was more like a side effect of something deeper.

Luis Von Ahn (00:20:23):
I think that's about right. In the case of Duolingo, it's always been a very mission-driven company. When we started, we knew that we wanted to do something related to education. I grew up in a poor country, I grew up in Guatemala, and I saw what the difference is between those who have access to a good education and those who don't. That's true in the US. I mean, if you have money, you end up having access, you end up going to a fancy university, et cetera. And then those who don't have very much money don't, and usually go to a lower end school, et cetera. And therefore never end up making money because they don't have a very good education. Whereas people who do-

Eric Ries (00:21:04):
Yeah. It becomes self-perpetuating

Luis Von Ahn (00:21:07):
And it's like 10 times worse in a poor country. And so I saw that and I was fortunate that I got the education of a rich person, even though I didn't grow up rich, I was fortunate that I was an only child of my mother who was a doctor, and doctors in Guatemala are not rich, but at least they make an okay living. But she spent all of her salary and everything on my education. So I got the education of a rich person, and I saw the difference between the education I got versus the education that other people in my neighborhood got, which was way worse.

(00:21:38):
And so we wanted to do something that would give equal access to education to everybody. That was the idea with Duolingo. And so my co-founder, Severin, and I were big believers in this and we started that. Now, the first few people that we hired with Duolingo, they were all Carnegie Mellon students, and then we hired them to be at Duolingo. We loved the mission, but they were zealots because these people, they came in and they had the chance to go get a job at, at the time it was Google or Facebook making, I don't know, two or three times what we were offering them. But they came to work for us for the mission. So the first, call it 10 to 15 employees put their money where their mouth was. They were really, really zealots for the mission. So what that did is over the years, there was just this group of early employees who was very gung ho on the mission.

(00:22:35):
And so everybody who came in, they were either really bought into the mission or quickly got bought into the mission because these early employees were really into that. So I think that just perpetuated, and by now Duolingo was still quite a mission oriented company and I think we're all big believers in that. And I think most everybody that works here, this is not true of a hundred percent of the people, but I would say most everybody that works here is a big believer in the mission, big believer in education. And they're very happy that it's hard to argue against what we're doing. So if you work, of course I'm a big fan of a lot of big tech companies, but you can argue against, oh, I work for Meta. Well, I don't know, are you interfering with elections? You can argue against those where it's hard to argue against the stuff that we do because it's just teaching people something. So I think a lot of the people that work here really like that.

Eric Ries (00:23:32):
I appreciate you giving a shout-out to those early employees because in the press, the way that startups are written about, it's always the founder-

Luis Von Ahn (00:23:38):
The founder does this amazing thing.

Eric Ries (00:23:42):
Yeah, you're an official genius TM founder, so it's even worse for you. Genius founder, knew everything, figured everything out. And there's always these people in companies, I call them the torch-bearers. They're like the people who really, as you say, put their money where their mouth is, put their life on the line to make the thing a reality. And of course early employees, you tend to have those people. But in a strong culture company, you'll have new people enter into that status where they have the moral authority internally to just say, look, this is what we stand for, this is what we're doing. And they're not always the people with the biggest title, they don't always get paid the most. They very often do not get the public recognition. But internally, if you ask around who are those [inaudible 00:24:19], they're known, and that interaction between founders and those people, so much of the value of startups comes from that moment.

Luis Von Ahn (00:24:25):
A hundred percent. I couldn't agree more. I mean, there's a group of, I don't know the number exactly, but there's a group of, call it 30 to 50 people inside Duolingo that are really, this company is built on their backs. I mean, every employee has contributed of course, but there's this group of people that have been around for a decade or so, and I'm good friends with all of them. I am extremely thankful for everything that they've done. And honestly, I've done only a small fraction when you compare it with what this whole group has done.

Eric Ries (00:24:57):
Oh, yeah. I remember hearing that it was a big part of Steve Jobs's leadership style is that he would have a regular retreat off campus from the Apple campus with top executives from the company, but it was not chosen by rank or hierarchy, but just by his personal sense of who he thought the really up and coming key people when it was a major political battle to make sure you were one of those people. And it was very controversial, I mean, it still is a very controversial practice, but it really established this idea that there are these key people who are really driving the Apple way. I always liked that-

Luis Von Ahn (00:25:30):
I think that's 100% true of Duolingo. What may not be true is my opinion of who they are versus reality may not be a hundred percent-

Eric Ries (00:25:38):
Yeah, that is the downside-

Luis Von Ahn (00:25:38):
... aligned.

Eric Ries (00:25:38):
... to doing it.

Luis Von Ahn (00:25:39):
That is the downside. I have my own opinion. Reality may be a little different, but I think I'm probably at least 80% right.

Eric Ries (00:25:46):
Yeah. Well, and the important thing, the lesson for me is that you take responsibility for it. That is, you're not just like, well, whatever happens with the people I hire, they go off and do whatever. You have a real sense that the company has a certain integrity that has to be maintained.

Luis Von Ahn (00:26:01):
Yeah. Yeah.

Eric Ries (00:26:03):
And your customers can feel it. It's one of those things going back to the theme of authenticity, you can't fake it.

Luis Von Ahn (00:26:09):
Yeah, I mean, that's how I feel.

Eric Ries (00:26:11):
Talk a little bit about the early days. What was the first product like? How did you get to the early traction that let you, for example, raise money from our friends at USB or have a early breakthrough as it got you to product market fit?

Luis Von Ahn (00:26:23):
It took us about a year and a half to launch something publicly. So we worked on it internally for about a year and a half. We ran into a big problem actually when we were working on it early on. We knew that we were going to teach languages and we knew that it was going to be a free way to teach languages and it was going to be the computer that taught you the languages. We knew that. We had this great idea that I'm a native Spanish speaker, so I was going to make the first Spanish course, and then my co-founder, Severin, is a native German speaker, and he was going to make the first German course.

(00:27:00):
Turns out, actually he's not a native German speaker. Turns out he's a native Swiss-German speaker. And I did not know that those were two different languages. I thought it was British-English, it's not. It's like a different language. Anyways, he does know German, so he made the first German course. I'm going to assume that it sounded a lot like Swiss-German. It's now completely been redone, by the way. But anyways, we made those courses and we thought we had this great idea that I was going to learn German and he was going to learn Spanish, so we were going to learn each other's language through our software.

(00:27:35):
So we started doing that. We ran into this huge problem that was, I would come into the office every day and then I would say, "Hey, Severin, did you do your Spanish?" And then he would say, "No, man, so boring." And I had the same problem that it was, my God, learning German on that thing was so boring. And then, that was the problem we had. We were like, "Well, we can't get ourselves to use our own thing. We're pretty committed to this and we can't get ourselves to do it, so how can we expect others to do it?" So we started freaking out, and this is when we made it a big thing to make it as enjoyable as possible, and this is when we learned something that I still believe today, is the hardest thing about learning something by yourself is staying motivated.

Eric Ries (00:28:25):
Yeah.

Luis Von Ahn (00:28:25):
So we spent a lot of effort trying to make the thing be fun enough that we would like it, and that just meant turning it into a game as much as possible. So the lessons used to be 30 minutes long. We shortened them to be like three minutes long. We added progress bars everywhere. We started adding points every time you finish a lesson, et cetera. And we made enough changes that we found ourselves enjoying it. And so that was a big thing. And so by the time we launched, we were fortunate that we launched a thing that was pretty good in that it taught you languages, it was free, and it was enjoyable. I think that combo was pretty powerful, and that's what got us to grow at first, it's just this combo that was free and fun. You got to understand at the time when we launched, which was 2012, the thing that was the standard was this thing called Rosetta Stone.

Eric Ries (00:29:25):
Sure. I remember Rosetta Stone. Yeah.

Luis Von Ahn (00:29:28):
Yeah. They were mainly selling CDs and it was about 500 US dollars. We came up with this thing that was an app and it was free. It just turned out that that combo, app, free, and also fun, was just a lot better of an offering for people than a CD that costs $500. And so that got us to grow quite a bit.

Eric Ries (00:29:54):
That's a classic your margin is my opportunity moment.

Luis Von Ahn (00:29:59):
Yeah. They didn't have to sell that for $500.

Eric Ries (00:30:03):
The CDs are not gold-plated. Yeah, exactly.

Luis Von Ahn (00:30:05):
Yeah. Also, they probably should have gone on mobile a little sooner than that.

Eric Ries (00:30:10):
Well, yeah, it's just Innovator's Dilemma 101.

Luis Von Ahn (00:30:12):
Yeah, yeah. And by the way, yes it is because later they saw what we were doing, but they were stuck because they spent a lot of money on marketing, and in order to make that work, they needed to charge that much money. And so they were stuck. We were like-

Eric Ries (00:30:29):
I could feel the pain of those meetings where they must have been trying to figure it out. Yeah.

Luis Von Ahn (00:30:32):
But I really think there was nothing they could have done. I really think they were just like, well, what are you going to do? And so we were there. We were not spending any money on marketing, we were just growing a lot and we were free.

Eric Ries (00:30:43):
They had to have bought Duolingo at that time. That was their only solution.

Luis Von Ahn (00:30:47):
I mean we were not interested in selling, but that is probably what they would've had to do.

Eric Ries (00:30:52):
No, I think it's very important to the story. For people who don't realize is that one of the advantages of being a mission-driven company is you can make promises to your customers that they can believe because the typical mercenary company that is for sale, you never know when some horrible company's going to come in and buy it. So even if you like the company, even if you believe in what they're doing, they're one phone call away from one very rich person becoming really crappy. So it's really difficult to have that kind of trust with customers if you do that.

Luis Von Ahn (00:31:23):
Yeah, yeah. We were definitely, I mean everybody that came in, came in thinking we are going to do free education for the world, that's what we came in. And by the way, there's another interesting thing that happened was we weren't making any money at first, it was just free. And the way we supported ourselves was through venture capital. We had enough venture capital we could pay for salaries. But-

Eric Ries (00:31:45):
Did I read right that it was Laela Sturdy, our mutual friend who was-

Luis Von Ahn (00:31:48):
Laela Sturdy is the person that forced me to start making money.

Eric Ries (00:31:52):
Okay, tell that story. Tell that story. Explain who she is, first of all.

Luis Von Ahn (00:31:55):
That's what happened. So we grew up, we had grown, we had, I don't know how many users, but we had millions of active users at some point. At the time it was called Google Capital, but now it's called CapitalG, it was this venture firm, later stage venture firm invested in us when we had zero revenue. And so Laela invested in us when we had zero revenue and the valuation was something like half a billion dollars or something like that and zero revenue.

Eric Ries (00:32:20):
For those who don't know, there's some people who I'm sure are listening who are not familiar with startup economics and their mind is exploding right now. How can a company with no revenue be worth half a billion dollars?

Luis Von Ahn (00:32:29):
Yes. I think that was different times in that I think it was more okay to have no revenue back then.

Eric Ries (00:32:36):
I don't know. I know a fair bit of AI companies with no revenue right this past second.

Luis Von Ahn (00:32:45):
AI is now back, you're right about that. But I think it was more okay to have no revenue. And I think also we had a lot of user traction. We were growing organically, no marketing spend and a lot of user traction. So I think the thesis was, look, at some point these people are going to figure out how to make money, but they have a lot of user traction. I think that was the thesis. And so they invested in us, but what I didn't know is that quickly after they invested in us, Laela flew, she's based in San Francisco, she flew here to Pittsburgh, we're in Pittsburgh, and she took us to a bar, Severin and me, and it turns out she can out drink us, and she out drank us, and after enough drinks, she just starts basically telling us, "Look, I don't care what you do, but tomorrow you need to start figuring out how to make money."

(00:33:35):
She said this amazing line. She said, "Look, there is no bigger fool than Google." You're not going to find a bigger fool, another venture firm that gives you a higher valuation. You have to make money. And so anyway, she scared us enough that the next day, hung over and all, we started trying to figure out how to make money, which by the way was actually quite hard in that not figuring out how to make money exactly, but culturally for the company, almost everybody who came in here came in with a promise of free education. When I said, "We need to start making money," they were like, "But why?"

Eric Ries (00:34:12):
Alarmed, I'm sure.

Luis Von Ahn (00:34:13):
And they thought making money was evil. That was the premise. And it took me about six months to convince everybody. At the time we had maybe, I don't know, a hundred to 150 employees. It took me about six months to convince them, particularly that old guard of employees that came here, the early ones.

Eric Ries (00:34:35):
Yeah, that's what I was going to ask about because why isn't it evil to take a free education thing and make money off it? How did you make the moral case that this is the right thing to do?

Luis Von Ahn (00:34:46):
I don't think it's evil. I don't think making money is evil. And the case, and I think I am a big believer and I believe it's been true, is look, we have about 150 people and we can't really grow very fast in terms of employees and therefore in terms of what we work on and in terms of what our impact is if we don't have more money.

(00:35:08):
So making money actually will allow us to satisfy our mission much faster. And boy, it's been true. I mean as soon as we started making significant amounts of money, we were able to start hiring way more and we were able to start working on many more things. And so we really expanded our courses, by now, we started teaching other things like math and music and it's all because we have this nice revenue stream that allows us to do that. I think if you're not making money, you can't do that. And I think it is not true that there are no successful nonprofits out there. That is not true. However, I will say it is rare to find nonprofits out there that have that much impact. There are some, but it is rare to find.

Eric Ries (00:35:57):
Yeah. There are some exceptions.

Luis Von Ahn (00:35:57):
And I think it's because if you don't have this revenue stream, it's just hard to have that much impact. I think that's just how it is.

Eric Ries (00:36:04):
It's a very rare thing when you have this true alignment between the mission and the business model such that the more the mission is accomplished, the more money the organization makes and the more that can be reinvested. And one of the most common ways that that alignment gets broken is through greed. One of the things I really liked about the Handbook and talking about the allergy to short-termism is there's lots of ways you could make money today, you already mentioned one, you could have way more ads, you could dramatically increase the number of notifications, you could raise prices, you could reduce the quality of the freemium product. I could give you 25 ideas right now, no problem.

Luis Von Ahn (00:36:39):
And they all work in three months. They all work for three months.

Eric Ries (00:36:43):
If you only look at your metrics three months from now, you'll be great, stock price up. Most companies with for-profit investors simply cannot resist the lure of that behavior. So much so that Cory Doctorow coined the term literally enshittification to describe what happens to most internet platforms when they get put under this financial gravity. And I get how you reassured them in those early days, but now you've been at this quite a few years since then. You've obviously faced the temptation of the one [inaudible 00:37:15] many times, you've found a way to resist it. I think the reason people talk about making money as being evil is there's this widely accepted idea in our culture that that resistance is impossible, that eventually the system will win and you'll give up. Talk about structurally how have you built the company to resist that temptation and to create that alignment and protect it?

Luis Von Ahn (00:37:35):
We have, but I'll start by admitting that we haven't been 100% good at it. We've definitely been tempted, and it's even more stark when you're a public company and you have a stock price. And it's true for everybody involved, including me. You look at it and you're like, man, that stock. I could be worth more money. There's definite temptation. I think the thing that has worked best for us is just this knowledge that for most people who work at Duolingo, this is certainly true for me, and I think most everybody who's making decisions, we have realized that this is probably the last job we'll have. And I don't really want to, I'll be bored retiring, I want to be here for a very long time. And I think that's true for most everybody. I won't say everybody and I'm sure that some of these people will leave, I'm sure. But I think most everybody that is high up, that is making decisions at Duolingo really sees themselves as, we call them lifers. I think a lot of us are lifers.

(00:38:45):
And I think when you realize that, you're like, well, I don't know. I don't want to shoot myself in the foot two years from now. And I like to say the problem with these short-term things is that the short-term passes, even if it's like two years, two years pass and then two years later you're like, "Crap. Shouldn't have done that." And so there have been definite cases where we've done things and then a year later we're like, "We knew what we were doing was bad and we shouldn't have done that."

(00:39:22):
So I think the biggest thing is just, this is where this principle, take the long view comes from, it's like we really mean that we want this company to be thriving in many years and whatever many means. I mean we like to throw around that it's a hundred-year company. I don't know if a hundred is the right timeframe, 50 years, 20 years, I don't know. But it's not two years. And I think that that's what's done it. I got to tell you, I'll give you an example just so that people don't think that it's easy when you hear, a lot of times you hear these podcasts or read books and like, oh my God, they've done everything perfect. And I'll give you an example where we didn't follow this.

(00:40:05):
There was one time a few quarters ago, this was, I know two years ago, I don't don't know how long ago, a year, two years ago where we thought we weren't going to hit the quarterly guidance. So far that hasn't happened so far. We have hit, we have a perfect streak. We have every time beat the guidance.

Eric Ries (00:40:25):
Beat and raise. Beat and raise. Yeah.

Luis Von Ahn (00:40:28):
We've done that and we've done well in the stock market because of that. So we've done. There was one quarter where we thought that wasn't going to happen. It turned out we were wrong actually. It turned out we surpassed it, but we thought it wasn't going to happen. Things weren't looking so good. And my CFO comes to me and he's quite worried, of course this is also an amazing CFO, Matt Skaruppa, his ass is on the line in some sense because this is the CFO who's never missed a quarter. And then he's like, "What are we going to do?" I went home, I thought about it for a second and I came back and I'm like, "I know what to do. We are going to show two ads instead of one." I actually thought that.

Eric Ries (00:41:08):
Oh, yeah.

Luis Von Ahn (00:41:10):
And you know what? For a month we did.

Eric Ries (00:41:12):
No way.

Luis Von Ahn (00:41:12):
And I felt so bad about myself, we turned it off after about a month, but I felt so bad about myself. By the way, it worked. It fricking worked. But that's not the only reason we beat the quarter. It turned out we were going to beat the quarter anyways. But-

Eric Ries (00:41:29):
What a great parable.

Luis Von Ahn (00:41:30):
I was tempted. I was tempted and I did it and it was me who did it. And by the way, my head of product, Jim, who I freaking love, he looked at me and he basically told me he was disappointed, but I still did it. And then a month later it got to me and I'm like, "Guys, we got to turn this off." But yeah, so it happens.

Eric Ries (00:41:55):
Yeah, I think this is originally a Howard Schultz-ism, he called it the reservoir of trust. And you have a similar concept in the Handbook called the Trust Battery, which I want to ask you about. But for those that don't know the reservoir of trust, there's something that's called the culture bank, basically the idea is that if you want to build this strong culture company, purpose company, you're making deposits in an account, you're building a reservoir, it's an actual asset that is a huge source of competitive advantage if you have it. And every time you do the right thing for your customers, for your employees, for someone who matters to you and it costs you something. Obviously saying the right thing's easy, doing the right thing, when things are going well, it's easy. But this is a perfect example. Let's say you were at risk of missing the quarter.

(00:42:31):
Taking that risk is a huge deal. When you make the sacrifice, say no, we're going to stick to our principles even though it's hard, that's like making a deposit in the reservoir. And whenever you do the opposite, you're making a withdrawal. And I see so many companies that have depleted that reservoir. And everything, life is so hard, but they don't really understand why things have gotten hard and they tend to get their butt kicked by companies that have that trust, especially with customers because customer loyalty comes from believing the promises that a company can make. So I really like seeing the trust battery. But, yeah.

Luis Von Ahn (00:43:04):
A huge, huge believer in this. Huge believer in this. And you're right, and I'm fortunate that I think me and most of the executive team, we're all at the moment with very full trust batteries, we call it trust batteries, because most of the times we've done the right thing, but it's not a hundred percent of the time. And that time, I think I took a withdrawal from that one.

Eric Ries (00:43:32):
If it's high, you can feel the pain of it.

Luis Von Ahn (00:43:34):
Oh, I felt it.

Eric Ries (00:43:34):
And I'll never forget, I think this advice I got from Todd Park, the CEO of Devoted Health, and he was saying to me, "Look, it's very simple. If you need advice, only ever make deposits, never make withdrawals." I've given that advice to many people over the years and they always look at me with this horrified look like, that's too hard, man, come on. Everyone makes mistakes. And I'm like, no, no, you don't understand because we're human and we make mistakes. You're going to F it up anyway, so don't F it up extra times. Don't do it consciously. Just don't ever do it and make the company culture such that it's not possible for people to do that.

Luis Von Ahn (00:44:09):
That's really good advice, actually. That's really good advice. Just do that.

Eric Ries (00:44:14):
Yeah. One of the easiest things to say, incredibly hard to do.

Luis Von Ahn (00:44:18):
Yeah.

Eric Ries (00:44:19):
Do you have governance protections and other structural protections that protect the company from investor or activist pressure to do the wrong thing?

Luis Von Ahn (00:44:27):
I mean, we have dual class shares.

Eric Ries (00:44:29):
Talk about why.

Luis Von Ahn (00:44:30):
We were worried that because of how mission-driven this company has been, we were worried that at some point there would be strong pressure to monetize the app more. By the way, a lot of public market investors have told us their thesis on Duolingo is great product, undermonetized. At some point it'll get monetized more. And so-

Eric Ries (00:44:55):
What a comment.

Luis Von Ahn (00:44:57):
Well, they're right. They're right from their point of view.

Eric Ries (00:45:03):
Well, compared to what? Compared to what, and by what standard?

Luis Von Ahn (00:45:06):
Compared to apps that monetize more. If you look at the amount of money we make per user, et cetera, surely we can monetize more.

Eric Ries (00:45:13):
Of course.

Luis Von Ahn (00:45:15):
So we were worried that the pressure would be too high and that there would be some aggressive takeover or some sort, et cetera. So we have this dual class. So it turns out as long as I am still employed here and Severin is still employed here, as long as that's true, we have control of the board and the company.

Eric Ries (00:45:37):
Well, do you think that that has been... Let me ask you this question. Do you think your investors have made more money because those protections are in place than they would've made if you had been subject to this pressure all this time?

Luis Von Ahn (00:45:50):
I don't know the answer to your question. It's very hard to answer these hypotheticals. What I can tell you is this. Not exactly your question, what I can tell you is this. If you look at certainly education apps, most education apps basically charge you to learn. So they just charge for learning. 90% of our users don't pay us, about 10% pay us, but we make, it's something like this, I don't know the exact number, but it's about a 30% of the revenue of all education apps is Duolingo and there's like a million of these education apps. So we make way more money than every other app. And we are the ones that have 90% of our users not paying us. So I'm pretty sure that even though the reason that most of our users don't pay us is a very mission reason, even though that's true, I'm pretty sure that it's also the case that it is the better business model.

Eric Ries (00:46:52):
You may not be sure, but I'm quite sure about it, not-

Luis Von Ahn (00:46:55):
I'm pretty sure.

Eric Ries (00:46:56):
[inaudible 00:46:56] I've seen this pattern in many, many companies.

Luis Von Ahn (00:46:58):
I'm pretty,

Eric Ries (00:47:00):
Yeah, it's interesting how companies are sometimes a little bit sheepish about it because there's this idea in our culture, it's called the theory of shareholder primacy that shareholders should call the shots. And so many founders I know at least privately will tell me, "Look, investors are like the people who kill the golden goose all the time. They can't help themselves and actually they're sub-optimizing how much money they make by putting that pressure." And so yeah, to your point, I love the phrase under-monetized because if your goal really is to maximize the amount of long-term profit this asset, the asset of Duolingo will generate over its lifetime, I would say it's probably much more closely to optimally monetized than these other assets that are going away. I mean, think about how many ed tech companies have come and gone in the years you've been working on Duolingo.

Luis Von Ahn (00:47:43):
I think you're completely right. One thing that I was struck by is the timeframe of public market investors is pretty funny. I mean some of them are like, we're long-term investors we hold for a year.

Eric Ries (00:47:59):
Yeah. Yeah. I've heard that.

Luis Von Ahn (00:48:00):
I'm like, okay. I think your definition of long-term and my definition of long-term are not the same. So if your timeframes are months to years, sure we're under monetizing. Sure, I agree with that assessment. But if your timeframe is 30 years, then yeah, that's different.

Eric Ries (00:48:20):
So let me get back to the Handbook because I'm really excited about the Duolingo Handbook. And when we were talking about arranging, having this conversation, I didn't know it was about to be coming out. So the timing has just serendipitously worked out really well. And part of the reason I'm excited about it, you don't know this, I [inaudible 00:48:35] reveal it to you now. One of the concepts that I'm advocating for with companies that I work with, I call it the Leader's Guide, the name of the first one of these documents I ever helped the company write. But basically in order for a company to maintain its culture, you have to write down what good looks like. What do you actually expect your leaders to do? And companies really get this wrong. If you build up the reservoir of trust, one of the reasons of the reservoir of trust works is because a lore or a mythology starts to evolve around the company about times when the company did this thing and operates this way. And how many employees are you at Duolingo now? It's like-

Luis Von Ahn (00:49:12):
About 900.

Eric Ries (00:49:14):
Yeah, I would say about almost a thousand, especially when you get to a thousand person mark, it's really like a sea change. And one of the things that's really important about that is the vast majority of people that work at the company, they worked somewhere else before.

Luis Von Ahn (00:49:25):
Yep. Yep.

Eric Ries (00:49:25):
So they got their professional training somewhere else. And companies as they grow, especially fast-growing companies, they import all this foreign cultural DNA into their company and they're often shocked at the incoherence that that can lead to because now you have people who all think different things. And in a lot of companies, unfortunately, the language about mission, purpose, whatever, is actually just bullshit, just straight up BS. They don't mean it, but they like to say stuff like that. It helps with recruiting. It makes them feel better. I don't really know why they do it. And so people think, oh, I get it. This is the kind of place where I have to pay lip service to the mission, but I actually still do the exploitative things. And the more you say, no, no, no, please don't do that, the more people interpret that sometimes as to mean like, oh, hide it really well. A lot of the worst corporate scandals in history, think about Boeing, Wells Fargo, Silicon Valley Bank, the recent blow up, a lot of those have their origins in this idea that anti-mission activities were being hidden from the top executives themselves.

(00:50:19):
So anyway, so I bring this up because by formalizing, by writing it down, now you have an opportunity for people to see that you're serious about it and to really establish that standard. So talk about the motivation to write the Handbook and then talk about why you've made the decision to share it publicly.

Luis Von Ahn (00:50:36):
Yeah, I mean the motivation is exactly what you said. Early on as a company, everybody's in the same office or nearby and culture is just there. You absorb it because you're just sitting there and you're next to the next person and everybody that's just the culture and everybody has the same set of beliefs. As you get bigger, you have multiple offices, et cetera, that starts getting harder. So we thought it would be a good idea to write this down. We actually tried to write this down a few years ago, but failed because this was not that high of a priority giving everything else we were doing. So we gave up.

(00:51:19):
But about a year ago, enough things came together that we decided to start really writing this down. It's gone through a lot of iterations and part of the problem is internally and also for me, the stakes were so high for this because I'm like, but how am I going to capture the essence? How is this going to happen? One thing that helped a lot is that we hired an external person to really help us. This guy, Adam, came and watched us for six months and he just watched us do stuff. And then he told me, he was like, "Look, these things that you do, they're the essence. These other things that you say, they're not even," because we had some operating principles before and he's like, "that one's like, you don't even do that, man."

Eric Ries (00:52:09):
That is really important. Having an outside ear is really valuable.

Luis Von Ahn (00:52:14):
It helped to have an external person looking at what we did. And so he helped with this and then a lot of other people participated in writing this. And in the end, I think we're very proud of the outcome. And I think it really captures at least the most important things. It doesn't capture everything we do, but at least it captures most important things. And I'm very happy with that. And I think it comes across the way I want it to come across. You may or may not like it, but at least this is how I want it to come across.

Eric Ries (00:52:53):
You are going to find it's going to have a tremendously positive effect on hiring just because of the selection bias of people who will read it and say, "This isn't for me, why would I ever want to work somewhere like that?"

Luis Von Ahn (00:53:02):
If that is the only thing that this does, I would love that. And I think that's exactly right and I think that's what, we didn't put this in the book in the end, but I wanted something like that in the book. It has taken me years to be okay with the fact. With saying this, Duolingo is not for everyone. It's taken me years. At first I'm like, "No, no, no. We have to be a company that is great for everybody." Et cetera. Turns out there are people who thrive here and there are people who don't. And there are people who don't thrive here, who thrive elsewhere.

Eric Ries (00:53:35):
Yeah.

Luis Von Ahn (00:53:36):
And so I think you're exactly right. If all it does is it gets people who are not right for this to not come here, that would be amazing.

Eric Ries (00:53:47):
So do you have a plan for keeping it up to date over time?

Luis Von Ahn (00:53:51):
Yeah. I mean the plan is that we're going to be looking at this every couple of years. I think that's the plan. And we'll see what happens in a couple of years. But that's the plan. I think the cadence is going to be two years. We wrote it in such a way that we think it'll last five years, not stale, but we're going to look at it in two years.

Eric Ries (00:54:15):
Do you have a plan to incorporate it into your employee review and promotions, compensation review process?

Luis Von Ahn (00:54:22):
Not at the moment. We don't have that plan. The plan at the moment is to obviously give it to all employees. We're going to share it publicly and also give it to all candidates. We're just going to give it to them and we're not going to quiz them on it. We're just going to be like, if you want to read-

Eric Ries (00:54:37):
No, no, no. I mean, in your performance matrix where you're outlining what the promotion criteria are, are you going to align it with these principles?

Luis Von Ahn (00:54:44):
I would like to. We have one of the criteria for promotion is that it's just like how good of a citizen they are. And I think in that place I think it makes sense to align it with that. We haven't quite done that, but I would like to.

Eric Ries (00:54:59):
So having done this a couple of times now, can I pitch you on an idea?

Luis Von Ahn (00:55:02):
Please.

Eric Ries (00:55:03):
Okay. I have this concept I call the two-way review because one of the funny things about bigger companies, when you get to 2,000, 5,000 employees and beyond, especially, you tend to start to differentiate between HR, corporate communications, employee communications, and the employee performance management become separate functions. And sometimes recruiting even is separate. So anyway, so the Leader's Guide is really important for communicating externally about why you should come work at this company. And of course it's a really important part of employee onboarding, you communicate it internally.

(00:55:37):
But in some companies the performance review matrix is based on a different set of values. And in a lot of companies, if there is alignment, the alignment goes stale because the problem with mythology is it gets old. So the story, the legendary story about the time that you and your co-founder picked green because you thought he hates it, that becomes a legendary story that ceases to seem that relevant as you get older. And of course I've worked for companies like GE where Thomas Edison is long dead or even at Walmart, I remember at Walmart huge problem, Sam Walton is not there anymore. His picture is everywhere. And people are like, "Well, could he even get hired at this company anymore?" It's-

Luis Von Ahn (00:56:17):
Probably not.

Eric Ries (00:56:18):
Probably not. It actually becomes very stressful when that comes to be the case. So anyway, so companies are always asking me, "Well, how do we keep this thing up to date? How do we source stories to include?" And the funny part is of course most companies already spend an incredible amount of money every year sourcing lots and lots and lots of stories of employee behavior in the employee review process.

Luis Von Ahn (00:56:37):
That's funny. Yeah.

Eric Ries (00:56:37):
Which they promptly throw away.

Luis Von Ahn (00:56:40):
That's funny. That's good.

Eric Ries (00:56:41):
So my suggestion, my pitch to you would be every year have your managers select the stories that they think best illustrate a principle from the Handbook and submit them to whoever's in charge of the Handbook.

Luis Von Ahn (00:56:52):
I really like that. I really like because you killed two birds with one stone on that one.

Eric Ries (00:56:56):
And then yeah, update the book with those. Take the top 2% of the stories, include those, revise the book, and then you do still have to do what you're describing every two to five years. You have to also realign your compensation standards to make sure they're in alignment with the changed book. You got to keep it synchronized, but if you do that, the two-way nature of it, you're reviewing employees but they're also reviewing you. It's incredibly powerful as an alignment tool.

Luis Von Ahn (00:57:19):
I really like that. I really like that.

Eric Ries (00:57:22):
Excellent. Okay, good. I'm glad to hear you say that. When you do a leader's guide, saying your statements of intentions and you're a high lofty language about your purpose and excellence and stuff, that's pretty easy. And I don't mean to make fun of it. It's actually really, really important. And in companies that don't hold people to high standards run into huge problems. So I really think it's important to have a high concept, really intense idea about what the company is all about. But if you don't translate that into an operating model that works, you create huge problems.

(00:57:53):
First of all, people don't know how to implement the ideas, but even worse than that, people think, okay, anything that makes the numbers go up and to the right must be good. And if there are ways to make the numbers go up and to the right that are not aligned with the model, they'll gravitate because their compensation is tied to performance, they'll gravitate towards those behaviors and not to the ones that you want. So the only solution to that is to have an actual model that is prescriptive, it says, look, this is how we accomplish things. And it also has to be right. It has to work to accomplish the company's goals. And that's not that easy to come up with. The Handbook really goes into, I thought, a surprising amount of detail about what you call the green machine, which is great. It's a great term. Explain what the green machine is and why you wanted to share it in the Handbook.

Luis Von Ahn (00:58:33):
Yeah, I mean it's a model that we found is behind a lot of our most successful things we've done. It's not true that a hundred percent of the things we do at Duolingo are done to the green machine. That is not true. But if you just look at some of the biggest, most successful things we've done, it turns out we operate pretty similarly across the company in those ways. And it's basically a continuous iteration process and we're just very good at that. And it basically starts by forming a team around the thing you want to do, at first, of course it's a small team, setting up what success metrics and it's important to come up with some sort of metric, so that means what good looks like, and then just letting them innovate, but in this iterative hill climbing model where you're doubling down on what works. So you're trying many things, whatever it is.

(00:59:42):
For example, our TikTok, the first TikTok video that we put out was not that successful, but it was like, oh, we tried a few different formats and then this one's a little more successful than the other one. Okay, we're going to double down on that one. And then we tried more iterations there and we keep doubling down on what works. And as you're doubling down on what works, for us as a company, the team, if the team is succeeding, we also are doubling down on that team. We're putting more and more people on that team.

(01:00:16):
So we have this green machine, which is this continuous improvement process that we have the Duolingo app, most of the features in there certainly are marketing. A lot of the stuff has gone through this process of just basically doubling down on what works. So we decided to write it down. And there's a number of things that matter there, but what I'll say is at first, if we had written this a long time ago, we would've just written down just do more of what works. There's one extra thing that I think really matters, which is do more of what works, but always take the long view. If you just double down on what works, this is what gets you to doubling the number of ads because guess what? Putting ads works.

Eric Ries (01:01:08):
Oh my god, I was just talking to a CEO whose company had gotten to sending an average of 20 email notifications per day to its customers, because every time they added an email notification-

Luis Von Ahn (01:01:20):
It works.

Eric Ries (01:01:20):
Somebody's OKR went a little bit up and it was becoming absolutely unbearable. And he was like, "I can't understand how this happened." And I walked him through this exact thing. He's like, "Oh, I see how." He was so embarrassed, "Like, oh my god, I can't believe we did this. So stupid."

Luis Von Ahn (01:01:33):
That is exactly what we've learned. And that's exactly why for example, every change that we make on the product has to go through this thing called, we do product review, that's where we look for this type of stuff. We're like, okay, you improving your metric. Good, good. Now we are going to look into this and think about it from the long-term perspective. And we're like, yeah, don't do that. And in our product organization, it's basically the same story you have. We have this apocryphal story of Groupon where, this is the founder of Groupon told us this story, which was basically that he said, "Look, at first I had this rule that we were only going to send one email per day of offers-

Eric Ries (01:02:20):
A daily deal. Yeah.

Luis Von Ahn (01:02:22):
Because of course nobody wants to receive more than one email per day. And what happened there was there a product manager that was an enterprising product manager and said, "Have we ever thought about sending two?" The founder said, "No, I don't want to send two because more than one is too much." And then the program manager used this amazing argument which now no longer works at Duolingo, but it is an amazing argument. He said, "Well, why don't we just try it? Are you against knowledge? Just try it." And of course they tried it, turned out sending two emails is better than sending one. So they started sending. And once they knew that, it was very hard even for the founder to say, no, we cannot send two emails because look, all the data suggests you should send two emails. And so they started sending two emails, then of course tested three, they started sending three and then they had A/B tested their way to whatever, and suddenly the whole channel died because people started marking them as spam. And the problem is-

Eric Ries (01:03:23):
Yeah, I've seen that happen. Listen, I've been a huge advocate of A/B testing obviously in my career, but I have seen it go really horribly wrong, and in just the way that you're describing, basically. You'd basically be selling a psychic hotline or pornography or something in no time.

Luis Von Ahn (01:03:34):
Yes.

Eric Ries (01:03:35):
You have all [inaudible 01:03:37] click-through rates. There are things that can get click-through rates if that's all you care about. And I think it's a fundamental misunderstanding of what analytics is for because, yeah. Yeah.

Luis Von Ahn (01:03:45):
I completely agree. And the other thing that exacerbates it is if you're A high performing company and you're A/B testing like hell, which we do, you get really good at A/B testing.

(01:03:56):
And so you're able to move things very fast, you're able to A/B test a lot. And if you do not have these guardrails, these long-term guardrails, you can A/B test your way very quickly to complete garbage. And so we noticed that that was happening with us. We really, we got to the point where we had A/B tested our way. It never got to that bad where a whole channel died for us. But we noticed that, we were like, "Oh my God, it's absurd that we got to this point." But very smart people can get themselves to absurd spots if they just A/B test their way to it. This is one of those ridiculous things. We A/B tested our way to asking the same question three times in our onboarding flow. It turned out that performed better.

Eric Ries (01:04:45):
Yeah. Yeah. Yeah.

Luis Von Ahn (01:04:46):
And I'm like, "Why are we asking the same question three times?" And they're like, "I don't know what to tell you. This performs better. I don't know what to tell you." And so we tried really hard through this product review process where basically we don't do absurd things. And the argument whenever somebody says, "Don't you just want to know?" There are cases where we say, "Actually, we don't want to know because it would be too tempting." I don't want to know how much more money we would make if we, whatever, some random thing. I don't want to know because if I'm told it's $50 million, I'm like, "That's nice. I'll take the extra $50 million." Or whatever it is. And so I don't want to know. I don't want to know.

Eric Ries (01:05:30):
No, I've developed all these metaphors over the years for this because this way of operating is dangerous.

Luis Von Ahn (01:05:37):
Yeah. It is.

Eric Ries (01:05:38):
A lot of things in business are dangerous. Flying an airplane is dangerous, making a rocket ship is dangerous. This is dangerous. Driving an F1 car at high speed is extremely dangerous. And if someone's like, "Hey, I have an idea, let's close our eyes and floor the accelerator, because we'll go faster." And if you're like, "I don't really want to try that," they're like, "Well, don't you want to know?"

Luis Von Ahn (01:05:56):
Exactly.

Eric Ries (01:05:57):
Not really.

Luis Von Ahn (01:05:58):
Actually, no. Actually, no.

Eric Ries (01:06:02):
It really requires that every experiment has a why, and that's why having a purpose to the company is such a critically important thing, not just morally and ethically, but just as a matter of company performance. If you could say, look, we have a goal, then that's like having a hypothesis in the scientific method. We're not just throwing chemicals around in the lab to see, I wonder if anything will explode if I do this or do that. You're testing in line with some vision and that's critically important.

Luis Von Ahn (01:06:23):
Yeah, I think we've gotten good at that. It took us years to learn this, but we learned it eventually.

Eric Ries (01:06:28):
That's excellent. Okay. I do have a bone to pick with one thing in the Handbook, because you say that you don't do MVPs, you don't believe in minimum viable product.

Luis Von Ahn (01:06:36):
I knew you would have a bone with that.

Eric Ries (01:06:38):
Of course. How could you...? Listen, I can't. There's no way we can not talk about it. And look, if anyone reads the Handbook and they're like, "You didn't ask them about that? What's gives?" I had to ask you about it.

Luis Von Ahn (01:06:46):
You'd have a bone with that. Yeah.

Eric Ries (01:06:47):
Yeah. But actually I thought it was really interesting because the version of minimum viable product that you articulate in the Handbook is not one that I would advocate for. It's basically saying that your minimum viable product is like a crappy product or a subjugated product. And you did, I thought, a very good job of explaining that the V1, we call it the V1, is a version of the product that is high quality enough that even if it's feature poor, the thing that it does do is high quality enough that it represents the company and its brand well. So talk about why that happens and then we can talk about the concept of MVP itself.

Luis Von Ahn (01:07:23):
Yeah, I mean, look, personally I'm a big believer in MVP, especially if you're earlier, in an early stage company, I'm a big believer. Look, just get the thing out there. I am a big believer of that. However, what we found is that we were getting a number of particularly product managers who are coming from other places, sometimes from scrappier startups who would use the term MVP to justify shitty work. They would, "Look look, it's an MVP, it's an MVP." And it'd be like, it had typos. And you're just like, "Nah."

(01:08:02):
So this is when we were like, "Look, let's not use this anymore. Let's not use this." So it's a bit of, I think if everybody was reasonable, we could use this term, but it's a bit of an overcorrection from people who come in and they're like, "Ah, whatever. Let's just do it." So here we tell every new product manager, every new designer that comes in, we don't do MVPs so that they cannot justify like, oh, it's okay. It doesn't work for half the users, but that's an MVP, right? It's like, no, no, it has to be polished. It doesn't need to have all the bells and whistles, we're okay with that, but it has to be polished. No typos, no weird design flaws or anything. So I think that's what we meant by that.

Eric Ries (01:08:51):
This is going to sound strange to people, I think, maybe even to you, but I liked it in the Handbook. Even though that definition of MVP is not one that I would endorse, and for the record, I mean product managers listening right now, if you use that argument and say what Eric said to build an MVP, then you do not understand my work, and you should probably reread, I was extremely clear about the definition of MVP and it does not include making crap.

(01:09:10):
But what I thought was really interesting about it is a very common criticism that I've gotten in my life is that lean startup is associated with all this jargon. We have all these terms, MVP, build, measure, learn, pivot, go on and on. And I don't take that criticism. I think it's really important to have conceptual vocabulary to name the concepts that we're talking about so that we can talk to each other across company boundaries about what we mean. But people often think that to do lean startup means to adopt the terminology of lean startup and they get confused between the thing and the name of the thing, basically confusion between signifier and sign for those who know what that means.

(01:09:46):
And so I would much rather have a company that adopts its own terminology and uses the concept correctly than one that uses my, I don't care if you use my terminology or not. So if you want to call it a V1, that's totally fine as long as you do it correctly. And the second thing I really liked about the Handbook, even though I had this disagreement, is that you define V1 very crisply in a way that makes sense for your size and stage.

Luis Von Ahn (01:10:09):
Yes.

Eric Ries (01:10:10):
Because one of the hardest issues in entrepreneurship is that people want to do stuff in the name of quality. So in this case, you define quality very strongly. You create a very clear expectation in a customer's mind what things are supposed to look like on Duolingo. But one of the things I learned in my career is that if you don't know who the customer is, then you can't define quality because quality is in the mind of the customer. Well, by the time you have $350 million in revenue, you damn well better have a really good idea of who the customer is and what they want. At an earlier stage, that's not possible. And so people often jump to assumptions about what's going to be a high quality. And because of that, I think there's been this misconception out there that MVP is like anti-quality or anti-craftsmanship, which I think is a really pernicious and wrong idea. So I'm glad to have the chance to clarify.

Luis Von Ahn (01:10:52):
Yeah, and we're very big. I mean, every feature we put out is basically in your definition of an MVP, an MVP, not in the crappy definition. But every feature we put out, we never put out first thing all the bells and whistles. We never do that. So I'm a big fan of starting small, but look, no typos, no misaligned tables, no button that you can't click, none of that.

Eric Ries (01:11:18):
That's great. I hope in a future version we'll get a little asterisk, like footnotes like by the way da-da-da-da-da. That'll be my aspiration. Duolingo probably has the most famous engagement loop, certainly in education products, but I think maybe among all consciously designed products, I hear product designers talk about it all the time. There's been a lot of written about just how you actually manage to get this incredibly engaging product to the scale that you've gotten to. And of course, using the green machine process, you could never have... You see, there's so many counterintuitive things about it, you couldn't have designed it just at the whiteboard one day, no matter how much of a genius you were. So just maybe because such a famous event in the history of apps and consumer products. Talk about just how did you get to that engagement loop?

Luis Von Ahn (01:12:09):
It's basically the result of 10 years of work.

Eric Ries (01:12:13):
Another overnight success, 10 years in the making?

Luis Von Ahn (01:12:14):
Yeah, exactly. It's a couple of things. The first few years of the company after we launched, two things were true. We were not making any money and because we were not making any money, so there was no way to pay us, we didn't have ads. Money wasn't coming in period. And there was no way for money to come in because we hadn't implemented any of that. Because of that, we also weren't doing any performance marketing. So in fact, we weren't doing any paid marketing. We were not acquiring users because we thought it was silly to spend 10 bucks acquiring a user when we can't make any money from that user. We thought that was silly.

(01:12:55):
So because we didn't do performance marketing and we weren't making any money, the only thing we could work on was improving user retention. That's the only thing. We weren't spending any time figuring out how to do better performance marketing through Google or Facebook. We weren't figuring out how to get people to subscribe by whatever. We were just like, the only thing we can control in the product is, is the user going to come back tomorrow?

(01:13:29):
And so we spent about five years where that's the only thing we did. So for five years, the entire product team, it was of course much smaller than it is now, but our entire product team was just what can we do to improve user retention?

(01:13:49):
And we moved metrics a lot in that time. So for example, day one retention, so that is when people sign up, what is the probability that they come back the next day, went from, in that span of those about five years, went from about, I remember in the early days, it was about 13%, went from about 13% to about 50 in those five years. And every retention metric that you can, it saw a similar improvement. And it's because we were A/B testing like crazy and everything we were doing, we were doing that. By now, of course, we continue improving that, et cetera, I just saw a thing. We have our own internal experiment framework. We have run, now, of course, many of these are small. We have run 16,000 experiments, A/B tests over the history of Duolingo. So that's a lot. 16,000 experiments-

Eric Ries (01:14:42):
That's a lot.

Luis Von Ahn (01:14:44):
This engagement loop is the result of 16,000 experiments.

Eric Ries (01:14:49):
Off the top of your head, what percentage of experiments were successful by the definition of they actually improved the metric, do you think?

Luis Von Ahn (01:14:56):
I know. It's not off the top of my head. I know, and it's a weird answer. Almost exactly to the point, 50%.

Eric Ries (01:15:04):
I knew you were going to say that. Yeah. That's just to me, the most amazing thing. Incredible.

Luis Von Ahn (01:15:10):
IT's almost exactly, it's weirdly accurate. It's not 49.1. No, no, it's 49. 995 or something. It's almost exactly 50%. And we've looked at that and we're like, that is so weird that it's so... Some people have come up with random explanations because the reality is we run experiments. Sometimes, there are certain things that we run them until they win. We keep redoing them until they win. And it turns out that you have these mathematical things that if you're going to have children until you have a female, how many children are you going to have? And there's all this stuff, and I think there's some explanations for why it's very close to 50%, but basically it's 50%.

Eric Ries (01:15:52):
You're not the first person I've seen talk about that. And actually, if you look at professions that have to deal with this kind of probability theory professionally, like professional poker players and stuff, everyone agrees the best experiment is one that cleaves your uncertainty in half.

Luis Von Ahn (01:16:08):
Yeah. Yeah. Exactly.

Eric Ries (01:16:08):
So it means you're actually pushing the efficient frontier of gaining information.

Luis Von Ahn (01:16:14):
Yeah, we've gotten pretty good at this.

Eric Ries (01:16:15):
It's a very healthy experimentation engine. I'm going to ask you what you're going to consider to be the dumbest question you've ever been asked. And I'm going to do it on purpose because I've actually had people ask me this question, I really want to see how you respond to it. People look at that and say, if you had 16,000 experiments of which half were a waste of time, why don't just eliminate the 50% that doesn't work? Why are you spending twice as much money as you have to on experimentation? Why not just get better at doing things that work?

Luis Von Ahn (01:16:43):
Look, if you're that good a product, and you can come and tell us how to do that, I would like to hire you. But look, the truth is when we run an experiment, we think it's going to win. It just turns out we're not that good at that. Every product manager, every designer, every engineer that proposes an experiment, sometimes we don't know if it's going to win, but the majority of the times we're like, "I don't know. It's worth trying." And it's worth the effort because of course, experiments are not free. I mean, each experiment takes us however long depending on the size of experiment from a day to sometimes months. So it's not free. So we have a belief on it.

(01:17:24):
But this is, by the way, one of the reasons why we started running so many experiments is because early on I thought of myself as somebody who was good at making product decisions. And we had a few people, our head designer at the time, we all thought we were good at making product decisions. And early on what we did is when we started A/B testing, we also started betting on whether something was going to win or lose. And we found that none of us were much better than 50%, that we are a little better, and of course, if you're going to run something dumb, I can tell you if something's going to lose if it's stupid. Like I don't know, make the button not work or something. If it's stupid, I can tell you it does not work, but if it's reasonable, I have a hard time telling you if it's going to win or not, any reasonable idea. So that's why we started running experiments.

Eric Ries (01:18:09):
You got time for the lightning round?

Luis Von Ahn (01:18:10):
Uh-huh.

Eric Ries (01:18:11):
Okay. One of my favorite things doing the research for these episodes is just coming across some funny things that you've said over the years and I get to ask you about.

Luis Von Ahn (01:18:16):
Oh, no.

Eric Ries (01:18:17):
So just, yeah, nothing too crazy. But you said, for example, that people are good at figuring out what's attractive and computers are good at quickly searching and finding. You put them together and bang, bang. What happens when you have that magic?

Luis Von Ahn (01:18:33):
I think that was a million years ago. I was talking about a project that we were working on to figure out what images were prettier than other images. I think that's what that was about. Barely remember that.

Eric Ries (01:18:51):
Yeah. Well maybe talk a little bit about, we're obviously living in a time now where machine learning and AI's in the news every five seconds and people are really trying to figure out how do you combine human and machine intelligence in a way that's positive? You've always been a pioneer of doing that at Duolingo. How do you feel about it now these years later?

Luis Von Ahn (01:19:09):
Yeah, I mean things are of course very different. We're still, it's an interesting thing, and I want to make an example with our artists. We hire a lot of people. Our design department, we call it the design department, has about 150 people. So it's pretty large. And some of them are animators, some of them are illustrators, but they're all pretty much artistic people. Over the last few years, some of their work can be done by computers now. And in the next few years, probably a larger fraction of the work will be able to, computers will be able to do it. But what I love about them and I think they'll have a job for a long time is that computers can do the average thing. I don't know, I just can't imagine that we're there yet where the computer can really come up with a cool unexpected thing. And I think our artists are just really good at that. I mean, I'll give you an example. Our session end screen, just recently one of our artists decided that they were going to make the owl look like a rainbow unicorn.

Eric Ries (01:20:31):
As one does.

Luis Von Ahn (01:20:32):
Exactly, but it's just how did you freaking come up with that? I don't know. But it was super successful to put that at the end of the session. So I think our artists are just able to come up with things that have soul in a way that I don't think computers are able to do that just yet. And I think that'll take a while.

Eric Ries (01:20:55):
You said that one thing that's changed as you've gotten older, you said that early on you were motivated by surprising, non-obvious, clever ideas, but now you find yourself more motivated by large scale impact and you find that a lot more motivating. Talk about that change.

Luis Von Ahn (01:21:07):
Yeah, that's definitely true. The older I get, the more it turns into that. I mean, when I was a teenager or when I was in my twenties or so, I mainly just wanted to prove that I was smart. And whatever surprising, clever idea or whatever. At this point, I still appreciate clever ideas, but I am just not all that into them anymore. I am more interested in pragmatic things that actually have large scale impact and that's what I'm most interested in.

(01:21:40):
And the other thing is that I have found, and I've learned this from other people mainly, but I have found that especially if you're working on a product, that it gets used by hundreds of millions of people, the simpler the better. I used to really be like, oh, this super clever idea that like a Rube Goldberg machine kind of thing where it's just like this implies that and implies that and it is just super clever. And at this point I am a lot more like, "Does the screen only have one button? Good. That's good." It's just what happens if you're designing for hundreds of millions of people, the simpler the better.

Eric Ries (01:22:24):
You talked a lot about coming from Guatemala and the impact education had on your life. I know you do philanthropy still tied to Guatemala. I had to ask you, given the global sentiment against migration and immigration and obviously the recent world events, there's so many things, this has been a controversial... What's your perspective as an immigrant who's enriched this country so much? What are your thoughts about that recent phenomenon?

Luis Von Ahn (01:22:47):
I'm trying to figure out whether to deport myself or not. Since I'm from Guatemala and I don't know what the legal status is. I'm kidding. I'm a US citizen.

Eric Ries (01:22:57):
Please do not. Please do not. We need you. We need you.

Luis Von Ahn (01:22:59):
I'm a US citizen. Look, it's a complicated situation. I fully understand that illegal immigration is not something anybody wants. I mean, it's illegal. There's that. At the same time, countries like mine, countries like Guatemala are for a number of reasons, so mismanaged and such,, essentially non-functioning countries that a very large number of people, a very large fraction of people just can't make a living. So they're looking for a better place to be and so they're going to find other places. Of course, the US being a land of such opportunity does attract a lot of these people. Interestingly here, inside Duolingo, we use this term immigrant energy. People who left their family, their country, the food that they were used to, their roots and move somewhere else are usually major risk-takers and also hard workers. Of course, this is not a hundred percent true, but on average they are.

(01:24:05):
And this is even true of illegal immigrants. And so I have a hard time knowing if it's net positive or not negative, certainly immigration, without talking about illegal immigration, certainly immigration can't possibly argue that it has been bad for this country if you look over a long period of time because essentially everybody here is an immigrant. I mean, if you look over a long enough period of time, everybody here is an immigrant or the descendant of immigrants. And in addition to that, immigration has allowed this country to attract essentially the brightest minds in the planet. And it's not a hundred percent true, and it's not as true as it was 50 years ago where really it was almost the only destination. Now, there are other destinations, but we still attracting some of the brightest minds in the planet, and I think that's good for the US.

(01:24:54):
So how do I feel about this? One thing I don't like about the anti-immigration rhetoric is that I understand that technically a lot of these people are saying they don't like illegal immigration. I understand that. But that line gets blurred and it gets understood as against all immigration. And of course I'm not going to like that being an immigrant, but it's also the case that a large fraction of this country is an immigrant or the sons or daughters of immigrants. And do you really want to live in a place where you're being called out or discriminated or something because of that? So this is why I don't like that rhetoric.

(01:25:33):
Last time Trump got elected in 2016 here in Pittsburgh, the day after we had an Asian-American employee cross the street by the Duolingo office and somebody stopped their car to yell at them, "Go back home." Now, this is a person that was born in Queens, but so this is what I don't like about that rhetoric, and this is a very diverse country, so I would like to stop that rhetoric. I don't know what to say other than that.

Eric Ries (01:26:03):
No, I appreciate you being willing to talk about it. Like I said, immigrants like you have enriched this country so immeasurably, and I think it's sad when that rhetoric gets overheated.

Luis Von Ahn (01:26:13):
Yeah, I don't eat dog or cat.

Eric Ries (01:26:18):
That was a very upsetting moment I have to say. Yeah.

Luis Von Ahn (01:26:24):
I don't do that. Maybe others do, but I don't.

Eric Ries (01:26:26):
Yeah, so many of us owe our ancestry and our very lives to the melting pot aspects of this country. I hope that we'll stand up to defend it in the years to come. All right, last one. This is just so funny to me. You said after you win the MacArthur Fellowship, people always make the same joke that since you're a genius, you can't do anything stupid anymore, but you do stupid stuff all the time anyway. And I just like, what's it like for people to make that joke? But more importantly, I feel like you have shared with great humility, not just the successes, but the failures in this conversation. But I've known you a long time, you've always been very open about that. I think that's such an important part of what's become the culture of Duolingo. Just talk about where that comes from and how it feels to be valorized in this way, knowing that you're still human.

Luis Von Ahn (01:27:10):
Our culture at Duolingo is quite humble, actually. We really are quite open with our mistakes, and I think it's important that we're open with our mistakes in a way that is not pointing fingers, but just like we screwed that up. Can we do better next time?

Eric Ries (01:27:23):
Yeah.

Luis Von Ahn (01:27:24):
Certainly for me, I mean it's less now, but certainly the first few years after having won the MacArthur Award, every time I did something dumb, which again happens often, somebody makes that stupid joke and like, "Well, I thought you were a genius." And I'm like, "I don't know what to tell you. I screw things up all the time." So we all make a lot of mistakes. For certain things, I am quite good at making mistakes. But yeah, I think the most important thing is, and we have it in the Handbook by the way, mistakes will happen. I think the most important thing is to learn from them. And by the way, sometimes people will tell you, "Well, look, I make the same mistake, but I don't make the same mistake twice." Or, whatever. Nah. At Duolingo, we have made the same mistake probably a dozen times and we'll probably make it time number 13 this year. Some mistakes are so tempting to make that you just keep making them. It happens.

Eric Ries (01:28:27):
It happens. Well, on behalf of the millions of people, learners all over the world who have benefited from your mistakes and what you've learned from them, thank you and thanks for coming on the show.

Luis Von Ahn (01:28:37):
Yeah, thank you. Thank you for having me.

Eric Ries (01:28:39):
My pleasure. You've been listening to The Eric Ries Show. The Eric Ries show is produced by Jordan Bornstein and Kiki Garthwaite, research by Tom White and Melanie Rieback, visual design by Reform Collective, title theme by DB Music. I'm your host, Eric Ries. Thanks for listening and watching. See you next time.