Warmerdam: I’ve been doing data science for a couple years. What I want to do is explain how I got this role at Rasa, and also, some stuff I learned over the last couple years. Because I think, maybe, if you’re optimizing for accuracy, you’re actually doing it wrong. I want to explain why because it seems counterintuitive. I think it’s something we should talk about. Also, this will be a presentation with lots of references to Pokémon. I’ve got this blog called koaning.io. I have also been organizing PyData Amsterdam. I’m also the YouTuber for the spaCy project. The video just went live yesterday. I have a couple of open source projects that I started. There are some other projects that I start on the side. Recently, I started working at Rasa. I’m this open source community guy. It’s what I do. It’s what I intend on doing. It’s my job.
I got here, and the story was eight years old now. Eight years ago, I graduated with a degree in operations research. I taught myself how to code. I was part of that first batch of the first MOOC, Sebastian Thrun, Andrew Ng, Peter Norvig. That first thing was a part of it. That made me consider, I really got to do this programming thing. What happened was, I taught calculus at a university for about half a year. Then the other half a year I would travel. What I did was I called myself an independent contractor back in the day when no one called themselves a data scientist. Something I found out was you can actually do most of that work remotely. I did. Life was actually pretty good as long as there was Wi-Fi. It was also around that time that I became an internet meme. I started sending this photo around. People noticed. Then you start trending on Reddit a bit.
Then I figured I should start this blog, because at some point, you’re done with world travel. You’re settling down. I wanted to figure out what this data science thing was going to be. Six years ago, data science was super new. I figured whenever there’s something non-obvious, I should write about that and that should go to the blog. Then I went to this big data conference in London, Strata. This was my first time at a big data conference. It was also, literally, a big conference. I met this guy. A guy in a suit, nerdy, but he wouldn’t shut up about it. He kept on bragging about how he calculated the optimal portfolio of bonds using big data. The guy wouldn’t shut up about it. I thought it was going to be funny if I wrote a blog post that was going to calculate the optimal portfolio of Pokémon, as a joke. I figured it might be funny.
Pokémon
You research, what is Pokémon? I’ve never played it. It sounds like something that’s funny to check. Turns out it’s this video game where you select a couple of animals and they battle some other animals. I figured, let’s try to find the best Pokémon. Around that time, there was actually a research paper called, “Classic Nintendo Games are Computationally Hard.” I think this was Cambridge. I’m not exactly sure. It was an actual fundamental 10 page proof that Pokémon is NP-hard, just like Zelda, Mario Brothers, and Super Mario World, and Donkey Kong. I knew I was solving a hard problem. The funny thing here was that there were actually these fan websites. These fan websites, they were people who were really enthusiastic about the game. They had all of these formulas that would explain if you have one Pokémon, and it deals damage to the other Pokémon, how much damage does it do? It depends if they’re a fire Pokémon, or a water Pokémon. I still don’t know what Pokémon is, but I found the formula. Then there was also this other website, which was the Pokéapi, you can download any statistics you want from Pokémon. I have no domain knowledge about Pokémon, but I do have data. The game that I’m playing here is going to give me a competitive edge, nonetheless.
The first thing I did was I did what a BI person would do, because if you’re doing stuff with data back in those days, you would do the business intelligence thing, which usually means that you’re making a giant visualization. What you’re looking at here is a matrix where all the Pokémon are listed, and they’re battling it out against another Pokémon. This is the result of the simulation. It’s 150 by 150 matrix. You should see something symmetrical here. There’s this red line at the bottom here, that’s Magikarp. Magikarp loses against everyone and everyone wins against Magikarp. The same thing with Diglett. Using BI you quickly find out that Diglett and Magikarp are the worst Pokémon to have, which is great. It’s not what you’re interested in. You want to find the best combination of Pokémon. This is one of the moments where BI has its limits. If you have a good BI tool, then what you can do is you can click a button and the fancy thing happens where all the Pokémon are clustered together. You can see that all the fire Pokémon are weak against the water Pokémon. All the grass Pokémon are weak against the fire Pokémon. This was I think seven years ago, this is what people did with data. This was BI. This is what they did.
#chatbots #qcon london 2020 #data model #transcripts #ai #data analysis
