Summary How I became a machine learning practitioner blog.gregbrockman.com
1,640 words - html page - View html page
One Line
Greg Brockman overcame a mental barrier to become a machine learning expert and realized software engineers with basic linear algebra and probability knowledge can become ML engineers in a few months.
Key Points
- Greg Brockman transitioned from being a machine learning beginner to a practitioner, and overcame the mental barrier by working on OpenAI Gym and Universe.
- In 2017, I began work on a machine learning project using behavioral cloning, and eventually achieved a feeling of mastery.
- I made changes to GPT-1, fine-tuned it on chat datasets, and implemented GPU caching with help from OpenAI’s experts.
- I learned that software engineers can become ML engineers in a few months with basic linear algebra and probability knowledge.
- I joined Stripe as an engineer in 2010, and worked on backend infrastructure.
Summary
340 word summary
Greg Brockman dreamed of becoming a machine learning expert for the first three years at OpenAI, but made little progress. After nine months, he finally transitioned to being a practitioner. He found that the biggest challenge was overcoming the mental barrier of being a beginner again. During this time he worked on OpenAI Gym and Universe before turning the game Dota into an RL environment without source code access. He found success in combining software and ML skills, which was best demonstrated by Jakub Pachocki and Szymon Sidor's breakthrough that powered the Dota bot. In July 2017, I began work on a machine learning project using behavioral cloning to teach a neural network from human training data. Despite feeling like a beginner and encountering various workflow issues and bugs, I made some progress and my code was used as the starting point for creep blocking by Jie Tang. He later figured out how to get better results without my code and I never tried machine learning on the project again. In November 2018, I started self-studying NLP-related modules and read many papers to build a chatbot. With support from my partner, I kept pushing, wanting to gain a full understanding of the project, and eventually achieved a feeling of mastery. After overcoming a mental barrier, I made changes to GPT-1, fine-tuned it on chat datasets, and implemented GPU caching, which helped me understand the codebase. With advice from OpenAI's Jakub and Ilya Sutskever, I started to get exciting results and Jakub and Szymon joined the project full-time. With self-study, I realized I could make progress without constantly learning new primitives and motivated myself to spend more time doing ML work. I learned that software engineers with basic linear algebra and probability knowledge can become ML engineers in a few months. It's important to give yourself space and time to fail and OpenAI is a great place to be surrounded by experts. Joined Stripe as engineer in 2010; worked on backend infrastructure (server architecture, credit card vault, internal abstractions); loved it.