Meta Learning: Addendum or a revised recipe for life

In 2021 I published Meta Learning: How To Learn Deep Learning And Thrive In The Digital World.

The book is based on 8 years of my life where nearly every day I thought about how to learn machine learning and how to do machine learning efficiently and at a high level.

The book can be best thought of as a recipe. It describes how I learned machine learning by taking the fast.ai courses. It focuses on the actions (project-based learning) but also – which is probably more valuable – describes the mindset and ideas that I found extremely helpful along the way.

Following the recipe I began to share my work, won a Kaggle competition, and worked for exciting startups from Silicon Valley, Dubai, and Perth.

Ultimately, I moved to Australia thanks to my work in AI and an unbelievable amount of help from the community and began working for NVIDIA as a Senior Data Scientist.

But as what I did for work and what I pursued out of passion merged, I began to lose my tempo.

Problems started to arise with my first professional ML engagements but they seemed to culminate in the best job I ever had, that is my current role at NVIDIA.

Now, don't get me wrong.

NVIDIA is a dream job and a life-transforming experience. Based on what I know about tech companies, there is no other tech company I would rather work for.

I couldn't have asked for a better outcome.

The real problem was with my thinking. The more I pursued ML professionally, the more I started to think that the recipe that led me up to this point no longer applied.

On one hand, this hasn't impacted my work one bit. I have been an employee for nearly 20 years and that's plenty of time to figure things out.

However, I didn't appreciate how big of a role being part of the fast.ai community and continuous learning played in my life. How important following the recipe was for me on a personal level.

I share my experience in the hopes that should you find yourself in a similar situation, you might have an easier time balancing your personal growth trajectory and your work.

The recipe vs a tech job

In the professional world, very few people follow the Fast.ai Lifestyle™️.

In some sense, that is understandable. The success of the companies we work for comes first. Often, we use highly specialized tools (docker, SLURM) or hardware (compute clusters). We share knowledge internally by passing around Google docs listing arcane instructions to keep the machinery running.

All this takes up a lot of our mental space and if you add family life to it, there is very little space left for anything else.

How to square into this picture:

• following your curiosity
• or sharing your experience publicly

becomes a dilemma.

Of course, some people manage to pull it off. Eugene Yan is an engineer who talks publicly about topics related to his job at Amazon using a deeply personal voice, but also one that would be welcome at a team Zoom meeting.

But those examples are few and far between and somehow are impossible for me to copy.

So while I couldn't figure out what recipe would apply in this new world and would work for me as a person, the previous recipe that I had followed seemed to be completely out of place.

The salvation

It turns out that I was wrong all along! But I only figured this out recently while talking to a friend.

There have also been some memorable public exchanges that might hint at what the answer might be:

And so I had an epiphany!

I thought my new life began upon taking the fast.ai course part 1 v2.

But in fact, I entered a new path in my life many years before.

As I was sitting in an open-space office, bored out of my mind, an idea came to me – I would learn new things, without ever hoping that they might lead to anything practical in my life, purely in the hopes that doing so would make my life more interesting and enjoyable.

I learned about the history of the world, about programming languages, and at some point I ran into machine learning.

But the motivation was never to achieve this or that professionally.

I learned for the sake of learning which turned out to be a source of great happiness. All the good things that followed later in my life were a side product of this approach.

This simple but magical formula led me to stumble into fast.ai a couple of years later.

Don't get me wrong – fast.ai gave me the tools to accelerate my journey 1000x fold. It took me from not being able to apply machine learning to even the simplest of problems to winning a Kaggle competition within 9 months of taking the course.

The experience of taking the courses and participating in the discussions on the forums have been nothing short of transformative.

But somewhere along the way as my passion merged with my professional life, as my family's subsistence started to rely on me having a good tech job, much of that magic went puff.

I started to approach learning in a more calculated way.

"If I do A, B, and then C, this should lead me to D".

But it just doesn't work that way. Life doesn't accept counterfeit currency.

You either follow the things only you can see, the ideas you find genuinely fascinating, or you run out of steam.

And, fundamentally, the conflict between this approach and my work only existed in my head!

At work, you strive to do what's best for the collective good. And that in itself can be very rewarding. There is a component of mastery to this that I deeply appreciate.

But even though my passion is ML and I also do ML for a living, there doesn't need to be a merger between the two worlds!

One can inform the other, there will be a natural overlap at times that will arise spontaneously, but in most circumstances, it is a fool's errand to try to combine the two.

And yes, I do have to cut myself some slack here.

In retrospect, all this sounds much more obvious than it has been.

For once, it is perfectly natural that when you join a group of people you admire you want to be more like them.

is that I would sometimes write blog posts for work and

Probably not many.

Another confusing touch point is that I would sometimes write blog posts for work and continue to blog here, on my personal blog. I somehow imagined that there should be a common thread between the two.

That my work self should have the same voice as my personal self.

No, no, no, and no.

Even at a fantastic place like NVIDIA, which is extremely supportive of its employees, both will become diminished if you attempt to merge your professional and personal lives.

So, first of all, you need to separate in your mind your personal and professional lives, even if important components of both might happen online, even across the same media (for instance, I tweet about work-related things but also about unrelated things that I find fascinating and useful).

Secondly, the important idea that I tried to circumvent to no avail is the importance of following my curiosity. It is great to achieve wonderful things in life, but focusing on the outcome instead of learning, which as a side product might lead to neat things in my life, doesn't work for me.

Last but not least, another related thought here is that attempting to change my voice to seem like I am less of a newb and more of a professional is a hopeless endeavor.

When I write, I care about honesty and communicating ideas as well as I can. And that often requires admitting that something has been a struggle for me.

For instance, I got my first two professional ML engagements in January 2019. That means that it took me 5 years to identify that I was experiencing a problem covered in this blog post and only now do I feel like I have maybe found a solution.

The recipe redux

In summary, the recipe as I outlined it in Meta Learning hasn't changed one bit.

I have just rediscovered for the trillionth time that it works and have hopefully adapted my thinking to continue to follow it given my new circumstances.

Still, it can be good to put down in words what's on your mind from time to time, and how you plan to approach the next leg of your journey (I have done so numerous times in the past, for instance here).

Here is what's on my mind right now as I ponder what to give my spare time to:

1. Learning is enough for interesting things to happen in your life. Follow your curiosity for extra fun and to make it all worthwhile!
2. Project-based learning is key. It makes learning more fun and is the only way to validate the castles in the clouds you build in your head.
3. Give, give, give. We tend to think transactionally – if I do this, I hope to receive this in return. But that is not a fun nor constructive way to live your life. The number of doors that opened for me, because I did something for someone without thinking of getting anything in return, is unbelievable. Give freely in whatever way you choose, but if someone wastes your time, run for the hills!
4. Look inside yourself. We don't get to choose our circumstances. We might have less or more mental or physical energy. We might be younger or older. Nearly everything that matters in our lives is outside our control, even in the long run. This doesn't make us powerless. It only signifies the importance of acceptance. It seems to me that I can make more progress in my life these days by looking inwards than by looking outwards.
5. Communicate, communicate, communicate. You need to tell the world what you can do! Sure, it might feel unpleasant at first but there is no other way. Plus, when you write or record a video, you learn so much in the process. It is through reflection on our experience that we learn.
6. You can't do it alone. This ties in with #5 but you need people that can enrich your thinking. If you share your work, there is a near certainty that you will stumble into such people one way or another. But the other thing that works well is finding the hackers that are doing something exciting and following them. For instance, answer.ai folks and Teknium for LLMs, or Alexander Koch for robotics. I get an immediate boost of energy when I see the awesome things they can do. Sure, they are at the cutting edge and there is a lot of insight that goes into being able to do what they do. But with a bit of applying yourself, you can sense that you could do something amazing as well, even if on a smaller scale. These folks don't portray themselves as figures larger than life, they use hardware and software that is (for the most part) not that hard to get a hold of. And, what is most important, they either share it all about how they do things or they leave enough breadcrumbs to get you on the right path. This kind of inspiration is invaluable.
7. Be inefficient. The Western conception of efficiency doesn't seem to work all that well, even if you are a machine. But probably, especially so, if you are a human! Act in a way that seems inefficient to make faster progress and have more fun along the way.
8. Just give it a go. I can't believe the amount of time I still waste by just thinking about stuff. "Oh, maybe I should do this, or maybe I should do that." It is much better to quickly give something a go and see how it feels. The experience will be much different from how you imagined it to be but if it doesn't work for you, try something else. Why does this happen? There is little overlap between our internal map of the world and the real terrain. The only way to overcome this is by actually doing things.
9. Please spend the money, pretty please... This ties in with #7 above. If I want to try something new – say, signing up for a $20 subscription service – the time I waste in overthinking the decision is worth way more even if the $20 would get sucked into a black hole immediately upon spending it! Of course, the $20 is an arbitrary number and will depend on your life situation. But overall, we humans do such a poor job at valuing our time against our money! Don't know if I will ever cure myself of this completely, but skimping on a certain class of things just doesn't make sense for me anymore.
10. There is no need to hurry. When is enough enough? I live in one of the best places to live on planet Earth. I have one of the best jobs I could imagine. What do I need to rush to? Being in a hurry, hustling, strikes me as a defense mechanism from the world around you. What are you running away from? It is probably (nearly) always fundamentally wrong to live in a rush, no matter your circumstances. It is fun and useful to generate momentum and achieve something extraordinary in life, even at a smaller scale than SOTA or world-best. But you get there by doing something regularly over weeks, months, and years. Consistency is the key, not the hustle. I am convinced that I am doing my very best given the circumstances, so why hurry? I cannot will more than 24 hours into a day nor am I willing to compromise other areas of my life (such as family or work), so what is the point of stressing out or setting unrealistic expectations for myself? I don't need to pour hours into following my curiosity, but even if it is just 2-4 hrs a week, I need it like a fish needs water.

PS. I added more background on sharing your work in a Twitter thread where I shared this blog post.

There was also an excellent observation from Mat Miller shared as a comment – learning is ultimately always a leap of faith. It is a belief that it is a worthwhile activity in its own right that can also lead to a better life. The process, however, is extremely stochastic, unlike for instance graduating from a particular university with a specific degree, which has a narrower distribution of outcomes.

This is an interesting and important conundrum. On the one hand, learning is +EV without any qualifiers (what you study, how you study, etc), on the other hand, we live in a world where we need to generate income to sustain ourselves and our families, and sometimes the link between learning new things and the latter is elusive.

But we need to keep life simple. I don't know about you, but I am not a great decision-maker. Plus, learning can take you to destinations far outside your current environment, so it is tough to be more selective about the direction of your learning.

It is precisely because of my limited ability to make decisions and my limited understanding of the world around me that I remain committed to dedicating a portion of my time to learning new things.