There is something weird about the current generation of AI — better pay attention

The hype is strong around GPT-4.

• If you want to develop websites and games, coding is no barrier. GPT-4 can take in your instructions and spit out the final product (source).
• Make thousands in hours using GPT (source).

Of course, all these claims are sensationalized bullshit.

This is a race to the bottom of the stem cell (a term I learned from The Undivided Attention podcast). A way to exploit the lowest human instincts to garner publicity on social media.

But hype aside, there is something very uncanny about the most recent generation of AI models.

In the GTP-4 release stream Greg Brockman showed two amazing applications:

• developing a working Discord ChatBot (GPT-4 had to parse Discord API documentation for some of the functionality)
• calculating tax liabilities for a relatively complex situation by relying on fed in tax code

And the progress in Generative Art is equally stunning.

Or take a look at these generated shoe ads below.

To be honest, I feel caught off guard.

A quality threshold has been crossed that I didn't expect AI to cross anytime soon.

And I suspect I understand why I feel surprised.

Our human mind evolved to extrapolate from data linearly.

If we found 5 mushrooms in this area of the forest this year, we can expect there to be 6 or 7 mushrooms next year, should the weather be particularly good.

We don't expect to find 13 or 36 mushrooms there.

GPT-2 was fun and impressive. It could write coherent short stories about unicorns.

GPT-4 should write just slightly better stories.

And yet a whole new set of capabilities have been unlocked!

To understand what is happening, we need to take a look at a slide from Jeremy Howard's TED talk from 8 years ago.

The line for "Computer Performance" is self-accelerating, the pace builds on itself. It is exponential.

This is the reality I have missed.

So what am I doing about all this now?

First and foremost, I want to spend more time with this cutting-edge technology. To learn how to integrate it into my life.

To learn to code with it.

That is the best way to future-proof me.

Sure, I could go back to studying transformers. Or stable diffusion.

But the ground-breaking applications and functionality live at the top of the stack.

You want to become an expert F1 driver and for that you don't need to understand exactly how the engine works. You need to understand what it can do and its limitations.

And be able to squeeze its capabilities to the limits.

This is a scary reality for many of us in tech. What good is my understanding of how BatchNorm works if a teenager who has been using the OpenAI API for months now can run circles around me when it comes to the quality of output she can generate?

But I welcome this reality. This is the great democratization of AI.

In 2012 you had to be an expert coder to leverage the GPU.

In 2017 you only needed a year of programming experience to be able to implement SOTA models thanks to Keras and later PyTorch.

And now all you need is to be able to use natural language and think.

I am not bitter. I am just sad I missed this reality, that these models have so much potential. That I didn't start to capitalize on this trend earlier.

But I welcome this new technology.

And will do my best to not miss the boat on this paradigm change.

This text first appeared in the newsletter I send out each Tuesday. If ideas like these are of interest to you, please consider subscribing.