My 5 takeaways from the NVIDIA GTC keynote
“God, I love NVIDIA!!”
He paused and took in the whole moment, with thousands sitting in the dark auditorium waiting for him to continue. He gulped, paced a few steps forward, living every bit of the spotlight, and then continued his keynote.
Wearing his now-famous black jacket, his face was as radiant as ever. This was one keynote that NVIDIA CEO Jensen Huang knew could possibly rattle the pillars on which the industry was built for decades.
The stakes were high, and he knew it.
This was GTC, an annual global artificial intelligence conference where NVIDIA brings together developers, engineers, researchers, inventors, and IT professionals to hear about the most cutting-edge developments in AI. It goes without saying that the cutting edge AI stuff shown here is all built by NVIDIA.
The Brief History
A little bit of history first.
NVIDIA?? Who are they and where did they come from?
For a company that began in a garage in the 90’s, NVIDIA made their fortune by selling graphic processing units, which are specialized chips for gaming consoles, in an era where Intel was not only inside everything but also the everything of silicon chips.
It was the AI revolution of the last decade that propelled NVIDIA to the forefront. These Graphical Processing Units (GPUs) that they build are what power the AI infrastructure of today. Today, as the demand for AI applications continues to grow exponentially, their products are in high demand. So much so that their GTC conference is now one of the premier tech conferences on the planet, with 11,000 in-person attendees!!
Ever since the pandemic, NVIDIA has been on a tear. Today, NVIDIA powers most of the generative AI infrastructure. In 2016, Jensen Huang hand-delivered their GPU to Sam Altman from OpenAI to build ChatGPT. In fact, the ChatGPT moment was also NVIDIA’s moment.
They are now a $2 trillion company and have a seat at the high table alongside names like Apple Computer, Microsoft, and even Google. Over the past year, its stock price has skyrocketed to previously unheard-of levels, up by 200% in 2023 alone.
This two-hour GTC keynote was like a roller-coaster ride, taking the audience on a journey of both awe and fear. There was awe when what seemed to be science fiction until now (robots) were prancing about on stage, emitting strange sounds straight out of Star Wars, and taking instructions from Jensen. There was also fear — fear that the future we thought was still a distant 5–10 years away is in fact already here. For once, we could not only read the future; we could see it right in front of us.
Here are my five key observations.
The First Pillar — Blackwell
Although Jensen tried to change the impression in his keynote about NVIDIA being a platform company, I’m not sure how many people bought this narrative.
A year ago, at their last conference, NVIDIA unveiled Grace Hopper, its most sophisticated GPU chip. It was faster than anything they had ever built. It could perform calculations faster than anything in the industry, almost seemingly built for an AI era.
The largest AI firms then started placing orders for this chip. They ran out of supplies. In fact, NVIDIA is still trying to service their demand backlog. If you thought that this would keep them content with product releases, you were clearly mistaken!!
Enter the new beast — Grace Blackwell.
So what is Blackwell?
This is NVIDIA’s latest most advanced chip. Well, they’re calling it a platform, but at the end of the day, it’s a very, very advanced chip. The most advanced they have till date, until the next GTC conference, of course. 😃
What’s so special about Blackwell?
At one point during his keynote, Jensen held both the chips side by side — the grace hopper and the blackwell. He then enacted a brief yet witty conversation between the two of them.
This conversation had very strong undertones. The bigger chip was telling the smaller chip that the laws formed in its era would not work. What laws?
- Small isn’t necessarily always good.
- Small doesn’t mean more throughput.
- Small isn’t always good for the environment.
Blackwell, in comparison, can deploy real-time generative AI on trillion-parameter large language models at significantly reduced costs and energy consumption compared to previous chip designs.
This is the huge pillar that they shook. Within 8 years, NVIDIA has increased the performance of their latest GPU chip by 1000 times. Moore’s Law (although its not a scientific law) states that the speed and capability of computers can be expected to double every two years!!
NVIDIA just upended one of the most foundational laws of computing. It just declared open season for the semiconductor industry.
The Second Pillar — The New AI Factory
Once the foundations are shaken, you don’t stop there!!
The next phase was the unveiling of the Blackwell platform and unraveling the intricate hardware improvements that they have made to their infrastructure, right from GPUs to networking technology such as Infiniband, that make them the ideal first-class AI workhorse.
Accelerated computing is here. If there was any message Jensen was making throughout the two hours, this was it. The era of general-purpose computing is over.
In the future, according to NVIDIA, AI factories will spring up all over town. These factories will host thousands of systems that are powered by GPUs. Foundational models (ex: - ChatGPT) of all shapes and sizes will reside here. Since the future will be dominated by AI agents in almost every aspect of technology, the need for such infrastructure will rise, as will the demand for GPUs.
Guess who’s going to make a killing?
Today, data centers globally still have a mixture of CPUs (for general-purpose computing) and GPUs (specialized for AI and graphics). While it remains to be seen how much of the GPU data center market gets cornered by NVIDIA, there are other players in the GPU market.
The Third Pillar —NVIDIA Applications
It would be foolish for NVIDIA to think that they could conquer and dominate the entire GPU market without eventually meeting their match. To ensure customer retention, they built an entire layer of applications fueled by their superior infrastructure (GPUs).
These applications run on platforms that handle some of the most sophisticated AI challenges of the future, all areas where NVIDIA is strong.
NVIDIA gains a significant advantage by expanding its influence across all industries that AI will influence. Let’s take a look at two of them::
- The Omniverse: — NVIDIA’s Omniverse was first launched with much fanfare a few years ago. This is their play for the Digital Twin market. Using USD (Universal Scene Description), the Omniverse platform gives developers the ability to build digital twins of real world objects that do not exist to simulate them in their entirety, before they are produced physically. Firms such as Siemens, Ansys, and even Rockwell Automation are among the many who are collaborating with NVIDIA on this.
- BioNeMo: — They couldn’t leave healthcare out of the picture with so much money on the table. Here too, NVIDIA has gone for the gold. With BioNeMo, its supercomputing platform, it allows for training biomolecular large language models specifically meant for therapeutics. In short, the platform accelerates the development of new molecules for drug discovery.
By sinking its teeth deeply into a future driven by IoT, NVIDIA creates a stickiness in its products that would be difficult for its customers to shake off.
The Fourth Pillar — Groot
So what were those robots doing on the stage with Jensen Huang just minutes before the end?
Make no mistake, this is another one of NVIDIA’s application platforms, like the ones mentioned above. The only reason I give it another section here is because I feel it is really super-ambitious.
Robotics has been moving in fits and starts since science fiction started appearing on our black-and-white television screens decades ago. The two robots that walked on to the stage were of the same height as R2D2 and were making similar, strange sounds.
So Groot is another one of its general-purpose foundational models, but this time specifically for humanoid robots. Alongside this came the release of Thor (a system on a chip for robots), and finally topping it off was the Isaac Robotics simulation platform, which has all the tools needed to help customers develop their own robots.
These robots, apart from being designed to understand natural language, also learn to emulate movements by observing human actions. Adaptation to the real world is what humans do better than robots, and this is where NVIDIA thinks Groot and its Isaac Platform will give them an edge over the rest.
This really is ambitious, as some of their own partners, such as Amazon, already have their own robotics division.
The Fifth Pillar — The rest of us?
So much technology in just two hours, and multiple deep-dive sessions lined up over an entire week.
What am I missing here?
The cost of the Grace Blackwell GPU ranges from between USD 30,000 and USD 40,000; even the Grace Hopper (its predecessor) was slightly less. Needless to say, when companies look to build AI factories (data centers powered by GPUs), they will invest in hundreds of such chips.
This clearly puts this infrastructure in the hands of the few that can afford it. Surprisingly, when Jenson flashed a slide on the screen that had the customers who were presenting at the conference, it was mostly the Fortune 500 brand names.
This looks like a market built for the likes of hyperscalers (this is the term given to large cloud service providers such as Amazon, Microsoft, and Google that can provide services such as computing and storage for a cost).
So how do smaller firms gain access to this GPU infrastructure?
Well, one way would be to pay for their services from these cloud providers for a fee, since building out their own infrastructure for AI could be prohibitively expensive, as seen from above.
This is where the other vendors such as AMD could possibly make a dent. Their GPU’s are priced considerably lower than NVIDIA but they lack the platforms that NVIDIA come with and certain critical components like the software that powers it.
Nevertheless NVIDIA still surges, unrelentingly. They have sunk their chips, their network, and their AI infrastructure into too many cutting-edge, world-class products now to be dismissed lightly.
Jensen Huang’s NVIDIA has arrived.
In fact, many are now calling it the Huang Law. This is the forecast that the performance of graphics processing units (GPUs) will more than double every two years.
Have they become too big to be displaced?
