©2026 Business Next Media Corp. All Rights Reserved. No.102, Guangfu S. Rd., Da'an Dist., Taipei City 106, Taiwan
How Nvidia won AI
In this video, we are going to look at how Nvidia turned the humble graphics card into a platform that dominates one of tech’s most important fields: Artificial Intelligence.
When we last left Nvidia, the company had emerged victorious in the brutal graphics card Battle Royale throughout the 1990s.
Very impressive. But as the company entered the 2000s, they embarked on a journey to do more. Moving towards an entirely new kind of microprocessor - and the multi-billion dollar market it would unlock.
In this video, we are going to look at how Nvidia turned the humble graphics card into a platform that dominates one of tech’s most important fields: Artificial Intelligence.
What is a GPU?
In 1999, Nvidia released the GeForce 256 graphics card. Its processor had 17 million transistors and was fabbed by TSMC.
Most notably, when Nvidia unveiled the GeForce, they called it "the world's first Graphics Processing Unit", or GPU.
A new term that must have struck people as a bit of clever marketing. At its introduction, Nvidia defined the term GPU to mean:
"a single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines that is capable of processing a minimum of 10 million polygons per second."
To understand what was so special or groundbreaking about this particular Nvidia product, we need to dive into the graphics pipeline. Yes, we need to talk about graphics.
The Graphics Pipeline: Geometry
The job of a 3D graphics system is to turn a scene into images. 60 times a second, sometimes more, with minimal lag. Here is a very simplified example of how they did it. Like, super simple.
Early real-time graphics systems broke everything down into triangles. More complicated squares and quadrilaterals? Triangles. Curved patches? Triangles. It's triangles all the way down, baby!
At the start of the pipeline, you have a bunch of 3D coordinates data and 3D model or shape data.
The pipeline first ingests those 3D coordinates and turns them into 2D window coordinates. This is called the "geometry stage".
The geometry stage can be further subdivided into two sub-stages: The first is "Transform and Lighting", and the second is "Triangle Setup and Clipping".
In Transform and Lighting, the pipeline needs to adjust the 3D coordinates data and apply lighting effects to take into account the user's own personal perspective.
The Triangle Setup and Clipping stage is where we process the coordinate data into something that the rendering engine can render.
This especially matters in scenes where multiple 3D objects are overlapping one another. The overlapped object would need to be clipped, ergo the second part of the name.
Enter the Matrix
I want to pause here for a quick rest stop. When the graphics pipeline is processing and manipulating these coordinates, they use a mathematic structure called a "matrix".
A matrix is a rectangular array of numbers. You can perform operations on these matrices - like multiplication or addition - in a standard defined way.
We don't need to know exactly how matrices figure into manipulating 3D coordinates within the overall graphics process. Fear not, this is not a math class.
But we do need to know that they exist. And that the graphics pipeline - and thus graphics cards running them - were heavily optimized to process and run mathematical operations on them. It will come up again later.
The Graphics Pipeline: Rendering
Okay. Next, is the "rendering stage". Here, the pipeline fills the pixel space between the translated 2D coordinates with image pixels representing the object's surface.
The setup engine passes the render engine all the information about the object's color and texture at a particular spot as well as how those are affected by lighting conditions and perspective.
The final result is then sent to the monitor for displaying to the user.
As recently as 1996, Nvidia's graphics cards handled just a subset of the graphics pipeline - the render stage.
The CPU handled the rest.
This made the most sense for the company's early hardware since rendering was simple and repetitive. You receive the coordinates of the triangles' lines. Then you just draw the right lines with the right colors, many times over.
Evolving Up the Chain
But very quickly, Nvidia's cards started to take on more of the graphics pipeline on its own. In 1997, their cards took on Triangle Setup and Clipping.
If you recall, this stage was about computing the actual triangle coordinates for the render stage to process.
The CPU previously handled this, but the work was repetitive and became a bottleneck in the pipeline.
Now freed of the lame work of setting up triangle coordinate data, the CPU could focus on the geometry stage.
And it did this part quite well, throwing more triangles at the graphics card than it could handle. At that point, the card became the bottleneck in the pipeline.
Finding Parallelism
The solution Nvidia's engineers found for this bottleneck was parallelism. They converted the work into a pipeline of sequential steps. Then they added many identical pipelines and lined them up to work in parallel.
For some reason, I am reminded of bees in honeycombs, making honey. Each individual bee in its little comb works independently of its fellow parallel bees. No need to wait for this other bee to finish up his bee work first.
And no, I have no idea how honey is actually made.
3D graphics work was ideally suited for parallelism because the task is repetitive and data-independent. The system's processing speed is almost linearly proportional to the number of available honeycombs, or in the GPU’s case, processing cores.
This finally opened the door for Nvidia to take on the last part of the graphics pipeline: Transform and Lighting calculations.
With the GeForce 256, Nvidia succeeded in taking the entire graphics pipeline onto a single chip. No more need to transfer data back and forth with the CPU. This major structural change is why the company deemed it worthy of a new name.
More Programmable GPUs
The GeForce 256, the first GPU, was a major achievement and the market started to really take off.
However, this generation of GPUs ran what were known as "fixed function" pipelines. Once the programmer sent data to the GPU, that data could not be modified.
Kind of like an escalator, you cannot get on or off in the middle. This fixed functionality prevented developers from taking advantage of new features in graphics APIs and the like.
So the next natural step in GPU evolution would be to introduce additional customizability into that pipeline. This came about in 2001, two years later, with the GeForce 3.
To do this, Nvidia broke apart the fixed graphics pipeline. Programmers can now send data to customized programs called shaders.
Vertex shaders replaced the Transform and Lighting stage of the pipeline.
Pixel shaders operate at a pixel level and help with rendering the image. Programmers can now swap that bland artificial look with more organic surfaces.
These shaders were written in a low-level language, and had their limits. But it opened the door for more generalized programming capabilities down the line. That line came in 2006.
CUDA
In 2006, Nvidia introduced the GeForce 8 series GPU as well as a new, proprietary software framework.
Compute Unified Device Architecture, later dubbed just CUDA, gave programmers a set of libraries, debuggers, and APIs that made it substantially easier to program a GPU.
With this, the old fixed function graphics pipeline was now completely wiped away. The GPU was no longer a piece of specialized graphics hardware with bits for textures, bits for lighting, and bits for triangles.
Along with its open source cousin OpenCL, CUDA turned millions of commodity GPUs from game machines into generalized processors. Each equipped with many thousands of cores capable of manipulating large chunks of data in parallel.
Compare this with the other major category of general purpose processor in the consumer market, the CPU. The CPU has far fewer cores, and is best at running operations that need to go sequentially.
GPUs versus CPUs
GPU performance have scaled much faster than CPUs as of late. This is for several reasons. First, Nvidia has been able to benefit from TSMC's progression in commercializing new process nodes.
Because a GPU's processing power is linearly proportional to the number of processing cores it has, Nvidia could scale its GPUs' processing power by scaling its transistor counts. Some recent Nvidia GPUs have over 20 billion transistors.
Second, Nvidia was able to scale up frequencies, with the biggest jump coming in 2016. That year, Nvidia's Pascal GPUs used TSMC's 16nm process node for the first time.
That node was TSMC's first to use a new type of transistor - the 3D transistor called a FinFET. It allowed chips to run at a much higher clock speed.
Third, Nvidia ramped up their GPUs' power use while at the same time benefitting from improving per-transistor energy usage trends.
This scaling law is referred to as Dennard Scaling and it means that new chips use the same amount of energy despite having more transistors on the same sized space.
Dennard Scaling has slowed starting in 2006, but the trend continues to some extent and this allowed Nvidia GPUs to get much more powerful.
And fourth and most lastly, Nvidia found a new market demand for insanely powerful GPUs. One unrelated to the gaming market.
Neural Networks
ImageNet is a large dataset of one million images maintained by Stanford. Starting in 2010, they held a contest between various computer vision models to see who has the best image classifier.
Like as in, identify if the thing in the image was a strawberry, blueberry, or pigeon. So on.
In 2010 and 2011, the top performing models used hand-engineered computer vision methods. Those didn't perform too well, with an error rate of 25% in the top-5 performers. A normal human with a little bit of training can probably do about 5%.
Then in 2012, three researchers Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton participated in the contest using something called a deep neural network.
Trained over 5-6 days on two GTX 580 3GB GPUs, it took first place by a mile with a 15% error rate - ten percentage points better than the runner-up.
Since then, the winning model's error rate has dropped to the point where it regularly performs better than a trained human.
Then in 2016, four years later, deep learning hit the mainstream when Google Deepmind's AlphaGo won a 5 game match of Go against 18-time world champion Lee Sedol.
The Math Behind
As you might expect, a deep neural network is based on one conceptual understanding of how the biological neural network works. I use that phrase carefully because nobody knows how the brain really works.
But underneath all the philosophizing is simply a titanic mathematical function. It uses a network of simple processing elements to model complex relationships between many inputs.
The network is made up of many small "neurons". Each neuron takes in many inputs, applies weights to them, and like some sort of judge, spits out an output.
The network can have multiple layers of these neurons, all feeding information to one another.
Structuring these in different ways leads to varying results.
In the field, people represent a network's layers of neurons using matrices. The inputs and weights are basically just numbers within a big matrix. This takes advantage of the GPU's relative strength in handling matrices to train networks vastly faster than ever before.
GPUs and Neural Networks
Neural networks as a concept have been around since the 1950s. But researchers never got good results with neural networks before. The reason has to do with the sheer amount of data required to get reliably good results.
For instance, HBO’s Silicon Valley trained a neural network for its “Not Hot Dog” mobile app. In order to train that simple app the developer had to collect 3,000 photos of hot dogs. And 147,000 photos of things that were not hot dogs.
For many years, researchers had to use CPUs to train their neural networks. This took an impractically long amount of time, because CPUs are not meant for parallel operations. They have too few cores for doing that work.
Furthermore, they have limited amounts of memory cache. Neural network training is quite memory intensive. You have to store the input data, parameters, and more as the inputs work through the network's layers.
Anyway, training a network with 128 images using a dual ten-core CPU from 2012 took about 124 seconds. Extrapolate that out to a million images or more and it takes 11 days to train a single machine learning model. That is not practical. You would never be able to iterate on your work.
GPUs on the other hand can do it 8.5 times faster than a CPU - a million images in about a day. A massive improvement. In other use cases, the speed improvement can get even better - up to 40 times faster.
Scientists had noticed this possible application for years as GPUs' power - as measured by floating operations per second - vastly scaled up. But prior to CUDA's release in 2006, there was no easy way to actually program these operations on a GPU. Researchers had to fit the neural network into the confines of graphical operations, which was about as hard as it sounds.
Revenue Generation
Machine learning first helped computers read handwriting, recognize images, and play the game of Go. Okay, who cares about that right?
Companies quickly realized however that they would be able to apply the same techniques to more financially lucrative tasks. Things like autonomous driving, content moderation, revenue optimization, and more.
As you might expect, these killer apps motivated businesses to quickly adopt deep learning techniques into their revenue models. Almost all of this work has been done on Nvidia GPUs and coded up in CUDA.
The company has invested substantial resources developing the space, correctly believing that it would lead to more high margin GPU sales.
Their flagship GPU for enterprise-level neural networks is the A100, with some 54 billion transistors and fabbed by TSMC. These are sold to data centers at something like $20,000 a pop. The data centers then rent access to their hardware through the cloud for as much as $33 an hour.
The company's biggest revenue generators are still consumer GPUs for gaming or crypto. But starting in 2017, its data center revenues started accelerating too.
In the third quarter of 2021, the company made $3.2 billion from gaming, and $2.94 billion from data centers.
But the data center division is growing much faster, 55% year over year compared to 42% in Gaming.
Competition
Nvidia GPUs helped enable the deep learning revolution, but competition in the hardware space remains pretty tough.
On the GPU side, AMD bought Nvidia's former graphics card competitor ATI. They then evolved their own line of GPUs, along with a CUDA of their own - Radeon Open Compute, or ROCm.
Unlike CUDA, AMD has opted to make ROCm open source, which offers programmers a greater amount of access to internal GPU details.
On another front, a variety of startups have emerged offering ASICs - chips designed for one purpose only, like machine learning. Cerebras and Graphcore are a few notable names. Both companies have raised multi-million dollar funding rounds from big VCs, looking for a piece of the Nvidia deep learning lunch.
Other companies have opted to create their own deep learning hardware, mostly for economic or strategic reasons. Google, Amazon, and Tesla for instance.
This is an option few companies have the expertise and scale for. And the tech giants in particular are fortunate since so many control their own popular deep learning frameworks.
Despite all of this competition, the machine learning revolution remains centered on Nvidia and their ecosystem of hardware and software products - both of which have been integrated together to deliver the best experience and performance.
Furthermore, the company’s close collaboration with AI researchers in the industry has helped make its products the de facto standard.
Conclusion
This evolution reminds me of how Apple evolved the iPhone. When it first came out, there was no App Store and third party apps could not exist. Steve Jobs imagined it to be a phone with a few narrowly defined extra functions.
The first iPhone sold well, but it didn’t become a phenomena until Apple opened up the iPhone hardware to outside developers, turning it into a powerful computing platform that you can fit in your pocket.
Likewise with Nvidia and its GPUs. It took longer than the iPhone’s opening up, but the results have been just as ground shaking. Nvidia probably didn't see that its newly generalized hardware would eventually power the latest AI revolution.
But they didn't really need to. That's why a platform can be so powerful.
©2026 Business Next Media Corp. All Rights Reserved. No.102, Guangfu S. Rd., Da'an Dist., Taipei City 106, Taiwan