How CPU caching speeds processor performance
CPU caching keeps recently (or frequently) requested data in a place where it is easily accessible. This avoids the delay associated with reading data from RAM.
The key processor specs are clock speed, number of cores and sizes of the level 1 and level 2 caches. It's easy to see how clock speed and the number of cores affect CPU performance, but what impact do level 1 and level 2 cache sizes have? Why is CPU caching important?
A CPU typically has a much higher clock speed than the system bus. For example, my computer has a clock speed higher than 3 GHz and a system bus speed of 400 MHz. This means when the CPU needs to read data from the system's RAM, it must conform to the system bus speed, even though that speed is ridiculously slow compared to the CPU's clock speed.
It's bad enough that a CPU has to spend so much time waiting on data to be read from memory, but often the same data gets read multiple times as a program executes. If the CPU did not have a cache, it would have to stop and wait for the data to be read each time it was requested.
A CPU cache places a small amount of memory directly on the CPU. This memory is much faster than the system RAM because it operates at the CPU's speed rather than the system bus speed. The idea behind the cache is that chip makers assume that if data has been requested once, there's a good chance it will be requested again. Placing the data on the cache makes it accessible faster.
The reason for two CPU caches
Why not just create one large cache on a CPU instead of two small ones?
Using two small caches increases performance. The data requested most recently is typically the data most likely to be needed again. Therefore, the CPU will always check the level 1 cache first. If the requested data is found in the level 1 cache, there's no need for the CPU to even bother checking the level 2 cache. This saves the CPU time because it does not have to search through the majority of the cache memory.
To further explain CPU caching, let's use the analogy of a library. The books in the library represent data. The librarian represents the CPU.
If someone asks the librarian for a particular book, she must get up, go to the shelves, find the requested book and bring it back to the person waiting for it. This takes some time, and there are other people waiting on the librarian (just as there'd be other instructions waiting on the CPU).
After the person reads the book and returns it, the librarian could put it back on the shelves, but instead she decides to put it on a small shelf next to her desk. This small shelf represents a level 1 cache. If another person were to ask for this book, the librarian could just reach over and pull the book off the shelf rather than having to hunt for it on the main shelves.
As more people return books, the shelf next to the librarian fills up. To make room for more books, she moves books from the shelf next to her desk to a larger shelf a few steps away. This shelf represents the level 2 cache. If someone asked the librarian for a book on this shelf, it would take the librarian a bit of time to find it because she'd first check the shelf next to her desk, then check the second shelf. Even so, it would still take her less time to retrieve the book than if the book were located somewhere in the library's main shelves.
As you can see, CPU caching is designed to keep recently (or frequently) requested data in a place where it is easily accessible. This avoids the delay associated with reading data from RAM.
About the author:
Brien M. Posey, MCSE, is a Microsoft Most Valuable Professional for his work with Windows 2000 Server, Exchange Server and IIS. He has served as CIO for a nationwide chain of hospitals and was once in charge of IT security for Fort Knox. He writes regularly for SearchWinComputing.com and other TechTarget sites.
