ISA, Microarchitecture and Implementation

The word "processor" hides three separate things: the instruction set that software sees, the microarchitecture that organizes the hardware, and the physical implementation in silicon. Keeping those layers apart is what lets one program run on many different chips, and it lets engineers change one layer without breaking the others. This article explains each layer, shows what each one can and cannot change, measures the effect of changing the bottom layer alone, and looks at where the layers leak into each other.

12 min read | Beginner

What You'll Learn

• Distinguish instruction set, microarchitecture and implementation
• Explain why the separation matters in industry
• Say which layer each design decision belongs to
• Identify cases where a lower layer becomes visible to software

Three Layers of One Processor

Every processor can be described at three levels:

The instruction set architecture (ISA) is the contract with software. It defines the registers, the instructions, how they're encoded, how memory behaves and how traps work. A compiler needs nothing else. What an ISA Defines covers it in detail.

The microarchitecture is how one design carries out that contract: the number of pipeline stages, the branch predictor, the cache sizes, whether instructions run out of order. Software can't see any of it directly, though it can feel the difference in speed.

The implementation is how that organization becomes a physical chip: the manufacturing process, the library of logic gates, the layout, the supply voltage and the clock frequency.

These are independent choices, within limits. Many microarchitectures can implement one ISA, and one microarchitecture can be built in many implementations.

One Program, Many Processors

The great benefit of this separation is binary compatibility. A program compiled for RV32I runs on a tiny two-stage microcontroller, and on a large out-of-order server core, and gives identical results. The first might take 1 million cycles, and the second 300,000, at ten times the clock. Only the timing differs.

That allows a software ecosystem to outlive any particular chip. An x86 program from the 1990s still runs on a modern desktop, though the processor inside has changed beyond recognition. It lets vendors sell a range of products, from cheap to fast, all running the same software. And it means one team can work on a better pipeline while another builds compilers, with only the ISA between them.

Which Layer Is This?

Here is how to place common decisions:

Changing the Microarchitecture

A microarchitecture change leaves the program the same, and changes CPI and perhaps the clock. Forwarding is a clear example. A small five-stage core was built with and without it, running the same programs with the same compiler. The speedup ranged from about 1.17 to about 1.7, for about 2.5 percent more area, as PPA Trade-offs and Amdahl's Law shows. Not one instruction changed, and no program needed recompiling. The pipeline chapter and everything after it in this tutorial describe techniques of this kind: prediction, caches, out-of-order execution. They are all at this layer.

Changing the Implementation

The same RTL can also be built in different ways. Take one small core and synthesize it, turning it into gates, using four cell libraries from the same 130 nm process:

A standard cell library is a chip maker's catalog of ready-made logic gates, characterized for speed and power. A high-density library packs them more tightly: the same core came out about 29 percent smaller than in the other libraries, whose cells share one footprint. A high-speed library has lower gate delays and higher leakage, and a low-speed library the reverse. The RTL did not change at all, so the instructions, the cycle counts, and the CPI are all unchanged. Only the area, the frequency and the power moved.

Moving to a new process node is a stronger version of the same thing. It usually gives more speed and lower energy per operation, or a smaller die, with no change to the microarchitecture. For decades, a good share of each generation's performance came from this alone. That slowed around 2005, and today gains have to be earned at the upper layers.

Where the Layers Interact

The layers are independent in principle, but each constrains its neighbors.

The ISA shapes the microarchitecture. RISC-V's fixed instruction format makes decoding several instructions per cycle easy, while x86's variable lengths need much more logic. Delay slots, once built into an ISA for a five-stage pipeline, became a burden on later designs with different pipelines. An ISA lasts for decades, so a decision that suits today's design can be a nuisance for a long time.

The implementation shapes the microarchitecture. Clock frequency depends on the logic depth per stage, which is a microarchitecture choice, and also on gate speed, which is an implementation fact. If the wires in a process are slow, the design may need an extra pipeline stage just to cross a block. Memory macros come in certain sizes, so a cache size may be chosen by what the process offers.

The microarchitecture shapes the ISA as well, in the long run. New instructions are often added because of what hardware can do cheaply. ISA Design Choices gives examples.

When the Layers Leak

The separation isn't perfect, and some lower-layer details become visible to software.

• Timing. A program can measure how long things take, and then it can see a cache or a predictor. Performance tuning does that on purpose. Attacks such as Spectre do it to steal data.
• Memory ordering. The consistency model exists because hardware buffers change what other cores see.
• Pipeline-specific tuning. A compiler's -mtune option schedules code for one microarchitecture, as Compiler Optimization Levels describes. The code runs anywhere, but it runs best on that one.
• Cache management. Drivers that share buffers with DMA need to know about caches, even though the ISA tries to hide them.
• Errata. A bug in one implementation may need a software workaround on that chip alone.

A Worked Example: Same Program, Two Chips

Take a program of 1 million instructions, and two chips that implement the same ISA. Chip A is a small three-stage core with a CPI of 1.3 at 200 MHz. Chip B is a large out-of-order core with a CPI of 0.5 at 2 GHz. These numbers are assumptions for this example. Using the performance equation:

chip A:  1,000,000 x 1.3 x 5 ns    = 6.5 ms
chip B:  1,000,000 x 0.5 x 0.5 ns  = 0.25 ms

Chip B is 26 times faster, and the program didn't change at all. The instruction count is the same on both, because that belongs to the ISA and the compiler. The CPI differs because of the microarchitecture. The clock differs because of both the microarchitecture and the implementation. So the speed difference breaks down neatly: a factor of 2.6 from CPI, and a factor of 10 from the clock. If a third chip had B's design at 1 GHz in an older process, B would beat it by exactly 2, a gain that comes only from the clock. This kind of breakdown tells you which layer gets the credit for a result.

How an ISA Grows

An ISA doesn't stay fixed forever. It must change slowly, though, because old software has to keep running. The usual rule is that new instructions may be added, but old ones are never removed or changed. x86 has grown this way for more than forty years, and Arm has done something similar. RISC-V is built around the idea: a small frozen base, with optional extensions, each with its own letter or name.

Options create a new problem. If each chip picks its own set of extensions, software cannot know what it can use. RISC-V's answer is profiles: named sets of extensions that a whole class of processors agree to provide, so that an operating system can target one profile and run on all of them. Software can also check at run time which features exist, and choose a faster routine if it can. A library function such as memcpy often has several versions, and picks one when the program starts.

Why Vendors Keep the Layers Apart

The separation also shapes how companies work. Because software depends only on the ISA, a chip company can replace its microarchitecture every generation without asking anybody's permission. It can use the same design in a laptop and in a server, at different voltages and frequencies. And it can move a design to a new foundry with no change to the software. Each layer can be improved, licensed or bought separately. Without those clean boundaries, every new chip would need all of its software rewritten.

Layers Below and Above

Real systems have even more layers. Below the implementation are transistor physics and the manufacturing process. Above the ISA come the ABI, the operating system and the language. Each layer provides a contract to the one above, and hides the one below. That is how a system of billions of transistors can be built by people who only understand a few layers each.

Software can even cross the ISA boundary. Binary translation rewrites a program for one ISA into instructions for another, which is how Apple kept x86 programs running when its computers moved to Arm.

Common Mistakes