Memory Bandwidth Budgets

A memory bandwidth budget adds up how much data every block of an SoC needs to move to and from DRAM in a given use case, and compares the total with what the memory system can really deliver. It is one of the first calculations an architect makes, because DRAM bandwidth is expensive, shared by everyone, and very hard to add once the chip is built. This article shows how to build a budget step by step, from the requirements of each block, through the efficiency of the memory, to the margin and the worst case, with a complete worked example.

11 min read | Intermediate

What You'll Learn

• Calculate the bandwidth of display, camera, video and processor traffic
• Convert a DRAM's peak bandwidth into a realistic sustained figure
• Build a budget for a use case and judge its margin
• Identify ways to reduce bandwidth when the budget does not close

Why a Budget

Latency and Bandwidth showed how to calculate the bandwidth a display needs. A real SoC has many such clients at once: the display reading frames, the camera writing them, a video encoder reading and writing, the GPU drawing, and the CPUs running software. All of them share a few DRAM channels. If together they ask for more than the memory can deliver, something must wait, and for real-time clients, waiting means visible or audible failure.

The budget answers three questions early, when the answers are cheap to act on: how many DRAM channels, at what speed? Is there enough margin for the worst case? And which clients need priority, as Arbitration and Quality of Service describes?

Step 1: List the Use Cases

Bandwidth depends on what the product is doing, so the budget is built per use case, from From Use Cases to Requirements. A phone might have: video playback, video recording, gaming, a video call, and standby. Each has different clients active at different rates. The budget must close for every one of them, and the hardest one, often recording 4K video while showing a preview and running the camera's processing, sets the requirement.

Step 2: Calculate Each Client

Most real-time clients are simple to calculate from their format:

display read     = width x height x bytes per pixel x frames per second
camera write     = width x height x bytes per pixel x frames per second

A 1,920 x 1,080 display at 4 bytes per pixel and 60 frames per second reads about 0.5 GB/s. A camera writing frames of the same size at the same rate also needs about 0.5 GB/s. These numbers rise quickly: a 4K display (3,840 x 2,160) at 120 frames per second needs about 4 GB/s on its own.

Other clients need more thought:

• Video encoders and decoders read reference frames, often several times each, and write the output. Their traffic is usually a few times the raw frame rate, and depends on the codec and its settings. Vendors provide figures for their IP.
• GPUs read textures and geometry and write rendered images. Their need depends on the content, and they will use as much as they get, so they are budgeted as what's left, with a floor for acceptable frame rates.
• CPUs are measured by running real software on a model, or estimated from their cache misses: misses per second times the line size.
• DMA and I/O, such as storage or network transfers, are set by their interface rates.

Write traffic often costs more than its byte count suggests, because a partial write to a line may force a read first, and DRAM pays a penalty for switching between reads and writes, as Memory Controllers and Scheduling explains.

Step 3: Find the Real Supply

The memory's peak bandwidth comes from its width and data rate. A 32-bit interface at 4,266 million transfers per second gives 4,266,000,000 x 4 bytes, about 17 GB/s. But as DRAM Organization explains, peak is reached only if the data bus is busy every cycle. Refresh, row conflicts, bank conflicts and read-write turnarounds leave gaps. A realistic efficiency on mixed traffic is often 60 to 80 percent, and much lower for random access. In a model of one DRAM channel, a single sequential stream reached about 84 percent, random accesses about 39 percent, and two interleaved streams between 13 and 76 percent depending on the scheduler. These numbers show why the efficiency must be estimated from the real traffic mix, not assumed.

For a budget, architects use a sustained figure, such as 70 percent of peak, checked with simulations of the actual traffic. In the example above, that is about 12 GB/s.

Step 4: Add It Up

Here is a worked budget for one use case, video recording with a live preview, on that 17 GB/s memory. The client figures are assumptions for this example:

Demand is 8.5 GB/s against a sustained supply of about 12, so the budget closes with about 30 percent margin. That margin isn't waste. It absorbs the bursts, the variation in traffic, and the effects that the simple numbers leave out, such as the extra traffic when a cache thrashes. Many teams want at least 20 to 30 percent margin at this stage, since the numbers tend to grow as the design firms up.

Step 5: Check the Peaks

Averages hide moments. A display reads continuously, but a GPU may need its whole frame's work in a fraction of each frame period, and a camera may deliver a frame in a burst. If several bursts coincide, the instantaneous demand may exceed the supply for a while. Real-time clients survive that only if their buffers cover the gap. So the budget is checked with a time-based simulation of the traffic, and the buffer sizes of real-time clients are chosen to cover the longest expected delay.

When the Budget Doesn't Close

If demand exceeds supply, there are two directions: more supply, or less demand.

More supply means more channels, a faster DRAM type, or wider interfaces. Each costs pins, power and money.

Less demand is often cheaper, and there are several well-known ways to get it:

• Compression. Frame buffers are compressed losslessly by the blocks that write them and decompressed by the blocks that read them. Display and GPU traffic can often be cut substantially this way.
• On-chip memory. A system cache, or a dedicated buffer, keeps data that is used again near the blocks, so it never goes to DRAM, as On-Chip SRAM, Scratchpads and System Caches explains.
• Direct paths. Blocks pass data to each other directly, rather than writing it to DRAM and reading it back, such as a camera pipeline feeding the encoder.
• Lower resolution or rate where the user won't notice, such as a preview at lower resolution than the recording.
• Better locality, through tiled data layouts that make DRAM accesses more efficient.

Power Follows Bandwidth

Every byte moved to and from DRAM costs energy, in the controller, the PHY, the wires and the DRAM itself, and moving data off chip costs far more energy per bit than moving it on chip. In a phone, DRAM traffic can be a significant part of the power when the screen and camera are active. So a bandwidth budget is also a power budget. Reducing traffic for performance reasons usually saves power too, which is why compression and on-chip buffers appear even on chips with spare bandwidth.

Keeping the Budget Alive

A budget isn't done once. It starts as a spreadsheet early in the project, with estimates for every client. As IP is chosen, the vendors' figures replace the estimates. As the design matures, a performance model of the interconnect and memory controller replaces the spreadsheet's simple efficiency factor, as System Performance Modeling describes. Finally, the real chip is measured, with hardware counters in the memory controller that report the traffic of each client. Each stage catches errors in the one before it.

Real chips also have hardware that helps check the budget in the field: counters in the interconnect and memory controller that count bytes per client, which firmware can read while the product runs a real use case. Those measurements often reveal clients that use more bandwidth than anyone expected.

The budget should be owned by one person, reviewed whenever a feature changes, and kept with the reasons for each number. A frame rate raised by marketing, or a GPU workload that turns out heavier than expected, should show up in the budget the same day, not in a failed test months later. Treated this way, the budget is one of the most useful documents an SoC team keeps.

Common Mistakes