This chapter moves from verifying individual SoC mechanisms - the address map, the interconnect, DMA engines, interrupts - to verifying what happens when several CPU cores are active on that same fabric at once. This article covers the basics: what a multi-core cluster actually looks like from a verification standpoint, how cores boot, how they talk to each other, and which resources are private versus shared. Cache coherency, the hardest part of that picture, gets its own dedicated article next.
10 min read | Intermediate Level
What You'll Learn
- How a multi-core cluster is organized: private per-core resources versus resources shared across the cluster
- How cores are identified and how that identity affects routing and testing
- The multi-core boot sequence - why only one core starts running out of reset, and how the others are released
- Inter-processor interrupts as the basic building block of cross-core software coordination
What a Core Cluster Actually Looks Like
A typical multi-core cluster groups two, four, or eight CPU cores behind a shared interconnect port. Each core keeps its own private L1 instruction and data cache, its own register file, and its own pipeline - none of that is visible to any other core. Above the private L1s sits a cluster-level resource that is shared: an L2 cache, a snoop-capable interconnect, and often a shared scratch memory or tightly-coupled memory region used for fast inter-core communication.
That split matters directly for verification. A bug in a core's private L1 behaves exactly like a single-core bug - the existing per-core testbench infrastructure already covers it. A bug in anything shared - the L2, the snoop logic, an arbiter deciding whose request gets the shared bus this cycle - can only be triggered by activity from more than one core at the same time, which means it can only be found by a test that actually runs multiple cores concurrently.
Core and Cluster Identification
Every core in a cluster needs a unique identity so the rest of the SoC - the interrupt controller, the debug infrastructure, software itself - can tell them apart. This is usually a small, read-only ID value each core exposes, often split into a cluster ID and a per-core affinity ID within that cluster. Software reads it early in boot to decide which code path to take, and the interrupt controller uses it to route a given interrupt or inter-processor signal to the correct destination core.
From a verification standpoint, this ID is a small thing with an outsized blast radius if it is wrong. If two cores in simulation ever report the same ID, every downstream mechanism that depends on it - interrupt targeting, debug core selection, per-core test bring-up - breaks in a way that can look like a completely unrelated failure. A basic sanity check that reads back each core's ID and confirms it is unique and matches its physical position belongs in the very first multi-core test written for a design.
Multi-Core Boot: Who Fetches First, and Who Waits
Out of reset, an SoC almost never lets every core start fetching instructions simultaneously. Typically only one designated core - conventionally core 0 - comes out of reset and begins executing, while every other core is held in a low-power wait state or spins on a small piece of boot ROM, waiting to be released.
The primary core's boot software is responsible for releasing the others once the system is ready: it writes a start address into a per-core mailbox or holding-pen register, then signals the waiting core - through a dedicated wake event, a doorbell register, or an inter-processor interrupt - to jump to that address. Verifying this sequence means checking two things independently: that a secondary core genuinely stays parked and produces no bus activity until released, and that once released, it fetches from exactly the address the primary core wrote, not a stale or default reset vector.
task verify_secondary_core_release(int core_id, bit[31:0] entry_addr);
// Confirm the secondary core is parked - no fetch activity yet
check_core_idle(core_id);
// Primary core writes the release address into this core's mailbox
write_reg(get_mailbox_addr(core_id), entry_addr);
// Primary core signals the release (doorbell / IPI / wake event)
send_core_release_signal(core_id);
// Confirm the core's first fetch lands exactly on entry_addr
wait_first_fetch(core_id);
if (last_fetch_addr[core_id] !== entry_addr)
`uvm_error("CORE_BOOT", $sformatf("Core %0d fetched from 0x%0h, expected 0x%0h",
core_id, last_fetch_addr[core_id], entry_addr))
endtask
A core that is supposed to be parked but still generates bus transactions is its own class of bug - it can quietly corrupt shared state before software even knows that core exists.
Letting a parked secondary core share the same boot ROM entry point and stack region as the primary core.
If every core reads from the same address-0 boot vector before each core's own stack pointer is set up, two cores fetching from that shared code can end up writing to the same stack memory and corrupting each other's state - a failure that shows up as a random-looking crash with no obvious connection to boot. Verifying multi-core boot means confirming each core establishes its own private stack before doing anything else, not just confirming it eventually reaches the right entry address.
Inter-Processor Interrupts: How Cores Talk to Each Other
Once every core is running, software still needs a way for one core to get another core's attention - to hand off work, signal that shared data is ready, or ask a core to stop what it is doing. The mechanism is an inter-processor interrupt (IPI): one core writes to a register that asks the interrupt controller to raise an interrupt on a specific target core, rather than reacting to a hardware event the way an ordinary peripheral interrupt does.
An IPI is still an interrupt, so every failure mode already discussed for cross-block interrupt verification applies here too - a targeted IPI has to actually reach the intended core and no other, and back-to-back IPIs sent close together must not let one get lost the way a clear-versus-reassert race can drop a peripheral interrupt. What is new at multi-core scale is the addressing: the test has to confirm an IPI aimed at core 2 never shows up on core 1, which requires driving traffic from multiple cores and checking each one's interrupt state independently.
Private Versus Shared: Drawing the Line for Testing
A useful habit when planning multi-core tests is to sort every resource in the design into one of two buckets. Private resources - a core's own registers, its L1 caches, its local performance counters - can be tested largely the same way a single-core bench already tests them, just repeated per core. Shared resources - the L2, the snoop-capable interconnect, any shared scratch memory, the interrupt controller's routing logic - need tests specifically built to run multiple cores concurrently, since a single active core can never expose a sharing bug.
Shared memory locations deserve particular attention even before coherency itself becomes the focus. A location one core writes and another core is expected to read is only interesting from a multi-core-basics standpoint if the read genuinely happens after the write in program order, using whatever synchronization the software protocol requires - a flag, a semaphore, an IPI. Whether the second core's cache actually reflects the first core's write correctly is a cache coherency question, and that is where this chapter goes next.
Published verification work on multi-core SoCs makes a consistent point: heavier random stimulus on a single core does not find shared-resource bugs, because one core's own randomization never varies the one thing that matters - the relative timing between different cores' requests hitting the same shared resource. A shared-resource test needs that cross-core timing coordinated on purpose, not left to chance.
Common Mistakes
Assuming a single-core regression suite exercises shared resources adequately.
A regression that runs one core at a time, even many times over, will never generate the concurrent, overlapping requests needed to stress an L2 arbiter or a snoop path. Multi-core coverage requires tests explicitly written to run cores in parallel.
Verifying core release with a fixed, single entry address.
A boot-release test that always parks and releases the secondary core at the same hardcoded address can pass even if the mailbox write path itself is broken. Randomizing the release address across runs is what actually exercises the mailbox logic rather than the reset vector default.
Treating an IPI exactly like a peripheral interrupt with no addressing check.
Reusing peripheral-interrupt tests without adding a check that the IPI reached only its intended core misses the one failure mode that is unique to inter-processor interrupts: correct interrupt behavior on the wrong core.
Summary
In this article, you learned:
- A core cluster separates private per-core resources like L1 caches from resources shared across the cluster, such as the L2 and the snoop-capable interconnect
- Each core needs a unique, verifiable ID that downstream logic like interrupt routing depends on
- Only one core typically fetches out of reset; secondary cores wait in a parked state until released to an address written by the primary core
- Inter-processor interrupts let cores signal each other directly, and need their own addressing checks beyond ordinary interrupt verification
- Multi-core test planning starts by sorting resources into private (test per-core) and shared (test with concurrent cores), setting up the coherency verification that follows in this chapter