Debugging Methodologies

A simulation failure gives you one piece of information: something went wrong, at some point, in some signal. Turning that single data point into a fixed RTL bug is a skill built on systematic methodology. Engineers who debug randomly — opening waveforms and browsing signals hoping to spot something unusual — waste hours. Engineers with a structured approach work through the same failures in minutes. This article covers the two core debugging techniques: divide and conquer to find when the bug occurs, and signal tracing to find where in the logic it originates.

9 min read  |  Intermediate Level

What You'll Learn

• How to bisect a simulation timeline to rapidly narrow the failing window
• How to trace a wrong signal backward through logic to its root cause
• How assertions act as proximity detectors that fire close to the bug source
• How to reproduce intermittent failures and recognise common bug categories

Divide and Conquer

A simulation that runs for a million clock cycles and fails somewhere in the middle presents a navigation problem before it presents a logic problem. You need to know when the design first went wrong before you can understand why. The divide-and-conquer technique solves this with binary bisection: rather than scanning from the beginning, you check the midpoint of the simulation, determine which half contains the earliest failure, discard the other half, and repeat.

In practice, bisection works by inserting a temporary assertion or checkpoint at the halfway point of the simulation and running to that point. If the design state looks correct there, the bug is in the second half — discard the first half and bisect the remainder. If the design state is already wrong, the bug is in the first half. Each iteration halves the search space. Ten bisections narrow one million cycles to fewer than a thousand. This is far faster than scrolling through a waveform from cycle zero.

You do not always need to run the full simulation to check a midpoint. Many simulators support saving a simulation snapshot — a checkpoint of the complete design state at a given time — that you can restore and continue from without re-running the earlier portion. Check your simulator's documentation for checkpoint and restore commands.

Bisection also applies at a higher level. If a test suite has hundreds of tests and one regression run produces failures across several of them, start by checking whether the failures share a common time window or a common design block. A failure that appears in every test involving the memory interface is not ten independent bugs — it is one bug in the memory interface that manifests across ten tests.

Signal Tracing

Once you have narrowed the failure to a specific time window, signal tracing finds the root cause. The fundamental rule is: start at the wrong output and follow the fanin backward. Never start at a suspected cause and look forward — you will waste time on signals that were not involved.

The trace follows a repeating pattern at each level of logic:

• Identify the wrong signal. You have a signal whose value is incorrect at the timestep of interest. This is your starting point.
• Check its inputs. For a combinational gate, look at all the input signals at the same timestep. For a register, look at its data input at the previous clock edge.
• Find which input is wrong. One or more inputs will be incorrect. Mark the correct ones and continue tracing only the wrong one.
• Repeat one level back. The wrong input becomes your new starting signal. Check its inputs. Keep going until you find a signal that is wrong but whose inputs are all correct — that is the root cause.

The trace terminates when you reach a signal that is wrong despite having all-correct inputs. At that point, the bug is in the logic that computes that signal — a wrong condition in an if statement, a missing reset, an off-by-one in a counter, or a wrong operator in an expression.

Tracing through a register requires stepping back one clock cycle. If a register output is wrong at cycle N, its input was wrong at cycle N-1 (the cycle when the register sampled its data input). Always account for this cycle boundary when tracing through sequential elements.

Using Assertions as Bug Proximity Detectors

The further a failure propagates through a design before being caught, the harder it is to trace back to its source. A wrong value in a counter on cycle 100 may not produce a visible wrong output until cycle 850, by which point the waveform contains hundreds of cycles of corrupted state to sift through. Assertions solve this by checking invariants directly in the RTL, close to the logic they monitor, so that violations fire at or near the cycle the bug actually occurred.

There are two kinds of assertions in SystemVerilog:

• Immediate assertions fire when the line of code executes, inside a procedural block. They check a condition at a specific point of execution and are useful for verifying assumptions about values mid-computation.
• Concurrent assertions evaluate at every clock edge and express temporal properties — things that must be true about signal relationships over time. They are written outside procedural blocks and are the primary tool for protocol checking and invariant monitoring.

// Immediate assertion — fires when this line executes
always @(posedge clk) begin
    count <= count + 1;
    assert (count < MAX_COUNT)
        else $error("[%0t] count overflow: %0d", $time, count);
end

// Concurrent assertion — evaluates every clock edge
assert property (@(posedge clk) valid |-> !$isunknown(data))
    else $error("[%0t] data is X while valid is asserted", $time);

When an assertion fires, the error message appears in the log with a timestamp. That timestamp is the start of your divide-and-conquer search — not the end of the simulation, but the cycle the property was first violated. This is almost always much closer to the actual root cause than a scoreboard mismatch caught thousands of cycles later.

Adding assertions to RTL is not only a debugging technique — it is a verification technique. A well-asserted design catches bugs at their source during any simulation, not just when you happen to be actively debugging. Assertions added during debug should be kept in the RTL permanently so they protect against future regressions.

Reproducing Failures

A failure you cannot reproduce reliably is a failure you cannot debug. Two categories of simulation failures require special attention for reproduction:

Seed-dependent failures arise in tests that use constrained random stimulus. Each run uses a different random seed to generate its input transactions, so the same test may pass with seed 42 and fail with seed 1983. When a failure is found, record the seed immediately — it is printed in the simulator log at the start of every run. To reproduce the failure deterministically, re-run with the same seed using your simulator's +ntb_random_seed or equivalent plusarg. Once you have a reproducible run, you can use divide and conquer and signal tracing normally.

// Re-run with a specific seed (simulator-dependent syntax)
// Cadence:   +ntb_random_seed=1983
// Synopsys:  +ntb_random_seed=1983
// Mentor:    -sv_seed 1983

Race-condition failures may disappear when you add $display statements or change the simulation options, because those changes alter the Active-region scheduling order. If a bug only appears sometimes and vanishes when you try to instrument it, suspect a race condition of the kind described in article 4.2. The fix is to eliminate the race in the testbench rather than to chase the symptom.

Common Bug Categories and Their Signatures

Certain classes of RTL bugs appear repeatedly across designs and have recognisable waveform signatures. Learning these patterns shortens the trace significantly.

Off-by-one errors are the most common category in counter and FSM logic. A counter that should wrap at value N wraps at N-1 or N+1. Symptoms: a burst that is one transaction too long or too short; a FIFO that overflows by one entry; a timer that expires one cycle early. Look for comparators using < vs <= or a reset value that is 0 vs 1.

Missing or incorrect reset causes registers to start in state X and propagate unknown values until something forces them to a known state. Symptom: X values appearing at the start of simulation that do not resolve after reset deasserts, or output signals that behave correctly in some runs and produce X in others depending on initial state. Check that every register that needs a defined reset value has an active reset condition in its always block.

Wrong clock domain bugs arise when a signal is sampled by a register clocked on a different domain without a synchroniser. In RTL simulation, CDC bugs are largely invisible — zero-delay simulation has no propagation delays, so there are no setup or hold violations. The receiving register simply captures whatever value the signal holds at the clock edge and the simulation passes cleanly. The bug only becomes apparent in gate-level simulation with SDF back-annotation, where real timing is modelled and setup/hold violations cause the register output to go to X, or in silicon where metastability can cause the device to malfunction. This is why dedicated CDC static analysis tools are used alongside simulation rather than relying on simulation alone to catch these issues.

Timing violations in gate-level simulation appear as X values on register outputs caused by setup or hold time violations flagged in the SDF-annotated netlist. These indicate that the synthesised design has a timing path that does not meet the target frequency. They are not RTL bugs — they are timing closure issues that require either RTL restructuring or physical design changes.

Common Mistakes