Pass Fail Criteria & Scoreboards

A test that runs to completion without crashing is not a passing test — it is an unverified test. For a simulation to produce a meaningful pass or fail result, the testbench must actively check that the DUT produced the correct output for the given input. Without that checking infrastructure, bugs can hide in plain sight: the RTL produces wrong results, the simulation finishes cleanly, and the regression turns green.

12 min read | Intermediate Level

What You'll Learn

• What constitutes a passing test: output correctness, assertion status, and timeout checks
• How a scoreboard works and how to build a simple one in SystemVerilog
• What reference models are and how to choose the right level of accuracy
• How to handle false failures and known-incorrect behavior during early bring-up

What Makes a Test Pass?

A test passes when three independent conditions are all satisfied simultaneously. Missing any one of them means the result is unreliable.

1. Output Correctness

The DUT must produce the correct output for every input it received during the test. This is checked by comparing the DUT's actual outputs against expected values — either pre-computed golden references or the outputs of a reference model running in parallel. A mismatch on any output at any point in time is a failure, even if subsequent outputs happen to be correct.

2. Assertion Pass

SystemVerilog assertions embedded in the testbench and DUT monitor protocol correctness, signal relationships, and timing properties throughout the simulation. A test passes only if no assertion fires during the run. An assertion failure indicates a protocol violation or an internal design invariant was broken — even if the final output happens to look correct, the design reached that output through an illegal state.

3. No Timeouts

Every test must complete within a defined simulation time limit. A DUT that hangs — waiting for an acknowledgment that never arrives, or stuck in a state it cannot exit — will run indefinitely without producing any output. Timeout watchdogs detect this condition and force a failure rather than letting the simulation run forever. A timeout is as much a failure as a wrong output.

All three conditions must be checked independently. It is possible for output checking to pass while an assertion fires (the DUT reached the right result through an illegal path), or for no assertion to fire while a timeout occurs (the DUT stalled without violating any monitored protocol property).

Scoreboard Architecture

A scoreboard is the component in the testbench responsible for output correctness checking. It sits between the DUT's outputs and the test result, collecting what the DUT produced and comparing it against what the DUT should have produced. A well-designed scoreboard is the difference between a self-checking testbench — one that can run unattended in regression and flag failures automatically — and a testbench that requires an engineer to eyeball waveforms after every run.

The architecture of a scoreboard has three stages:

Stage 1 — Capture DUT Inputs

The scoreboard monitors the DUT's input interface and records each transaction as it is applied. This record is used to compute the expected output. In a UVM environment, a monitor on the input side sends transaction objects to the scoreboard through a TLM port. In a simpler testbench, this can be a task that captures inputs at the relevant clock edge and pushes them into a queue.

Stage 2 — Compute Expected Output

Using the captured input, the scoreboard runs a reference model to compute what the DUT should produce. The reference model can be a SystemVerilog function, a C/C++ model called via DPI, or a pre-computed table of expected values. The result is stored in an expected-output queue, ordered to match the sequence in which the DUT will produce its outputs.

Stage 3 — Compare Against DUT Output

When the DUT drives a result on its output interface, the scoreboard captures it, dequeues the corresponding expected value, and compares them. A mismatch calls $error or $fatal with a message identifying the transaction, the expected value, and the actual value. At the end of simulation, the scoreboard also checks that its expected queue is empty — any remaining entries represent inputs the DUT received but never responded to.

The following example shows a minimal scoreboard for a combinational adder. It is intentionally simple to illustrate the structure clearly:

// Simple scoreboard for an 8-bit adder DUT
// Inputs: operand_a, operand_b, carry_in
// Outputs: sum, carry_out

class adder_scoreboard;

  // Queue to hold expected results until the DUT output arrives
  // Each entry is {expected_sum, expected_carry_out}
  logic [8:0] expected_q[$];  // 9 bits: 8-bit sum + carry_out

  // Called by the input monitor each time a new transaction is applied
  function void predict(
    input logic [7:0] operand_a,
    input logic [7:0] operand_b,
    input logic       carry_in
  );
    logic [8:0] result;
    // Reference model: simple addition
    result = operand_a + operand_b + carry_in;
    // Push expected result into the queue in transaction order
    expected_q.push_back(result);
  endfunction

  // Called by the output monitor each time the DUT drives a result
  function void check(
    input logic [7:0] actual_sum,
    input logic       actual_carry_out
  );
    logic [8:0] expected;
    logic [8:0] actual;

    // Make sure we have something to compare against
    if (expected_q.size() == 0) begin
      $error("SCOREBOARD: DUT produced output with no pending expected result");
      return;
    end

    expected = expected_q.pop_front();
    actual   = {actual_carry_out, actual_sum};

    if (actual !== expected) begin
      $error("SCOREBOARD MISMATCH: expected 0x%0h, got 0x%0h",
             expected, actual);
    end
  endfunction

  // Called at end of test — any remaining entries are missing DUT responses
  function void final_check();
    if (expected_q.size() != 0)
      $error("SCOREBOARD: %0d expected results never received from DUT",
             expected_q.size());
  endfunction

endclass

Use !== (four-state inequality) rather than != when comparing scoreboard results. The !== operator treats X and Z as distinct values — if the DUT drives X on an output that should be a known value, != might not flag it as a mismatch, but !== will.

Reference Models

The reference model is the ground truth that the scoreboard compares against. Choosing the right type of reference model is a design decision with real tradeoffs between accuracy, development effort, and simulation performance.

Behavioral Reference Models

A behavioral model captures the what of the design — the correct input-to-output mapping — without modeling the how of the implementation. For an 8-bit adder, the behavioral model is a single addition operation. For a CRC engine, it is a software CRC computation. For a sorting network, it is a call to a sort function.

Behavioral models are fast to write, fast to simulate, and easy to verify for correctness. They are the right choice when the DUT's output can be described by a well-understood algorithm. Most block-level scoreboards use behavioral models.

Transaction-Level Models

Some designs require the reference model to understand the protocol as well as the algorithm. A memory controller does not just compute correct data values — it must issue read and write commands in a legal sequence, honor timing constraints, and return data in the correct order for outstanding requests. A transaction-level reference model tracks the state of the interface and computes expected outputs based on both the current transaction and the pending transaction history.

Cycle-Accurate Models

A cycle-accurate model replicates the DUT's timing behavior: it produces the correct output on the exact same clock cycle that the DUT should. This is useful when the timing of the output is itself a correctness requirement — for example, a pipeline that must produce a result exactly N cycles after the input, with no early or late delivery permitted.

Cycle-accurate models are more complex to build and maintain, since they must be updated whenever the DUT's pipeline structure changes. They are appropriate when latency and throughput are first-class verification requirements, not just incidental implementation details.

Reference models do not need to handle every edge case the RTL handles — they need to define the correct behavior so the scoreboard can detect deviations. If the reference model is wrong, the scoreboard will miss real bugs or report false failures. A simple, clearly correct reference model is better than a complex one that might itself contain bugs.

Handling False Failures During Bring-Up

During the early stages of a project — when the RTL is first being written and the testbench is being developed in parallel — scoreboard failures are expected. The challenge is distinguishing between a real DUT bug, a testbench bug, and known incorrect behavior that will be fixed in a future RTL revision.

Testbench Bugs

A scoreboard mismatch caused by a bug in the reference model or monitor produces a false failure: the DUT is correct, but the testbench disagrees. False failures are expensive because they waste debug time and erode trust in the test infrastructure. During bring-up, every new failure should be triaged: check the reference model logic before assuming the DUT is wrong. If the scoreboard reference model cannot be easily verified by inspection, it needs its own verification — typically through comparison against a known-good software implementation.

Known Incorrect DUT Behavior

Early RTL is frequently incomplete. A feature may not yet be implemented, a bug may be known and filed but not yet fixed, or the spec may have changed since the RTL was last updated. Leaving the scoreboard active against these known-bad behaviors produces constant noise that makes it harder to spot new failures.

The standard approach is to use a directed disable or expected-failure mechanism: the test explicitly marks a specific check as expected to fail, with a reference to the bug ID. When that check fires, the scoreboard logs an informational message rather than a failure. When the bug is fixed, the expected-failure annotation is removed and the scoreboard resumes normal checking.

class pipeline_scoreboard;

  // Queue for expected outputs (filled by predict, drained by check)
  logic [7:0] expected_q[$];

  // TODO: implement predict()
  // Called when valid_in asserts — compute and store the expected output
  function void predict(input logic [7:0] data_in);
    // your code here
  endfunction

  // TODO: implement check()
  // Called when valid_out asserts — compare DUT output against expected
  function void check(input logic [7:0] data_out);
    // your code here
  endfunction

  function void final_check();
    if (expected_q.size() != 0)
      $error("SCOREBOARD: %0d responses still pending at end of test",
             expected_q.size());
  endfunction

endclass

  function void predict(input logic [7:0] data_in);
    logic [7:0] expected;
    // Reference model: multiply by 3 and mask to 8 bits
    // Truncation to 8 bits matches the DUT's output width
    expected = (data_in * 3) & 8'hFF;
    expected_q.push_back(expected);
  endfunction

  function void check(input logic [7:0] data_out);
    logic [7:0] expected;

    if (expected_q.size() == 0) begin
      $error("SCOREBOARD: unexpected output 0x%0h — no prediction pending",
             data_out);
      return;
    end

    expected = expected_q.pop_front();

    // Use !== to catch X/Z propagation from the DUT
    if (data_out !== expected)
      $error("SCOREBOARD MISMATCH: expected 0x%0h, got 0x%0h",
             expected, data_out);
  endfunction