Safety and Reliability

Safety and reliability are about what a chip does when something goes wrong inside it. A transistor can wear out, a particle from space can flip a memory bit, and a supply can droop for a moment. In a phone, a rare wrong result is an annoyance; in a car's brake controller or a medical pump, it can hurt someone. This article explains the kinds of faults a chip suffers, how designs detect and correct them with error-correcting codes, lockstep cores and monitors, how much of the chip such mechanisms cover, and how safety standards turn these ideas into numbers a team must meet.

12 min read | Intermediate

What You'll Learn

• Distinguish permanent, transient and wear-out faults
• Explain how ECC, parity and lockstep detect and correct faults
• Describe safety mechanisms, diagnostic coverage and FMEDA
• Plan the reaction when a fault is detected

Faults, Errors and Failures

Three words describe the chain of events. A fault is a physical defect or disturbance, such as a broken wire or a flipped bit. An error is a wrong value inside the system caused by the fault. A failure is when the system stops doing what it should, visible from outside. Not every fault becomes an error, since the flipped bit may be in a register that is overwritten before it is read, and not every error becomes a failure. Safety design tries to stop the chain before it reaches the end.

Faults come in three kinds:

• Permanent faults: a wire or transistor is broken for good, from manufacture or later damage. The same input gives the same wrong result every time.
• Transient faults, also called soft errors: a disturbance flips a value once, and the hardware is fine afterwards. The main cause is particles, neutrons from cosmic rays and alpha particles from traces of radioactive material in the package, which leave a trail of charge in the silicon. Memory cells, which store small charges, are most sensitive.
• Wear-out: transistors and wires age with heat, voltage and current, and slow down or break after years of use. Designers model it and add margins so that the chip lasts its intended life.

Soft errors are rare per bit but chips have many bits. A large memory can see an upset every few weeks or months in normal conditions, more at high altitude. Rates are often quoted in FIT, failures in time: the number of failures expected in one billion device-hours. A FIT rate of 1,000 means one failure per million hours, about once in 114 years for one chip, but about once every six weeks across a thousand cars that run all the time.

Detecting and Correcting Errors in Memory

Memories are protected by adding check bits. Memories in a System introduced the two common schemes. Parity adds one bit per word, which detects any single flipped bit but can't say which one, so it can't correct it. Error-correcting codes (ECC) add more bits: the common SECDED code, single error correction and double error detection, adds 8 check bits to 64 data bits. It corrects any one flipped bit and detects any two.

ECC costs about 12.5 percent more memory cells for 64-bit words, plus an encoder and decoder that add some delay to every access. Caches often use parity on lines that also exist in main memory, since a bad line can just be thrown away and fetched again, and ECC on lines that hold the only copy of modified data. Errors that ECC corrects are counted and reported, because a rising count can show a cell that is failing for good.

Correcting an error on read fixes the value returned, but the wrong bit is still in the memory. If a second bit in the same word flips later, the word can no longer be corrected. Scrubbing solves this: a background engine reads every word in turn, corrects it and writes it back, so that single errors don't sit long enough to become double ones.

Protecting Logic: Lockstep

Codes work well for data that is stored and moved, but logic that computes, such as a processor's pipeline, changes its values every cycle. The common protection for a processor is lockstep: two identical cores run the same program on the same inputs, and a comparator checks their outputs every cycle.

If one core suffers a fault, their outputs differ and the comparator raises an alarm within a cycle or so. The second core usually runs a couple of cycles behind the first, so that a single disturbance, such as a glitch on the shared supply, hits the two at different points of the calculation and is unlikely to change both in the same way. The cost is high: about twice the area and power for the cores. Lockstep detects faults but doesn't correct them; three cores with a majority vote, triple modular redundancy, can also mask one faulty core, at three times the cost, and is used mostly in space and aviation.

Cheaper techniques exist for less critical logic: parity on buses and registers, duplicated state machines, and software tests that run regularly and check that the hardware still computes known answers. A whole SoC uses a mix, chosen block by block.

Safety Mechanisms and Coverage

Each of these techniques is a safety mechanism: a piece of hardware or software that detects a fault, or corrects it, before it causes a dangerous failure. A watchdog, described in Timers and Watchdogs, is one; so are voltage and clock monitors, ECC, lockstep and memory built-in self-tests run at power-up.

No mechanism catches everything. The diagnostic coverage of a mechanism is the fraction of faults in the protected logic that it detects. SECDED on a memory covers essentially all single-bit faults in the cells, but not a fault in the address decoder that reads the wrong word with correct check bits. That is why some designs include the address in the check bits. Lockstep covers almost all faults in the cores, but not the comparator itself, so the comparator is designed to fail safely and is tested regularly.

An FMEDA, failure modes, effects and diagnostic analysis, adds this up for a whole chip. It lists each block, its estimated raw failure rate in FIT from its size and type, how it can fail, the effect of each failure, which safety mechanism covers it and with what coverage. The result is the rate of failures that are both dangerous and undetected, which the standard for the product sets a limit on.

Safety Standards

Cars follow a functional safety standard, ISO 26262, that defines four levels, called ASILs, from A, the least strict, to D, the most strict. A brake or steering controller is often ASIL D; a dashboard display may be ASIL B. The level sets targets for the chip's metrics: the share of single-point faults that are covered, the share of latent faults, which are faults in the safety mechanisms themselves, and the rate of undetected dangerous failures. Industrial machinery, railways and medical devices have similar standards.

The standard also governs the process: requirements traced to design and tests, reviews, controlled tools, and fault-injection simulations that flip bits in the design deliberately and check that the mechanisms catch them. A chip sold for safety use ships with a safety manual that tells system designers which mechanisms exist, what software must do, such as running periodic self-tests, and what assumptions the analysis made.

Reacting to a Fault

Detecting a fault is half the job; the system must then reach a safe state within a set time, the fault tolerance time interval. For a motor controller, the safe state might be to stop driving the motor; for a brake system, to hand control to a backup. A typical chain is: a mechanism detects the fault, a fault collection unit gathers alarms from all mechanisms, and it either raises an interrupt for software to handle, forces outputs to a safe value, or resets the chip, depending on how severe the fault is and how quickly the reaction must happen.

Corrected errors are reported too, but they don't need a reaction other than logging. Uncorrectable ones must not be silently ignored. Designs also consider what a reset does: a reset that clears the log of what went wrong makes the fault hard to diagnose, so fault records are often kept in a domain that the reset does not touch, as Reset Architecture and Reset Domain Crossings describes.

Reliability Over the Lifetime

Reliability also covers the chip's whole life. Wear-out mechanisms, such as electromigration, where current slowly moves metal atoms until a wire thins and breaks, and transistor aging, which slows gates over years, are checked during design against the expected temperature, voltage and years of use. A car chip may need to work for fifteen years at high temperature, far longer than a phone chip. The design adds margins, wider wires and sometimes on-chip monitors that measure aging, so that software can lower the speed before timing fails.

Testing at manufacture is part of reliability as well. A chip that leaves the factory with a weak defect is likely to fail early, so chips for demanding markets get extra tests and sometimes a burn-in, a period of operation at high temperature and voltage that makes weak parts fail before they ship. Design for Test at SoC Level covers how the chip is made testable.

Common Mistakes