Why a load needs one stall cycle, and why nothing else does

The instruction that reads x5 comes…	next (1 apart)	2 apart	3 apart
ALU op writes x5	No stall, from MEM	No stall, from WB	No stall, bypass in ID
Load writes x5	1 stall, then from WB	No stall, from WB	No stall, bypass in ID

1Five stages, one instruction in each

A pipeline works like an assembly line. Every clock cycle, each instruction moves one stage to the right, so up to five instructions are in flight at once.

A register is one of 32 storage slots inside the CPU, named x0 to x31. x0 always reads zero. The register file holds them. An instruction reads its source registers (rs1, rs2) in ID and writes its destination register (rd) in WB.

The five stages. The code on the right of each box is the name prefix of that stage's pipeline registers in the RTL. The yellow lines are forwarding paths. Each one carries a result back to an instruction that needs it before WB has written it.

A data hazard happens when an instruction needs a register that an older instruction has not written yet. The older instruction is still in the pipeline. There are two fixes. Forwarding takes the value from the stage where it already exists. A stall makes the younger instruction wait a cycle. Forwarding is free, and a stall costs a cycle.

The key fact is in the figure. An ALU result exists at the end of EX. A load's data exists only at the end of MEM, one stage later.

2An ALU result forwards from MEM with no stall

An ALU instruction computes its result in EX. One cycle later that result sits in the MEM stage register m_result. The next instruction is now in EX and needs the value at the start of this cycle. The forwarding logic copies m_result straight into EX, so nobody waits.

RTL trace forward_priority4 instructions · 9 cycles

Cycle

Real Icarus simulation of rtl/rv32_core.sv from reports/visual-traces.json. Rows are instructions and columns are clock cycles. A yellow cell sends a value, and an outlined cell receives it. In cycle 5, two older instructions both hold a new x1. The add takes 8 from MEM, not 5 from WB.

When two older instructions both write the same register, the newer one must win. The RTL checks WB first and then MEM, so a MEM match overwrites a WB match:

101// Newest producer wins; a load becomes available only in WB.102always @* begin103    a = x_a; b = x_b;104    if (w_valid && w_write && w_rd != 0) begin105        if (rs1 == w_rd) a = w_result;106        if (rs2 == w_rd) b = w_result;107    end108    if (m_valid && m_write && !m_load && !m_trap && m_rd != 0) begin109        if (rs1 == m_rd) a = m_result;110        if (rs2 == m_rd) b = m_result;111    end112end

3A load's data arrives one stage too late

A load uses EX only to add up its address. The data memory returns the value during MEM, so the value exists only at the end of MEM. The next instruction needs it at the start of its own EX, which is that same cycle. The value is not there yet.

So the core holds the next instruction in ID for one cycle and sends an empty slot, a bubble, into EX. One cycle later the load is in WB, its data sits in w_result, and the waiting instruction takes it from there.

RTL trace load_store5 instructions · 11 cycles

Cycle

Real RTL trace. lw x2 is followed right away by sw x2, which needs x2 as the data to store. Cycle 5 is the stall. IF and ID repeat (outlined), and the bubble row shows the empty slot moving through EX, MEM and WB.

Why not forward from MEM, as with an ALU result? In MEM, m_result holds the load's address, not its data. Line 45 skips loads (!m_load), so an address can never pass for data. The stall already keeps a consumer out of EX while its load is in MEM, so this term is a second guard.

4The stall rule, term by term

The whole rule is five lines. The hazard check looks at two stages: the instruction in EX (fields of x_insn) and the instruction behind it in ID (fields of f_insn).

213wire [6:0] f_op = f_insn[6:0];214wire f_csr = ZICSR && f_op==7'h73 && f_insn[13:12]!=0 && !f_insn[14];215wire uses_rs1 = f_op==7'h67 || f_op==7'h63 || f_op==7'h03 || f_op==7'h23 || f_op==7'h13 || f_op==7'h33 || (DOT4 && f_op==7'h0b) || f_csr;216wire uses_rs2 = f_op==7'h63 || f_op==7'h23 || f_op==7'h33 || (DOT4 && f_op==7'h0b);217wire hazard = f_valid && x_valid && ld && !trap && x_insn[11:7]!=0 &&218    ((uses_rs1 && f_insn[19:15]==x_insn[11:7]) || (uses_rs2 && f_insn[24:20]==x_insn[11:7]));

f_valid && x_valid: Both slots hold real instructions. A bubble or a flushed slot never causes a stall.
ld: The instruction in EX is a load. Only loads can stall, because every other result is ready by the end of EX.
!trap: A load that traps writes nothing, so there is nothing to wait for. Line 112 also clears ld on a trap, so this is a second guard.
x_insn[11:7] != 0: The load's rd is not x0. Writes to x0 are thrown away, so nothing can depend on them.
f_insn[19:15] == x_insn[11:7]: The rs1 field of the instruction in ID names the load's rd. The same check on bits 24:20 covers rs2.
uses_rs1, uses_rs2: The instruction in ID really reads that register. See below.

Why uses_rs1 and uses_rs2 exist. Bits 19:15 name a register only in some instruction formats. In lui and jal those bits are part of an immediate. For example, lui x7, 0x30 has the bits of x6 in its rs1 field. Without uses_rs1, it would stall after lw x6 for no reason. With it, the model runs that pair with 0 stalls. Lines 115-116 list the opcodes that read each register. Stores and branches read rs2. So does DOT4, the custom instruction.

The list must also be complete. A missing entry would let a real dependency through without a stall. The mutation test in reports/mutation.json deletes entries one at a time, and the tests caught each one: no_load_use_stall_for_store_data (caught by hazard_load_store_data), no_load_use_stall_for_jalr (caught by hazard_load_jalr_rs1), no_load_use_stall_for_dot4_rs2 (caught by hazard_load_dot4_rs2). The mutant stall_without_bubble, a stall without a bubble, was caught too: "stall did not hold IF/ID and insert a bubble".

5Try it

Pick what writes x5 and how far behind it the reader addi x6, x5, 1 sits. Empty slots are filled with nop. The answer and the chart come from the model of the RTL rules, not a recorded trace.

Producer

ALU op: addi x5, x0, 42 Load: lw x5, 128(x0)

Distance (instructions apart)

123

Model run

Cycle

At distance 3 the reader is in ID while the producer is in WB. The register file is written at the same clock edge as it is read, so a separate WB-to-ID bypass (lines 337-341) hands the value over. Past distance 3, the register file already holds the value.

6When a flush and a stall meet, the flush wins

A flush throws away the two instructions behind the one in EX: the one in ID, and the one being fetched in IF. In the RTL, clearing f_valid and x_valid means neither one ever enters EX. A flush happens when a branch or jump is taken in EX, because those two were fetched from the wrong place. It also happens when an instruction traps. The clock-edge logic checks the flush first:

328                if(redirect || fault) begin329                    f_valid<=0; x_valid<=0;330                    if(redirect) pc<=target;331                    if(fault && ZICSR) pc<=trap_target;332                    if(fault && !ZICSR) halted<=1;333                end else if(hazard) begin334                    x_valid<=0;

Flush (redirect or fault)	Hazard	What happens at the clock edge
no	no	Everything moves one stage forward.
no	yes	Stall: IF and ID hold, and a bubble goes into EX.
yes	no	Flush: the instructions in IF and ID are thrown away. On a branch, fetch restarts at the target.
yes	yes	Same as the row above. The stall is ignored.

The order is right because the instruction waiting in ID is on the wrong path. Holding it for one cycle would waste that cycle, since the flush removes it anyway. The stall counter follows the same rule, so a lost cycle is never counted twice:

228assign stall_event = advance && hazard && !redirect && !fault;229assign flush_event = advance && (redirect || fault);

In this core the last row cannot happen. Both signals look at the one instruction in EX. A hazard needs a load that does not trap, and a load is never a taken branch (lines 153-159 and 211). A model test checks this against random instruction words. The order still matters. If a later change let the two overlap, the flush would still win. You can see a flush in the trace in section 3: EBREAK traps in EX in cycle 8.

7The cost, one cycle at a time

reports/benchmark.json states the cycle count of every benchmark run as one sum:

cycles = retired + stalls + 2 × flushes + 3

Each retired instruction costs one cycle. Each load-use stall adds one. Each flush adds two, for the two thrown-away instructions. The 3 is fixed: the pipeline fills for five cycles before the first instruction retires, and the flush of the final EBREAK already accounts for two of them.

Worked example. The DOT4 kernel on 256 elements, read from the report:

454 + 0 + 2 × 64 + 3 = 585 cycles

The flushes cost 128 cycles, 22% of the run. They come from the taken loop branch, plus the final EBREAK. Stalls cost nothing: every one of the 16 benchmark runs reports 0 stalls, because no load in those kernels is read by the very next instruction.

The same sum holds for the trace in section 3: 5 + 1 + 2 × 1 + 3 = 11 cycles, and there you can count each cycle by hand. The 253 regression programs trigger the stall on purpose, 118 times in total (reports/verification.json).