Sparsr Architecture

Sparsr is a processor on a PCIe card that does bitwise work on very wide vectors. It has an ordinary 32-bit RISC-V core for control flow and, beside it, a wide datapath of 32 registers that are each 8,192 bits wide. One wide instruction combines two whole registers at once.

This chapter describes the architecture Sparsr is being built to. It is the design a kernel should be written for.

What runs today. The Sparsr VM in the SDK runs this architecture: the RV32I core, the wide instructions, the memory map, XMEM, the DMA and the host wire format. The FPGA hardware is still being built up to it. Today's card has 8,192-bit wide registers, 4 KiB of instruction and data memory, and a small row memory. XMEM, the DMA and the WMEM byte window are still to be built in the hardware. Each page in this chapter says what it covers.

The pages in this section

Memory Map - every memory a kernel can reach, its address and its size.
WMEM, XMEM and the DMA - the two memories that hold rows, and the engine that copies rows between them.
Host Wire Format - how the host sends rows to the card in frames and slots.
Glossary - the words these pages use.

1. The host and the card

The host runs your application. The card runs your kernel. The host and the card are connected by PCIe, and everything that crosses between them goes in fixed-size frames.

A program uses the card in the same order every time:

Load a kernel. The host library writes the kernel image into IMEM.
Place the data. The host writes scalar arguments into DMEM and rows into WMEM or XMEM.
Run a batch. The core starts at the first word of the kernel and runs until the kernel returns, exits or faults.
Read the results. The host reads the status word, scalar results from DMEM, and result rows.

The host library hides the frames. Your application calls functions declared in sparsr.h, and SPARSR_BACKEND chooses the device at run time, so the same program runs on the Sparsr VM or on a card.

2. The scalar core

The scalar core is a 32-bit RISC-V processor. It targets the ratified RVI20U32 profile: RV32I with no extensions. It has integer add, subtract, logic, shifts, compares, branches, jumps, loads and stores.

32 registers, x0 to x31, 32 bits each. x0 always reads as zero.
One flat, byte-addressed, little-endian address space. See the memory map.
A kernel is compiled with a stock RISC-V GCC. The SDK's link script places the code and data, and its startup file puts kernel_main first.
The core runs a kernel's loops, branches, calls and bookkeeping. It never does the wide work itself.

The core is a five-stage pipeline with no cache. A load from on-chip memory waits a fixed, small number of clock cycles, so every memory a kernel addresses directly is on the chip.

3. The wide datapath

The wide datapath holds 32 wide registers, w0 to w31, of 8,192 bits each. It works on whole registers at once.

Item	Value
Wide registers	32
Bits in one register	8,192
Bytes in one register	1,024
Lanes in one register	256, each 32 bits

A lane is a 32-bit piece of a register. The wide ALU treats each lane on its own: no carry passes from one lane to the next. So a lane-wise operation costs the same whether a register is sparse or dense.

A wide instruction does one of these things:

Combines two registers bit by bit: AND, OR, XOR, NOR, XNOR, NOT, AND-NOT.
Reduces a result to a number in a scalar register, such as a population count or a Hamming distance.
Loads or stores a whole row between WMEM and a wide register.

The `custom-0` opcode

The wide instructions use the RISC-V custom-0 major opcode (0x0B). That opcode is set aside by the RISC-V specification for exactly this, so a standard toolchain and a standard disassembler never mistake a wide instruction for something else. The other three custom opcodes are unused.

A C kernel emits them through sparsr_intrinsics.h, and an assembly kernel through sparsr.S. Both expand to one .insn per instruction, so no special assembler is needed. See Instruction Set Architecture for every encoding.

Wide load and store

WL copies one row from WMEM into a wide register, and WS copies a wide register into a row. The row number is a base register plus a signed 12-bit displacement, exactly as lw and sw address a byte. With a base of x0, the displacement is an absolute row number.

A wide load and store reach WMEM and nothing else. They move raw rows, with no compression and no density limit.

4. Where data lives

Memory	Holds	Size	Reached by
IMEM	the kernel's code	64 KiB	the core's fetch
DMEM	the kernel's stack, globals and scalar data	64 KiB	RV32I loads and stores
WMEM	rows the kernel works on now	4,096 rows	wide load and store by row; RV32I loads and stores by byte; the DMA
XMEM	a large data set, row by row	16 GiB	the DMA and the host only

The design keeps one rule for rows: a kernel computes on WMEM, and XMEM is storage behind it. When a data set is larger than WMEM, the kernel or the host asks the DMA to copy the next block of rows in, and the kernel keeps running while the copy is made. See WMEM, XMEM and the DMA.

5. Compression at the edge

Rows on the card are raw, and the card never compresses a row.

The host may send a sparse row compressed, in a format called LIL that lists only the lanes holding a 1-bit. A dense row is always sent raw. The card has one decoder, on the path into WMEM, which expands a LIL row to raw. Results leave the card raw.

This keeps compression out of the core's load path, so a wide load always takes the same time, and no row is ever refused because it is too dense. See Host Wire Format.