Architecture

Sparsr System Architecture

The Sparsr processing system is a high-performance heterogeneous computing platform designed to accelerate sparse matrix compression, decompression, and wide bitwise logical operations. It combines an x86/x64 host CPU with a custom 4096-bit soft-processor core deployed on an FPGA (e.g., AWS EC2 FPGA instances).

flowchart LR subgraph Application["your_host_app.c"] direction TB A["func_a()"] B["func_b()"] C["func_heavy()"] D["func_c()"] E["func_d()"] end subgraph CPU["CPU"] Application end subgraph MidCol[" "] direction TB RAM["RAM"] PCIe["PCIe Controller"] end subgraph Sparsr["Sparsr"] Backend["your_kernel.spex"] end subgraph FPGA_Memory_Kernel["Sparsr System"] direction TB CMEM["CMEM"] Sparsr end subgraph FPGA["FPGA"] direction TB FPGA_Memory_Kernel end subgraph EC2["AWS EC2 F2"] direction LR CPU MidCol FPGA end CMEM ~~~ Sparsr A ~~~ B B ~~~ C C ~~~ D D ~~~ E RAM ~~~ PCIe CPU ~~~ MidCol MidCol ~~~ FPGA RAM <--> PCIe style C stroke:#cc0000,stroke-width:4px style MidCol fill:none,stroke:none

1. Host-FPGA Physical Split & PCIe Interconnect

The system architecture is divided into two execution domains separated by a PCI Express (PCIe) bus: the Host CPU Domain and the FPGA Hardware Domain.

graph TD subgraph Host_CPU_Domain ["Host CPU Domain (x86 / x64 Host)"] HA[Host Application - C Language] SHL[Sparsr Host Library] RAM[Host System RAM] HA <--> SHL HA <--> RAM end subgraph PCIe_Bus ["PCIe Bus Interconnect"] CMD[256-Word Command Bursts] end subgraph FPGA_Domain ["FPGA Hardware Domain (AWS EC2 Instance)"] IP[Sparsr IP Core / Pipelined CPU] IMEM[IMEM - Instruction Memory] DMEM[DMEM - Data Memory] CMEM[CMEM - High-Bandwidth BRAM] IP <--> IMEM IP <--> DMEM IP <--> CMEM end SHL <== PCIe_Bus ==> IP

Host CPU Domain

  • Host Application: A C application running on standard Intel/AMD x86_64 architecture under Linux or Windows.
  • Sparsr Host Library: Exposes control, compilation, and data transfer API routines (sparsr_kernel_init(), sparsr_load_batch_from_bin(), sparsr_execute_batch(), sparsr_read_status(), sparsr_write_data_cmem(), sparsr_read_data_cmem()).
  • Host System RAM: Main memory used to buffer application input datasets, assembled bitcode, and output sparse matrices.

FPGA Hardware Domain

  • Sparsr Soft-Processor Core: A custom 5-stage pipelined processor featuring standard 32-bit MIPS scalar capabilities augmented with wide 4096-bit vector registers, a Parallel Logic Unit (pLU), and hardware compression/decompression units.
  • On-Chip Storage: Instruction Memory (IMEM), Data Memory (DMEM), and Co-processor Memory (CMEM) implemented via FPGA Block RAM (BRAM).

See Memory Model (CMEM & DMEM) for the geometry, addressing rules, and compression format of DMEM and CMEM.

PCIe Interface Protocol

Communication between the Host Application and the Sparsr IP core occurs in 256-word (32-bit word) command bursts over PCIe. The host CPU acts as the master, and the FPGA acts as the responder. High-level host applications interact with the protocol via the C library header interface (sparsr.h), which dispatches commands to the runtime execution backend (softemu, fpgasim, or fpgaf2 selected via SPARSR_BACKEND).

Command Name CMD Encoding Corresponding sparsr.h API Function Target Destination / Action Description
LoadBatch 1 sparsr_load_batch_from_bin(char[30], uint32_t) IMEM Uploads compiled kernel instruction streams (.spex) to Instruction Memory starting at start_address.
WriteData_CMEM 2 sparsr_write_data_cmem(uint8_t*, uint32_t) CMEM Writes wide sparse/dense matrix data buffer from host RAM into On-FPGA High-Bandwidth CMEM at target address.
WriteData_DMEM 3 sparsr_write_data_dmem(uint32_t*, uint32_t, uint32_t) DMEM Writes 32-bit scalar data into On-FPGA Data Memory.
ReadData_CMEM 4 sparsr_read_data_cmem(uint32_t) CMEM Reads wide vector/matrix outputs from CMEM block at address back to Host System RAM.
ReadData_DMEM 5 sparsr_read_data_dmem(uint32_t, uint32_t) DMEM Reads scalar output values from DMEM back to Host System RAM.
ExecuteBatch 6 sparsr_execute_batch(uint32_t) Processor Pipeline Initiates kernel execution on the active backend from start_address.
ReadStatus 7 sparsr_read_status() Control Unit Polls execution status register (0 = not completed / running, 1 = completed).

Backend Initialization & Selection: Before issuing command bursts, host applications invoke sparsr_kernel_init() to initialize the execution backend. The active backend is configured via the environment variable SPARSR_BACKEND_ENV_VAR ("SPARSR_BACKEND"), supporting "softemu" (default), "fpgasim", and "fpgaf2".

Burst Payload Structure: Every 256-word burst contains 1 command word, payload length, start address, and up to 253 data words. If a transfer exceeds 253 words, it is sent across sequential command bursts.

2. Register File Architecture

The Sparsr processor contains two distinct register files (groups of CPU registers) operating in parallel within Stage 2 (Instruction Decode / Register Fetch) of the pipeline.

flowchart LR subgraph Scalar_RF["32-Bit Scalar Register File"] GPR["32 x 32-bit General Purpose Registers ($0 - $31)"] end subgraph Wide_RF["4096-Bit Vector Register File"] WR["Wide Register Bank ($wrd, $wrs, $wrt)"] wA_Reg["wA Staging Reg (4096-bit)"] wB_Reg["wB Staging Reg (4096-bit)"] end subgraph Stage2["Stage 2: Instruction Decode & Register Fetch"] IR["Instruction Register"] Scalar_RF Wide_RF end IR --> GPR & WR WR --> wA_Reg & wB_Reg

Scalar Register Set (32-Bit)

  • Register Count & Size: 32 general-purpose registers, each 32 bits wide ($0 to $31).
  • Special Registers:
    • $0 ($zero): Hardwired to constant 0.
    • PC (Program Counter): 32-bit instruction pointer.
    • HI / LO: 32-bit multiply/divide target registers.
  • Instruction Set Compatibility: Supports MIPS-I 32-bit scalar instructions (ADD, SUB, AND, OR, XOR, LB, LW, SB, SW, BEQ, BNE, J, JAL).
  • Pipeline Execution: Feeds operand staging registers A and B in Stage 2, which route to the standard 32-bit ALU in Stage 3.

Wide Vector Register Set (4096-Bit)

  • Register Count & Size: Dedicated vector registers operating on 4096-bit wide bitvectors.
  • Internal Staging Registers: wA and wB (4096 bits wide).
  • Wide Instructions (WR & WI Formats):
    • WAND $wrd, $wrs, $wrt — 4096-bit bitwise AND: $wrd = $wrs & $wrt (Encoding 18, Format WR).
    • WOR $wrd, $wrs, $wrt — 4096-bit bitwise OR: $wrd = $wrs | $wrt (Encoding 19, Format WR).
    • WXOR $wrd, $wrs, $wrt — 4096-bit bitwise XOR: $wrd = $wrs ^ $wrt (Encoding 20, Format WR).
    • WL $wrd, CMem[$addr] — Wide Load from CMEM into $wrd (Encoding 21, Format WI).
    • WS CMem[$addr], $wrd — Wide Store from $wrd into CMEM (Encoding 22, Format WI).
    • AIC CMem[$addr], $wrd — Compress $wrd vector and store to CMEM (Encoding 23, Format WI).
    • AID $wrd, CMem[$addr] — Decompress CMEM entry into wide register $wrd (Encoding 28, Format WI).
  • Pipeline Execution: Feeds staging registers wA and wB in Stage 2, which route directly to the Parallel Logic Unit (pLU) in Stage 3.

3. Memory Subsystem Layout

The Sparsr system utilizes a segmented memory architecture separating host RAM from on-chip FPGA memory banks.

graph TD subgraph Host_Memory ["Host Domain"] HRAM["Host System RAM (Gigabytes)"] end subgraph PCIe_Interconnect ["PCIe Bus Interface"] DMA["Host Library PCIe Transfers"] end subgraph FPGA_Memory ["On-FPGA Memory Spaces (BRAM)"] IMEM["IMEM (Instruction Memory)
32-bit Instructions"] DMEM["DMEM (Data Memory)
32-bit Scalar Data"] CMEM["CMEM (Co-processor Memory)
4096-bit High-Bandwidth Storage"] end HRAM <== DMA ==> IMEM HRAM <== DMA ==> DMEM HRAM <== DMA ==> CMEM IMEM -->|Stage 1: IF| Core[Sparsr Core Pipeline] DMEM <-->|Stage 4: MEM| Core CMEM <-->|Stage 4: MEM / pLU| Core

Memory Space Specifications

Memory Space Location Access Width Primary Purpose Pipeline Stage
Host System RAM Off-Chip (Host PC) 64-bit / System bus Storing host application data, raw matrices, and driver buffers. N/A
IMEM On-FPGA (BRAM) 32-bit Stores assembled Sparsr/MIPS kernel machine code. Stage 1 (Fetch)
DMEM On-FPGA (BRAM) 32-bit Stores scalar data, indices, pointers, and loop counters. Stage 4 (Memory)
CMEM On-FPGA (BRAM) 4096-bit / Variable High-bandwidth storage for wide vectors, binary sparse matrices, and compressed streams. Stage 4 (Memory)

On-FPGA CMEM & Sparse Compression Pipeline

CMEM serves as the high-bandwidth memory block for vector and matrix operations:

  1. Direct pLU Interconnect: Connected directly to Stage 4 (Memory Access) of the 5-stage pipeline, allowing 4096-bit wide registers (wA, wB) to read or write full bitvectors without bottlenecks.
  2. Hardware Compression/Decompression Hardware (AIC / AID):
    • Compress (AIC): Takes a full 4096-bit vector from wide register $wrd, extracts non-zero bit indices/runs, and writes the compressed representation directly into CMEM.
    • Decompress (AID): Reads a compressed entry from CMEM, expands it in hardware into a full 4096-bit sparse representation, and loads it into wide register $wrd.