Memory Model (CMEM & DMEM)

Sparsr Memory Model: CMEM & DMEM

A Sparsr device exposes three separate on-chip memories. They share no address space with each other and none of them is the host's RAM — every byte a kernel touches must be placed there explicitly by the host before execution, and read back explicitly afterwards.

A note on the name. CMEM stands for Co-processor Memory, as declared in the public sparsr.h header. You will occasionally see it expanded as "Compressed Memory" in older material, which describes what it stores rather than what it is — the two names refer to the same memory.

The two memories a kernel author works with directly are DMEM (scalars) and CMEM (wide vectors). They differ in far more than width, and the differences are the source of most first-kernel surprises.

1. CMEM — Co-processor Memory

CMEM is the wide-vector store that feeds Sparsr's 4096-bit registers. It is the reason Sparsr exists: bulk sparse bit-vectors live here, in compressed form, and are decompressed on the fly as they enter a wide register.

Geometry

CMEM is slot-addressed, not byte-addressed. A CMEM address is an index selecting one whole block; there is no such thing as a partial or unaligned CMEM access.

Every slot is always physically 240 bytes wide. Compression here is a fixed-size encoding, not a variable-length one: a block occupies the same 240 bytes whether it holds one non-zero word or forty-eight.

Slot count is backend-dependent; block size is not. The 4096-bit logical block and its 240-byte stored form are fixed everywhere. How many slots exist is not: the software emulator provides 32. Portable kernels should stay within the smallest target's slot count rather than assuming the widest.

The LIL-32b codec

CMEM never stores a raw 4096-bit vector. Data is held in LIL-32b (list-of-lists, 32-bit granularity), and the conversion happens automatically at every boundary — on host writes, on host reads, and on every kernel wide load/store.

The 512-byte logical block is treated as 128 chunks of 32 bits. Each chunk that is not entirely zero is emitted as a 5-byte record:

byte 0     : chunk index + 1   (1-based, so 0 marks "no more records")
bytes 1..4 : the chunk's four data bytes

All-zero chunks are simply omitted — that is the entire compression scheme. Records are packed from the start of the slot and the remainder is zero-filled.

Density limit — the constraint to design around

A slot is 240 bytes and each record costs 5 bytes, so a CMEM block can represent at most 48 non-zero 32-bit chunks out of 128.

A block in which more than 48 of its 128 32-bit chunks are non-zero cannot be stored in CMEM. In practice that caps usable density at 37.5% of chunks. Note this counts chunks, not bits: a chunk with a single set bit costs exactly as much as a chunk with all 32 bits set, so what matters is how the set bits are clustered, not how many there are.

This is a hard property of the storage format, not a tunable. Callers are expected to check density before writing — torchhd-sparsr, for example, validates against this limit (kLilMaxNonzeroChunks = 48) and raises a descriptive error rather than writing a block that cannot round-trip. Code that writes CMEM directly through the C API should do the same; the codec itself does not currently reject over-dense input.

Host access

Both host-side CMEM calls operate on uncompressed 512-byte buffers and handle the codec internally — you never assemble LIL records by hand.

uint8_t block[512];              // exactly 512 bytes, always
// ... fill block ...

sparsr_write_data_cmem(block, 1);       // compress and store into CMEM slot 1

uint8_t *out = sparsr_read_data_cmem(3); // load slot 3, decompress, return 512 bytes

sparsr_read_data_cmem() returns a pointer to an internal static buffer that is overwritten by the next call — copy out anything you need to keep.

Kernel access

Wide loads and stores move a whole slot to or from a wide register, compressing and decompressing transparently.

• WL $wrd, addr — Wide Load. Decompresses CMEM slot addr into wide register $wrd. The address is a fixed instruction immediate.
• WS $wrd, addr — Wide Store. Compresses $wrd into CMEM slot addr. Also a fixed immediate.
• WLR $wrd, offset($rt) — Wide Load, register-indirect. The slot index is computed at run time as $rt + offset.
• WSR $wrd, offset($rt) — Wide Store, register-indirect.

WLR/WSR are a software-emulator-only extension. They exist so a kernel can be assembled and loaded once and then reused against different slots, taking the slot indices as runtime arguments instead of baking them into the instruction stream. They are implemented in the assembler and the softemu backend only — there is no RTL support yet. Kernels that must run on FPGA backends are limited to WL/WS with immediate addresses.

Under softemu, a WLR/WSR whose effective address lands outside the 32-slot range is skipped silently (it is traced, but does not fault) — a bug in address arithmetic shows up as stale data rather than an error.

2. DMEM — Data Memory

DMEM is the ordinary 32-bit MIPS core's data memory: plain scalar words, no compression, no wide access.

Geometry

Word addressing, not byte addressing

This is the single most important thing to know about DMEM, and it departs from standard MIPS:

DMEM addresses are word indices, not byte addresses. LW $t0, 1($zero) reads the second word of DMEM, not the byte at offset 1. Consecutive words live at consecutive addresses 0, 1, 2, …, and the usual MIPS habit of stepping addresses by 4 will skip three words each time.

The same convention applies to the host API: start_address in sparsr_write_data_dmem() / sparsr_read_data_dmem() is a word index, and numb_of_data is a count of 32-bit words.

Host access

uint32_t args[3] = { 1, 2, 3 };
sparsr_write_data_dmem(args, 0, 3);        // write 3 words into DMEM[0], DMEM[1], DMEM[2]

uint32_t *out = sparsr_read_data_dmem(0, 3); // read those 3 words back

As with the CMEM reader, the returned pointer refers to an internal static buffer reused by the next call.

Kernel access

Standard MIPS LW/SW, with the word-addressing caveat above:

LW $t0,0($zero)    # $t0 <- DMEM[0]
SW $t0,5($zero)    # DMEM[5] <- $t0

What DMEM is actually for

Because the wide data path never touches DMEM, its main job in practice is passing runtime arguments into a kernel. A kernel assembled with fixed CMEM addresses can only ever operate on those addresses; a kernel that reads its CMEM slot indices out of DMEM can be loaded once and reused for every call.

3. The two working together

The bind kernel from torchhd-sparsr is the canonical example of the intended division of labour: DMEM carries the addresses, CMEM carries the data.

The host writes three CMEM slot indices — operand A, operand B, destination — into DMEM[0..2], then triggers the same pre-loaded kernel:

uint32_t args[3] = { slot_a, slot_b, slot_out };
sparsr_write_data_dmem(args, 0, 3);
sparsr_execute_batch(kBindKernelAddress);

The kernel loads those indices into scalar registers, then uses them as register-indirect CMEM addresses:

LW  $t0,0($zero)   # $t0 <- CMEM slot of operand A   (DMEM[0])
LW  $t1,1($zero)   # $t1 <- CMEM slot of operand B   (DMEM[1])
LW  $t2,2($zero)   # $t2 <- CMEM slot of result      (DMEM[2])
WLR $w1,0($t0)     # wA <- CMEM[$t0]
WLR $w2,0($t1)     # wB <- CMEM[$t1]
WXOR $w3,$w1,$w2   # wC <- wA XOR wB
WSR $w3,0($t2)     # CMEM[$t2] <- wC

Nothing about the kernel changes between calls — only the three words in DMEM.

4. Summary of constraints

See also

• Instruction Set Architecture — full encoding reference for WL, WS, WLR, WSR, LW, and SW.
• Writing a Kernel in Sparsr Assembly — pipeline hazards and scheduling around wide loads.
• API Reference: C (Native ABI) — full host-side function signatures.