Spectra

A command-line tool for analyzing, transforming, and compressing tensor artifacts (.npy, .npz). Spectra profiles every tensor individually — measuring sparsity, entropy, and spectral structure — then routes each one to the optimal compression strategy (quantization, sparse COO storage, or truncated SVD decomposition). Every transform is audited in a per-tensor report and a machine-readable manifest stored alongside the data.

Designed for data scientists who need to shrink model weights, activation checkpoints, or any numerical array collection without black-box compression that hides what was done or how much was lost.

Installation

pip install stz

Or with uv:

uv add stz

Or directly from GitHub (latest unreleased):

uv add "git+https://github.com/ShivUCSD1104/spectra.git"

Optional extras:

pip install "stz[torch]"    # enable .pt / .pth loading (requires PyTorch)
pip install "stz[wavelet]"  # enable wavelet preconditioning (requires PyWavelets)

Commands

spectra inspect   <file>          Profile every tensor. Read-only.
spectra compress  <file>          Auto-route each tensor to optimal compression.
spectra transform <file>          Apply an explicit transform strategy.
spectra extract   <file.stz>      Reconstruct tensors from a .stz archive.
spectra info      <file.stz>      Show archive manifest without decompressing.

Usage

A complete inspect → transform → extract round-trip on a four-tensor synthetic model.

Build the model:

import numpy as np
 
rng = np.random.RandomState(42)
model = {
    "attn.weight":  (rng.randn(128, 10) @ rng.randn(10, 128)).astype(np.float32),  # low-rank
    "attn.bias":    np.zeros(128, dtype=np.float32),                                 # all zeros
    "embed.weight": rng.randn(256, 64).astype(np.float32),                           # dense
    "conv.kernel":  rng.randn(8, 8, 16).astype(np.float32),                          # 3D
}
np.savez("demo_model.npz", **model)

Step 1 — Inspect (read-only profile):

$ spectra inspect demo_model.npz

File: demo_model.npz  |  4 tensors  |  33.9K parameters  |  132.5 KB

 Tensor        Shape    Dtype     Size   Sparsity   Entropy  Recommendation
 attn.weight   128x128  float32  64.0 KB     0.0%   6.6 bit  fp16 safe
 embed.weight  256x64   float32  64.0 KB     0.0%   7.0 bit  fp16 safe
 conv.kernel   8x8x16   float32   4.0 KB     0.0%   7.1 bit  fp16 safe
 attn.bias     128      float32  512.0 B   100.0%  -0.0 bit  fp16 safe, int8 safe, sparse (100%)

Compression Potential Summary
  SVD/Tucker candidates:   1 tensors  (4.0 KB)
  Quantization only:       1 tensors  (512.0 B)
  Leave alone:             2 tensors  (128.0 KB)

Step 2 — Transform: fp16 quantize everything:

$ spectra transform demo_model.npz --quantize fp16 --out demo_fp16.stz

Transform Report
─────────────────────────────────────────────────────────────────
attn.weight   float32 [128x128]
  → quantize fp16
  → 64.0 KB → 32.0 KB  (2.0x)
  → MSE: 0.0000  |  Relative error: 0.02%

attn.bias     float32 [128]
  → quantize fp16
  → 512.0 B → 256.0 B  (2.0x)
  → MSE: 0.0000  |  Relative error: 0.00%

embed.weight  float32 [256x64]
  → quantize fp16
  → 64.0 KB → 32.0 KB  (2.0x)
  → MSE: 0.0000  |  Relative error: 0.02%

conv.kernel   float32 [8x8x16]
  → quantize fp16
  → 4.0 KB → 2.0 KB  (2.0x)
  → MSE: 0.0000  |  Relative error: 0.02%

Summary
  Tensors transformed:  4 / 4
  Original size:        132.5 KB
  Stored size:          66.2 KB
  Overall ratio:        2.0x
  Max relative error:   0.02%
  Written to: demo_fp16.stz    (65.1 KB)

Step 3 — Transform: int8 on weight matrices only:

$ spectra transform demo_model.npz --quantize int8 --select "*.weight" --out demo_int8.stz

attn.weight   float32 [128x128]  → quantize int8  →  64.0 KB → 16.0 KB  (4.0x)  MSE: 0.0014
attn.bias     float32 [128]      → Skipped (not selected)
embed.weight  float32 [256x64]   → quantize int8  →  64.0 KB → 16.0 KB  (4.0x)  MSE: 0.0001
conv.kernel   float32 [8x8x16]   → Skipped (not selected)

Summary
  Tensors transformed:  2 / 4
  Overall ratio:        3.6x    ← unselected tensors stored as-is
  Written to: demo_int8.stz    (35.5 KB)

Step 4 — Inspect the archive manifest (no decompression):

$ spectra info demo_fp16.stz

Spectra Archive: demo_fp16.stz
Storage Summary
  4 tensors
  Original:                132.5 KB
  After tensor transforms:  66.2 KB  (2.0x)
  After binary (zstd):      65.1 KB  (1.0x)
  Total ratio:              2.0x

Strategy Breakdown
  quantized_fp16     4 tensors  (132.5 KB original)

Lossy tensors: 0  |  Lossless tensors: 4

Step 5 — Extract and verify round-trip:

$ spectra extract demo_fp16.stz --out demo_extracted.npz --report

Extraction Report
─────────────────────────────────────────────────────
attn.weight   float32 [128x128]  lossless  storage: quantized_fp16
attn.bias     float32 [128]      lossless  storage: quantized_fp16
embed.weight  float32 [256x64]   lossless  storage: quantized_fp16
conv.kernel   float32 [8x8x16]   lossless  storage: quantized_fp16

Extracted 4 tensor(s) → demo_extracted.npz

File sizes after all steps:

spectra inspect

Profile every tensor in an artifact without writing anything. Reports shape, dtype, size, sparsity, Shannon entropy, and — for 2D matrices — a full spectral analysis including singular value decay rate, effective rank, intrinsic dimension, and condition number.

spectra inspect <file> [OPTIONS]

Inputs: .npy, .npz, .stz

Outputs: Terminal table, CSV, or JSON to stdout. Nothing written to disk.

Flags

Metrics computed per tensor

SVD analysis detail

For 2D tensors, Spectra computes the top-64 singular triplets via scipy.sparse.linalg.svds (a partial SVD — much faster than full SVD for large matrices). The spectral decay rate is derived by fitting a line to log(S/S[0]) as a function of index, then normalizing: rate = 1 - exp(-slope). A rate near 1.0 means singular values drop sharply (strong low-rank structure). A rate near 0.0 means the spectrum is flat (dense, information-rich).

Examples

# Basic table
spectra inspect model.npz
 
# Sort by entropy, show only top 5 tensors
spectra inspect model.npz --sort entropy --top 5
 
# Inspect one tensor with full detail block
spectra inspect model.npz --tensor attention.weight --depth full
 
# Export as JSON for scripting
spectra inspect model.npz --format json > analysis.json
 
# Export as CSV
spectra inspect model.npz --format csv > analysis.csv
 
# Inspect tensors previously compressed into a .stz
spectra inspect model.stz

spectra compress

The intelligent command. Automatically analyzes every tensor and routes it to the best compression strategy using the built-in routing engine. Produces a .stz archive with a full audit manifest.

spectra compress <file> [OPTIONS]

Inputs: .npy, .npz, .stz

Output: .stz archive (default: <input>.stz)

Flags

Routing engine

The router analyzes each tensor and selects a strategy based on its structure. The decision tree is:

if tensor.nbytes < min_size:
    → dense passthrough (no change)

if tensor.ndim == 1:
    → quantize_fp16
      Rationale: 1D tensors are bias vectors, position encodings, etc.
      fp16 is always lossless for values within ±65504 and halves the size.

if tensor.ndim == 2:
    Compute top-64 singular values (cached from inspect if available).
    Compute spectral decay rate and find the smallest SVD rank k where
    relative error < tolerance (using Eckart-Young theorem).

    SVD candidate if:
        decay_rate > 0.85        (fast exponential decay of singular values)
        OR rank_k / min(shape) < 0.3 AND compression_ratio(k) > 2.0
                                 (sharp rank cliff: few singular values explain
                                  most energy, even if they are similar in magnitude)
    if SVD candidate AND compression_ratio > 2.0:
        → SVD rank k
          Stored as float32 U, S, Vt factors.
          Reconstruction: U @ diag(S) @ Vt

    elif entropy < 5.0 AND int8_safe:
        → quantize_int8
          int8_safe: entropy < 5.0 bits AND dynamic_range < 20.0
          Rationale: low-entropy tensors have concentrated value distributions
          that map well onto 256 discrete levels.

    elif near_zero_fraction > 0.50:
        → sparse_coo
          Rationale: more than half of values are near-zero (|x| < 1e-6);
          COO format stores only non-zero indices + values.

    else:
        → quantize_fp16
          Conservative fallback for dense, high-entropy 2D tensors.

if tensor.ndim >= 3:
    → quantize_fp16
      Rationale: Tucker decomposition (Phase 14) not yet implemented.
      fp16 is always a safe 2x reduction.

Why these thresholds?

• decay_rate > 0.85 — An exponential fit slope of 0.85 on the normalized singular value curve corresponds to roughly 85% energy loss per step. At this rate the matrix is compressible with small rank. Values below 0.85 indicate that many singular values are significant and truncation would be lossy.
• rank_fraction < 0.3 — If the tolerance-satisfying rank is less than 30% of the smaller dimension, SVD will almost always exceed 2x compression. This catches matrices with a sharp spectral cliff (e.g. a true rank-10 matrix in a 256×256 space) that the exponential decay metric misses because the retained singular values are not themselves fast-decaying.
• compression_ratio > 2.0 — The break-even for SVD storage (m×k + k + k×n float32) vs. the original (m×n float32). Below 2× it is not worth the reconstruction overhead.
• entropy < 5.0 bits — Uniform float32 noise has entropy ≈ 8 bits. Values below 5 bits indicate a distribution that is clustered enough to be accurately represented with only 256 levels.
• dynamic_range < 20.0 — Defined as max(|x|) / mean(|x|). A ratio above 20 means outliers would dominate the int8 quantization scale, causing large errors on the common values.
• near_zero_fraction > 0.50 — COO storage costs nnz × (ndim × 8 + 4) bytes. Break-even vs. dense is at ~50% sparsity for a typical 2D float32 tensor, assuming float32 values and int64 indices.

Examples

# Auto-compress at 1% tolerance (default)
spectra compress model.npz
 
# 5% tolerance — more aggressive, smaller files
spectra compress model.npz --tolerance 0.05 --out model_compressed.stz
 
# Only quantize, no SVD
spectra compress model.npz --no-factorize
 
# Preview routing decisions without writing
spectra compress model.npz --dry-run
 
# Ignore tensors smaller than 1 MB
spectra compress model.npz --min-size 1MB
 
# Save report as JSON
spectra compress model.npz --report-file report.json
 
# Use xz binary compression for maximum space savings (slow)
spectra compress model.npz --binary-compress xz --binary-level 9
 
# Lossless only (fp16 — safe for values within ±65504)
spectra compress model.npz --lossless-only

spectra transform

Apply an explicit, user-specified transform to all (or selected) tensors. Unlike compress, you choose the strategy; Spectra applies it uniformly.

spectra transform <file> [OPTIONS]

Inputs: .npy, .npz, .stz

Output: .stz archive (default: <input>.stz)

Flags

Transform modes

--quantize fp16 Casts every value to float16. Float32 → float16 halves the byte count. Lossless for values within ±65504; values outside this range are clipped to ±inf. The manifest records fp16_overflow_detected: true if any value exceeds the representable range.

--quantize int8 Per-tensor affine quantization. Computes scale = (max - min) / 255 and zero_point such that the minimum value maps to -128 and the maximum to +127. Stores as int8 (4× compression from float32). Reconstruction formula stored in manifest: x ≈ q * scale + zero_point. Always lossy; error depends on value distribution.

--factorize svd --rank N Truncated SVD at a fixed rank N, applied to all 2D tensors. Stores three float32 arrays per tensor: U (m×k), S (k,), Vt (k×n). Reconstruction: U @ diag(S) @ Vt. Non-2D tensors are passed through unmodified. Compression ratio: (m×n) / (m×k + k + k×n).

--sparsify THRESHOLD Zeros out all values with |x| < threshold, then encodes as COO (coordinate list): int64 indices of shape (nnz, ndim) and float32 values of shape (nnz,). Lossless when threshold = 0. Can be combined with --quantize to first sparsify, then quantize the remaining values.

Selection

--select and --exclude use Python's fnmatch shell-style glob patterns:

# Only transform attention weight matrices
spectra transform model.npz --quantize fp16 --select "attention*"
 
# Transform everything except embedding layers
spectra transform model.npz --quantize fp16 --exclude "embed*"
 
# Skip tensors smaller than 100 KB
spectra transform model.npz --quantize int8 --min-size 100KB

Examples

# fp16 quantize everything
spectra transform model.npz --quantize fp16
 
# int8 quantize all weight matrices (not biases)
spectra transform model.npz --quantize int8 --exclude "*.bias"
 
# SVD at rank 32 for all 2D tensors
spectra transform model.npz --factorize svd --rank 32
 
# Sparsify: zero out values smaller than 1e-4
spectra transform model.npz --sparsify 1e-4
 
# Sparsify + quantize the non-zero values
spectra transform model.npz --sparsify 1e-4 --quantize fp16
 
# Dry run to preview what would happen
spectra transform model.npz --quantize int8 --dry-run
 
# Chain on an existing .stz file
spectra transform previous.stz --quantize fp16

spectra extract

Reconstruct tensors from a .stz archive, reversing all stored transforms. Handles all storage types: dense, quantized_fp16, quantized_int8, svd, sparse_coo.

spectra extract <file.stz> [OPTIONS]

Inputs: .stz

Output: .npz (default) or .npy (single tensor)

Flags

Reconstruction by storage type

Note: --no-original-dtype leaves tensors in their stored dtype (e.g. float16 or int8) rather than casting back to float32. Useful for memory-constrained environments.

Examples

# Reconstruct all tensors to recovered.npz
spectra extract model.stz --out recovered.npz
 
# Extract one tensor to a .npy file
spectra extract model.stz --tensor attention.weight --format npy
 
# Extract without restoring original dtype (keep as float16)
spectra extract model.stz --no-original-dtype
 
# Show reconstruction report (storage type and error per tensor)
spectra extract model.stz --report

spectra info

Display the manifest of a .stz archive. Reads only manifest.json from the zip; never decompresses tensors.npz. Sub-second even for large archives.

spectra info <file.stz> [OPTIONS]

Inputs: .stz

Outputs: Terminal summary or raw JSON to stdout.

Flags

Output sections (default mode)

• Archive header — filename, creation date, source file, Spectra version
• Storage summary — original size, size after tensor transforms (e.g. quantization/SVD), size after binary compression, each ratio
• Strategy breakdown — how many tensors used each storage type, and total original bytes per group
• Quality summary — count of lossy vs. lossless tensors; maximum stored MSE across all tensors

Examples

# Full manifest summary
spectra info model.stz
 
# Info for one tensor
spectra info model.stz --tensor attention.weight
 
# Raw manifest JSON (pipeable to jq)
spectra info model.stz --json | jq '.tensors | keys'
 
# Just the global stats
spectra info model.stz --json | jq '.global_stats'

Binary compression options

After tensor-level transforms, Spectra applies a second binary compression pass over the packed tensors.npz. The binary layer can be tuned independently of the tensor strategy.

Out-of-range levels are clamped with a warning rather than erroring. The method used is recorded in manifest.json → global_stats.binary_compression_method so extraction is always automatic.

The .stz format

A .stz (Spectral Tensor Zip) file is a standard ZIP archive containing exactly two entries:

archive.stz
├── tensors.npz      ← all transformed tensor arrays, optionally binary-compressed
└── manifest.json    ← metadata, routing decisions, error metrics

manifest.json structure:

{
  "spectra_version": "0.1.0",
  "created_at": "2026-06-09T...",
  "source_file": "model.npz",
  "source_format": "npz",
  "global_stats": {
    "total_tensors": 4,
    "total_parameters": 85000,
    "original_size_bytes": 340000,
    "tensor_transformed_size_bytes": 42000,
    "compressed_size_bytes": 38000,
    "compression_ratio_tensor_aware": 8.1,
    "compression_ratio_binary": 1.1,
    "compression_ratio_total": 8.9,
    "binary_compression_method": "zstd"
  },
  "tensors": {
    "attention.weight": {
      "storage_type": "svd",
      "original_shape": [256, 256],
      "original_dtype": "float32",
      "lossless": false,
      "reconstruction_method": "U @ diag(S) @ Vt",
      "rank_used": 10,
      "rank_full": 256,
      "spectrum_decay_rate": 0.21,
      "reconstruction_error_mse": 0.0,
      "reconstruction_error_relative": 0.0,
      "keys": [
        "attention.weight__U",
        "attention.weight__S",
        "attention.weight__Vt"
      ],
      "strategy_reason": "low intrinsic rank (10/256), SVD rank=10"
    },
    "output.bias": {
      "storage_type": "quantized_fp16",
      "original_dtype": "float32",
      "fp16_overflow_detected": false,
      "lossless": true,
      "reconstruction_method": "cast_to_original_dtype",
      "keys": ["output.bias"]
    }
  }
}

Array key naming conventions inside tensors.npz:

.stz files are readable by any ZIP tool (e.g. unzip -l model.stz) and the manifest is always plain JSON — no custom binary headers or proprietary structures.

Benchmarks

Spectra was benchmarked on two standard transformer models using the compress → extract pipeline at three tolerance levels. All tests run on CPU.

BERT-base-uncased (109M parameters, 438 MB)

BERT's weight matrices have broadly distributed singular value spectra — no weight matrix has a spectral decay rate above 0.85 and all require far more than 64 singular values to reach 1% reconstruction error. The routing engine correctly falls through to quantize_fp16 for every tensor, giving a clean 2× reduction with zero information loss.

• All 199 tensors routed to quantized_fp16 (lossless for float32 values within ±65504)
• 0 tolerance violations at any level ✓
• Sentence embedding cosine similarity vs. original: 1.000 — indistinguishable
• Binary (zstd) adds no further savings — the data is already well-entropy-coded after fp16

Interpretation: BERT weight matrices are informationally dense. Their singular values do not fall off sharply, meaning low-rank approximation cannot yield meaningful compression without large errors. fp16 quantization is the correct choice: it halves the storage footprint and is lossless in all practical ranges.

GPT-2 small (124M parameters, 498 MB)

GPT-2 shows the same spectral pattern as BERT — all 50 weight matrices have decay rates below 0.85. At 5% and 10% tolerance, one tensor passes the rank-fraction threshold and is compressed with SVD. All other tensors route to fp16.

• 147–148 tensors → quantized_fp16, 0–1 tensor → svd
• 0 tolerance violations at any level ✓
• KL divergence between original and compressed next-token distributions: 0.000 — identical outputs
• Top-5 predicted token overlap across 6 prompts: 100%
• The token embedding (wte.weight, 154 MB) routes to fp16 — its 50 257-word vocabulary requires a dense representation

Interpretation: Like BERT, GPT-2's transformer layers do not exhibit strong low-rank structure. fp16 compression achieves a stable 2.17–2.18× ratio across all tolerance settings. The tolerance parameter's primary effect is on whether edge-case tensors trigger SVD routing; for most real-world transformer weights it does not change the result.

Key takeaway

Both models achieve ~2.18× lossless compression via fp16 quantization with zero downstream quality degradation. The routing engine's SVD path activates on matrices with genuine low-rank structure (e.g. outputs of A @ B factorizations), not on trained transformer weights, which are informationally dense by design.

Roadmap

The following features are not yet implemented and will be added in future releases:

Until Tucker is available, 3D+ tensors (e.g. convolutional weights) are routed to quantize_fp16 as a safe 2× fallback.

Analysis module reference

Size notation

The --min-size flag accepts human-readable sizes: