Shiv Mehta

Optimization, Geometry, Inference, and Interpretability → Emergent Complexity

Philosophy of Study

Pre-undergrad work and writing lives here: archive.shivmehta.com

Worth your time:

The Bitter Lesson

Why You Can't Solve the Quintic

Group Theory & Positional Encodings

A Shakuntala GemmaScope Feature

A New Kind of Science

The Unreasonable Effectiveness of Mathematics

Moravec's Paradox & the Robot Olympics

More Is Different

Verbalizable Representations Form a Global Workspace in Language Models

The Future Worth Building Is Human

Modular Manifolds

Recent History

Graviton Research Capital
Quantitative Research InternJun 2026 – Aug 2026
UCSD Halıcıoğlu Data Science Institute
Undergraduate Researcher2026 – Present
Studying feature representations and latent geometry across diffusion timesteps with Dr. Yusu Wang, and steering methods for more controllable, efficient diffusion models.
Astera Holdings
Quantitative Research InternJun 2025 – Aug 2025
Built BART/XGBoost ensembles for MLB outcomes and player props with time-split validation and online Bayesian fair-value updates; a Kelly-criterion strategy reached <0.2 Brier.
SEELab, UCSD CSE Department
Undergraduate Researcher2025 – 2026
Improved numerical reasoning in quantized LLMs via chain-of-thought over sensor time-series, and built a DPO-tuned multi-agent pipeline with a Bayesian-signal grader under Dr Tajana Rosing and Dr Xiaofan Yu.
Multi Commodity Exchange (MCX)
Machine Learning InternJun 2024 – Aug 2024
Kernel-PCA factor models on commodity futures (60% dimensionality reduction), served through a PyTorch inference pipeline on a low-latency C++ (Treelite) backend.
bundl.ai
Machine Learning InternJun 2023 – Aug 2023
Built a RAG pipeline for a Llama model with hybrid vector/graph retrieval (PyTorch, Neo4j) and cut inference time 40% via distributed fine-tuning and TensorRT.
University of California, San Diego
B.S. in Data Science (Machine Learning & AI)2023 – Jun 2027
Coursework spanning linear algebra, vector calculus, probability & statistics, signal processing, spectral methods, stochastic processes, data structures & algorithms, and graph theory.

Highlighted Work

RoPE in the Residual Stream

2026 – 2026

Tracked RoPE's implicit positional manifold through the residual stream of Qwen2.5-3B: a multiscale, Fourier/RoPE-like geometry that is approximately linearly transported (rotated, stretched, and occasionally reorganized) across depth.

GitHub ↗write-up →

Partition Geometry

May 2026 – Present

A framework comparing trees, ReLU networks, and polyhedra on shared labeled grids via alignments; proved a general profile lower bound that gives exact tree complexities.

GitHub ↗write-up →

Spectra

Jan 2026 – Present

\begin{bmatrix} \color{#a5cfaf}{\blacksquare} & & & & \\ & \color{#a5bccf}{\blacksquare} & & & \\ & & \color{#aea5cf}{\blacktriangle} & & \\ & & & \color{#cfa5b3}{\bullet} & \\ & & & & \color{#c7a5cf}{\cdot} \end{bmatrix}

A command-line tool that profiles tensor artifacts (.npy, .npz) for sparsity, entropy, and spectral structure, then routes each to the best compression strategy: quantization, sparse COO storage, or truncated SVD.

GitHub ↗PyPI ↗write-up →

Prediction Market: Market Making

Dec 2025 – Present

Market-making BTC prediction markets: $10k+ volume across 100+ markets. Survival analysis for fills, markout estimation through random forest quantile regression (Treelite for inference), all optimized over a thin plate spline for spread, sizing, and timing.

write-up →