Travis Waters

I study how language models can do useful work with fewer neurons.

I’m working on contextual sparsity and efficient LLM inference. My current research asks whether we can predict which neurons a token needs early enough to help a system prepare the right weights.

I’m especially interested in inference on memory-constrained hardware. I share selected findings and working notes here, and discuss the work on X.

research

Mistral-7B HellaSwag simulated sparsity comparison: planner 79.11% accuracy at 39.66% eligible MLP retention; GRIFFIN 70.93% at 40.04% retention.

November 2026 · Work in progress

Ahead-of-time neuron planning

Different tokens need different computation. I’m exploring a planner that selects eligible MLP neurons before the system needs their weights.

In our simulated Mistral-7B HellaSwag evaluation, the planner scored 79.11% at 39.66% retention, versus GRIFFIN’s 70.93% at similar retention. The dense model scored 81.18%.

Retention refers to eligible MLP neurons, not the whole model. Actual acceleration is still unmeasured.

Explore the selected results
Selected results

How much quality survives?

Mistral-7B / HellaSwag

Normalized accuracy (%) · simulated sparsity

My planner39.66% kept
79.11
GRIFFIN40.04% kept
70.93
Dense modelUnmasked reference
81.18
+8.18 percentage points over GRIFFIN at similar retention. GRIFFIN was the stronger of the two sparse baselines tested in this evaluation.
Evaluation details & limitations

These are results from our evaluation, not GRIFFIN’s published scores. Our HellaSwag evaluation applies sparse computation across answer tokens; GRIFFIN’s published protocol sparsifies only the final token’s forward pass.

Most results are single-seed. The comparisons shown here cover one model and one downstream benchmark. They do not establish a global ranking or broad task performance.

The planner adds overhead. Sparse kernels, weight transfers, and scheduling still need to produce a measured net gain on real hardware.

Open the comparison figure

working notes

Systems

Why predicting earlier matters

The value of a decision depends on when you can use it.

If the weights a token needs have already been transferred, deciding to skip them comes too late to save that transfer. An early prediction gives the system a chance to act before the cost is paid.

That is what interests me about planning. The question is whether the extra work of predicting buys enough time—and avoids enough movement—to be worthwhile.

It is a systems hypothesis. The accuracy results motivate testing it; they do not prove the hardware payoff.

Evaluation

Good sparsity doesn’t automatically mean fast inference

A smaller mask is a starting point, not a speed measurement.

A method can preserve quality while selecting fewer neurons, then lose the apparent savings to prediction overhead, irregular memory access, or inefficient kernels.

I want to keep two questions separate: can the model make useful selections, and can a system execute those selections efficiently? The first is promising here. The second still needs to be measured.

Next questions

What changes when the model doesn’t fit in memory?

Why memory-constrained machines are an interesting test.

When model weights exceed accelerator memory, inference has to deal with moving them. That makes neuron selection interesting beyond the arithmetic it might avoid.

A memory-constrained laptop is a compelling place to investigate the idea: could early selections reduce transfers enough to improve the experience? I have not measured a laptop speedup. That is a question for the next stage of the work.

contact

If you work on contextual sparsity or efficient inference, I’d be interested to compare notes. Find me on X: @traviswaterxpx.