Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

Ben Merbaum, Mohammad Amin Raeisi, Wenhao Wang, Charalampos Papamanthou, Katerina Sotiraki, Fan Zhang

In submission
AI summary

Maverick lets a client outsource costly LLM computations while hiding its inputs and checking the results. Its matrix-vector verification and input masking speed up inference with very little extra work for the server.

Abstract

Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inference without exposing user inputs to a service provider. However, running large-scale models locally requires substantial computational resources. In practice, users may still resort to a third-party provider, giving rise to privacy and correctness concerns. Existing solutions that address these problems often impose substantial server overhead or introduce additional trust assumptions.

In this paper, we present Maverick, a novel approach to private and verifiable LLM inference based on a protocol for delegating matrix-vector multiplication, a dominant operation in LLMs. At its core, Maverick provides, to our knowledge, the first information-theoretically sound verification protocol for matrix-vector multiplication delegation with transparent preprocessing, efficient (batch) verification, and virtually no server overhead. We combine this verification primitive with LPN-based pseudorandom masking to provide input privacy.

We implement our matrix-vector delegation primitive and use it to build an end-to-end prototype of Maverick, which we evaluate on Qwen3-4B by measuring throughput in tokens per second. We evaluate client configurations with 1-8 threads. With one client thread and a CPU server using up to 128 threads, Maverick achieves throughput gains over local inference of up to 17x when privacy masks are generated online, 45x when they are precomputed, and 44x when only verification is required. With four client threads, the corresponding gains are 13x, 18x, and 17x. When server computation is no longer the bottleneck, client-side microbenchmarks with simulated network delay show speedups of 12x-20x, 34x-135x, and 38x-157x.

Cite this paper

@article{DBLP:journals/corr/abs-2609-10264,
  author       = {Ben Merbaum and
                  Mohammad Amin Raeisi and
                  Wenhao Wang and
                  Charalampos Papamanthou and
                  Katerina Sotiraki and
                  Fan Zhang},
  title        = {Maverick: Private and Verifiable {LLM} Inference Made Practical via
                  Matrix-Vector Multiplication Delegation},
  journal      = {CoRR},
  year         = {2026},
  url          = {https://doi.org/10.48550/arXiv.2609.10264},
  doi          = {10.48550/ARXIV.2609.10264},
  eprinttype   = {arXiv},
  eprint       = {2609.10264}
}