A research team published the Hibana study on November 7, looking at how AI agents run on computers where the processor and graphics share memory. On a single NVIDIA DGX Spark, the authors report up to 3.4 times as many agent tasks completed per hour and up to 3.9 times the decoding throughput of the fastest baseline in the respective tests. The study is a preprint, and the figures come from the authors' evaluation.
Why shared memory changes the inference problem
DGX Spark uses the GB10 chip with 128 GB of unified LPDDR5X memory and peak bandwidth of 273 GB/s, according to NVIDIA's specifications. The processor and graphics unit share that capacity. The amount of memory makes it possible to load large models onto a compact system. When generating tokens in succession, however, reading weights and transferring them to the compute units can limit how quickly responses are produced.
An agent calls the same model multiple times. In a coding task, it might ask for a proposed change, run a test through a tool, and send the result back to the model. Each return extends or changes material that has already been processed. The inference engine can keep a representation of the context in memory and reuse it. With several active sessions, one session's context can be evicted to make space for another, so returning to the first requires recomputing part of it.
Hibana addresses both weight transfers and the preservation of state between steps. In the implementation described, some weights used during generation are stored in a three-bit representation with a lookup table of FP8 values. The authors also adapt expert selection in models that activate only part of their network at each step. The aim is to let several requests processed together reuse the same loaded data, reducing repeated reads from shared memory.
The engine can move some session state to the SSD when working memory fills up, then load it again on the next request. A third technique concerns speculative decoding. A smaller component proposes multiple tokens and the target model checks them. Hibana adapts the number of proposals to recent acceptance rates and the cost of verification. At low acceptance rates, an overly long draft would consume resources on proposals that are then rejected.
How speed and completed tasks were measured
The authors tested Qwen3.8-27B, Qwen3.8-Flash-Next, and two models from NVIDIA's Nemotron family, Bolt and Super. They included vLLM, NVIDIA Flare, llama.cpp, and TensorFold as baselines. SPEED-Bench measures decoding throughput as the number of simultaneous requests rises. On Qwen3.8-Flash-Next, Hibana's peak at 64 requests was 3.9 times Flare's at 32 requests. At the same level of 32 requests, the ratio was 3.1 times, according to the paper.
For complete tasks, the researchers replayed recorded sessions from AgentPerf-Local. In that test, throughput also depends on how much context the engine can reuse when an agent returns. At each system's best concurrency, Hibana completed between 1.5 and 3.4 times as many tasks per hour as the fastest baseline, depending on the model. With Qwen3.8-Flash-Next, the authors report 168 tasks per hour at 32 agents under that protocol.
In some replayed sessions, the final portion of the request changes from one turn to the next. An engine that can retrieve a state saved before that portion avoids recomputing the shared material. The researchers also ran a version that removed the rewrite from those sessions. At 16 agents with Qwen3.8-Flash-Next, the advantage over vLLM fell from 3.1 times to 1.9 times in that version. The experiment separates part of the gain from reusing context from the difference in decoding throughput.
The paper separately measures the effect of resuming a session under memory pressure. One experiment replayed 192 conversations, with 32 agents sending requests back to back. The combined context grew to about twice the graphics memory's cache capacity. In that test, saving state to the SSD reduced the amount of context that had to be recomputed on resumption. The experiment used recorded conversations and excluded time waiting for tools.
Model accuracy and the status of the project
The study also reports the cost of certain optimizations in quality scores. For Qwen3.8-Flash-Next, the speed-oriented configuration stayed within 3.6 percentage points of the exact baseline in the evaluations reported in the corresponding table. Altering weights and accepting some drafts more permissively can change the distribution of outputs. The authors report those figures separately from throughput because a task can finish faster and produce a different result.
In these evaluations, LiveCodeBench covers programming tasks and IFBench measures instruction following. The scores describe model behaviour under each test set's conditions. Throughput measures the number of tokens produced or tasks completed in a period. For an agent that returns frequently after intermediate results, the time to the first token of the new response also matters.
The paper points to a public repository for the Hibana code. When checked on November 11, the repository contained no files. The results describe the configurations evaluated by the authors, with the specified models, precision settings, and requests. A business installation also depends on actual software availability, the number of sessions, and time spent in the applications the agents call. A decoding test on one workstation and an automation that waits on other systems measure different components of the total time.
