How Paper Instruments uses E2B to cut idle GPU time for RL rollouts

Paper Instruments trains frontier models to do knowledge work: consulting, finance, investment banking, and law. They partnered with Perplexity to develop DRACO, a benchmark for its flagship Advanced Deep Research product. Most recently, they released DiligenceBench with Thoughtful Lab, which is a benchmark that grades agents against equity-research tasks with rubrics built from thousands of weighted criteria.

Training frontier models against these benchmarks requires infrastructure that can scale. In practice, that means running thousands of concurrent rollouts, each one with an agent operating on its own filesystem and external application state over many steps. These rollouts need to be isolated from each other to keep training signals clean. Additionally, the data generated by rollouts (the agent's actions, file changes, intermediate outputs, rewards/grades from the benchmark rubrics) is wired directly into the pipeline in a structured and traceable way so that there is visibility into how data actually influences the models. That visibility starts with the sandbox each rollout runs in.

"Our core IP centers on plumbing data generation through training so that we have visibility into how each task and its provenance actually influences the models we train. On the infrastructure side, running RL with a policy gradient method like GRPO or CISPO means orchestrating thousands of concurrent rollouts at once."

- Joey Besgen, Co-Founder, Paper Instruments

‍

The challenge

Isolating thousands of RL rollouts 

Each RL rollout needs its own file system and persistent state for the length of the trajectory, and it’s important that the solver and the grader are completely isolated from one another to prevent reward hacking. Additionally, parallel rollouts running in parallel must not affect each other. Cross-contamination between rollouts, or exposure to systems outside the sandbox, can corrupt the reward signal the model is training against.

Latency is a GPU bill

RL training has an inherently spiky workload: hundreds of sandboxes spin up together, run agent actions and tool calls against the file system/external state, then grade the results and tear down. Every second a rollout spends waiting on a slow command or a slow cold start impacts GPU utilization, and long waits for sandbox actions accumulate to drive up training cost directly.

Build vs. buy on Firecracker

Paper Instruments evaluated all of the top sandbox providers, and E2B’s command execution latency and resilience to production spikes proved to be the best for their workflows.

The other alternative to a managed sandbox layer was building one, but standing up and operating that orchestration layer in-house would have pulled engineering time away from the model and data work that gives Paper Instruments its edge.

‍

The solution

Paper Instruments runs its RL rollouts inside E2B sandboxes, giving every concurrent trajectory its own isolated file system and its own state for the length of the run.

Cutting cold-start and command latency

In Paper Instruments' own benchmarking, E2B's tool execution came in up to 3 times faster than other sandbox providers they evaluated. Faster startup and faster command execution both translate directly into less idle GPU time and lower training cost.

Driving faster startup with snapshots

In RL training, every rollout needs to start from the exact same "world": the file system, running services, and any external state the agent operates on, so that differences in outcomes reflect the policy being trained and not inconsistencies in the starting environment. Paper Instruments starts with an E2B sandbox template, initializes each environment and any required services, then snapshots the running sandbox. Rollouts can then spin up directly from the appropriate snapshot, restoring the captured file system and memory state instead of repeating the setup process. This lets hundreds of concurrent rollouts launch from consistent, pre-initialized environments while minimizing idle GPU time.

"Another thing E2B is great at is that you can snapshot both the file system and the memory. That drives faster startup, because we can snapshot running servers inside the sandbox. When a job launches hundreds of rollouts against the same starting state, that snapshot is what eliminates most of the cold-start work for every rollout.”

- Joey Besgen, Co-Founder, Paper Instruments

Developer experience for humans and agents

Besgen points to onboarding as a differentiator that showed up twice: once for his own team reading the docs, and once for the agents that also read them. That meant they could go from zero to one a lot faster. Spending less time on perfecting sandbox infrastructure allowed the team to focus more efforts on improving training data and the training recipe, which meant they could iterate on training results in a much shorter timeframe.

"E2B provides a very good developer experience. They make it easy to spin up sandboxes, easy to snapshot the running state so we can resume quickly for later training runs, and gives us good intelligence into what's happening inside those sandboxes so we can debug during training."

- Joey Besgen, Co-Founder, Paper Instruments

Looking ahead

Paper Instruments' bet is that synthetic data, sampled from existing policies and pushed slightly out of distribution, can drive high Pareto performance. That bet depends on being able to run large sweeps of RL experiments cheaply and repeatedly, which is what E2B helps solve today. As Paper Instruments scales up training and evaluation for its next models, the volume of concurrent runs is only set to multiply on E2B.