Harbor / Terminal-Bench
Run Harbor trials, including Terminal-Bench, in E2B sandboxes instead of local containers. One flag moves a job to E2B, so you can raise concurrency without loading your machine.
- Harbor
- Built into Harbor
- harbor[e2b]
- Python
Get started
Paste the prompt into your coding agent, or follow the steps yourself.
Paste this into your coding agent
Run this project's Harbor evals on E2B sandboxes.
- 1.Install Harbor with the e2b extra ("harbor[e2b]"), for example with uv tool install.
- 2.Harbor needs the E2B_API_KEY environment variable. If it isn't set, ask me for my API key from https://console.e2b.dev/?tab=keys. Never hardcode it.
- 3.Follow the sandbox guide at https://docs.harborframework.com/sandboxes/pre-integrated-sandboxes and run the job with -e e2b, or set environment.type to e2b in the job config.
- 4.Run one small job and show me the trial results that came back from the sandboxes.
Or set it up yourself
1. Install Harbor with E2B
2. Set your API key
3. Run Terminal-Bench on E2B
4. Or keep it in a job config
What runs in the sandbox
Each Harbor trial runs in its own E2B sandbox, away from the machine running the job.
- The agent under test and the task's verifier, inside the environment the task defines.
- Harbor builds the task's environment into an E2B template once and reuses it across trials.
- Tasks can turn internet access off or allow only listed hostnames, and change that while a trial runs.
- Sandboxes are deleted after each trial by default. --no-delete keeps them for inspection.
More training & eval tools
Other RL and eval frameworks that run rollouts on E2B.
