Hugging Face TRL
Score the code your model writes during TRL GRPO training in E2B sandboxes. Generated programs run remotely, away from your training data and credentials.
- E2B
- Async reward function for GRPOTrainer
- trl 1.14.2
- Python
Get started
Paste the prompt into your coding agent, or follow the steps yourself.
Paste this into your coding agent
Add E2B sandboxes to this TRL project.
- 1.Get the trl-grpo-python example from the E2B cookbook and install it with the project's package manager.
- 2.The reward pool needs the E2B_API_KEY environment variable. If it isn't set, ask me for my API key from https://console.e2b.dev/?tab=keys. Never hardcode it.
- 3.Follow the TRL guide at https://docs.e2b.dev/agents/trl and pass the pool's async code_reward to GRPOTrainer as reward_funcs, so generated programs run in E2B sandboxes. Close the pool in a finally block.
- 4.Run the reward smoke test once and show me the scores that came from the sandbox.
Or set it up yourself
1. Get the example and set E2B_API_KEY in .env
2. Score fixed programs without a GPU
3. Pass the reward to GRPOTrainer
4. Check the baseline, then train
What runs in the sandbox
Generation and weight updates stay on your training machine. E2B only runs the candidate programs.
- Each generated program runs on a pooled worker sandbox, in a fresh isolated process for every test case.
- Expected outputs and the reward calculation stay on the host. Candidates only receive the test inputs.
- Two seconds per test, 64 KiB of output, and outbound internet disabled on every worker.
- Workers have a 600-second TTL, renewed while the pool lives. Close the pool in finally to kill them.
More training & eval tools
Other RL and eval tools that run their rollouts on E2B.
