verl

0.7.1 verified Mon Apr 27 auth: no python

verl is a reinforcement learning framework for large language models (LLMs) developed by Volcano Engine. It provides end-to-end RL training (e.g., PPO, GRPO) with distributed compute and vLLM integration. Current version 0.7.1, actively developed and released on PyPI.

pip install verl

Common errors

error ModuleNotFoundError: No module named 'verl' ↓

cause verl is not installed or installed in a different environment.

fix

Run pip install verl and ensure the virtual environment is activated.

error TypeError: 'NoneType' object is not callable when calling WorkerGroup.generate() ↓

cause The inference engine was not properly initialized.

fix

Use from verl.utils import create_offload_inference_engine and pass engine=create_offload_inference_engine() to WorkerGroup.

Warnings

breaking verl 0.7.x dropped support for Python 3.9 and below. Only Python >=3.10 is supported. ↓

fix Upgrade Python to 3.10 or higher.

gotcha WorkerGroup.generate() returns a list of strings, not a dictionary. New users often expect a dict with keys like 'text' or 'response'. ↓

fix Access completions directly via list indexing.

gotcha The inference engine must be created via `create_offload_inference_engine()` (or similar) before constructing WorkerGroup. Passing a raw vLLM engine will cause silent errors. ↓

fix Always use the utility function provided by verl to create the engine.

Imports

WorkerGroup
```
from verl import WorkerGroup
```
Main entry point for creating a group of workers for distributed RL
create_offload_inference_engine
```
from verl.utils import create_offload_inference_engine
```
Helper to create an inference engine with offloading support

Quickstart

Initialize a WorkerGroup with an offloaded inference engine and generate text.

from verl import WorkerGroup
from verl.utils import create_offload_inference_engine

model_path = "Qwen/Qwen2.5-0.5B-Instruct"
worker_group = WorkerGroup(
    num_gpus_per_worker=1,
    model_path=model_path,
    inference_engine=create_offload_inference_engine()
)

# Example: generate completions
prompts = ["What is RL?"]
outputs = worker_group.generate(prompts)
print(outputs)