{"id":8561,"library":"pytorch-ranger","title":"PyTorch Ranger Optimizer","description":"Ranger is a synergistic PyTorch optimizer that combines Rectified Adam (RAdam) and LookAhead techniques to improve training stability and convergence in deep learning models. The PyPI package, `pytorch-ranger`, provides an implementation of this optimizer, though its last update was in March 2020. More recent developments and features are primarily found in the original author's GitHub repository or the `Ranger21` project.","status":"maintenance","version":"0.1.1","language":"en","source_language":"en","source_url":"https://github.com/mpariente/Ranger-Deep-Learning-Optimizer","tags":["pytorch","optimizer","deep-learning","radam","lookahead"],"install":[{"cmd":"pip install pytorch-ranger","lang":"bash","label":"Install with pip"}],"dependencies":[{"reason":"PyTorch is the core deep learning framework this optimizer is built for.","package":"torch"}],"imports":[{"note":"The PyPI package `pytorch-ranger` uses `pytorch_ranger` as its top-level module, not `ranger`, unless you manually copied the `ranger.py` file from the original GitHub repository.","wrong":"from ranger import Ranger","symbol":"Ranger","correct":"from pytorch_ranger import Ranger"}],"quickstart":{"code":"import torch\nimport torch.nn as nn\nfrom pytorch_ranger import Ranger\n\n# 1. Define a simple PyTorch model\nclass SimpleModel(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.linear = nn.Linear(10, 1)\n\n    def forward(self, x):\n        return self.linear(x)\n\nmodel = SimpleModel()\n\n# 2. Define dummy data and target\ninputs = torch.randn(32, 10) # Example batch of 32 samples, 10 features\ntargets = torch.randn(32, 1) # Example batch of 32 targets\n\n# 3. Instantiate the Ranger optimizer\n# Pass model.parameters() to the optimizer\noptimizer = Ranger(model.parameters(), lr=0.001)\n\n# 4. Define a loss function\ncriterion = nn.MSELoss()\n\n# 5. Perform a single training step (in a real scenario, this would be in a loop)\noptimizer.zero_grad() # Zero the gradients\noutputs = model(inputs) # Forward pass\nloss = criterion(outputs, targets) # Compute loss\nloss.backward() # Backward pass (compute gradients)\noptimizer.step() # Update model parameters\n\nprint(f\"Loss after one step: {loss.item():.4f}\")","lang":"python","description":"This quickstart demonstrates how to initialize a `Ranger` optimizer with a simple PyTorch model's parameters and perform a single forward and backward pass, followed by an optimization step."},"warnings":[{"fix":"For the latest features and fixes, consider copying the `ranger.py` file directly from the `lessw2020/Ranger-Deep-Learning-Optimizer` GitHub repository or exploring the `lessw2020/Ranger21` project.","message":"The `pytorch-ranger` PyPI package (v0.1.1) has not been updated since March 2020. It may lack critical bug fixes, performance improvements, and newer features (like Gradient Centralization v2) present in the original author's more actively maintained GitHub repository's `ranger.py` file or the `Ranger21` project.","severity":"deprecated","affected_versions":"<=0.1.1"},{"fix":"Ensure you are using the most up-to-date `ranger.py` from the GitHub repository if you frequently save and load models, as this issue was addressed in later revisions of the original project.","message":"Older versions of Ranger (likely including `pytorch-ranger==0.1.1`) may have issues with optimizer state management, specifically, 'save and then load may leave first run weights stranded in memory, slowing down future runs'.","severity":"gotcha","affected_versions":"<=0.1.1"},{"fix":"The author often recommends specific learning rate strategies, such as a 75% flat learning rate followed by a step-down or cosine annealing for the final 25% of training. Experiment with different schedules.","message":"Ranger, like other advanced optimizers, can be sensitive to learning rate schedules. Suboptimal schedules can lead to slow convergence or unstable training, even though Ranger aims for stability.","severity":"gotcha","affected_versions":"all"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure both your model and all input tensors are explicitly moved to the same device (e.g., `model.to('cuda')`, `inputs.to('cuda')`).","cause":"Your model and input data are on different devices (e.g., one on GPU, one on CPU), leading to a device mismatch during computation.","error":"RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu"},{"fix":"Carefully inspect the `forward` method of your model and the shapes of your input data. Use `tensor.shape` or `print(tensor.size())` to debug tensor dimensions.","cause":"A shape mismatch occurred between tensors, often when the output of one layer doesn't match the expected input shape of the next, or when input data doesn't align with the model's first layer.","error":"RuntimeError: The size of tensor a (X) must match the size of tensor b (Y) at non-singleton dimension Z"},{"fix":"Review your learning rate schedule (Ranger benefits from specific patterns). Consider if your `pytorch-ranger` version is too old for your PyTorch version, leading to unpatched issues. Double-check your loss function and data preprocessing.","cause":"This can stem from various issues, including an incorrect learning rate, inappropriate learning rate schedule for Ranger, or using an outdated `pytorch-ranger` version with subtle bugs.","error":"Loss is not decreasing or model is not learning effectively, despite seemingly correct setup."}]}