Clusterscope

Common errors

• ModuleNotFoundError: No module named 'clusterscope'

  cause The `clusterscope` package is not installed in the current Python environment.
  fix Install the package using pip: `pip install clusterscope`

• cscope: command not found

  cause The `cscope` command-line entry point is not in your system's PATH, or the package was installed into an environment not activated in your shell.
  fix Ensure your Python environment is activated (e.g., `source .venv/bin/activate`) or that pip's script directory is in your system's PATH. If using `pipx`, ensure `pipx ensurepath` has been run.

• (CLI output is empty or shows 'Unknown cluster')

  cause Clusterscope could not detect the cluster type or retrieve information from the underlying HPC system (e.g., Slurm). This often happens if Slurm commands are not available or if the output format is unexpected.
  fix Verify that you are on a node with Slurm client tools installed and configured. Try running native Slurm commands like `sinfo`, `squeue` to confirm they work. Ensure necessary environment variables (e.g., `SLURM_CONF`) are set correctly.

Warnings

• breaking Changes in v0.0.31 removed direct `init` calls to methods. Code relying on implicit initialization or specific method call order in constructors might break.
  Fix: Review code for `clusterscope` object instantiation and method calls within `__init__`. Ensure methods are called explicitly after object creation if they were implicitly called by the constructor in prior versions.
• gotcha Clusterscope heavily integrates with Slurm. Misconfigurations in the Slurm environment (e.g., missing `sinfo`, `squeue` commands, incorrect `srun` environment variables, or unexpected Slurm output formats) can lead to incorrect or incomplete information being extracted.
  Fix: Ensure a properly configured and accessible Slurm environment on the head node. Verify that standard Slurm CLI commands (`sinfo`, `squeue`, `scontrol`, `srun`, `sbatch`) are functional and produce expected output. Consult Slurm documentation and cluster administrators for environment setup.
• gotcha The library's functionality might be affected by specific cluster hardware or cloud configurations, especially regarding GPU visibility or resource allocation details. For instance, the changelog mentions `getting max gpus, cpus out of sinfo output` and updates related to `aws nccl defaults`.
  Fix: For unexpected results, particularly with `gpus`, `cpus`, or `aws` commands, check the clusterscope documentation or GitHub issues for known compatibility notes with specific cluster setups (e.g., different AWS instance types, Slurm versions, or NVIDIA driver configurations).

Install

• pip install clusterscope Install with pip

Imports

• cluster

  import clusterscope; clusterscope.cluster()

  The `cluster()` function directly returns the detected cluster name.
• get_gpus

  from clusterscope.gpus import get_gpus

  Used to retrieve detailed GPU information. The CLI command `cscope gpus` is a common way to access this functionality.

Quickstart

Demonstrates how to import the `clusterscope` library and use its `cluster()` function to identify the current HPC cluster name. It also hints at the available command-line interface (CLI) tools like `cscope gpus` for detailed hardware information.

import clusterscope

# Get the detected cluster name
cluster_name = clusterscope.cluster()
print(f"Detected cluster: {cluster_name}")

# You can also use the CLI for more specific information:
# print("\nTry running these commands in your terminal:")
# print("  $ cscope gpus") # Show GPU information
# print("  $ cscope cpus") # Show CPU counts per node
# print("  $ cscope mem")  # Show memory information per node
# print("  $ cscope aws")  # Check if running on AWS and show NCCL settings