{"id":23778,"library":"garak","title":"Garak - LLM vulnerability scanner","description":"Garak is a framework for probing large language models (LLMs) for security vulnerabilities. It automates the process of generating adversarial inputs and evaluating model outputs for issues like jailbreak, hallucination, data leakage, toxicity, and more. Current version is 0.14.1, requiring Python >=3.10. Active development with monthly releases.","status":"active","version":"0.14.1","language":"python","source_language":"en","source_url":"https://github.com/NVIDIA/garak","tags":["security","llm","penetration-testing","red-teaming","adversarial"],"install":[{"cmd":"pip install garak","lang":"bash","label":"Install from PyPI"}],"dependencies":[],"imports":[{"note":"Garak exposes a CLI; programmatic usage uses submodules like garak.probes, garak.generators, etc.","wrong":"","symbol":"garak","correct":"import garak"}],"quickstart":{"code":"import garak\nfrom garak.probes.base import Probe\n\n# Quick scan with default config\n# Run from command line: garak --model_type openai --model_name gpt-3.5-turbo\nprint(\"Garak imported successfully - run CLI for full functionality\")","lang":"python","description":"Basic import test. Garak is primarily used via CLI; this verifies the installation."},"warnings":[{"fix":"Remove `--generate_autodan` from command lines. Use other probe options instead.","message":"In v0.14.0, CLI option `--generate_autodan` was removed. Scripts using this flag will break.","severity":"breaking","affected_versions":">=0.14.0"},{"fix":"Replace references to 'failure' with 'attack_success' in custom reporting or parsing scripts.","message":"In v0.12.0, the `failure` concept was renamed to `attack success`. Output reports and APIs using 'failure' need updating.","severity":"breaking","affected_versions":">=0.12.0"},{"fix":"Continue using YAML config files unless you specifically need JSON. Monitor future releases.","message":"JSON config support (v0.14.0) is experimental; YAML config remains default and recommended for stability.","severity":"deprecated","affected_versions":">=0.14.0"},{"fix":"Set environment variables like OPENAI_API_KEY or use local generators (e.g., `--model_type test` for testing).","message":"Garak requires a model to be accessible via API or local. Running without a valid model will throw an error.","severity":"gotcha","affected_versions":"all"},{"fix":"Use `--max_workers 1` to reduce load or increase for faster scanning on capable hardware.","message":"Probe performance can be slow on large models; set `--max_workers` to control parallelism.","severity":"gotcha","affected_versions":"all"}],"env_vars":null,"last_verified":"2026-05-01T00:00:00.000Z","next_check":"2026-07-30T00:00:00.000Z","problems":[{"fix":"Run `pip install garak` in the correct Python environment (Python >=3.10).","cause":"Garak not installed or installed in a different environment.","error":"ModuleNotFoundError: No module named 'garak'"},{"fix":"Provide `--model_type` (e.g., openai, huggingface, test) and optionally `--model_name`.","cause":"CLI invoked without specifying the model type.","error":"garak: error: the following arguments are required: --model_type"},{"fix":"Set environment variable OPENAI_API_KEY before running garak, or use a different model type.","cause":"OpenAI API key not set when using OpenAI generator.","error":"openai.error.AuthenticationError: No API key found"},{"fix":"Check the YAML file syntax. Use `--config` with a valid .yml file or remove config flag to use defaults.","cause":"Malformed YAML config file (e.g., missing colon or space).","error":"yaml.scanner.ScannerError: mapping values are not allowed here"},{"fix":"Update parsing code to target 'attack_success' instead of 'failure'.","cause":"Script expects 'failure' key but garak v0.12+ uses 'attack_success'.","error":"KeyError: 'failure' in report parsing"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"install_tag":null,"quickstart_score":null,"quickstart_tag":null}