{"id":8357,"library":"nmslib","title":"NMSLIB (Non-Metric Space Library)","description":"NMSLIB (Non-Metric Space Library) is an efficient cross-platform similarity search library with Python bindings, designed for approximate nearest neighbor search in high-dimensional and non-metric spaces. It is widely used for vector similarity tasks, often in conjunction with embeddings from libraries like Gensim. The current version is 2.1.2, and the library receives updates focused on performance, new features, and broader platform/Python support.","status":"active","version":"2.1.2","language":"en","source_language":"en","source_url":"https://github.com/nmslib/nmslib","tags":["similarity search","nearest neighbors","vector search","indexing","approximate nearest neighbors","machine learning","high-dimensional data"],"install":[{"cmd":"pip install nmslib","lang":"bash","label":"Standard installation"},{"cmd":"pip install --no-binary :all: nmslib","lang":"bash","label":"Installation from source (for optimal performance)"}],"dependencies":[{"reason":"Required for data handling (e.g., converting data to float32 arrays) in most use cases.","package":"numpy","optional":false},{"reason":"Core dependency for Python bindings (typically installed automatically but can be a source of build issues).","package":"pybind11","optional":false},{"reason":"Used by nmslib, often installed automatically.","package":"psutil","optional":false}],"imports":[{"symbol":"nmslib","correct":"import nmslib"},{"symbol":"numpy","correct":"import numpy"}],"quickstart":{"code":"import nmslib\nimport numpy\n\n# Create a random matrix to index\ndata = numpy.random.randn(10000, 100).astype(numpy.float32)\n\n# Initialize a new index using HNSW on Cosine Similarity\nindex = nmslib.init(method='hnsw', space='cosinesimil')\nindex.addDataPointBatch(data)\nindex.createIndex({'post': 2}, print_progress=True)\n\n# Query for the nearest neighbours of the first datapoint\nids, distances = index.knnQuery(data[0], k=10)\nprint(f\"Nearest neighbors for data[0]: {ids}, distances: {distances}\")\n\n# Get all nearest neighbours for all the datapoints using multiple threads\n# neighbours = index.knnQueryBatch(data, k=10, num_threads=4)\n# print(f\"Batch query results (first entry): {neighbours[0]}\")","lang":"python","description":"This quickstart demonstrates how to initialize an HNSW index, add a batch of data points (random float32 vectors), build the index, and perform a k-nearest neighbor query. It uses `numpy` for data generation and `nmslib` for indexing and querying. The `post` parameter in `createIndex` is a common HNSW tuning parameter."},"warnings":[{"fix":"Upgrade to `nmslib>=2.1.1` by running `pip install --upgrade nmslib`.","message":"NMSLIB versions between 2.0.6 and 2.1.1 had deployment issues and were deleted. Users who installed these versions are advised to delete them and install a more recent version (>=2.1.1) to avoid potential instability or missing features.","severity":"breaking","affected_versions":"2.0.7 - 2.1.0"},{"fix":"Install with `pip install --no-binary :all: nmslib`. This requires C++ development tools (e.g., `python3-dev` on Ubuntu, Visual Studio 2019+ on Windows) and potentially CMake for full features.","message":"Pre-compiled binaries installed via `pip install nmslib` might be slower. For optimal performance, especially when using optimized spaces (e.g., negdotprod, l1, linf), it's recommended to install from source.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Review `save_data` and `load_data` parameters to avoid duplication. If `getDistance` is needed, ensure data is either loaded with `load_data=True` or added separately to the loaded index.","message":"When saving an HNSW index for `l2` or `cosinesimil` spaces, `saveIndex` can store an optimized copy of the data. If both `save_data=True` and `load_data=True` are used, this can lead to data duplication. Additionally, `getDistance` might not work properly on a loaded index if data is not explicitly reloaded.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If installation fails, try `pip install --use-pep517 nmslib` or install from source directly via `python -m pip install .` from the `python_bindings` directory of the cloned repository.","message":"The `nmslib` package currently uses the legacy `setup.py install` method. Pip versions 23.1 and later may enforce changes, potentially leading to installation failures.","severity":"deprecated","affected_versions":"All versions with pip >= 23.1"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure you have the necessary C++ build tools (e.g., `python3-dev` on Linux, Xcode command line tools on macOS, Visual Studio 2019 or later on Windows). For Windows, ensure your Visual Studio installation matches the Python version's build requirements. Consider installing from source using `pip install --no-binary :all: nmslib` after installing build tools, or directly from a cloned Git repository.","cause":"This generic error during installation often indicates missing C++ build tools, an outdated Python development environment, or incompatibility with the current Python version or operating system's compiler.","error":"error: subprocess-exited-with-error × Running setup.py install for nmslib did not run successfully."},{"fix":"Update your C++ compiler (e.g., GCC to version 4.7+, Clang to 3.4+, Visual Studio to 2015/VC14 or later). On Linux, ensure `g++` is up-to-date. On macOS, ensure Xcode command-line tools are installed and updated (`xcode-select --install`).","cause":"The C++ compiler available on your system (or detected by `setuptools`) does not meet the C++11 standard requirement for building NMSLIB.","error":"RuntimeError: Unsupported compiler -- at least C++11 support is needed"},{"fix":"This often requires patching the NMSLIB source or using a specific compiler version. Check the NMSLIB GitHub issues for workarounds or newer releases that address this specific build problem. Building from source with specific compiler flags or a different compiler environment (e.g., via `conda` with a known working C++ toolchain) might be necessary.","cause":"Specific compiler flags or features required for NMSLIB might be incompatible with newer versions of Apple Clang or architectures like Aarch64, particularly on macOS Sonoma.","error":"Build error with Apple clang: error: unknown type name 'thread_local' (Sonoma / aarch64)"},{"fix":"Ensure you are using the latest stable version of NMSLIB (2.1.2 or newer) which often includes fixes for Python compatibility. If the issue persists, review GitHub issues for specific patches or temporary workarounds related to your Python version.","cause":"A bug in specific NMSLIB versions or their interaction with newer Python runtime versions (>=3.9) can cause incorrect query results, such as repeatedly returning the same item.","error":"Incompatibility with python >= 3.9 -- repeating only one item."}]}