NCNN Neural Network Inference Framework

1.0.20260114 · active · verified Thu Apr 16

NCNN is a high-performance neural network inference framework optimized for mobile platforms. The `ncnn` Python library provides official Python bindings, allowing users to load and run NCNN models from Python applications. It enables efficient deep learning inference on devices with limited computational resources. The library's versioning (`1.0.YYYYMMDD`) reflects a frequent release cadence, often aligned with new features or fixes in the core C++ NCNN project.

Common errors

```
ModuleNotFoundError: No module named 'ncnn'
```
cause The `ncnn` Python package is not installed in the current environment or the Python interpreter cannot find it.

fix Ensure `ncnn` is installed: `pip install ncnn`.
```
ImportError: libncnn.so: cannot open shared object file: No such file or directory
```
cause The underlying C++ NCNN shared library (`libncnn.so` on Linux, `.dll` on Windows, `.dylib` on macOS) was either not built, not properly linked during Python package installation, or cannot be found by the system's dynamic linker (e.g., not in `LD_LIBRARY_PATH`). This often happens when `pip install ncnn` fails to complete the C++ compilation step or if custom builds are attempted.

fix Verify `ncnn` installed successfully. If building from source, ensure `libncnn.so` (or equivalent) is generated and its directory is in `LD_LIBRARY_PATH` (Linux/macOS) or `PATH` (Windows). Re-installing `ncnn` after ensuring CMake and C++ compilers are present might resolve it: `pip install --no-cache-dir --upgrade ncnn`.
```
ncnn.Net: failed to load param model
```
cause The `.param` file for the NCNN model is either missing, has an incorrect path, is corrupted, or is not in the correct NCNN model format.

fix Double-check the path to your `.param` file. Ensure the file exists and is readable. Verify that both `.param` and its corresponding `.bin` file are present and correctly generated using `ncnn`'s model conversion tools.
```
TypeError: descriptor 'load_param' for 'ncnn.Net' objects doesn't apply to a 'str' object
```
cause Attempting to call an instance method (e.g., `load_param`, `load_model`) on the `ncnn.Net` *class* instead of an *instance* of the `ncnn.Net` class.

fix You must first create an instance of `ncnn.Net` before calling its methods: `net = ncnn.Net(); net.load_param('model.param')`.

Warnings

gotcha Installing `ncnn` via pip on certain platforms (e.g., Linux without pre-built wheels) requires a C++ compiler (like GCC/Clang) and CMake to be installed system-wide for the Python bindings to build successfully.
Fix: Ensure CMake and a compatible C++ compiler are available in your system's PATH. E.g., `sudo apt install build-essential cmake` on Ubuntu or `brew install cmake` on macOS (after installing Xcode Command Line Tools).
gotcha NCNN operates on its proprietary `.param` (network structure) and `.bin` (weights) model formats. You cannot directly load models saved from frameworks like PyTorch, TensorFlow, or ONNX. Conversion is required.
Fix: Use the `ncnn` tools (e.g., `onnx2ncnn`, `torch2ncnn`) or third-party converters to transform your models into the `.param` and `.bin` format before attempting to load them.
gotcha To utilize GPU acceleration with NCNN (via Vulkan), the library must be specifically compiled with Vulkan support. A standard `pip install ncnn` might provide a CPU-only build.
Fix: If pre-built wheels with Vulkan are not available for your platform, you may need to build `ncnn` from source, ensuring `NCNN_VULKAN=ON` during the CMake configuration step. Then set `net.opt.use_vulkan_compute = True` in your Python code.
gotcha When converting input data (e.g., NumPy arrays) to `ncnn.Mat`, ensure the channel order (e.g., HWC vs CHW) matches what your `ncnn` model expects. Common image processing libraries often output HWC, while many NCNN models might expect CHW.
Fix: Inspect your model's input requirements. Use `numpy.transpose` or `ncnn.Mat.from_pixels` with correct `pixel_type` and `num_channels` arguments to match the expected layout (e.g., `ncnn.Mat.from_pixels(img, ncnn.PIXEL_BGR2RGB, img.shape[1], img.shape[0])` for HWC BGR to NCNN RGB).

Install

pip install ncnn Install ncnn Python package

Imports

ncnn
```
import ncnn
```
The main module for accessing NCNN functionalities like Net, Mat, etc.
Net
```
from ncnn import Net
```
Used to create and manage the neural network model.
Mat
```
from ncnn import Mat
```
The primary data structure for NCNN tensors, used for inputs and outputs.

Quickstart

This quickstart demonstrates the basic API flow for initializing an NCNN network, loading (dummy) model files, and preparing input data using `ncnn.Mat`. It uses placeholder model files to be runnable without actual model conversion, but you would replace them with your own `.param` and `.bin` files for real inference.

import ncnn
import numpy as np
import os

# NCNN requires model files (.param and .bin)
# For a runnable quickstart, we'll demonstrate the API flow.
# In a real scenario, you'd replace these with actual converted model files.

# Create dummy files - these are NOT functional NCNN models
# but allow the API calls to proceed without immediate file not found errors.
# A real NCNN model conversion would generate proper .param and .bin.
with open("dummy_model.param", "w") as f:
    f.write("7767517\n0\n") # Minimal valid param content for an empty net
with open("dummy_model.bin", "wb") as f:
    f.write(b'') # Empty bin content

try:
    # 1. Initialize NCNN network
    net = ncnn.Net()
    print("NCNN Net initialized.")

    # Optional: Configure options (e.g., enable Vulkan if available)
    # net.opt.use_vulkan_compute = True 

    # 2. Load model structure (.param) and weights (.bin)
    # Note: These dummy files will load but won't perform actual inference.
    # Replace "dummy_model.param" and "dummy_model.bin" with your converted NCNN model paths.
    ret_param = net.load_param("dummy_model.param")
    ret_bin = net.load_model("dummy_model.bin")

    if ret_param == 0 and ret_bin == 0:
        print("Dummy NCNN model files loaded successfully.")
    else:
        print(f"Failed to load dummy model. param_ret={ret_param}, bin_ret={ret_bin}")
        # A non-zero return code means failure, e.g., malformed model files.

    # 3. Prepare input data (e.g., from an image or NumPy array)
    # This step is for demonstration; actual inference won't happen with dummy model.
    dummy_input_array = np.random.rand(224, 224, 3).astype(np.float32) * 255
    mat_in = ncnn.Mat.from_pixels(dummy_input_array, ncnn.PIXEL_RGB, 224, 224)
    print(f"Dummy input Mat created with shape: {mat_in.w}x{mat_in.h}x{mat_in.c}")

    # 4. Create an extractor and push input (for a real model)
    # ex = net.create_extractor()
    # ex.input("data", mat_in) # "data" is a common input blob name

    # 5. Run inference and extract output (for a real model)
    # ret, mat_out = ex.extract("output") # "output" is a common output blob name

    print("NCNN API usage demonstrated. For real inference, replace dummy files with actual NCNN models.")

except Exception as e:
    print(f"An error occurred: {e}")
finally:
    # Clean up dummy files
    if os.path.exists("dummy_model.param"):
        os.remove("dummy_model.param")
    if os.path.exists("dummy_model.bin"):
        os.remove("dummy_model.bin")

view raw JSON →