{"id":4074,"library":"langchain-nvidia-ai-endpoints","title":"NVIDIA AI Endpoints for LangChain","description":"This integration package connects NVIDIA AI Endpoints with the LangChain framework, providing seamless access to NVIDIA's state-of-the-art AI Foundation Models. It enables robust conversational AI and semantic embedding capabilities through classes like `ChatNVIDIA` and `NVIDIAEmbeddings`. Currently at version 1.2.1, the library maintains an active development pace with frequent updates.","status":"active","version":"1.2.1","language":"en","source_language":"en","source_url":"https://github.com/langchain-ai/langchain-nvidia","tags":["LLM","Embeddings","NVIDIA","LangChain","AI Models","Generative AI"],"install":[{"cmd":"pip install langchain-nvidia-ai-endpoints","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core LangChain abstractions; specific versions may be required for full compatibility with newer features.","package":"langchain-core","optional":false}],"imports":[{"symbol":"ChatNVIDIA","correct":"from langchain_nvidia_ai_endpoints import ChatNVIDIA"},{"symbol":"NVIDIAEmbeddings","correct":"from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings"},{"symbol":"NVIDIARerank","correct":"from langchain_nvidia_ai_endpoints import NVIDIARerank"},{"note":"Used for Dynamo KV cache optimization support, introduced in v1.1.0.","symbol":"ChatNVIDIADynamo","correct":"from langchain_nvidia_ai_endpoints import ChatNVIDIADynamo"},{"note":"Connects to NVIDIA RAG Blueprint /v1/search endpoint, introduced in v1.2.1.","symbol":"NVIDIARAGRetriever","correct":"from langchain_nvidia_ai_endpoints import NVIDIARAGRetriever"}],"quickstart":{"code":"import os\nfrom langchain_nvidia_ai_endpoints import ChatNVIDIA, NVIDIAEmbeddings\n\nnvapi_key = os.environ.get('NVIDIA_API_KEY', '')\nif not nvapi_key.startswith('nvapi-'):\n    print(\"Please set the NVIDIA_API_KEY environment variable. You can get one from the NVIDIA API Catalog.\")\nelse:\n    # Initialize ChatNVIDIA for conversational AI\n    chat_model = ChatNVIDIA(model=\"nvidia/nemotron-3-super-120b-a12b\", nvidia_api_key=nvapi_key)\n    chat_response = chat_model.invoke(\"Explain the concept of large language models.\")\n    print(\"Chat Model Response:\", chat_response.content)\n\n    # Initialize NVIDIAEmbeddings for semantic embeddings\n    embed_model = NVIDIAEmbeddings(model=\"nvolveqa_40k\", nvidia_api_key=nvapi_key)\n    embedding_output = embed_model.embed_query(\"What are vector embeddings?\")\n    print(\"Embedding Model Output Length:\", len(embedding_output))","lang":"python","description":"This quickstart demonstrates how to initialize `ChatNVIDIA` for text generation and `NVIDIAEmbeddings` for creating text embeddings. It requires `NVIDIA_API_KEY` to be set as an environment variable, obtainable from the NVIDIA API Catalog. Replace 'nvidia/nemotron-3-super-120b-a12b' and 'nvolveqa_40k' with desired model names available on NVIDIA AI Endpoints."},"warnings":[{"fix":"Review LangChain's v1 migration guide. For `langchain-nvidia-ai-endpoints`, ensure `langchain-core` is updated to a compatible version (e.g., `0.1.15` or newer as per examples).","message":"With the release of LangChain 1.0, the broader LangChain ecosystem underwent significant changes, streamlining its core and moving some legacy functionality to `langchain-classic`. While `langchain-nvidia-ai-endpoints` adapted, ensure your overall `langchain` and `langchain-core` dependencies are compatible.","severity":"breaking","affected_versions":">=1.0.0 (langchain ecosystem), >=1.0.0 (langchain-nvidia-ai-endpoints)"},{"fix":"Obtain your `NVIDIA_API_KEY` from the NVIDIA API Catalog and ensure it is correctly set as an environment variable or passed directly during class instantiation. Verify it starts with `nvapi-`.","message":"Incorrect or missing `NVIDIA_API_KEY` will lead to authentication failures. The key must start with `nvapi-`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always keep `langchain-nvidia-ai-endpoints` updated to the latest version to ensure compatibility with the newest NVIDIA models and features. Refer to GitHub releases for model-specific support.","message":"Using older versions of `langchain-nvidia-ai-endpoints` with newer NVIDIA AI Foundation Models may result in uninformative errors or lack of support for latest model features. For example, specific models might require a minimum package version.","severity":"gotcha","affected_versions":"<1.2.1"},{"fix":"Ensure the `base_url` parameter points to the correct endpoint (e.g., `http://localhost:8081` for RAG server) and that the NIM container is running and accessible.","message":"When self-hosting NVIDIA NIM microservices, incorrect `base_url` or port configuration for models like `NVIDIARAGRetriever` can lead to `NVIDIARAGConnectionError`.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}