Embedchain

Warnings

• gotcha ChromaDB, the default vector database, can sometimes get corrupted. This often manifests as unexpected errors during operations.
  Fix: Delete the `./chroma_db` folder (or your configured ChromaDB path) and re-index your data.
• gotcha The first query to an Embedchain application can be significantly slower than subsequent ones. This is due to initial loading of models and data into memory.
  Fix: Be aware of this initial delay, especially in user-facing applications. Subsequent queries will be faster.
• gotcha Large document sets can consume a substantial amount of RAM. For example, processing 4GB of documents might require a minimum of 8GB of RAM.
  Fix: Monitor memory usage, especially when dealing with extensive datasets. Consider breaking down extremely large datasets or optimizing your environment.
• gotcha Embedchain does not provide a straightforward method to update individual documents. If source data changes, the existing indexed document remains.
  Fix: The safest approach to update data is to re-index the entire dataset after making changes, which involves deleting the old index and adding the data again.
• gotcha When using OpenAI's API, you might encounter rate limiting issues during intensive embedding or querying operations.
  Fix: Implement delays between API calls, switch to a different embedding provider, or explore self-hosted LLM/embedding solutions with `OpenSourceApp`.
• breaking Embedchain can introduce unexpected transitive dependencies when integrated into projects already using other GenAI frameworks (e.g., `llama-index`, `crewai`), leading to version conflicts and compatibility issues.
  Fix: Carefully manage your project's dependencies. If conflicts arise, try to isolate Embedchain or manually pin conflicting package versions. Regularly check for known incompatibilities.
• breaking Incorrect configuration of embedding dimensions or incompatible settings between custom embedding models and vector stores (e.g., Qdrant, Ollama) can lead to silent failures or incorrect retrieval, often without clear error messages.
  Fix: Ensure that the embedding dimension configured for your embedding model precisely matches the schema or expected dimension of your vector store. Rebuilding indexes after model changes is recommended. Consult the specific vector store and embedding model documentation.

Install

• pip install embedchain Install Embedchain

Imports

• App

  from embedchain import App

  The primary class for creating and interacting with an Embedchain application.
• OpenSourceApp

  from embedchain import OpenSourceApp

  Used to initialize an Embedchain application with open-source LLMs and embedding models (e.g., GPT4All, Sentence Transformers).

Quickstart

This quickstart demonstrates how to create an Embedchain application, add web page data, and query it. By default, Embedchain uses OpenAI's models, requiring the `OPENAI_API_KEY` to be set as an environment variable.

import os
from embedchain import App

# Ensure your OpenAI API key is set as an environment variable
# Or replace os.environ.get with your actual key for testing.
# For production, always use environment variables.
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-YOUR_OPENAI_KEY")

# Create an Embedchain app instance
app = App()

# Add data sources (e.g., URLs, PDFs, YouTube videos, local files)
app.add("https://en.wikipedia.org/wiki/Elon_Musk")
app.add("https://www.forbes.com/profile/elon-musk")

# Query the app
response = app.query("How many companies does Elon Musk run and name those?")
print(response)

# You can also use the chat interface for conversational queries
# app.chat("Tell me more about Tesla.")