FlagEmbedding

1.3.5 · active · verified Wed Apr 15

FlagEmbedding is an actively developed Python library providing a powerful toolkit for text embedding and reranking, primarily featuring BGE (BAAI General Embedding) models. It specializes in multi-linguality, multi-granularities (up to 8192 tokens), and multi-functionality (dense, lexical, multi-vec retrieval). It achieves state-of-the-art performance in various benchmarks and is designed for applications like retrieval, classification, clustering, semantic search, and enhancing vector databases for Large Language Models (LLMs). The library is currently at version 1.3.5 and sees regular updates.

Warnings

gotcha For BGE v1.5 models, using `query_instruction_for_retrieval` is generally recommended for short queries in retrieval tasks for optimal performance. For other tasks (e.g., semantic similarity of short texts), instructions might not be needed or could even degrade performance. While v1.5 models are improved to work without instructions with only slight degradation, explicit instruction is often best practice for retrieval.
Fix: Experiment with `query_instruction_for_retrieval` parameter. For retrieval tasks with short queries, pass a descriptive instruction like `Represent this sentence for searching relevant passages:` to `from_pretrained` or `__init__`. For other tasks, omit or use an empty string.
gotcha Similarity scores from BGE models, especially those prior to v1.5, are often concentrated in a narrow range (e.g., [0.6, 1]). An absolute score greater than 0.5 does not necessarily indicate strong similarity. For downstream tasks like retrieval, the *relative order* of scores is usually more important than their absolute values. If filtering by threshold, determine an appropriate threshold (e.g., 0.8-0.9) based on your specific data's similarity distribution.
Fix: Focus on the ranking of similarity scores rather than absolute values. If filtering is required, empirically determine a suitable threshold for your dataset.
breaking Users upgrading from FlagEmbedding v1.2.x to v1.3.x have reported significant inference performance regressions (up to 100% slower) for BGE-M3 models and FlagReranker. This degradation can occur in subsequent calls to `model.encode` or `compute_score`.
Fix: Investigate GitHub issues for potential workarounds or official fixes. Consider pinning to a 1.2.x version if inference speed is critical and you're encountering this issue. The issue points to `self.model.to(device)` and `self.model.eval()` being invoked multiple times in `encode_single_device`.
gotcha Dependency conflicts, particularly with `transformers` and `accelerate`, have been reported. For instance, `transformers==4.44.2` has caused conflicts when other packages require a newer version (e.g., `transformers<5.0.0,>=4.45.2`). This can lead to installation failures or runtime issues.
Fix: Use a dedicated virtual environment. If conflicts arise, try explicitly installing a compatible version of `transformers` or `accelerate` that satisfies all your dependencies, or install `FlagEmbedding` in isolation.

Install

pip install -U FlagEmbedding Base Install
pip install -U FlagEmbedding[finetune] With Finetuning Dependencies

Imports

FlagAutoModel
```
from FlagEmbedding import FlagAutoModel
```
Recommended high-level class for unified embedding model inference.
FlagReranker
```
from FlagEmbedding import FlagReranker
```
For loading and using reranker models.
FlagModel
wrong:
```
from FlagEmbedding.FlagModel import FlagModel
```
correct:
```
from FlagEmbedding import FlagModel
```
While functional, `FlagAutoModel` is often preferred for its unified interface. Older examples or specific use cases might directly use `FlagModel`.

Quickstart

This quickstart demonstrates how to load an embedding model using `FlagAutoModel.from_pretrained()` and generate embeddings for queries and passages. It then shows how to compute similarity scores. Additionally, it provides an example of using `FlagReranker` to compute scores for query-passage pairs. Remember to replace placeholder model names with actual Hugging Face model IDs.

import os
from FlagEmbedding import FlagAutoModel

# You can replace 'BAAI/bge-base-en-v1.5' with other BGE models like 'BAAI/bge-m3'
# Consider setting query_instruction_for_retrieval for optimal performance in retrieval tasks.
# Use use_fp16=True for faster inference on compatible hardware.

# Example for embedding queries and passages
model = FlagAutoModel.from_pretrained(
    'BAAI/bge-base-en-v1.5',
    query_instruction_for_retrieval="Represent this sentence for searching relevant passages:",
    use_fp16=True
)

queries = ["What is FlagEmbedding?", "How to use embedding models?"]
passages = [
    "FlagEmbedding maps text to low-dimensional dense vectors for tasks like retrieval.",
    "Embedding models can be used to generate vector representations of text.",
    "The BGE models are state-of-the-art embedding models."
]

# Encode queries and passages
query_embeddings = model.encode_queries(queries)
passage_embeddings = model.encode_corpus(passages)

print(f"Query embeddings shape: {query_embeddings.shape}")
print(f"Passage embeddings shape: {passage_embeddings.shape}")

# Compute similarity scores (e.g., dot product)
scores = query_embeddings @ passage_embeddings.T
print("Similarity scores:")
print(scores)

# Example for reranking using FlagReranker
from FlagEmbedding import FlagReranker

# Replace with a reranker model like 'BAAI/bge-reranker-base'
reranker = FlagReranker('BAAI/bge-reranker-base', use_fp16=True)

query_passage_pairs = [
    ['What is AI?', 'Artificial intelligence (AI) is intelligence demonstrated by machines.'],
    ['What is AI?', 'The quick brown fox jumps over the lazy dog.']
]

ranks = reranker.compute_score(query_passage_pairs)
print("Reranker scores:")
print(ranks)

view raw JSON →