fasttext-numpy2

Warnings

• breaking The original `fasttext` Python bindings are incompatible with NumPy 2.0 and newer versions, leading to a `ValueError: Unable to avoid copy while creating an array as requested`. The `fasttext-numpy2` library specifically provides the necessary patches to resolve this issue.
  Fix: Use `pip install fasttext-numpy2` instead of the original `fasttext` package to ensure compatibility with NumPy 2.x and later.
• deprecated The original Facebook Research `fastText` GitHub repository (github.com/facebookresearch/fastText) was set to a read-only archive on March 19, 2024, indicating that it is no longer actively maintained by Meta. While the core C++ library remains functional, new features or official patches are unlikely from the original source.
  Fix: For continued compatibility, especially with evolving Python and NumPy versions, consider using community-maintained forks like `fasttext-numpy2` that address specific compatibility issues.
• gotcha FastText relies heavily on correctly preprocessed and encoded text. It assumes UTF-8 encoding, and inconsistent tokenization or encoding conventions between training and inference can significantly degrade model performance or lead to errors. Ensure all text data is consistently encoded and prepared.
  Fix: Always ensure your input text files are UTF-8 encoded. Review FastText's documentation on preprocessing data and encoding conventions, especially regarding tokenization and handling of word boundaries.
• gotcha Model binary files (`.bin`) are highly sensitive to the specific library version and compilation settings used to train them. While `fasttext-numpy2` aims for drop-in compatibility, loading a model trained with a significantly different version of fastText (e.g., the original `fastText` vs. `fasttext-numpy2`, or different underlying C++ compiler versions) can lead to unexpected behavior or errors.
  Fix: Whenever possible, load `.bin` model files with the exact same version of the `fasttext` library (and ideally, the same environment) that was used to train them. If transferring models, verify compatibility with your `fasttext-numpy2` version.

Install

• pip install fasttext-numpy2 Install latest version

Imports

• fasttext

  import fasttext

  The official fastText Python module on PyPI uses the lowercase 'fasttext' import name, merging with previous unofficial versions.

Quickstart

This quickstart demonstrates how to train a simple text classification model using `fasttext-numpy2` and then make a prediction. It first creates a dummy dataset in a file named `data.txt` in the format expected by fastText for supervised learning. It then trains a model using `train_supervised` and finally predicts the label for a new piece of text.

import fasttext
import os

# Create a dummy training data file
with open('data.txt', 'w') as f:
    f.write('__label__sports This is a great game.\n')
    f.write('__label__politics The election results are in.\n')
    f.write('__label__sports I love playing basketball.\n')
    f.write('__label__politics Debates are important for democracy.\n')

# Train a supervised model
model = fasttext.train_supervised('data.txt', epoch=5, lr=0.1, dim=100)

# Predict a label for a new text
text_to_predict = 'I watched a thrilling football match.'
labels, probabilities = model.predict(text_to_predict)

print(f"Text: '{text_to_predict}'")
print(f"Predicted label: {labels[0][0]}")
print(f"Probability: {probabilities[0]:.4f}")

# Clean up the dummy file
os.remove('data.txt')