UniDic for Python

1.1.0 · active · verified Wed Apr 15

UniDic is a dictionary for the MeCab morphological analyzer, specifically designed for modern written Japanese. The `unidic` Python package provides this dictionary data, allowing it to be easily integrated with MeCab wrappers like `fugashi` or `mecab-python3`. The current version is 1.1.0, and it generally releases new versions to incorporate updated UniDic data or minor quality-of-life improvements.

Warnings

breaking Version 1.1.0 changed the 'latest' or default UniDic version from 2.3.0 to 3.1.0. If your application relied on the older default dictionary, its behavior may change.
Fix: To use the previous default (UniDic 2.3.0), you need to explicitly install it using `pip install unidic['2.3.0+2020-10-08']` or specifically reference that version's DICDIR if multiple are installed.
gotcha The `unidic` package itself is small, but after `pip install`, you MUST run `python -m unidic download` to fetch the actual dictionary data. This download can be large (around 770MB-1GB on disk). If this step is skipped, the package will not function.
Fix: Always execute `python -m unidic download` after installation and before first use. Consider `unidic-lite` if disk space is a major concern, as it's a smaller version of UniDic.
gotcha UniDic is a dictionary, not an analyzer. It requires a separate MeCab wrapper like `fugashi` or `mecab-python3` to perform morphological analysis. Without one of these, `unidic` alone cannot process text.
Fix: Install either `fugashi` (`pip install fugashi`) or `mecab-python3` (`pip install mecab-python3`) in addition to `unidic`.
gotcha The `unidic` dictionary is specifically for Japanese language processing. It is unnecessary and incorrect to install and use it if your application does not require Japanese text analysis.
Fix: Only install `unidic` if Japanese NLP is a requirement for your project. Consider making it an optional dependency if your library supports multiple languages.

Install

pip install unidic Install UniDic package
python -m unidic download Download dictionary data (required after install)

Imports

unidic
```
import unidic
```
DICDIR
```
import unidic
unidic.DICDIR
```
This constant provides the path to the installed UniDic dictionary, to be passed to a MeCab Tagger.

Quickstart

After installing `unidic` and a MeCab wrapper like `fugashi` (or `mecab-python3`), you must first download the actual dictionary data using `python -m unidic download`. Then, you can import `unidic` and pass `unidic.DICDIR` to your MeCab tagger to enable Japanese morphological analysis.

import unidic
import fugashi # or mecab-python3
import subprocess
import os

# Ensure dictionary is downloaded (important step!)
try:
    subprocess.run(['python', '-m', 'unidic', 'download'], check=True, capture_output=True)
    print("UniDic dictionary downloaded successfully.")
except subprocess.CalledProcessError as e:
    if 'already exists' in e.stderr.decode():
        print("UniDic dictionary already exists.")
    else:
        print(f"Error downloading UniDic dictionary: {e.stderr.decode()}")
        # Handle error or exit if download failed

# Initialize MeCab Tagger with UniDic
tagger = fugashi.Tagger(f'-d "{unidic.DICDIR}"')

# Analyze a Japanese sentence
sentence = "今日の天気は晴れです"
result = tagger.parse(sentence)
print(f"Sentence: {sentence}")
print(f"Analysis: \n{result}")

# Access individual tokens (fugashi specific)
words = tagger.parseToNodeList(sentence)
for word in words:
    if word.surface == '': # Skip empty node for EOS
        continue
    print(f"Surface: {word.surface}, Lemma: {word.lemma}, POS: {word.pos1}")

view raw JSON →