tiktoken-rs for Node.js

JSON →
library 1.0.6 ·javascript
verified May 27, 2026

tiktoken-rs-node provides performant NAPI Rust bindings for the `tiktoken-rs` library, which implements OpenAI's `tiktoken` BPE tokenizer for encoding and decoding text to and from token integers. As of version 1.0.6, it offers a zero-copy encode mechanism and avoids WebAssembly (WASM), differentiating it from other Node.js `tiktoken` implementations that often rely on WASM. The package is actively maintained with frequent patch releases, indicating ongoing support for its current stable API. Its primary use case is for applications requiring efficient tokenization of text for large language models, particularly in Node.js environments where high performance and minimal overhead are critical. It supports a range of encodings including `cl100k_base`, `gpt2`, and `o200k_base`.

total hits 18
actors 7 distinct systems
last hit 2d ago AhrefsBot
GPTBot
5
MetaBot
4
ChatGPT-User
3
Script
1
Search engines
1

top countries 🇺🇸 United States · 🇨🇦 Canada · 🇫🇷 France · 🇩🇪 Germany