Preparing for system design interviews?  Try bugzed.com →

tiktoken-rs for Node.js

JSON →
library 1.0.6 ·javascript
verified Jun 17, 2026

tiktoken-rs-node provides performant NAPI Rust bindings for the `tiktoken-rs` library, which implements OpenAI's `tiktoken` BPE tokenizer for encoding and decoding text to and from token integers. As of version 1.0.6, it offers a zero-copy encode mechanism and avoids WebAssembly (WASM), differentiating it from other Node.js `tiktoken` implementations that often rely on WASM. The package is actively maintained with frequent patch releases, indicating ongoing support for its current stable API. Its primary use case is for applications requiring efficient tokenization of text for large language models, particularly in Node.js environments where high performance and minimal overhead are critical. It supports a range of encodings including `cl100k_base`, `gpt2`, and `o200k_base`.

total hits 23
actors 8 distinct systems
last hit 17d ago human
GPTBot
4
Amazonbot
4
Script
1
ChatGPT-User
1
Search engines
3
Humans
7

top countries 🇺🇸 United States · 🇨🇦 Canada · 🇨🇳 China · VN · BD