gpt-tokenizer

niieani
esm cjs
gpt-tokenizer JS library on GitHub gpt-tokenizer JS library on npm Download gpt-tokenizer JS library

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

Version 2.8.1 License MIT
gpt-tokenizer has no homepage
gpt-tokenizer JS library on GitHub
gpt-tokenizer JS library on npm
Download gpt-tokenizer JS library
Keywords
BPEencoderdecodertokenizerGPTGPT-2GPT-3GPT-3.5GPT-4GPT-4oNLPNatural Language ProcessingText GenerationOpenAIMachine Learning
INSTALL
Version:
Learn more
Readme Files Statistics Browse CDN