infinityloop-dev/graphpinator-tokenizer
Tokenizer subproject for GraPHPinator: GraphQL server implementation for PHP.
时间：2026-01-04 06:38
h4kuna/latte-php-tokenizer
Transform macro args to array.
时间：2026-01-04 03:39
guttedgarden/tiktoken
PHP 7.4 version of tiktoken
时间：2026-01-04 03:10
codewithkyrian/tokenizers
Fast, pure-PHP tokenizer library compatible with Hugging Face tokenizers for encoding and decoding text
时间：2025-12-10 19:28
droath/laravel-text-chunker
Flexible, strategy-based text chunking for Laravel with support for character, token, sentence, and markdown-aware splitting. Perfect for AI/LLM applications and RAG systems.
时间：2025-11-16 03:13
sterzik/tokenizer
Library for creating an universal tokenizer
时间：2025-10-14 22:59
purewater2011/tiktoken-php7
PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP
时间：2025-09-10 08:06
hosmelq/search-syntax-parser
Parse search queries into structured data with field searches, boolean logic, ranges, lists, and adapters.
时间：2025-08-12 04:25
bermudaphp/tokenizer
PHP tokenizer for finding class, interface, trait, and enum declarations
时间：2025-06-11 13:35
pierotto/tokenizer-bundle
The package is used for secure data exchange between two parties using a JWT token.
时间：2025-04-19 07:21
hike/hike
A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages
时间：2025-04-06 15:53
your1/tokenizer-x
TokenizerX calculates required tokens for given prompt
时间：2025-03-28 08:48