与「tokenizer」相关的 PHP Composer 包
本页汇总了 Packagist 中所有与 tokenizer 方向相关的 PHP Composer 包，按下载量倒序排列，便于你快速找到最主流、活跃度最高的包。
如果你正在做 tokenizer 相关项目的开发、二次开发、性能优化或 BUG 排查，欢迎联系我们的技术团队 —— 我们承接「tokenizer」相关的定制开发、私有化部署、 对接及长期维护外包，全程专人跟进。
robinthehood/tokenizer
With Tokenizer you can easily convert a string to a stream or array of tokens.
时间：2017-11-08 13:32
soloproyectos-php/text-tokenizer
A library to split a string into smaller tokens. This library is part of the SoloProyectos PHP API.
时间：2015-06-28 18:41
neuron-core/tokenizer
High-Performance Tokenizer implementation in PHP
时间：2025-09-16 13:00
hosmelq/search-syntax-parser
Parse search queries into structured data with field searches, boolean logic, ranges, lists, and adapters.
时间：2025-08-12 04:25
apemsel/attributed-string
A collection of fast and easy to use attributed string classes for PHP. Attributed strings can have multiple attributes for each character of the string and are for example used for word processors and natural language processing.
时间：2016-02-20 15:53
textualization/ropherta-tokenizer
GPT3Tokenizer (BPE) with Roberta-base vocabulary.
时间：2023-08-08 13:24
frolaxhq/laravel-llm-tokenkit
Token estimation, cost calculation & context-window utilities for LLM-powered Laravel apps
时间：2026-04-01 03:35
yuubit/tokenizer
PHP Library to tokenize various inputs.
时间：2019-01-03 12:32
crazzy501/gpt-3-encoder-php
PHP BPE Text Encoder/Decoder for GPT-2 / GPT-3. Based on coderevolutionplugins/gpt-3-encoder-php
时间：2023-06-06 11:09
everstu/gpt3-tokenizer
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
时间：2023-04-12 03:07
your1/tokenizer-x
TokenizerX calculates required tokens for given prompt
时间：2025-03-28 08:48
droath/laravel-text-chunker
Flexible, strategy-based text chunking for Laravel with support for character, token, sentence, and markdown-aware splitting. Perfect for AI/LLM applications and RAG systems.
时间：2025-11-16 03:13
scheldejonas/phpunit-php-token-stream
Wrapper around PHP's tokenizer extension.
时间：2020-11-12 13:30
sincco/tokenizer
Create and validate token for user data
时间：2016-05-17 22:44
需要「tokenizer」方向的技术支持？
- 基于以上任意 Composer 包做二次开发 / 深度定制，对接你的业务系统；
- 「tokenizer」相关模块的BUG 修复、性能优化、私有化部署；
- 整套「tokenizer」相关项目从零开发 & 长期维护外包；
- 已有系统的代码审计、安全加固、架构升级。
联系方式：邮箱 yvsm@zunyunkeji.com | QQ 316430983 | 微信 yvsm316