Packages
Get Gentoo!
gentoo.org sites
gentoo.org
Wiki
Bugs
Forums
Packages
Planet
Archives
Sources
Infra Status
Home
Packages
Maintainers
USE flags
Architectures
About
sci-libs
/
tokenizers
Implementation of today's most used tokenizers
https://github.com/huggingface/tokenizers
Overview
Dependencies
QA report
Pull requests
0
Bugs
0
Security
0
Changelog
Available Versions
Version
amd64
x86
alpha
arm
arm64
hppa
mips
ppc
ppc64
riscv
sparc
0.20.1
: 0
EAPI 8
~amd64
?x86
?alpha
?arm
?arm64
?hppa
?mips
?ppc
?ppc64
?riscv
?sparc
Package Metadata
Upstream
Remote-Id
https://github.com/huggingface/tokenizers
USE flags
Global Use Flags
debug
test
python_single_target (Use Expand)
python3_10
python3_11
python3_12
License
Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD ISC MIT MPL-2.0 Unicode-DFS-2016
Maintainer(s)
Tupone Alfredo
External Resources
Related bugs
CI Report
(
verbose
)
Repology
Open Pull Requests
Documentation
Forums posts
Git repository browser
Git log
(
short
)
Changes Feed