Packages
Get Gentoo!
gentoo.org sites
gentoo.org
Wiki
Bugs
Forums
Packages
Planet
Archives
Sources
Infra Status
Home
Packages
Maintainers
USE flags
Architectures
About
sci-libs
/
tokenizers
Implementation of today's most used tokenizers
https://github.com/huggingface/tokenizers
Overview
Dependencies
QA report
Pull requests
0
Bugs
1
Security
0
Changelog
Latest Commits
Atom feed
Commits on 2025-01-31
sci-libs/tokenizers: add 0.21.0
97a85c9
Alfredo Tupone
committed on 2025-01-31 23:16:47 UTC
sci-libs/tokenizers/tokenizers-0.21.0.ebuild
sci-libs/tokenizers/Manifest
Commits on 2024-11-09
*/*: update for slotted Rust
01ba4dc
Matt Jolly
committed on 2024-11-09 06:25:41 UTC
sci-libs/tokenizers/tokenizers-0.20.1-r1.ebuild
sci-libs/tokenizers/tokenizers-0.20.1.ebuild
Commits on 2024-10-27
sci-libs/tokenizers: drop other network tests
ea95407
Alfredo Tupone
committed on 2024-10-27 13:07:36 UTC
sci-libs/tokenizers/tokenizers-0.20.1.ebuild
Commits on 2024-10-26
sci-libs/tokenizers: drop failing test
9ca7be1
Alfredo Tupone
committed on 2024-10-26 21:05:42 UTC
sci-libs/tokenizers/tokenizers-0.20.1.ebuild
Commits on 2024-10-20
sci-libs/tokenizers: add 0.20.1, drop 0.20.0
af80d44
Alfredo Tupone
committed on 2024-10-20 08:16:09 UTC
sci-libs/tokenizers/tokenizers-0.20.1.ebuild
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/tokenizers-0.20.0.ebuild
Commits on 2024-10-12
sci-libs/tokenizers: drop 0.19.1
6bec593
Alfredo Tupone
committed on 2024-10-12 20:24:52 UTC
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/tokenizers-0.19.1.ebuild
Commits on 2024-10-07
sci-libs/tokenizers: drop 0.14.1-r1, 0.15.2-r1
491a20e
Alfredo Tupone
committed on 2024-10-07 12:05:16 UTC
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/tokenizers-0.14.1-r1.ebuild
sci-libs/tokenizers/tokenizers-0.15.2-r1.ebuild
Commits on 2024-10-05
sci-libs/tokenizers: add 0.20.0
7ee8b3a
Alfredo Tupone
committed on 2024-10-05 19:37:18 UTC
sci-libs/tokenizers/tokenizers-0.20.0.ebuild
sci-libs/tokenizers/Manifest
sci-libs/tokenizers: update SRC_URI
45db697
Alfredo Tupone
committed on 2024-10-05 12:01:10 UTC
sci-libs/tokenizers/tokenizers-0.19.1.ebuild
sci-libs/tokenizers: upstream removed cli binary
c298684
Alfredo Tupone
committed on 2024-10-05 11:34:12 UTC
sci-libs/tokenizers/tokenizers-0.19.1.ebuild
Commits on 2024-10-04
sci-libs/tokenizers: add 0.19.1
6ca94af
Alfredo Tupone
committed on 2024-10-04 20:15:29 UTC
sci-libs/tokenizers/tokenizers-0.19.1.ebuild
sci-libs/tokenizers/Manifest
Commits on 2024-04-14
sci-libs/tokenizers: enable test
99e70c7
Alfredo Tupone
committed on 2024-04-14 13:03:26 UTC
sci-libs/tokenizers/files/tokenizers-0.15.2-test.patch
sci-libs/tokenizers/tokenizers-0.15.2-r1.ebuild
sci-libs/tokenizers/tokenizers-0.15.2.ebuild
Commits on 2024-04-05
sci-libs/tokenizers: add 0.15.2
b722fdc
Alfredo Tupone
committed on 2024-04-05 19:40:16 UTC
sci-libs/tokenizers/tokenizers-0.15.2.ebuild
sci-libs/tokenizers/Manifest
Commits on 2024-02-23
sci-libs/tokenizers: drop QA_CHECK for musl build, too
48f3ec2
Alfredo Tupone
committed on 2024-02-23 19:39:02 UTC
sci-libs/tokenizers/tokenizers-0.14.1-r1.ebuild
Commits on 2024-01-18
sci-libs/tokenizers: PythonCompatUpdate
3b66eb3
Alfredo Tupone
committed on 2024-01-18 22:16:13 UTC
sci-libs/tokenizers/tokenizers-0.14.1-r1.ebuild
sci-libs/tokenizers/tokenizers-0.14.1.ebuild
Commits on 2023-12-31
sci-libs/tokenizers: drop 0.13.3
21e300f
Alfredo Tupone
committed on 2023-12-31 10:51:14 UTC
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/tokenizers-0.13.3.ebuild
Commits on 2023-12-30
sci-libs/tokenizers: add QA_FLAGS_IGNORED. It's rust
bea229f
Alfredo Tupone
committed on 2023-12-30 21:21:39 UTC
sci-libs/tokenizers/tokenizers-0.13.3.ebuild
sci-libs/tokenizers/tokenizers-0.14.1.ebuild
Commits on 2023-11-16
sci-libs/tokenizers: add 0.14.1, drop 0.14.0
6d96574
Alfredo Tupone
committed on 2023-11-16 21:38:46 UTC
sci-libs/tokenizers/tokenizers-0.14.1.ebuild
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/tokenizers-0.14.0.ebuild
Commits on 2023-11-15
sci-libs/tokenizers: add 0.14.0
792001c
Alfredo Tupone
committed on 2023-11-15 20:07:41 UTC
sci-libs/tokenizers/tokenizers-0.14.0.ebuild
sci-libs/tokenizers/Manifest
Commits on 2023-04-12
sci-libs/tokenizers: add dep on setuptools_rust
570cd0f
Alfredo Tupone
committed on 2023-04-12 09:03:26 UTC
sci-libs/tokenizers/tokenizers-0.13.3.ebuild
Commits on 2023-04-11
sci-libs/tokenizers: new package, add 0.13.3
bb2231b
Alfredo Tupone
committed on 2023-04-11 21:10:53 UTC
sci-libs/tokenizers/Manifest
sci-libs/tokenizers/metadata.xml
sci-libs/tokenizers/tokenizers-0.13.3.ebuild