This website requires JavaScript.
Explore
Help
Sign In
phyer
/
avante.nvim
Watch
1
Star
0
Fork
0
You've already forked avante.nvim
Code
Issues
Pull Requests
Actions
Packages
Projects
Releases
Wiki
Activity
122bc3287b
Branches
Tags
View all branches
avante.nvim
/
crates
/
avante-tokenizers
History
Yang Jun
6c10081899
fix: failed to download tokernizer from HF (
#1060
)
2025-01-09 18:09:33 +08:00
..
src
fix: failed to download tokernizer from HF (
#1060
)
2025-01-09 18:09:33 +08:00
Cargo.lock
feat: tokenizers (
#429
)
2024-08-31 13:39:50 -04:00
Cargo.toml
feat(tokenizers): support parsing from public URL (
#765
)
2024-10-27 02:17:35 -04:00
README.md
feat: tokenizers (
#429
)
2024-08-31 13:39:50 -04:00
README.md
A simple crate to unify hf/tokenizers and tiktoken-rs
