phyer/avante.nvim
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8a3d5f30ec
avante.nvim/crates/avante-tokenizers
History
Aaron Pham 41c78127e5
chore(rust): fix current clippy lint () 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-09-03 22:18:53 -04:00
..
src
chore(rust): fix current clippy lint ()
2024-09-03 22:18:53 -04:00
Cargo.lock
feat: tokenizers ()
2024-08-31 13:39:50 -04:00
Cargo.toml
feat(templates): avanterules filetype support (closes ) ()
2024-09-03 04:09:13 -04:00
README.md
feat: tokenizers ()
2024-08-31 13:39:50 -04:00

README.md

A simple crate to unify hf/tokenizers and tiktoken-rs