7 Commits

Author SHA1 Message Date
yetone
d5a4db8321
Revert tokenizers ()
* Revert "fix: add missing "with" in README"

This reverts commit bf1e7f1f17c8b82687b15c41e5a8104ea03f70cf.

* Revert "feat: tokenizers ()"

This reverts commit d2095ba267abbfe169582708a4449e609aa9709b.
2024-08-31 22:45:31 +08:00
Aaron Pham
d2095ba267
feat: tokenizers ()
* feat: autobuild tiktoken lib and schenanigans

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* chore: revert readme changes

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* fix(build): windows

Signed-off-by: Hanchin Hsieh <me@yuchanns.xyz>

* chore(plugin): early load commands and base setup

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* fix(build): make sync

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* feat: rust go vroom vroom

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* feat: scuffed afaf implementation binding go brrrr

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* chore: remove dups

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* fix(tokens): calculate whether we should do prompt_caching (fixes )

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* chore: ignore lockfiles

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* Update README.md

* Update crates/avante-tokenizers/README.md

* chore: remove unused

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* chore: remove auto build

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

---------

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Hanchin Hsieh <me@yuchanns.xyz>
Co-authored-by: yuchanns <me@yuchanns.xyz>
2024-08-31 07:19:59 -04:00
Aaron Pham
5c1861d93f
feat(azure): support vision ()
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-08-29 01:48:40 -04:00
Aaron Pham
d7be4a59c2
fix(openai): enable image support on OpenAI platform only (fixes ) ()
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-08-27 11:35:25 -04:00
Aaron Pham
a7d3defa3d
feat(llm): add support for parsing secret vault ()
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-08-24 17:52:38 -04:00
Aaron Pham
6475407d0d
fix(llm): persistent key check for override class ()
* fix(llm): make sure to allow passing custom module

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* fix: correct custom class

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

* fix: correct attribute

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

---------

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-08-22 23:52:49 -04:00
Aaron Pham
8d52229f16
refactor(llm): cleanup providers for future ops (closes ) ()
support allow_insecure and proxy ops

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
2024-08-22 01:48:40 -04:00