mirror of
https://git.boos.lat/aibo/BossChat.git
synced 2026-10-30 07:06:08 +00:00
fa9f64752b
nomic-embed-text tokenizes Chinese characters differently from the estimateTokenCount heuristic (3-5 real tokens per character vs 1 estimated). Large chunks overflow the 8192 token context. Reduce TargetTokens/MaxTokens to keep chunks within limits.