h UddlmZddlmZddlmZiddddddd dd dd dd dd dddddddddddddddddddZded<iddddddddddd dd!dd"dd#dd$dd%dd&dd'dd(dd)d*d+d*d,d-id.d-d/d-d0d-d1d-d2d-d3d-d4d-d5d*d6d*d7d*d8d*d9d*d:d*d;d<d=d<d>d-d?d-d-d-d-d-d-d-d-d-d@d@dA ZdedB<dFdCZ dGdDZ yE)H) annotations)Encoding) get_encodingzo1- o200k_basezo3-zo4-mini-zgpt-5-zgpt-4.5-zgpt-4.1-z chatgpt-4o-zgpt-4o-zgpt-4- cl100k_basezgpt-3.5-turbo-z gpt-35-turbo-zgpt-oss- o200k_harmonyz ft:gpt-4ozft:gpt-4zft:gpt-3.5-turbozft:davinci-002zft:babbage-002zdict[str, str]MODEL_PREFIX_TO_ENCODINGo1o3zo4-minizgpt-4.1zgpt-4ozgpt-4z gpt-3.5-turbozgpt-3.5z gpt-35-turboz davinci-002z babbage-002ztext-embedding-ada-002ztext-embedding-3-smallztext-embedding-3-largeztext-davinci-003 p50k_baseztext-davinci-002ztext-davinci-001 r50k_baseztext-curie-001ztext-babbage-001z text-ada-001davincicuriebabbageadazcode-davinci-002zcode-davinci-001zcode-cushman-002zcode-cushman-001z davinci-codexz cushman-codexztext-davinci-edit-001 p50k_editzcode-davinci-edit-001ztext-similarity-davinci-001ztext-similarity-curie-001gpt2) ztext-similarity-babbage-001ztext-similarity-ada-001ztext-search-davinci-doc-001ztext-search-curie-doc-001ztext-search-babbage-doc-001ztext-search-ada-doc-001zcode-search-babbage-code-001zcode-search-ada-code-001rzgpt-2MODEL_TO_ENCODINGcd}|tvr t|}n1tjD]\}}|j|s|cS|t d|dd|S)zrReturns the name of the encoding used by a model. Raises a KeyError if the model name is not recognised. NzCould not automatically map z_ to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.)rr items startswithKeyError) model_name encoding_name model_prefixmodel_encoding_names L/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/tiktoken/model.pyencoding_name_for_modelrWs M&&)*5 2J1O1O1Q + -L-$$\2** +*:,7] ]   c*tt|S)zfReturns the encoding used by a model. Raises a KeyError if the model name is not recognised. )rr)rs rencoding_for_modelr"ps / ; <r*s"", <, <, ,  l ,  , ,<,|, m,m,],,,  !," #,$m%,&m',.,7%,7% ,7%| 7% | 7% l7% ]7%]7%}7%M7%=7%=7% m!7%"m#7%$m%7%* +7%, -7%. /7%0k17%2 37%4K57%6{77%8 [97%:{;7%< ;=7%@ A7%B C7%D E7%F G7%H[I7%J[K7%N[O7%P[Q7%T";U7%V W7%X$/*#.!,#.*$/ +  m7%>7t2=r