hMddlmZddlZddlZddlZddlmZmZmZm Z m Z m Z m Z m Z ddlZddlmZddlmZddlmZerddlZdZGdd eZy) ) annotationsN) TYPE_CHECKINGAnyDictIterableListOptionalTupleType)Document) Embeddings) VectorStoreceZdZdZdZdZdeeef ddZeddZe deeef ddZ e ddeeef dd Z d d dd d dd gf ddZ d ddZ d d!dZd"dZe deeef d#dZef d$dZef d%dZef d&dZ d' d&dZ d d(dZ d d)dZy)*Vearchlangchain_vearchcluster_client_dbNstrc  |rddl}nddl}|rw| td|sC|j}|dz }|t t jjddz }||_ ||_ j||_n|%tjjdd } n|} tjj!| stj"| tjj%| d } tjj!| stj"| j&| | |_| |_|sC|j*}|dz }|t t jjddz }||_||_||_y#t$r tdwxYw) zSInitialize vearch vector store flag 1 for cluster,0 for standalone rNzhCould not import suitable python package. Please install it with `pip install vearch or vearch_cluster`.zPlease input url of cluster_-\/log)vearch_clustervearch ImportError ValueError_DEFAULT_CLUSTER_DB_NAMEruuiduuid4split using_db_nameurl VearchClusterosgetcwdreplacepathisdirmakedirsjoinEngineusing_metapath_DEFAULT_TABLE_NAMEusing_table_nameembedding_funcflag) selfembedding_function path_or_url table_namedb_namer4kwargsrr metadata_pathlog_paths e/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/vearch.py__init__zVearch.__init__s % " !>??7733tzz|,2237;;!(D "DH(66{CDK" " 3 3D# > + 77==/ M*ww||M59H77==* H%'&-- x@DK"/D 11J # J #djjl+11#6r: :J *0 G Q  s F11Gc|jSN)r3r5s r= embeddingszVearch.embeddingsNs"""c |Dcgc]}|j} }|Dcgc]}|j} }|jd| || ||||d|Scc}wcc}w)Return Vearch VectorStore)texts embedding metadatasr7r8r9r4) page_contentmetadata from_texts) cls documentsrGr7r8r9r4r:drFrHs r=from_documentszVearch.from_documentsRsq*33A33)23AQZZ3 3s~~  #!    43s A Ac H|||||||} | j||| S)rE)r6rGr7r9r8r4)rFrH) add_texts) rMrFrGrHr7r8r9r4r: vearch_dbs r=rLzVearch.from_textsms;(#!   %9=rCtext)fieldtyperKc tjjtjjd}dddddd}|Dcgc]!}tj|d||d #}}tj d tjj d |d d ddid}|jj||j||}|Scc}w)z Create VectorStore Table Args: dim:dimension of vector fields_list: the field you want to store Return: code,0 for success,1 for failed )intri'IVFPQi ) ncentroids nsubvector) index_sizeretrieval_typeretrieval_paramrVrWtext_embeddingT MemoryOnly cache_sizeF)namerWis_index dimensionmodel_id store_type store_param has_source)refields vector_field) rdataTypeINTSTRINGGammaFieldInfoGammaVectorInfoVECTOR create_tabler2) r5dim field_list type_dict engine_infofirlrm response_codes r= _create_tablezVearch._create_tables"#OO//8N8NO %.2"E !   ! !"W+yF/D E  --!''#%u-   00 &&% 1 ) s&C c |jdddddddidddiddid d |d d d d}|jj|j|}|S)z Create VectorStore space Args: dim:dimension of vector Return: code,0 failed for ,1 for success rgammaFLAT metric_typeL2)rer^r_r`rWstringvectorTrc)rWindexrgri)rUrKra)re partition_num replica_numengine properties)r2r create_spacer%)r5ru space_configrzs r= _create_spacezVearch._create_spaces))"(!4$ HH%!!$". #  6 001C1C\R rCc  d}|j$|jjt|}| td|jr|j j }|j|vr2|j j|j}|s td|j j|j}|j|vr*|jt|d}|s tdg} ||t|||D]\} } } i} | | d<| d| d<tj| }d |tj j#|z j%i| d <|j j'|j|j| }|d d k(r| j)|d |j j'|j|j| }| j)|d | St*j,j/|j0|jdz}t*j,j3|s,t|d}|j5|}|r td||g}t|||D]`\} } } i}| |d<| d|d<tj| }|tj j#|z |d <|j)|b|j j7|} d}t| t|k7r8t9j:d|dkDrn|dz }t| t|k7r8|j j= S)z^ Returns: List of ids from adding the texts into the vectorstore. Nzembeddings is Nonezcreate db failed!!!rzcreate space failed!!!rUsourcerKfeaturerastatus_id.schemazcreate table failed!!!g?r)r3embed_documentslistr r4rlist_dbsr% create_db list_spacesr2rlenzipnparraylinalgnormtolist insert_oneappendr(r+r.r0existsr{addtimesleepdump)r5rFrHr:rBdbs_listcreate_db_code space_listcreate_space_codedocidrUrKembedprofilesembed_np insert_res retry_insert table_pathrurz doc_items profiles_vt_times r=rRzVearch.add_textssX    *,,<>E%)*?-0 :-N!)D(E/1H'+HV$+3H+=HZ(!xxH!Hryy~~h/G$G#O#O#Q2H-."&!7!7**D,A,A8"J"(+s2 Z%67 '+{{'='= ..0E0Ex(  \%%89 '!^ 5##T%:%:Y%FJ77>>*-*Q-( $ 2 23 7  $%=>>%)*? -0 :-N1)D(E13J)-Jv&-5h-?Jz*!xxH3;biinnX>V3VJ/0$$Z0 1  2%jC O3JJsOzaKF %jC O3    " rCc8|jjy)z: load vearch engine for standalone vearch N)rloadrAs r=_loadz Vearch._load*s rCc |s td|s tdtjj||dz}tjj |s td||||||}|j |S)zLoad the local specified table of standalone vearch. Returns: Success or failure of loading the local specified table zNo metadata path!!!zNo table name!!!rz$vearch vectorbase table not exist!!!)r6r7r8r9r4)r r(r+r.rr) rMrGr7r8r9r4r:rrSs r= load_localzVearch.load_local0s23 3/0 0WW\\+zI/EF ww~~j)CD D(#!   rCc |j td|jj|}|j||}|S)z5 Return docs most similar to query. embedding_func is None!!!)r3r embed_querysimilarity_search_by_vector)r5querykr:rBdocss r=similarity_searchzVearch.similarity_searchPsI    &89 9((44U; // A> rCc tj|}|jrvdd|tjj |z j dgi|ddgd}|j j|j|j|}|dd}nRd|tjj |z dggdd d d |d }|j j|}|d d}g}|D]S} d} i} |jr| d} | D]} | dk(r| | } | dk(s| | | d<|jt| | U|S)The most k similar documents and scores of the specified query. Args: embeddings: embedding vector of the query. k: The k most similar documents to the text query. min_score: the score of similar documents to the text query Returns: The k most similar documents to the specified text query. 0 is dissimilar, 1 is the most similar. sumrarVrrUrKrsizerlhitsr InnerProductrnproberrlis_brute_searchr`topnr result_itemsrb_sourcerrJrK) rrr4rrrrsearchr%r2rr ) r5rGrr:r query_data query_resultresritemcontent meta_dataitem_keys r=rz"Vearch.similarity_search_by_vector`s# 99%5(- u0E(E'M'M'O!:. J ;;--""D$9$9:Lv&v.C "2#(299>>%+@#@ #$3AR#P J ;;--j9Lq/.1C LDGIyyI  v%"8nGz)*.x.Ih'   KKg J K L rCc 8|j td|jj|}tj|}|j rvdd|tj j|z jdgi|gdd}|jj|j|j|}|dd}nRd|tj j|z dggdd d d |d }|jj|}|d d}g} |D]w} d} i} |j r | d} | d} | D]6}|dk(r| |} |dk(r | || d<|j dk7s,|dk(s2| |} 8t| |  f}| j|y| S)rrrrar)rarUrKrrrrrrrrrrb_scorerrUrKrscorer)r3r rrrr4rrrrrr%r2r r)r5rrr:rBrrrrresultsrrrrrtmp_ress r=similarity_search_with_scorez#Vearch.similarity_search_with_scores    &89 9((44U; $ 99%5(- u0E(E'M'M'O@ J ;;--""D$9$9:Lv&v.C "2#(299>>%+@#@ #$3AR#P J ;;--j9Lq/.1C02 $DGIyyXI v%"8nGz)*.x.Ih'99>h'&9 NE  WyI5QG NN7 ## $$rCc *|j||fi|Sr@)r)r5rrr:s r=(_similarity_search_with_relevance_scoresz/Vearch._similarity_search_with_relevance_scoress 1t00DVDDrCc Bd}g}||jdk(r|S|D]l}|jr2|jj|j|j |}n|jj |}|j|ntd|D}|S)aIDelete the documents which have the specified ids. Args: ids: The ids of the embedding vectors. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful. False otherwise, None if not implemented. Nrc3&K|] }|dk( yw)rNrI).0is r= z Vearch.delete.. s*Q!q&*s) __len__r4rdeleter%r2del_docrall)r5idsr:retrrs r=rz Vearch.deletes# ;#++-1,J Cyykk((););T=R=RTWXkk))#. NN3   *'** rCc i}||jdk(r|S|jrdd|ii}|jj|j|j |}|D]J}|ddur d}i}|dD]!} | dk(r |d| }| d k(s|d| |d <#t || ||d <L|S|D]Z} |jj| }|ik(r$d}i}|D]} | dk(r|| }| d k(s|| |d <t || ||d <\|S) zReturn docs according ids. Args: ids: The ids of the embedding vectors. Returns: Documents which satisfy the input conditions. rrrfoundFrbrrUrKrrr)rr4r mget_by_idsr%r2r get_doc_by_id) r5rr:rr docs_detailrecordr meta_inforVids r=getz Vearch.get s(* ;#++-1,N 99!E3<0J++11""D$9$9:K& '?e+ #I.!E"("3E": *,.4Y.?.F (+ !*2!(9*u & @! "kk77; "$ (!E"-e"4 *,.9%.@ (+ !/7!(9/ E*+  rC)r6r r7 Optional[str]r8rr9rr4rYr:rreturnNone)rzOptional[Embeddings])rM Type[Vearch]rNList[Document]rGr r7rr8rr9rr4rYr:rrr)rMrrF List[str]rGr rHOptional[List[dict]]r7rr8rr9rr4rYr:rrr)rurYrvz List[dict]rrY)rT)rurYrrYr@)rFz Iterable[str]rHrr:rrr)rr)rGr r7rr8rr9rr4rYr:rrr)rrrrYr:rrr)rGz List[float]rrYr:rrr)rrrrYr:rrzList[Tuple[Document, float]])r)rOptional[List[str]]r:rrzOptional[bool])rrr:rrzDict[str, Document])__name__ __module__ __qualname__r1r!_DEFAULT_VERSIONr>propertyrB classmethodrPrLr{rrRrr DEFAULT_TOPNrrrrrrrIrCr=rrs,2 &*-/$ 4&4#4 4  4  44 4l## &*-/$  !  #           4 +/%)-/$ (  #    4e , % 0" + ++ +^( ( (Z+/KK(K K  KZ &*-/$ #      D    &>> > >  >FFF F F & FVEE E E & E$(   <$(5 55  5rCr) __future__rr(rr"typingrrrrrr r r numpyrlangchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresrrrrrIrCr=r s?" RRR-03 n[nrC