h]~ddlZddlmZmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdZGdd eZy) N)AnyDictIterableListOptionalTuple)Document) Embeddings) VectorStore)maximal_marginal_relevancec eZdZdZdededdfdZd(dZede efdZ d)d e e d e e edede e fd Z d)d e e d e e edede e fd Z d)d e e e dede efdZ d)d e e e dede efdZefddde edede ee e fdede eeeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde dede ee e fdede eeeff dZefddde edede ee e fdede ef dZefddde edede ee e fdede ef dZ efddde dede ee e fdede ef dZ!efddde dede ee e fdede ef dZ"eddfddde eded ed!ede ee e fdede efd"Z#eddfddde eded ed!ede ee e fdede efd#Z$ d*ddde ded ed!ede ee e fdede efd$Z%eddfddde ded ed!ede ee e fdede efd%Z&e' d)d e e ded e e ededdf d&Z(e' d)d e e ded e e ededdf d'Z)y)+SurrealDBStorea/ SurrealDB as Vector Store. To use, you should have the ``surrealdb`` python package installed. Args: embedding_function: Embedding function to use. dburl: SurrealDB connection url ns: surrealdb namespace for the vector store. (default: "langchain") db: surrealdb database for the vector store. (default: "database") collection: surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Example: .. code-block:: python from langchain_community.vectorstores.surrealdb import SurrealDBStore from langchain_community.embeddings import HuggingFaceEmbeddings model_name = "sentence-transformers/all-mpnet-base-v2" embedding_function = HuggingFaceEmbeddings(model_name=model_name) dburl = "ws://localhost:8000/rpc" ns = "langchain" db = "docstore" collection = "documents" db_user = "root" db_pass = "root" sdb = SurrealDBStore.from_texts( texts=texts, embedding=embedding_function, dburl, ns, db, collection, db_user=db_user, db_pass=db_pass) embedding_functionkwargsreturnNc  ddlm}|jdd|_|jdddk(r||j|_n t d|jd d |_|jd d |_|jd d|_ ||_ ||_ y#t$r}td|d}~wwxYw)Nr)SurrealzZCannot import from surrealdb. please install with `pip install surrealdb`.dburlzws://localhost:8000/rpcwsz6Only websocket connections are supported at this time.ns langchaindbdatabase collection documents) surrealdbr ImportErrorpoprsdb ValueErrorrrrrr)selfrrres h/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/surrealdb.py__init__zSurrealDBStore.__init__5s   )ZZ)BC ::a?d "tzz*DHUV V**T;/**T:. **\;?"4 # @  sB(( C1 B==CcK|jjd{d|jvrjd|jvr\|jjd}|jjd}|jj ||dd{|jj |j |jd{y77?7 w)zr Initialize connection to surrealdb database and authenticate if credentials are provided Ndb_userdb_pass)userpass)r!connectrgetsigninuserr)r#r*passwords r% initializezSurrealDBStore.initializeOs hh    # T[[(@;;??9-D{{y1H((//4"BC C Chhll477DGG,,, ! D,s4CCA7CC7CCCCCcRt|jtr |jSdSN) isinstancerr )r#s r% embeddingszSurrealDBStore.embeddings[s.$11:>  # #  texts metadatascZK|jjt|}g}t|D]p\}}|||d}||t |kr |||d<ng|d<|j j |j|d{} |j| ddr|S7w)zAdd list of text along with embeddings to the vector store asynchronously Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents )text embeddingNmetadatarid) rembed_documentslist enumeratelenr!createrappend) r#r7r8rr5idsidxr:datarecords r% aadd_textszSurrealDBStore.aadd_textscs,,<Z #%Z 88??F JJvay ' ( sBB+ B) B+c ddttdtttdt dttffd }t j|||fi|S)zAdd list of text along with embeddings to the vector store Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents r7r8rrc|Kjd{j||fi|d{S7 7wr3r1rH)r7r8rr#s r% _add_textsz,SurrealDBStore.add_texts.._add_textss= //# # #( DVDD D $D<8<:<<r3)rstrrrdictrasynciorun)r#r7r8rrLs` r% add_textszSurrealDBStore.add_textssd"/3 EC= ET + E E#Y  E{{:eYA&ABBr6rDcK|.|jj|jd{yt|tr$|jj|d{yt|t r?t |dkDr1|Dcgc]%}|jj|d{'}}yy77W7cc}ww)aDelete by document ID asynchronously. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. NTrF)r!deleterr4rNr?rA)r#rDrr=_s r%adeletezSurrealDBStore.adeletes ;((//$//2 2 2#s#hhooc***c4(SX\=@Artxxr222AAA 3+3AsE+CB73C!B9"'C #B=,B; -B=3C9C;B==Cc dtttdtdttffd }t j ||fi|S)a Delete by document ID. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. rDrrc|Kjd{jdd|i|d{S7 7w)NrD)r1rV)rDrr#s r%_deletez&SurrealDBStore.delete.._deletes<//# # #%8#888 8 $8rM)rrrNrboolrPrQ)r#rDrrZs` r%rTzSurrealDBStore.deletesE  9xS 2 9c 9htn 9{{731&122r6)filterr;kr\c >K|j|||jddd}d}|r=|D]8}t||ttfvr d||d}n||}|d|d|dz }:d |d d |d } |j j | |d {} t| dk(rgS| d} | ddk7r ddlm } | jdd} | | | dDcgc]5}t|dd|di|jdxsi|d|df7c}S7cc}ww)aRun similarity search for query embedding asynchronously and return documents and scores Args: embedding (List[float]): Query embedding. k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with scores score_thresholdr)rr;r]r_'z and metadata.z =  u select id, text, metadata, embedding, vector::similarity::cosine(embedding, $embedding) as similarity from ⟨rub⟩ where vector::similarity::cosine(embedding, $embedding) >= $score_threshold z4 order by similarity desc LIMIT $k; NstatusOK)SurrealExceptionresultz Unknown Errorr:r=r<) page_contentr< similarityr;) rr-typerNr[r!queryrA surrealdb.wsrer )r#r;r]r\rargs custom_filterkey filter_valuerjresultsrfreerrdocs r%(_asimilarity_search_by_vector_with_scorez7SurrealDBStore._asimilarity_search_by_vector_with_scores(//"%zz*;Q?    Is $d 3%&vc{m1#5L&,Sk]L=Sa!HH  Il#$% /   ud33 w<1 I ( t # 5**X7C"3' 'h'  !$V"CIM#''*2E2KML!K    4  s%BDDAD:DDDrjcK|jj|}|j||fd|i|d{Dcgc] \}}}||f c}}}S7cc}}}ww)afRun similarity search asynchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores r\Nr embed_queryrs r#rjr]r\rquery_embeddingdocumentrhrUs r%(asimilarity_search_with_relevance_scoresz7SurrealDBStore.asimilarity_search_with_relevance_scores}$11==eDDdCC#Q/59?  '*az "   !5AAAAAAc dttttfffd }t j |S)aeRun similarity search synchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores rcKjd{jfdid{S7"7wNr\)r1rzr\r]rrjr#sr%(_similarity_search_with_relevance_scoreszhSurrealDBStore.similarity_search_with_relevance_scores.._similarity_search_with_relevance_scoresFsS//# # #FFFq!'+1  $>:><>>rrr floatrPrQ)r#rjr]r\rrs````` r%'similarity_search_with_relevance_scoresz6SurrealDBStore.similarity_search_with_relevance_scores3s;&  (E/ "A   {{CEFFr6cK|jj|}|j||fd|i|d{Dcgc] \}}}||f c}}}S7cc}}}ww)anRun similarity search asynchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores r\Nrurws r%asimilarity_search_with_scorez,SurrealDBStore.asimilarity_search_with_scorePr{r|c dttttfffd }t j |S)amRun similarity search synchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores rcKjd{jfdid{S7"7wr)r1rrsr%_similarity_search_with_scorezRSurrealDBStore.similarity_search_with_score.._similarity_search_with_scoresQ//# # #;;;q!'+1  $rr)r#rjr]r\rrs````` r%similarity_search_with_scorez+SurrealDBStore.similarity_search_with_scorels7& T%%:P5Q   {{8:;;r6c~K|j||fd|i|d{Dcgc]\}}}| c}}S7cc}}ww)adRun similarity search on query embedding asynchronously Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query r\N)rs)r#r;r]r\rryrUs r%asimilarity_search_by_vectorz+SurrealDBStore.asimilarity_search_by_vectors[()V(U(U1)%+)/5)# !Q   # s=5= 7==c ndttffd }tj|S)aURun similarity search on query embedding Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcKjd{jfdid{S7"7wr)r1r)r;r\r]rr#sr%_similarity_search_by_vectorzPSurrealDBStore.similarity_search_by_vector.._similarity_search_by_vectorsQ//# # #:::1%+/5  $rrr rPrQ)r#r;r]r\rrs````` r%similarity_search_by_vectorz*SurrealDBStore.similarity_search_by_vectors-& DN   {{79::r6cK|jj|}|j||fd|i|d{S7w)aDRun similarity search on query asynchronously Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query r\N)rrvr)r#rjr]r\rrxs r%asimilarity_searchz!SurrealDBStore.asimilarity_searchsR$11==eD6T66 Q '- 17    s 5><>c ndttffd }tj|S)a5Run similarity search on query Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcKjd{jfdid{S7"7wr)r1rrsr%_similarity_searchz._similarity_searchsB//# # #000S&SFSS S $Srr)r#rjr]r\rrs````` r%similarity_searchz SurrealDBStore.similarity_searchs0& T$x. T T{{-/00r6?fetch_k lambda_multc:K|j||fd|i|d{}|Dcgc]}|d } }|Dcgc]}|d } }ttj|tj| ||} | D cgc]} | |  c} S7mcc}wcc}wcc} ww)aHReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r\Nr)dtype)r]r)rsr nparrayfloat32) r#r;r]rrr\rrfsubdocsr5 mmr_selectedis r%(amax_marginal_relevance_search_by_vectorz7SurrealDBStore.amax_marginal_relevance_search_by_vectors8EtDD w '- 17   #))3A)))/0#c"g0 01 HHYbjj 1 #  "..AQ..! *0/s6BB B B B B8B; BB Bc vdttffd }tj|S)aIReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcKjd{jfdid{S7$7wr)r1r)r;rr\r]rrr#sr%(_max_marginal_relevance_search_by_vectorzhSurrealDBStore.max_marginal_relevance_search_by_vector.._max_marginal_relevance_search_by_vector6sU//# # #FFF1g{;AEK  $A<A>AAr)r#r;r]rrr\rrs``````` r%'max_marginal_relevance_search_by_vectorz6SurrealDBStore.max_marginal_relevance_search_by_vectors-: X   {{CEFFr6cK|jj|}|j||||fd|i|d{}|S7w)a@Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r\N)rrvr) r#rjr]rrr\rr;rs r%amax_marginal_relevance_searchz-SurrealDBStore.amax_marginal_relevance_search>sZ:++77> BTBB q'; 7= AG    s7AAAc vdttffd }tj|S)a?Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcKjd{jfdid{S7$7wr)r1r)rr\r]rrrjr#sr%_max_marginal_relevance_searchzTSurrealDBStore.max_marginal_relevance_search.._max_marginal_relevance_search}sU//# # #<<<q';7=AG  $rr)r#rjr]rrr\rrs``````` r%max_marginal_relevance_searchz,SurrealDBStore.max_marginal_relevance_searchas-8 d8n   {{9;<$(3 d3i 33 $ 32I ,0 I ;I  I c3h( I  I  eHeS() *I \ ,0     c3h(     eHeO$ % >G ,0 GG G c3h( G  G eHeO$ %G@ ,0     c3h(     eHeO$ % >< ,0 << < c3h( <  < eHeO$ %<< ,0  ;   c3h(     h 8; ,0 ;;; ; c3h( ;  ; h;< ,0     c3h(     h 41 ,0 11 1 c3h( 1  1 h18 ,/,0,/;,/ ,/ ,/  ,/c3h(,/,/ h,/b #G,0#G;#G #G #G  #Gc3h(#G#G h#GP !,0!! ! !  !c3h(!! h!L "=,0"="= "= "=  "=c3h("="= h"=H +/ CyDJ'     > +/ CyDJ'     r6r)rPtypingrrrrrrnumpyrlangchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresr &langchain_community.vectorstores.utilsr rrrYr6r%rs0==-03M q [q r6