h|addlmZddlZddlZddlZddlZddlmZmZm Z m Z m Z m Z m Z ddlZddlmZddlmZddlmZddlmZddlmZd Zdd Zdd ZGd d eZy)) annotationsN)AnyCallableDictIterableListOptionalType)Document) Embeddings guard_import) VectorStore)maximal_marginal_relevancectdS)zImport lancedb package.lancedbr f/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/lancedb.pyimport_lancedbrs  ""rc ~dj|jDcgc] \}}|d|dc}}Scc}}w)z2Converts a dict filter to a LanceDB filter string.z AND z = '')joinitems)filterkvs rto_lance_filterrs5 <"> ?)D   S'***?*?$@ A8  ||##*??3#7 c3'~~g.+2??dkk,;,(,3??3+?( =   !GJJ117>>3G3G3S3ST$ ")%"8EJJn ...FDK "  @ s A(JJ-c6|jj}d|vrd}n d|vrd}nd}d|v}||sctt|Dcgc]E}t ||j |j |r|d|j niGc}S|r||rytt|Dcgc][}t ||j |j |r|d|j ni|||j f]c}Syycc}wcc}w)N _distance_relevance_scoremetadata) page_contentr_)schemanamesrangelenr r5as_py)rQresultsscorecolumns score_col has_metadataidxs rresults_to_docszLanceDB.results_to_docss>..&& ' !#I 7 **II!W,  E !W.   !(!8!=!C!C!EAMWZ05;;=SU 5!W. %,T^^% MM; < rc|jr |jS|jdk(r |jS|jdk(r |jS|jdk(r |jSt d|jd)a8 The 'correct' relevance function may differ depending on a few things, including: - the distance / similarity metric used by the VectorStore - the scale of your embeddings (OpenAI's are unit normed. Many others are not!) - embedding dimensionality - etc. cosiner'ipzANo supported normalization function for distance metric of type: z=.Consider providing relevance_score_fn to Chroma constructor.)r:r9_cosine_relevance_score_fn_euclidean_relevance_score_fn%_max_inner_product_relevance_score_fnrArns r_select_relevance_score_fnz"LanceDB._select_relevance_score_fns  + +33 3 ==H $22 2 ]]d "55 5 ]]d "== =115@OO rc | |j}|j||f||d|}|j||jddS)zD Return documents most similar to the query vector. rr-rgFrg)r;rrlpop)rQrSrrr-r~ress rsimilarity_search_by_vectorz#LanceDB.similarity_search_by_vectorsQ 9 Adkk)QKvDKFK##Cvzz'5/I#JJrc | |j}|j}|j||fddi|}|D cgc]\}} ||t| fc} }Scc} }w)zZ Return documents most similar to the query vector with relevance scores. rgT)r;rrfloat) rQrSrrr-r~rZdocs_and_scoresdocrgs r1similarity_search_by_vector_with_relevance_scoresz9LanceDB.similarity_search_by_vector_with_relevance_scoress| 9 A!<<>:$:: q $ (. GV 8BUS$U5\2 3   sAc | |j}|jdd}|jdd}|jdd}|j td|dk(s|d k(r|j|j |j |}|j|jd |_|d k(r |jj|} | |f} n|} |j| |f||d |} |j| | Std |jj|} |j| |fd|i|} |j| | S)zAReturn documents most similar to the query with relevance scores.NrgTr-rr"z4search needs an emmbedding function to be specified.ftsr)replacerrz?Full text/ Hybrid search is not supported in LanceDB Cloud yet.r) r;rr2rAr+r<rPcreate_fts_indexr5 embed_queryrrlNotImplementedError) rQrrrr~rgr-rrrSrrs rsimilarity_search_with_scorez$LanceDB.similarity_search_with_scoresZ 9 A 7D)zz&$'ZZ h7 ?? "ST T  *"8||#(?nnT*"%"6"6t~~t"6"T) $ ; ;E BI'/F"F!dkk&!PFPP++Cu+==)U33E:I$++iD6DVDC''5'9 9rc 8|jd|||||dd|}|S)apReturn documents most similar to the query Args: query: String to query the vectorstore with. k: Number of documents to return. filter (Optional[Dict]): Optional filter arguments sql_filter(Optional[string]): SQL filter to apply to the query. prefilter(Optional[bool]): Whether to apply the filter prior to the vector search. Raises: ValueError: If the specified table is not found in the database. Returns: List of documents most similar to the query. F)rrr-rrrgr)r)rQrrr-rrr~rs rsimilarity_searchzLanceDB.similarity_searchs700d// 14Cu PV  rc | |j}|j td|jj|}|j |||||}|S)a?Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. zBFor MMR search, you must specify an embedding function oncreation.) lambda_multr)r;r2rAr'max_marginal_relevance_search_by_vector) rQrrfetch_krrr~rSrs rmax_marginal_relevance_searchz%LanceDB.max_marginal_relevance_searchsm4 9 A ?? "T OO//6 ;;  # <  rc F|jd|||d|}ttj|tj|dj |xs |j |}|j|} t| D cgc] \} } | |vs | } } } | Scc} } w)aHReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. )rrr)dtyper")rrr) rrnparrayfloat32 to_pylistr;rlrw) rQrSrrrrr~rf mmr_selected candidatesirselected_resultss rrz/LanceDB.max_marginal_relevance_search_by_vectorCs6$++     2 HHYbjj 1 H  ' ' )o4::#  ))'2 *3J*?U$!Q1 CTAUUVs  BBc \td||||||| | | | | |d |}|j|||S)N) rRrSrUrVrWrXr+r,r8r9rYrZ)r|r)r!r)clsr{rSr|rRrUrVrWrXr+r,r8r9rYrZr~instances r from_textszLanceDB.from_textsps[& !!!1   5I6rc ^|j|}|r|j|y|r=|j|jdjdj |y|r)|j t d|j|y|r|jdytd)a Allows deleting rows by filtering, by ids or drop columns from the table. Args: filter: Provide a string SQL expression - "{col} {operation} {value}". ids: Provide list of ids to delete from the table. drop_columns: Provide list of columns to drop from the table. delete_all: If True, delete all rows from the table. z in ('{}'),Nz;Column operations currently not supported in LanceDB Cloud.truez6Provide either filter, ids, drop_columns or delete_all) rPdeleter4formatrr+r drop_columnsrA)rQr} delete_allrrr-r~rs rrzLanceDB.deletes$nnT"  JJv   JJ$,,|4;;CHHSMJ K ||')Q  .  JJv UV Vr)rR Optional[Any]rSOptional[Embeddings]rT Optional[str]rUrrVrrWrrXrr+rr,rr8rr1rr9rrYrrZ"Optional[Callable[[float], float]]r;int)F)rfrrgboolreturnr)rr)NN) r{z Iterable[str]r|Optional[List[dict]]r}Optional[List[str]]r~rr List[str])NF)r-rr/Optional[bool]rr)NN`NL2N)rrrrr Optional[int]rrrrrrr-rrNone)rTrBrrB) rrr|rr}rr~rrr)NNN) rrrrrrr-rr~rrr)rzCallable[[float], float]) rS List[float]rrrOptional[Dict[str, str]]r-rr~rrr) rrBrrrrr~rrr)NNNF)rrBrrr-rrrrrr~rrList[Document])Ng?N)rrBrrrrrrrrr~rrr)rSrrrrrrrrrr~rrr) NNr"r#r$r%NNr&r'NN)"rz Type[LanceDB]r{rrSr r|rrRrrUrrVrrWrrXrr+rr,rr8rr9rrYrrZrr~rrr!)NNNNN)r}rrrrrrrr-rr~rrr)__name__ __module__ __qualname____doc__ DEFAULT_Kr[rlpropertyrorrPrrrrrrrrrrr classmethodrrrrrr!r!s:%)*.+$, $"($1!% $)#"&"&AE!VG!VG(VG VG " VG  VG VG"VGVGVGVGVG VG VG?VG !VGp D +/#' ..(.! .  .  .bIN!7E F#'$((+)+*. $"(F(F"(F& (F ' (F ( (F(F(F (FTG+/#' 22(2! 2  2  2n $" $$ $ $  $  $ $L8 +/" KK K) K  K  K K( +/"     )        2 +/ &:&: &:) &:  &: &:V " $#        @  +/ ** * *  * ) ** *^  +/ + +  +  +  + ) + +  + Z +/$($, $"($1!% $)"&"&AE# ###( # " # " ## #"#### # #?# !#" ###N$(%) $,0" !W !W#!W !W * !W  !W!W !Wrr!)rr)rzDict[str, str]rrB) __future__rrr6rsrItypingrrrrrr r numpyrlangchain_core.documentsr langchain_core.embeddingsr langchain_core.utilsrlangchain_core.vectorstoresr&langchain_community.vectorstores.utilsrrrrr!rrrrsO" FFF-0-3M # E Y WkY Wr