h FddlmZddlZddlZddlZddlmZddlmZddl m Z m Z m Z m Z mZmZmZddlZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lm Z e!gd Z"dZ#ddZ$GddeZ%y)) annotationsN) ConfigParser)Path)AnyCallableDictIterableListOptionalTuple)Document) Embeddings guard_import) VectorStore)Docstore)InMemoryDocstore)maximal_marginal_relevance)angular euclidean manhattanhammingdotrctdS)z1Import annoy if available, otherwise raise error.annoyrd/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/annoy.pydependable_annoy_importrs   rcBeZdZdZ ddZeddZ d ddZ ddZ d ddZ d dd Z d d d Z d d!d Z d d"d Z d d#d Z d$ d%dZ d$ d&dZededdf d'dZededdf d(dZededdf d)dZd*d+dZedd d,dZy)-Annoya`Annoy` vector store. To use, you should have the ``annoy`` python package installed. Example: .. code-block:: python from langchain_community.vectorstores import Annoy db = Annoy(embedding_function, index, docstore, index_to_docstore_id) cJ||_||_||_||_||_y)z%Initialize with necessary components.N)embedding_functionindexmetricdocstoreindex_to_docstore_id)selfr#r$r%r&r's r__init__zAnnoy.__init__*s)#5    $8!rcyNr)r(s r embeddingszAnnoy.embeddings9srNc td)Nz=Annoy does not allow to add new data once the index is build.)NotImplementedError)r(texts metadataskwargss r add_textszAnnoy.add_texts>s " K  rcg}t||D]c\}}|j|}|jj|}t |t st d|d||j||fe|S)aTurns annoy results into a list of documents and scores. Args: idxs: List of indices of the documents in the index. dists: List of distances of the documents in the index. Returns: List of Documents and scores. Could not find document for id , got )zipr'r&search isinstancer ValueErrorappend)r(idxsdistsdocsidxdist_iddocs rprocess_index_resultszAnnoy.process_index_resultsHsT5) %IC++C0C--&&s+Cc8, #B3%vcU!STT KKd $  %  rcj|jj|||d\}}|j||Sa}Return docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. search_k: inspect up to search_k nodes which defaults to n_trees * n if not provided Returns: List of Documents most similar to the query and score for each Tsearch_kinclude_distances)r$get_nns_by_vectorrB)r( embeddingkrGr;r<s r&similarity_search_with_score_by_vectorz,Annoy.similarity_search_with_score_by_vector\s?jj22 q8t3 e))$66rcj|jj|||d\}}|j||SrE)r$get_nns_by_itemrB)r(docstore_indexrKrGr;r<s r%similarity_search_with_score_by_indexz+Annoy.similarity_search_with_score_by_indexns?jj00 AD1 e))$66rcN|j|}|j|||}|S)a~Return docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. search_k: inspect up to search_k nodes which defaults to n_trees * n if not provided Returns: List of Documents most similar to the query and score for each )r#rL)r(queryrKrGrJr=s rsimilarity_search_with_scorez"Annoy.similarity_search_with_scores-++E2 ::9aR rc ^|j|||}|Dcgc]\}}| c}}Scc}}w)aReturn docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. search_k: inspect up to search_k nodes which defaults to n_trees * n if not provided Returns: List of Documents most similar to the embedding. )rL)r(rJrKrGr1docs_and_scoresrA_s rsimilarity_search_by_vectorz!Annoy.similarity_search_by_vectors5EE q( #22Q222 )c ^|j|||}|Dcgc]\}}| c}}Scc}}w)azReturn docs most similar to docstore_index. Args: docstore_index: Index of document in docstore k: Number of Documents to return. Defaults to 4. search_k: inspect up to search_k nodes which defaults to n_trees * n if not provided Returns: List of Documents most similar to the embedding. )rP)r(rOrKrGr1rUrArVs rsimilarity_search_by_indexz Annoy.similarity_search_by_indexs5DD Ax #22Q222rXc ^|j|||}|Dcgc]\}}| c}}Scc}}w)alReturn docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. search_k: inspect up to search_k nodes which defaults to n_trees * n if not provided Returns: List of Documents most similar to the query. )rS)r(rRrKrGr1rUrArVs rsimilarity_searchzAnnoy.similarity_searchs0;;E1hO"12Q222rXc |jj||dd}|Dcgc]}|jj|}}tt j |gtj |||} | Dcgc] }|dk7s ||} }g} | D]^}|j|} |jj| } t| tstd| d| | j| `| Scc}wcc}w)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. fetch_k: Number of Documents to fetch to pass to MMR algorithm. k: Number of Documents to return. Defaults to 4. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. rCFrF)dtype)rK lambda_multr4r5)r$rIget_item_vectorrnparrayfloat32r'r&r7r8r r9r:)r(rJrKfetch_kr_r1r;ir, mmr_selectedselected_indicesr=r@rAs r'max_marginal_relevance_search_by_vectorz-Annoy.max_marginal_relevance_search_by_vectors 2zz++ wu, >BBdjj003B B1 HHi[ 3 #  .:EQ"WDGEE! A++A.C--&&s+Cc8, #B3%vcU!STT KK     #CFs"C:? C? C?c R|j|}|j||||}|S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. )r_)r#rh)r(rRrKrdr_r1rJr=s rmax_marginal_relevance_searchz#Annoy.max_marginal_relevance_searchs90++E2 ;; q'{<  rdc |tvrtd|dtttd} |s tdt |d} | j | |} t |D]\} } | j| | | j||g}t |D]*\} }|r|| ni}|jt||,tt |D cic] } | ttj"}} tt |D cic] \} }|| | c}} }||j | |||Scc} wcc}} w) NzUnsupported distance metric: z. Expected one of rz/embeddings must be provided to build AnnoyIndexrr%)n_jobs) page_contentmetadata) INDEX_METRICSr9listrlen AnnoyIndex enumerateadd_itembuildr:r rangestruuiduuid4r embed_query)clsr/r,rJr0r%treesrnr1rfr$reemb documentstextrp index_to_idrAr&s r__fromz Annoy.__fromsf  &3F8<''+M':&;=  W%NO O  1   6 2 + #FAs NN1c " # E& )  ' MGAt'0y|bH   X4(K L M6;3y>5JKq#djjl++K K#/8/C DVQ[^S D 9((%;OO L Ds (%E"E c V|j|}|j|||||||fi|S)aConstruct Annoy wrapper from raw documents. Args: texts: List of documents to index. embedding: Embedding function to use. metadatas: List of metadata dictionaries to associate with documents. metric: Metric to use for indexing. Defaults to "angular". trees: Number of trees to use for indexing. Defaults to 100. n_jobs: Number of jobs to use for indexing. Defaults to -1. This is a user friendly interface that: 1. Embeds documents. 2. Creates an in memory docstore 3. Initializes the Annoy database This is intended to be a quick way to get started. Example: .. code-block:: python from langchain_community.vectorstores import Annoy from langchain_community.embeddings import OpenAIEmbeddings embeddings = OpenAIEmbeddings() index = Annoy.from_texts(texts, embeddings) )embed_documents _Annoy__from) r}r/rJr0r%r~rnr1r,s r from_textszAnnoy.from_texts<s?H..u5 szz :y)VUF NT  rc |Dcgc]}|d } }|Dcgc]}|d } }|j| | |||||fi|Scc}wcc}w)aConstruct Annoy wrapper from embeddings. Args: text_embeddings: List of tuples of (text, embedding) embedding: Embedding function to use. metadatas: List of metadata dictionaries to associate with documents. metric: Metric to use for indexing. Defaults to "angular". trees: Number of trees to use for indexing. Defaults to 100. n_jobs: Number of jobs to use for indexing. Defaults to -1 This is a user friendly interface that: 1. Creates an in memory docstore with provided embeddings 2. Initializes the Annoy database This is intended to be a quick way to get started. Example: .. code-block:: python from langchain_community.vectorstores import Annoy from langchain_community.embeddings import OpenAIEmbeddings embeddings = OpenAIEmbeddings() text_embeddings = embeddings.embed_documents(texts) text_embedding_pairs = list(zip(texts, text_embeddings)) db = Annoy.from_embeddings(text_embedding_pairs, embeddings) r)r) r}text_embeddingsrJr0r%r~rnr1tr/r,s rfrom_embeddingszAnnoy.from_embeddingseseJ //!1//$34qad4 4szz :y)VUF NT  04s > AFct|}tj|dt}|jj |j d|d<|jjt|dz |t|dz d5}tj|j|j|f|d d d y #1swYy xYw) aSave Annoy index, docstore, and index_to_docstore_id to disk. Args: folder_path: folder path to save index, docstore, and index_to_docstore_id to. prefault: Whether to pre-load the index into memory. T)exist_ok)rr%ANNOY index.annoy)prefault index.pklwbN)rosmakedirsrr$rr%saveryopenpickledumpr&r')r( folder_pathrpath config_objectfiles r save_localzAnnoy.save_localsK  D4($ kk" g D=01HE $$d + Yt KK(A(A=QSW X Y Y Ys .CC )allow_dangerous_deserializationc|s tdt|}td}t|dz d5}t j |\}}} dddt  dd} | dd} |j| | } | j t|d z ||j| | S#1swYhxYw) aRLoad Annoy index, docstore, and index_to_docstore_id to disk. Args: folder_path: folder path to load index, docstore, and index_to_docstore_id from. embeddings: Embeddings to use when generating queries. allow_dangerous_deserialization: whether to allow deserialization of the data which involves loading a pickle file. Pickle files can be modified by malicious actors to deliver a malicious payload that results in execution of arbitrary code on your machine. aBThe de-serialization relies loading a pickle file. Pickle files can be modified to deliver a malicious payload that results in execution of arbitrary code on your machine.You will need to set `allow_dangerous_deserialization` to `True` to enable deserialization. If you do this, make sure that you trust the source of the data. For example, if you are loading a file that you created, and know that no one else has modified the file, then this is safe to do. Do not set this to `True` if you are loading a file from an untrusted source (e.g., some random site on the internet.).rrrbNrrr%rmr) r9rrrrloadintrtryr|) r}rr,rrrrr&r'rrr%r$s r load_localzAnnoy.load_locals(/ "  K W% $$d + t    $    g&s+ ,w'1  6 2 3tm+,-  " "E68=Q  !  s B44B=) r#rr$rr%ryr&rr'zDict[int, str])returnzOptional[Embeddings]r+)r/z Iterable[str]r0Optional[List[dict]]r1rr List[str])r;z List[int]r< List[float]rList[Tuple[Document, float]])rC)rJrrKrrGrrr)rOrrKrrGrrr)rRryrKrrGrrr) rJrrKrrGrr1rrList[Document]) rOrrKrrGrr1rrr) rRryrKrrGrr1rrr)rg?) rJrrKrrdrr_floatr1rrr) rRryrKrrdrr_rr1rrr)r/rr,zList[List[float]]rJrr0rr%ryr~rrnrr1rrr!)r/rrJrr0rr%ryr~rrnrr1rrr!)rzList[Tuple[str, List[float]]]rJrr0rr%ryr~rrnrr1rrr!)F)rryrboolrNone)rryr,rrrrr!)__name__ __module__ __qualname____doc__r)propertyr,r2rBrLrPrSrWrZr\rhrj classmethodDEFAULT_METRICrrrrrrrrr!r!s  9$ 9 9 9  9 - 9+/  (     &1 %*CE7$7),7rs[" %GGG-0-36CMQR !  K r