h;ddlmZddlZddlmZddlmZddlmZm Z m Z m Z m Z ddl ZddlmZddlmZddlmZdd lmZdd lmZej2eZGd d eZy) ) annotationsN)deepcopy)Enum)AnyIterableListOptionalTuple)Document) Embeddings)run_in_executor) VectorStore)maximal_marginal_relevancec:eZdZdZ d ddZeddZ d ddZe d ddZ Gdde Z d e jdf dd Z d e jdf dd Zd e jdf dd Zd e jdf dd Z d dd d!dZ d" d#dZd$dZd%dZd&d'dZ d& d'dZy)(Rockseta`Rockset` vector store. To use, you should have the `rockset` python package installed. Note that to use this, the collection being used must already exist in your Rockset instance. You must also ensure you use a Rockset ingest transformation to apply `VECTOR_ENFORCE` on the column being used to store `embedding_key` in the collection. See: https://rockset.com/blog/introducing-vector-search-on-rockset/ for more details Everything below assumes `commons` Rockset workspace. Example: .. code-block:: python from langchain_community.vectorstores import Rockset from langchain_community.embeddings.openai import OpenAIEmbeddings import rockset # Make sure you use the right host (region) for your Rockset instance # and APIKEY has both read-write access to your collection. rs = rockset.RocksetClient(host=rockset.Regions.use1a1, api_key="***") collection_name = "langchain_demo" embeddings = OpenAIEmbeddings() vectorstore = Rockset(rs, collection_name, embeddings, "description", "description_embedding") c2 ddlm}t||st dt |||_||_||_||_ ||_ ||_ |j jdy#t$r tdwxYw#t$rYywxYw)aNInitialize with Rockset client. Args: client: Rockset client object collection: Rockset collection to insert docs / query embeddings: Langchain Embeddings object to use to generate embedding for given text. text_key: column in Rockset collection to use to store the text embedding_key: column in Rockset collection to use to store the embedding. Note: We must apply `VECTOR_ENFORCE()` on this column via Rockset ingest transformation. r) RocksetClient]Could not import rockset client python package. Please install it with `pip install rockset`.z;client should be an instance of rockset.RocksetClient, got langchainN)rocksetr ImportError isinstance ValueErrortype_client_collection_name _embeddings _text_key_embedding_key _workspaceset_applicationAttributeError)selfclient embeddingscollection_nametext_key embedding_key workspacers h/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/rocksetdb.py__init__zRockset.__init__1s*  -&-0F|n&   /%!+#  LL ( ( 5) @  *   sA2B 2B BBc|jSN)rr#s r*r%zRockset.embeddingsasNc g}g}t|D]\}} t||k(r||j|z }g}i} |rt||kDrt||} |rt||kDr||| d<| | |j<|j j | | |j<|j| t|dkDr||j|z }g}|S)aRun more texts through the embeddings and add to the vectorstore Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. ids: Optional list of ids to associate with the texts. batch_size: Send documents in batches to rockset. Returns: List of ids from adding the texts into the vectorstore. _idr) enumeratelen_write_documents_to_rocksetrrr embed_queryrappend) r#texts metadatasids batch_sizekwargsbatch stored_idsitextdocs r* add_textszRockset.add_textses(  ' GAt5zZ'd>>uEE CS^a/y|,s3x!| VE "&C '+'7'7'C'CD'IC## $ LL   u:> $::5A AJEr/c  |Jd|sJd|sJd|sJd||||||} | j|||| | S)znCreate Rockset wrapper with existing texts. This is intended as a quicker way to get started. zRockset Client cannot be NonezCollection name cannot be emptyzText key name cannot be emptyzEmbedding key cannot be empty)rA) clsr7 embeddingr8r$r&r'r(r9r:r;rs r* from_textszRockset.from_textssk&!B#BB!A AA888x===}fi(MR%C<r/c eZdZdZdZdZddZy)Rockset.DistanceFunction COSINE_SIMEUCLIDEAN_DIST DOT_PRODUCTc$|jdk(ryy)NrIASCDESC)valuer.s r*order_byz!Rockset.DistanceFunction.order_byszz--r/N)returnstr)__name__ __module__ __qualname__rHrIrJrOr/r*DistanceFunctionrGs! )#  r/rVc `|j|jj||||fi|S)aPerform a similarity search with Rockset Args: query (str): Text to look up documents similar to. distance_func (DistanceFunction): how to compute distance between two vectors in Rockset. k (int, optional): Top K neighbors to retrieve. Defaults to 4. where_str (Optional[str], optional): Metadata filters supplied as a SQL `where` condition string. Defaults to None. eg. "price<=70.0 AND brand='Nintendo'" NOTE: Please do not let end-user to fill this and always be aware of SQL injection. Returns: List[Tuple[Document, float]]: List of documents with their relevance score )1similarity_search_by_vector_with_relevance_scoresrr5r#queryk distance_func where_strr;s r*'similarity_search_with_relevance_scoresz/Rockset.similarity_search_with_relevance_scoressA2FtEE    ( ( /       r/c `|j|jj||||fi|S)zaSame as `similarity_search_with_relevance_scores` but doesn't return the scores. )similarity_search_by_vectorrr5rZs r*similarity_searchzRockset.similarity_searchs@0t//    ( ( /       r/c d|j||||fi|}|Dcgc]\}}| c}}Scc}}w)zZAccepts a query_embedding (vector), and returns documents with similar embeddings.)rY) r#rDr\r]r^r;docs_and_scoresr@_s r*raz#Rockset.similarity_search_by_vectorsBQ$PP q- 6< #22Q222s ,c d}d|vr|d}|j|||||} |jjjd|i}g} |jD]} i} t| tsJdjt| | jD]\}} ||jk(r=t| ts*Jdj|jt| | }R|d k(r2t| tsJd jt| | }|d vs| | |<| j!t#| f| S#t$r"} t j d| gcYd} ~ Sd} ~ wwxYw) z|Accepts a query_embedding (vector), and returns documents with similar embeddings along with their relevance scores.Texclude_embeddingsr[)sqlz$Exception when querying Rockset: %s Nz;document should be of type `dict[str,Any]`. But found: `{}`zIpage content stored in column `{}` must be of type `str`. But found: `{}`distzDComputed distance between vectors must of type `float`. But found {})r1 _event_time_meta) page_contentmetadata)_build_query_sqlrQueriesr[ ExceptionloggererrorresultsrdictformatritemsrrQfloatr6r )r#rDr\r]r^r;rgq_strquery_responsee finalResultdocumentrmvrlscores r*rYz9Rockset.similarity_search_by_vector_with_relevance_scoress" 6 )!'(E93E>9E>)r^c r|jj|}|j|f||dd|}|D cgc]} | j|j} } t t j|| ||} | D]} || j|j=| D cgc]} ||  c} Scc} wcc} w)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. distance_func (DistanceFunction): how to compute distance between two vectors in Rockset. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. where_str: where clause for the sql query Returns: List of Documents selected by maximal marginal relevance. F)r\r^rg) lambda_multr\)rr5rarmrrnparray) r#r[r\fetch_krr^r;query_embedding initial_docsr@r%selected_indicesr>s r*max_marginal_relevance_searchz%Rockset.max_marginal_relevance_search0s:**66u=7t77  $    DPPCcll4#6#67P P5 HH_ % #  " >AQ(()<)<= >*::A Q::Q;s B/ B4c@djtt|}|jd|jd|d}|rd|dnd}|rd|jd nd}d |d |d |j d |j d|d|jdt|dS)zABuilds Rockset SQL query to query similar vectors to query_vector,(z, [z ]) as distzWHERE  z EXCEPT(z),zSELECT * z FROM .zORDER BY dist z LIMIT )joinmaprQrNrr rrO) r#rr]r\r^rgq_embedding_str distance_strselect_embeddings r*rnzRockset._build_query_sqlhs((3sO#<=)//0$2E2E1FG* .7fYKr*R 2Dht**+2 .#  !L>* ooa--./  %%'() 1vh  r/c|jjj|j||j}|j Dcgc]}|j c}Scc}w)N collectiondatar))r Documents add_documentsrr rr1)r#r< add_doc_res doc_statuss r*r4z#Rockset._write_documents_to_rocksetsRll,,::,,5DOO; 2=1A1AB: BBBs A!c  ddlm}|jjj |j |Dcgc] }|| c}|jy#t$r tdwxYwcc}w)z1Delete a list of docs from the Rockset collectionr)DeleteDocumentsRequestDatar)idrN)rockset.modelsrrrrdelete_documentsrr )r#r9rr>s r* delete_textszRockset.delete_textssv  A //,,rsK"77-043M   8 $RGkRGr/