h//ddlmZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z ddlZddlmZddlmZddlmZej*eZGdd eZy) ) annotationsN)ThreadPoolExecutor)AnyIterableListOptionalTuple)Document) Embeddings) VectorStorec$eZdZdZ d d dZ d d dZ d d dZ d ddZe d ddZ e d ddZ y)Clarifaia`Clarifai AI` vector store. To use, you should have the ``clarifai`` python SDK package installed. Example: .. code-block:: python from langchain_community.vectorstores import Clarifai clarifai_vector_db = Clarifai( user_id=USER_ID, app_id=APP_ID, number_of_docs=NUMBER_OF_DOCS, ) Nc>|xstjjd}|xstjjd}|| td||_ ddlm} | ||||||j|_ y#t$r} td| d} ~ wwxYw) aInitialize with Clarifai client. Args: user_id (Optional[str], optional): User ID. Defaults to None. app_id (Optional[str], optional): App ID. Defaults to None. pat (Optional[str], optional): Personal access token. Defaults to None. token (Optional[str], optional): Session token. Defaults to None. number_of_docs (Optional[int], optional): Number of documents to return during vector search. Defaults to None. api_base (Optional[str], optional): API base. Defaults to None. Raises: ValueError: If user ID, app ID or personal access token is not provided. CLARIFAI_USER_IDCLARIFAI_APP_IDNzCould not find CLARIFAI_USER_ID or CLARIFAI_APP_ID in your environment. Please set those env variables with a valid user ID, app IDrSearchXCould not import clarifai python package. Please install it with `pip install clarifai`.)user_idapp_idtop_kpattokenbase_url) osenvironget ValueError_number_of_docsclarifai.client.searchr ImportError auth_helper_auth) selfrrnumber_of_docsrrapi_base_user_id_app_idres g/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/vectorstores/clarifai.py__init__zClarifai.__init__#s.@bjjnn-?@=BJJNN+<=  wN  .  5    +  A  sB B BBc  ddlm}ddlm}t |}t |} | dkDsJd|| t |k(sJd|t |t |k(sJd|j|j } d } g} td| | D]} || | | z}|r|| | | znd}|>tt |Dcgc] }tjj"}}n|| | | z}|2g}|D]+}|}|j||j|-t|Dcgc]#\}}| j!||||r|nd %}}}| j#| }| j%|t&j)d | S#t$r}t d|d}~wwxYwcc}wcc}}w#t*$r7}t&j-d|t/j0Yd}~nd}~wwxYw)a'Add texts to the Clarifai vectorstore. This will push the text to a Clarifai application. Application use a base workflow that create and store embedding for each text. Make sure you are using a base workflow that is compatible with text (such as Language Understanding). Args: texts (Iterable[str]): Texts to add to the vectorstore. metadatas (Optional[List[dict]], optional): Optional list of metadatas. ids (Optional[List[str]], optional): Optional list of IDs. r)Inputs)StructrNz,No texts provided to add to the vectorstore.z1Number of texts and metadatas should be the same.z7Number of text inputs and input ids should be the same.)auth )input_idraw_textmetadata)inputszInput posted successfully.zPost inputs failed: )clarifai.client.inputr-google.protobuf.struct_pb2r.r!listlenfrom_auth_helperr#rangeuuiduuid4hexupdateappend enumerateget_text_input upload_inputsextendloggerdebug Exceptionwarning traceback print_exc)r$texts metadatasidskwargsr-r.r)ltextslength input_obj batch_size input_job_idsidx batch_textsbatch_metadatas_ batch_ids meta_listmeta meta_structitext input_batch result_iderrors r* add_textszClarifai.add_textsUsU&  4 9eVzIIIz  S^+ C + ?v;#c(* I *+++<   FJ/ &C &$S3+;< 9BIcC*$45 ;;@[AQ;R Sa!1!1 SI S #C# *: ;I". "I /6&,h #**40!((56$-[#9   4 ,,!*1!%1@1d- &33;3G $$Y/ 9:5 &@o A  <!T &!5eW=>##%% &sN F,G%F<*A G7(G;G F9( F44F9< G H,HHc * ddlm}ddlm}ddlm|xs j}|jj|} d|ig} |d |i} | j| | g } n| j| } | D cgc]} | jD]}|}} }td }dfd }|Dcgc]}|j||}}|Dcgc]}|j}}|S#t $r}t d|d}~wwxYwcc}} wcc}wcc}w)aRun similarity search with score using Clarifai. Args: query (str): Query text to search for. k (Optional[int]): Number of results to return. If not set, it'll take _number_of_docs. Defaults to None. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List[Document]: List of documents most similar to the query text. rr) resources_pb2) json_formatrN)r/rtext_rawr3)ranksfilters)re ) max_workersc 8j|jjj}t j j}t j|jjjj|}|j|_ |j}tjd|jdd|jj d|jj d|ddt#|||jfS) N)headersz Score z.2fz for annotation: z off input: z, text: }) page_contentr3) MessageToDictinputdatar3dictr#requestsrr\urlapparent_encodingencodingrDrEscore annotationidr )hitr3hrequestrequested_textrcr$s r*hit_to_documentz>Clarifai.similarity_search_with_score..hit_to_documents"001H1HIHTZZ(()All399>>#6#6#:#:AFG '88G $\\N LL399S/):3>>;L;L:MNIILL>.#2F1GI .8LciiX X)rxzresources_pb2.HitreturnzTuple[Document, float])r rclarifai_grpc.grpc.apirbgoogle.protobufrcr!rr9r#queryhitsrsubmitresult)r$rkrfrMrrbr)r search_objranksearch_metadatasearch_responserorxrexecutorr|futuresfuturedocs_and_scoresrcs` @r*similarity_search_with_scorez%Clarifai.similarity_search_with_scores3&  5 < 3)T)),,$**E,J U#$  )73O(..TOCT.UO(..T.:O!0E499ECEEE%"5 Y EIIS8???C8II9@Av6==?AAS A  &F&JAs)C(:D.D  D( D1 C==Dc b|j|fd|i|}|Dcgc]\}}| c}}Scc}}w)aGRun similarity search using Clarifai. Args: query: Text to look up documents similar to. k: Number of Documents to return. If not set, it'll take _number_of_docs. Defaults to None. Returns: List of Documents most similar to the query and score for each r)r)r$rrrMrdocrVs r*similarity_searchzClarifai.similarity_searchs8 <$;;EQQQ&Q"12Q222s +c H|d|||||d| } | j||| S)aCreate a Clarifai vectorstore from a list of texts. Args: user_id (str): User ID. app_id (str): App ID. texts (List[str]): List of texts to add. number_of_docs (Optional[int]): Number of documents to return during vector search. Defaults to None. pat (Optional[str], optional): Personal access token. Defaults to None. token (Optional[str], optional): Session token. Defaults to None. metadatas (Optional[List[dict]]): Optional list of metadatas. Defaults to None. kwargs: Additional keyword arguments to be passed to the Search. Returns: Clarifai: Clarifai vectorstore. )rrr%rr)rJrK)r`) clsrJ embeddingrKrrr%rrrMclarifai_vector_dbs r* from_textszClarifai.from_textssH>! )     $$5I$F!!r}c |D cgc]} | j} } |D cgc]} | j} } |jd||| ||| |d|Scc} wcc} w)aCreate a Clarifai vectorstore from a list of documents. Args: user_id (str): User ID. app_id (str): App ID. documents (List[Document]): List of documents to add. number_of_docs (Optional[int]): Number of documents to return during vector search. Defaults to None. pat (Optional[str], optional): Personal access token. Defaults to None. token (Optional[str], optional): Session token. Defaults to None. kwargs: Additional keyword arguments to be passed to the Search. Returns: Clarifai: Clarifai vectorstore. )rrrJr%rrKrr)rlr3r) r documentsrrrr%rrrMrrJrKs r*from_documentszClarifai.from_documents#ss6.77c!!77-67cS\\7 7s~~  )    87s A A)NNNNzhttps://api.clarifai.com)r Optional[str]rrr% Optional[int]rrrrr&rr~None)NN) rJz Iterable[str]rKOptional[List[dict]]rLzOptional[List[str]]rMrr~ List[str]) rstrrrrfzOptional[dict]rMrr~zList[Tuple[Document, float]])N)rrrrrMrr~List[Document])NNNNNNN)rJrrOptional[Embeddings]rKrrrrrr%rrrrrrMrr~r)NNNNNN)rrrrrrrrr%rrrrrrMrr~r) __name__ __module__ __qualname____doc__r+r`rr classmethodrrrr}r*rrs+$"& $()!#"<000& 0  0  0 0 0j+/#' MM(M! M  M  Md "& @@ @ @  @ & @J 33 3 3  3&+/*.!% $(,!#'"'"('"( '"  '"  '"&'"'"'"'" '"'"R+/!% $(,!#% !% (%  %  % & % % % %  % % r}r) __future__rloggingrrHr;concurrent.futuresrtypingrrrrr rqlangchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresr getLoggerrrDrrr}r*rsI"  177-03   8 $w {w r}