hSddlZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZddlmZmZGd d eZGd d eZy) N) TextIOWrapper)Path)AnyDictIteratorListOptionalSequenceUnion)Document) BaseLoader)detect_file_encodings)UnstructuredFileLoadervalidate_unstructured_versionceZdZdZ ddddeeefdeedeedee d eed e d eefd Z d e e fdZded e e fdZy) CSVLoaderaLoad a `CSV` file into a list of Documents. Each document represents one row of the CSV file. Every row is converted into a key/value pair and outputted to a new line in the document's page_content. The source for each document loaded from csv is set to the value of the `file_path` argument for all documents by default. You can override this by setting the `source_column` argument to the name of a column in the CSV file. The source of each document will then be set to the value of the column with the name specified in `source_column`. Output Example: .. code-block:: txt column1: value1 column2: value2 column3: value3 Instantiate: .. code-block:: python from langchain_community.document_loaders import CSVLoader loader = CSVLoader(file_path='./hw_200.csv', csv_args={ 'delimiter': ',', 'quotechar': '"', 'fieldnames': ['Index', 'Height', 'Weight'] }) Load: .. code-block:: python docs = loader.load() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Async load: .. code-block:: python docs = await loader.aload() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Lazy load: .. code-block:: python docs = [] docs_lazy = loader.lazy_load() # async variant: # docs_lazy = await loader.alazy_load() for doc in docs_lazy: docs.append(doc) print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} N)content_columns file_path source_columnmetadata_columnscsv_argsencodingautodetect_encodingrcn||_||_||_||_|xsi|_||_||_y)a Args: file_path: The path to the CSV file. source_column: The name of the column in the CSV file to use as the source. Optional. Defaults to None. metadata_columns: A sequence of column names to use as metadata. Optional. csv_args: A dictionary of arguments to pass to the csv.DictReader. Optional. Defaults to None. encoding: The encoding of the CSV file. Optional. Defaults to None. autodetect_encoding: Whether to try to autodetect the file encoding. content_columns: A sequence of column names to use for the document content. If not present, use all columns that are not part of the metadata. N)rrrrrrr)selfrrrrrrrs m/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/document_loaders/csv_loader.py__init__zCSVLoader.__init__cs=2#* 0   B #6 .returnc#~K t|jd|j5}|j|Ed{dddy7 #1swYyxYw#t$r}|j rt |j}|D]f} t|jd|j5}|j|Ed{7 dddn<#1swYnxYwY#t$rYdwxYwntd|j|Yd}~yYd}~yd}~wt$r}td|j|d}~wwxYww)N)newlinerzError loading ) openrr_CSVLoader__read_fileUnicodeDecodeErrorrr RuntimeError Exception)rcsvfileedetected_encodingsrs r lazy_loadzCSVLoader.lazy_loads; Idnnb4==I 5W++G444 5 54 5 5! M''%:4>>%J" 2!H!! NNBARAR"$'+'7'7'@@@! """" .! !!#^DNN3C#DE1L!" I/?@Aq H IsD="AA A A AD= A  AAD=A D: &D"C)C>C ?CC DC CD C(%D'C((D D= D:D55D::D=r)c#Ktj|fij}t|D]\}} j|jnt j }djfd|jD}||d}jD] } ||||< t||y#t$rtdjdwxYw#t$rtd|dwxYww)NzSource column 'z' not found in CSV file. c 3ZK|]\}}jr|jvrn|jvrt||jn|dt|tr|jn:t|t r)dj ttj|n|yw)Nz: ,)rrstrip isinstancestrlistjoinmap).0kvrs r z(CSVLoader.__read_file..s Aq++---$"7"77#$-QWWYQ7r!!S)GGI"!T*#cii"34 : sB(B+)sourcerowzMetadata column ') page_contentmetadata) csv DictReaderr enumeraterr3rKeyError ValueErrorr5itemsrr ) rr) csv_readerir<r;contentr>cols ` r __read_filezCSVLoader.__read_files0^^G=t}}=  + DFAs ))5**+T^^,ii  IIK G#)3H,, XX$'HHSM X (C CA D  %d&8&8%99QR . X$'8=U%VWWXs42D0B?&>D)NrNNF)__name__ __module__ __qualname____doc__r r3rr r rboolrrr r,rr%rrrrrsPj(,*,#'"&$)/*,/d#/ }/#3- / 4. / 3- /"/"#/BI8H-I*"D="DXh5G"Drrc@eZdZdZ ddededeffd ZdefdZxZ S) UnstructuredCSVLoadera|Load `CSV` files using `Unstructured`. Like other Unstructured loaders, UnstructuredCSVLoader can be used in both "single" and "elements" mode. If you use the loader in "elements" mode, the CSV file will be a single Unstructured Table element. If you use the loader in "elements" mode, an HTML representation of the table will be available in the "text_as_html" key in the document metadata. Examples -------- from langchain_community.document_loaders.csv_loader import UnstructuredCSVLoader loader = UnstructuredCSVLoader("stanley-cups.csv", mode="elements") docs = loader.load() rmodeunstructured_kwargsc Btdt|d||d|y)a Args: file_path: The path to the CSV file. mode: The mode to use when loading the CSV file. Optional. Defaults to "single". **unstructured_kwargs: Keyword arguments to pass to unstructured. z0.6.8)min_unstructured_version)rrQNr)rsuperr)rrrQrR __class__s rrzUnstructuredCSVLoader.__init__s% &wG O94O;NOrr cJddlm}|dd|ji|jS)Nr) partition_csvfilenamer)unstructured.partition.csvrXrrR)rrXs r _get_elementsz#UnstructuredCSVLoader._get_elementss"<QdnnQ8P8PQQr)single) rJrKrLrMr3rrrr[ __classcell__)rVs@rrPrPs<&+3 P P$' PKN PRtRrrP)r?iorpathlibrtypingrrrrr r r langchain_core.documentsr )langchain_community.document_loaders.baser ,langchain_community.document_loaders.helpersr1langchain_community.document_loaders.unstructuredrrrrPrrrresE GGG-@N kD kD\$R2$Rr