hddlZddlZddlmZddlmZmZmZddl m Z ddl m Z ejeZGdde Zy)N)Path)DictIteratorUnion)Document) BaseLoaderc deZdZdZ d deeefdeedfdeedfdeddf dZde e fd Z y) BSHTMLLoaderaS __ModuleName__ document loader integration Setup: Install ``langchain-community`` and ``bs4``. .. code-block:: bash pip install -U langchain-community bs4 Instantiate: .. code-block:: python from langchain_community.document_loaders import BSHTMLLoader loader = BSHTMLLoader( file_path="./example_data/fake-content.html", ) Lazy load: .. code-block:: python docs = [] docs_lazy = loader.lazy_load() # async variant: # docs_lazy = await loader.alazy_load() for doc in docs_lazy: docs.append(doc) print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Test Title My First Heading My first paragraph. {'source': './example_data/fake-content.html', 'title': 'Test Title'} Async load: .. code-block:: python docs = await loader.aload() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Test Title My First Heading My first paragraph. {'source': './example_data/fake-content.html', 'title': 'Test Title'} N file_path open_encoding bs_kwargsget_text_separatorreturnc ddl}||_||_|.tj j ds tdddi}||_||_y#t$r tdwxYw)ainitialize with path, and optionally, file encoding to use, and any kwargs to pass to the BeautifulSoup object. Args: file_path: The path to the file to load. open_encoding: The encoding to use when opening the file. bs_kwargs: Any kwargs to pass to the BeautifulSoup object. get_text_separator: The separator to use when calling get_text on the soup. rNzUbeautifulsoup4 package not found, please install it with `pip install beautifulsoup4`lxmlzBy default BSHTMLLoader uses the 'lxml' package. Please either install it with `pip install -U lxml` or pass in init arg `bs_kwargs={'features': '...'}` to overwrite the default BeautifulSoup kwargs.features) bs4 ImportErrorr r importlibutil find_specr r)selfr r r rrs j/var/www/html/eduruby.in/venv/lib/python3.12/site-packages/langchain_community/document_loaders/html_bs.py__init__zBSHTMLLoader.__init__Ss   #*  >>++F3!, $V,I""4% /  s AA(c#Kddlm}t|jd|j5}||fi|j }dddj |j}|jr t|jj}nd}t|j|d}t||y#1swYzxYww) z)Load HTML document into document objects.r) BeautifulSoupr)encodingN)sourcetitle) page_contentmetadata) rropenr r r get_textrr!strstringr)rrfsouptextr!r#s r lazy_loadzBSHTMLLoader.lazy_loadys% $..#0B0B C 6q 5dnn5D 6}}T445 :: ))*EE$..)1 D8<< 6 6s)CB8A9C8C=C)NNr) __name__ __module__ __qualname____doc__rr&rdictrrrr+rr r svCP+/'+"$ $5d#$5S$Y'$5t$ $5  $5  $5L=8H-=r2r )importlib.utilrloggingpathlibrtypingrrrlangchain_core.documentsr)langchain_community.document_loaders.baser getLoggerr,loggerr r1r2rr;s:((-@   8 $~=:~=r2