LangChain
Load any URL as clean Markdown Documents, or retrieve them straight from a live web search, inside a LangChain pipeline.
pip install langchain-wellmarkedexport WELLMARKED_API_KEY="wm_..."Loader
from langchain_wellmarked import WellMarkedLoader
loader = WellMarkedLoader("https://example.com/article")
docs = loader.load()
docs[0].page_content # clean Markdown
docs[0].metadata # {"source": ..., "title": ..., "author": ..., "retrieved_at": ...}Crawl a whole site instead — one Document per successfully extracted page. Requires Pro or
above.
loader = WellMarkedLoader("https://docs.example.com", mode="crawl", depth=2)
docs = loader.load()
docs[0].metadata["depth"] # how far from the root URL this page sitslazy_load() works too.
| Parameter | Default | Description |
|---|---|---|
url | required | Page to extract, or root URL to crawl from |
api_key | env var | Falls back to WELLMARKED_API_KEY |
mode | "extract" | "extract" (single page) or "crawl" (same-site BFS) |
depth | 1 | Crawl depth — mode="crawl" only |
render_js | False | Render JS-heavy pages with a headless browser (Pro and above) |
job_timeout | 300.0 | Seconds to wait for a crawl job; None waits forever |
In crawl mode, pages that fail to extract — timeouts, robots-disallowed, no content — are
skipped. Only successful pages become Documents.
Retriever
WellMarkedRetriever is the natural surface for RAG chains whose context should come from the
live web. Each retrieval searches, extracts the top results to clean Markdown, and returns them
as Documents — search and extraction in one round trip, with no URLs to wrangle.
Search runs on every plan. What the plan decides is how many results one search may
return — Free 5, Pro 10, Growth 50, Enterprise uncapped. Asking for more than your cap is a
422 search_cap_exceeded rather than a silent trim.
from langchain_wellmarked import WellMarkedRetriever
retriever = WellMarkedRetriever(num_results=5)
docs = retriever.invoke("best open-source vector databases")
docs[0].page_content # clean Markdown of a result page
docs[0].metadata # {"source": ..., "title": ..., "snippet": ...}| Parameter | Default | Description |
|---|---|---|
api_key | env var | Falls back to WELLMARKED_API_KEY |
num_results | 5 | Results to fetch and extract. Capped by plan — Free 5, Pro 10, Growth 50, Enterprise uncapped |