SDKs & integrations
LlamaIndex
A reader that loads any URL as clean Markdown Documents, crawls a site, or pulls results straight from a live web search.
pip install llama-index-readers-wellmarkedexport WELLMARKED_API_KEY="wm_..."Load a page
from llama_index.readers.wellmarked import WellMarkedReader
reader = WellMarkedReader()
docs = reader.load_data("https://example.com/article")
docs[0].text # clean Markdown
docs[0].metadata # {"source": ..., "title": ..., "author": ..., "retrieved_at": ...}Crawl a site
One Document per successfully extracted page. Requires Pro or above.
reader = WellMarkedReader(mode="crawl", depth=2)
docs = reader.load_data("https://docs.example.com")
docs[0].metadata["depth"] # how far from the root URL this page sitsSearch instead of naming URLs
Search plus extraction in one round trip. Runs on every plan — the plan caps how many results one search returns (Free 5, Pro 10, Growth 50, Enterprise uncapped), not whether you can search at all.
reader = WellMarkedReader(num_results=5)
docs = reader.search("best open-source vector databases")
docs[0].text # clean Markdown of a result page
docs[0].metadata # {"source": ..., "title": ..., "snippet": ...}Straight into an index
from llama_index.core import VectorStoreIndex
index = VectorStoreIndex.from_documents(docs)Options
All options are constructor arguments. load_data(url) takes a URL for extract and crawl;
search(query) takes a query.
| Parameter | Default | Description |
|---|---|---|
api_key | env var | Falls back to WELLMARKED_API_KEY |
mode | "extract" | "extract" (single page) or "crawl" (same-site BFS) |
depth | 1 | Crawl depth — mode="crawl" only |
render_js | False | Render JS-heavy pages with a headless browser (Pro and above) |
job_timeout | 300.0 | Seconds to wait for a crawl job; None waits forever |
num_results | 5 | Results for search(). Capped by plan — Free 5, Pro 10, Growth 50, Enterprise uncapped |
In crawl mode, pages that fail to extract — timeouts, robots-disallowed, no content — are
skipped, and the same applies to search(). Only successful pages become Documents.