Reading and extraction
Article extraction
Also called Content extraction, Boilerplate removal
Article extraction is the process of turning a raw HTML page into a structured article — title, author, publication date, and body text — while discarding navigation, advertising, comments, and related-story modules. It is the step every reader mode, summarizer, and archive depends on.
How extraction works
An extractor parses the document, scores candidate containers by signals such as text density, paragraph count, link ratio, and tag or class names, and selects the subtree most likely to hold the body. Metadata comes from structured data, Open Graph tags, and byline heuristics. The result is plain text or Markdown that no longer depends on the publisher's layout.
The hard part is not the common case; it is knowing when the result is wrong. An extractor can return a cookie notice, a subscribe prompt, or the first three paragraphs of a gated article and report complete success, which is why quality assessment matters as much as parsing.
Where it fails
Client-rendered pages, paginated articles, interactive features, and content assembled from multiple fragments all defeat naive extraction. Pages that gate the body server-side yield nothing at all, no matter how good the parser is.
In smry
smry runs more than one extraction strategy in parallel for a fresh article, waits for the attempts to settle, and serves the strongest readable result rather than whichever one returned first.
Common questions
Why does an extracted article sometimes miss images or captions?
Extractors optimize for the text body. Images loaded lazily, held in scripts, or placed outside the scored container are frequently dropped along with the page furniture.
Is extraction the same as scraping?
Scraping describes collecting data from pages at scale. Extraction is the parsing step that isolates one article's content from one page, and is what browsers do locally in reading view.
Why do two tools return different text for the same URL?
They score containers differently and may fetch different versions of the page. Publishers also vary responses by user agent, region, and traffic source.