HTML Extract

Pull values out of HTML with CSS selectors, from one value to a whole list of rows.

Try it

workflow · web-scrape-to-json.awf 4 steps

Web Scrape to JSON: a workflow with 4 steps, starting with Run now and ending with Show the data.

1. 1 Run now(When Started — open its doc page) When Started
2. 2 Fetch the page(Get HTML from Link — open its doc page) Get HTML from Link
3. 3 Pull out the items(HTML Extract — open its doc page) HTML Extract
4. 4 Show the data(Display Object — open its doc page) Display Object

Open in AWFlow Copy .awf

What it does

HTML Extract reads the HTML you put in its HTML field and finds elements in it with CSS selectors. It has two modes:

• Single value finds the first element that matches Selector and returns its text, its inner HTML or one of its attributes.
• Repeating list finds every element that matches Container Selector (each search result, table row or card) and builds one item per match, reading each of your Fields from inside that container.

Text is trimmed of surrounding spaces. Returned HTML is cleaned of scripts and other unsafe content. The HTML is only read, never run, and it can be up to about 2 MB.

When to use it

• Pull a title, price or link out of a page you fetched.
• Turn a listing page (search results, a table, a feed) into one item per row with clean fields.
• To read straight from the page open in your browser, Get Element Data may be simpler. To match text patterns rather than page structure, use Regex.

Inputs and settings

Setting	Type	What it does
HTML	Long text	The HTML to extract from, usually bound from a previous node.
Mode	Choice	Read one value, or one item per matched container. Options: Single value, Repeating list. Default: Single value
Selector	Text	CSS selector for the element to read (single value mode).
Return	Choice	Text, inner HTML, or an attribute of the matched element. Options: Text, HTML, Attribute. Default: Text
Attribute	Text	Attribute to read when Return is set to Attribute.
Container Selector	Text	CSS selector matching each repeating container (list mode).
Fields	List of entries	One mapping per column: name, selector, what to return (list mode).
Fields (each entry) › Field name REQ(required)	Text
Fields (each entry) › Selector REQ(required)	Text
Fields (each entry) › Return	Choice	Options: Text, HTML, Attribute. Default: Text
Fields (each entry) › Attribute	Text

• HTML: bind it with an expression. After an Http Request, the page is in {{ $input.data }}.
• Selector and Container Selector take standard CSS selectors, for example h1.title, #price or ul.results > li.
• In list mode, each field’s Selector is searched inside its container, so keep it short, for example a.title rather than the full path.
• Attribute names the attribute to read, for example href for links or src for images.

Outputs

• Output

• Single value returns one item, { "value": ... }. value is empty (undefined) when nothing matched or the element lacks the attribute.
• Repeating list returns one item per matched container, with one field per entry in Fields. The next node runs once per row. If no container matches, there are no items.

Dependencies and credentials

None. This node needs no connection and has no slots for other nodes.

Example workflow

1. Http Request with URL set to a search results page.
2. HTML Extract with HTML {{ $input.data }}, Mode Repeating list and Container Selector ul.results > li.
3. Under Fields, add title (Selector a.title, Return Text) and link (Selector a.title, Return Attribute, Attribute href).
4. Limit keeps the first 10 rows for the next steps.

Common issues

• “No selector given. Fill in the Selector field.” or “No container selector given. Fill in the Container Selector field.” Fill in the selector for the mode you chose.
• “Invalid CSS selector: “…”.“ The selector has a typo. Test it in your browser’s developer tools first.
• “Return is set to Attribute but no attribute name was given.” Fill in Attribute, for example href.
• “Input is … bytes, larger than the 2000000-byte limit this node will parse.” The HTML is over about 2 MB. Extract a smaller part of the page first.
• The value is empty though the element is visible in the browser. The page builds that part with scripts after it loads, so it is not in the fetched HTML. Read it from the live page with Get All HTML instead.

Recipes with this node

Turn a list on a web page into structured data intermediate · 7 min No setup

Related nodes

• Http Request: fetch the HTML to extract from.
• Get HTML from Link: get the HTML of a page by its URL.
• Markdown: turn HTML into readable Markdown.
• Regex: match text patterns instead of page structure.