Get HTML from Link

Download the HTML source of a web page by its address, without opening it in a tab.

Try it

Workflow · web-scrape-to-json.awf 4 Schritte

Web Scrape to JSON: ein Workflow mit 4 Schritten, beginnend mit Run now und endend mit Show the data.

1. 1 Run now(When Started – zur Dokumentationsseite) When Started
2. 2 Fetch the page(Get HTML from Link – zur Dokumentationsseite) Get HTML from Link
3. 3 Pull out the items(HTML Extract – zur Dokumentationsseite) HTML Extract
4. 4 Show the data(Display Object – zur Dokumentationsseite) Display Object

In AWFlow öffnen .awf kopieren

What it does

Get HTML from Link downloads the page at URL and returns its HTML exactly as the server sent it, as text, in the field named by Output field name.

Scripts on the page do not run, so anything a site adds afterwards with JavaScript is not in the result. Use it on pages whose content is in the HTML itself, such as articles, listings and simple shops.

When to use it

• Pull specific values (a price, a title, a table) out of a page with HTML Extract.
• Save a copy of a page’s source.
• Check a page for a meta tag or a script without opening it.

For just the readable text, use Get All Text from Link. To read the HTML of the page open in your tab, use Get All HTML.

Inputs and settings

Einstellung	Typ	Was es bewirkt
URL PFLICHT(erforderlich)	Text	The URL to get the HTML from.
Output field name PFLICHT(erforderlich)	Text	The name of the output field.
Timeout	Number	The timeout for the request. Standard: 10000
Options	Group	The options for the node.
Options › Headers	Key-value pairs	The headers to send with the request.
Options › Include input fields	Toggle	Whether to include the input fields in the output. Standard: Off

• URL must be a full address starting with https:// or http://, typed or as an expression such as {{ $input.url }}.
• Timeout is in milliseconds (10000 = 10 seconds).
• Headers add extra request headers, such as Accept-Language: en.

Outputs

• Output

The item holds the whole HTML as one text value under Output field name, for example { html: "<!doctype html><html>..." }. Turn on Include input fields to keep the incoming fields too.

Dependencies and credentials

None. This node needs no connection and has no slots for other nodes.

Example workflow

Check a product’s price every morning:

1. Schedule trigger, every day at 8:00.
2. Get HTML from Link, URL https://shop.example.com/product/123, Output field name html.
3. HTML Extract, HTML {{ $input.html }}, Mode Single value, Selector .price, Return Text.
4. Browser Notification with the extracted price.

Common issues

• “Invalid URL: …”. The address is malformed, or the request could not be sent. Check the address. If it is correct and the node runs in the web app, the site likely refuses requests from other websites; run the workflow from the extension.
• “HTTP error! status: 404” (or another number). The server answered with an error. Check the address; 403 often means the site blocks automated requests.
• “Request timeout”. The site took longer than Timeout. Raise it.
• The value you want is not in the HTML. The site builds it with JavaScript. Open the page and use Get All HTML or Get Element Data instead.

Recipes with this node

Turn a list on a web page into structured data Mittelstufe · 7 Min. Keine Einrichtung

Related nodes

• HTML Extract: pick values out of the HTML with CSS selectors.
• Get All Text from Link: get readable text instead of HTML.
• Get Links from Link: list the links on the page.
• Http Request: send a custom request with authentication.