Skip to content
Browse all functions

Scrape HTML

data/scrapehtml

scrapehtml (Scrape HTML) is a 1001fx API function in the group Data & JSON: POST https://api.1001fx.com/data/scrapehtml, 1 credit per call. It works as a ready-made module in Make and n8n or as a plain HTTP call.

1 credit per callHTTP APIn8nMake

Scrapes an HTML string or a website and returns the text or attributes of the elements your CSS selectors match.

Try it in the 1001fx app

Run scrapehtml in your browser with your API key – fill the form, click Run and get the answer or the file.

scrapehtml in the 1001fx app: the form with example values
Scrape HTML in the 1001fx app

Endpoint

POSThttps://api.1001fx.com/data/scrapehtml
Header
X-API-KEY: YOURSECRETAPIKEY How to
Content type
application/json

Parameters

  • htmlstringoptional

    HTML string to be scraped.

    Example:
    <div><a href='https://1001fx.com'>1001fx</a></div>
  • urlstringoptional

    Website to be scraped.

    Example:
    https://1001fx.com/blog
  • selectorsarray<object>required

    Array of selectors to search for.

    • selectorstringrequired

      The CSS selector to search for.

      Example:
      #__next > div > div > main > div > div > div > a
    • attributestringrequired

      The attribute to return. E.g.: href, src, text.

      Example:
      href
    • keyFieldstringrequired

      The field to store the found value in.

      Example:
      url

Examples

Scrape an HTML snippet

Reads the selected elements from the HTML passed in html.

Input

Expects a json body
{
  "selectors": [
    {
      "selector": "#__next > div > div > main > div > div > div > a",
      "attribute": "href",
      "keyField": "url"
    }
  ],
  "url": "https://1001fx.com/blog",
  "html": "<div><a href='https://1001fx.com'>1001fx</a></div>"
}

Output · HTTP 200

Response
{
  "statusCode": 200
}
Show curl command
curl -X POST 'https://api.1001fx.com/data/scrapehtml' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"selectors":[{"selector":"#__next > div > div > main > div > div > div > a","attribute":"href","keyField":"url"}],"url":"https://1001fx.com/blog","html":"<div><a href='\''https://1001fx.com'\''>1001fx</a></div>"}'

Scrape a web page

Loads the page from url and reads the selected elements.

Input

Expects a json body
{
  "selectors": [
    {
      "selector": "#__next > div > div > main > div > div > div > a",
      "attribute": "href",
      "keyField": "url"
    }
  ],
  "url": "https://1001fx.com"
}

Output · HTTP 200

Response
{
  "statusCode": 200
}
Show curl command
curl -X POST 'https://api.1001fx.com/data/scrapehtml' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"selectors":[{"selector":"#__next > div > div > main > div > div > div > a","attribute":"href","keyField":"url"}],"url":"https://1001fx.com"}'

Try scrapehtml now

Run it in your browser or create a free account and get 100 credits every month.

Changelog

  • 1.0.001.03.2023created
Ask 1001fx