Skip to content
Browse all functions

Remove Duplicates from an Array

array/removeduplicates

removeduplicates (Remove Duplicates from an Array) is a 1001fx API function in the group Arrays: POST https://api.1001fx.com/array/removeduplicates, 2 credits per call. It works as a ready-made module in Make and n8n or as a plain HTTP call.

2 credits per callHTTP APIn8nMake

Removes duplicates from an array. If the fields parameter is empty, all fields of the first array element are used for the comparison.

Try it in the 1001fx app

Run removeduplicates in your browser with your API key – fill the form, click Run and get the answer or the file.

removeduplicates in the 1001fx app: the form with example values
Remove Duplicates from an Array in the 1001fx app

Endpoint

POSThttps://api.1001fx.com/array/removeduplicates
Header
X-API-KEY: YOURSECRETAPIKEY How to
Content type
application/json

Parameters

  • dataarray<object>required

    Array of lines with mixed values.

  • fieldsarray<string>optional

    Array of fields to compare for uniqueness. If the fields parameter is empty, all fields of the first array item will be used for the comparison.

  • ignoreCasebooleanoptional

    Sets case sensitivity for operation.

    Default:
    false
    Example:
    false

Examples

Compare one field

Rows count as duplicates when Column2 is equal; returns 2 rows.

Input

Expects a json body
{
  "data": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "Germany",
      "Column2": "apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    }
  ],
  "fields": [
    "Column2"
  ],
  "ignoreCase": false
}

Output · HTTP 200

Response
{
  "result": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "Germany",
      "Column2": "apple",
      "Column3": "Dragonfruit"
    }
  ]
}
Show curl command
curl -X POST 'https://api.1001fx.com/array/removeduplicates' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"data":[{"Column1":"Germany","Column2":"Apple","Column3":"Dragonfruit"},{"Column1":"Germany","Column2":"apple","Column3":"Dragonfruit"},{"Column1":"US","Column2":"Apple","Column3":"Dragonfruit"}],"fields":["Column2"],"ignoreCase":false}'

Compare another field

Rows count as duplicates when Column3 is equal; returns 1 row.

Input

Expects a json body
{
  "data": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "Germany",
      "Column2": "apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    }
  ],
  "fields": [
    "Column3"
  ],
  "ignoreCase": false
}

Output · HTTP 200

Response
{
  "result": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    }
  ]
}
Show curl command
curl -X POST 'https://api.1001fx.com/array/removeduplicates' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"data":[{"Column1":"Germany","Column2":"Apple","Column3":"Dragonfruit"},{"Column1":"Germany","Column2":"apple","Column3":"Dragonfruit"},{"Column1":"US","Column2":"Apple","Column3":"Dragonfruit"}],"fields":["Column3"],"ignoreCase":false}'

Compare whole rows, ignore case

No fields given: all columns are compared, ignoring case; returns 2 rows.

Input

Expects a json body
{
  "data": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "Germany",
      "Column2": "apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    }
  ],
  "ignoreCase": true
}

Output · HTTP 200

Response
{
  "result": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": "Dragonfruit"
    }
  ]
}
Show curl command
curl -X POST 'https://api.1001fx.com/array/removeduplicates' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"data":[{"Column1":"Germany","Column2":"Apple","Column3":"Dragonfruit"},{"Column1":"Germany","Column2":"apple","Column3":"Dragonfruit"},{"Column1":"US","Column2":"Apple","Column3":"Dragonfruit"}],"ignoreCase":true}'

Compare an object field

Rows count as duplicates when the object in Column5 is equal; returns 1 row.

Input

Expects a json body
{
  "data": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": 1,
      "Column4": true,
      "Column5": {
        "a": "Value",
        "b": "value2"
      },
      "Column6": [
        "January",
        "February"
      ]
    },
    {
      "Column1": "Germany",
      "Column2": "apple",
      "Column3": 1,
      "Column4": true,
      "Column5": {
        "a": "Value",
        "b": "value2"
      },
      "Column6": [
        "January",
        "February"
      ]
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": 1,
      "Column4": false,
      "Column5": {
        "a": "Value",
        "b": "value2"
      },
      "Column6": [
        "January",
        "February"
      ]
    },
    {
      "Column1": "US",
      "Column2": "Apple",
      "Column3": 2,
      "Column4": true,
      "Column5": {
        "a": "Value",
        "b": "value2"
      },
      "Column6": [
        "January",
        "February"
      ]
    }
  ],
  "fields": [
… (4 more lines)

Output · HTTP 200

Response
{
  "result": [
    {
      "Column1": "Germany",
      "Column2": "Apple",
      "Column3": 1,
      "Column4": true,
      "Column5": {
        "a": "Value",
        "b": "value2"
      },
      "Column6": [
        "January",
        "February"
      ]
    }
  ]
}
Show curl command
curl -X POST 'https://api.1001fx.com/array/removeduplicates' \
  -H 'X-API-KEY: YOURSECRETAPIKEY' \
  -H 'Content-Type: application/json' \
  -d '{"data":[{"Column1":"Germany","Column2":"Apple","Column3":1,"Column4":true,"Column5":{"a":"Value","b":"value2"},"Column6":["January","February"]},{"Column1":"Germany","Column2":"apple","Column3":1,"Column4":true,"Column5":{"a":"Value","b":"value2"},"Column6":["January","February"]},{"Column1":"US","Column2":"Apple","Column3":1,"Column4":false,"Column5":{"a":"Value","b":"value2"},"Column6":["January","February"]},{"Column1":"US","Column2":"Apple","Column3":2,"Column4":true,"Column5":{"a":"Value","b":"value2"},"Column6":["January","February"]}],"fields":["Column5"],"ignoreCase":false}'

Try removeduplicates now

Run it in your browser or create a free account and get 100 credits every month.

Changelog

  • 1.0.010.03.2023created
Ask 1001fx