Tutorials Tonight
Follow Us

JavaScript Remove Duplicates From Array


In this tutorial, we will learn how javascript remove duplicates from array. We will look at various different ways to remove duplicates from the array with examples.

Sometimes, you need to work with an array of data that has duplicate values but you don't want to have duplicate values in your array. So you need to remove duplicate values from the array.

Below we have discussed 5 different ways to remove duplicate values from the array and also discussed their performance. Depending on the performance of the method you will get a better idea to use which method is best for you.

javascript remove duplicates from array

    Table Of Contents

  1. Remove Duplicates Using Set() Constructor
  2. Remove Duplicates Using filter() Method
  3. Remove Duplicates Using reduce() Method
    1. reduce() Method With indexOf() Method
    2. reduce() Method With includes() Method
  4. Remove Duplicates Using map() Method
  5. Performance Of Different Methods
  6. Conclusion

1. Remove Duplicates Using Set() Constructor

A set is a collection of unique values in JavaScript. A value in a set occurs only once.

A set can be created using the Set() constructor.

The Set() constructor takes an element as an argument and returns a set object.

If any element is repeated in the argument, it will be ignored and only unique values will be added to the set.

For example, if you add 1, 2, 2, 1, 3 to the set, it will return a set with 1, 2, 3.

Example

var unique = new Set();
unique.add(1);
unique.add(2);
unique.add(2);
unique.add(1);
unique.add(3);

// loop through the set and print out the values
for (let item of unique) {
  console.log(item);
}

Remove Duplicates From Array Using Set() Constructor

Pass the array to the Set() constructor and it will remove the duplicate values from the array.

Example

var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];

// create a set from array
var unique = new Set(arr);

// convert set to array
var uniqueArray = Array.from(unique);

console.log(uniqueArray); // [1, 2, 4, 3]

The duplicates from the array are removed when a set is created. You have to convert set to array to get an array.

You can notice one thing the set includes the first unique element and removes the next duplicate elements. The first unique 4 came before the first unique 3 so it includes 4 before 3 in the set.


2. Remove Duplicates Using filter() Method

The filter() method is used to remove any element from the array for which the return value of the function is false.

To remove duplicates from the array, create a function that returns true only for the first element and false for the rest of the elements.

You can use the indexOf() method here. It returns the index of the first occurrence of the element in the array.

Example

var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];

// using filter method to remove duplicates
var unique = arr.filter(function(element, index) {
  return arr.indexOf(element) === index;
});

console.log(unique);

Code Explanation: We have access to the index of the current element of the array within the filter function. We can use this index to check if the element is the first occurrence or not using the indexOf() method. If it is the first occurrence, it will return true and if it is not the first occurrence, it will return false.


3. Remove Duplicates Using reduce() Method

The reduce() method is used to reduce the array to a single value by executing a reducer function on each element of the array, resulting in a single output value.

We can use this to reduce the given array in an array that has no duplicate values.

The method takes an initial value to work with, so we give an empty array [] as the initial value.

Syntax:

array.reduce(function(accumulator, currentValue), initialValue)

array.reduce(function(accumulator, currentValue, currentIndex, array), initialValue)

We are going to use only 2 arguments in the reducer function: accumulator and currentValue.

3.1. reduce() Method With indexOf() Method

Start with an empty array and start adding the current element to the array if it is not already present in the array. Then return the array.

Example

var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];

// using reduce method to remove duplicates
var unique = arr.reduce(function(acc, current) {
  if (acc.indexOf(current) === -1) {
    acc.push(current);
  }
  return acc;
}, []);

console.log(unique);

Code Explanation: Execution starts with an empty array as the initial value. Then we check if the accumulator (initialValue) contains the current element. If it does not contain the current element, we add it to the accumulator array. Then we return the accumulator array. Then the same process is repeated for all the elements in the array.

If it contains the current element, we do not add it to the accumulator array. Then we return the accumulator array.


3.2. reduce() Method With includes() Method

Start with an empty array and start adding the current element to the array if it is not already present in the array. Then return the array.

Example

var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];

// using reduce method to remove duplicates
var unique = arr.reduce(function(acc, current) {
  return acc.includes(current) ? acc : [...acc, current];
}, []);

console.log(unique);

Code Explanation: This is the same as above but we are using the includes() method to check if the accumulator array contains the current element. The rest we are doing is the same as above.


4. Remove Duplicates Using map() Method

This is not the very best way to remove duplicates from an array but eventually, it can be used.

We can do a very simple thing we can map the array and return the item if it is not present in the array and if it is present in the array return null after that, we filter the array and remove the null values.

Example

var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];

// using map method to remove duplicates
var unique = arr.map((item, index) => {
               return arr.indexOf(item) === index ? item : null;
             }).filter(item => item !== null);

console.log(unique);

Performance Comparison All Discussed Methods

We can compare the performance of all the methods discussed above.

The graph below shows the performance of all the methods based on the number of operations and time taken to execute.

The method is executed for 100, 1000, 10000, 100000, 1000000, and 10000000 operations, and the time taken to execute is displayed in the graph.

Performance Comparison All Discussed Methods

Here is the program to compare the performance of all the methods discussed above.

Each method can be executed for a different number of operations. Since each execution gives a different result, so we ran each method 10 times for each number of operations.

Example

// remove duplicates from array
let arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
var operation = 1000, repeat = 10;
var avg = 0, sum = 0;
var start, end;

// check the performance of each method

// method 1
for (let i = 0; i < repeat; i++) {
  start = new Date().getTime();
  for (let i = 0; i < operation; i++) {
    let unique = [...new Set(arr)];
  }
  end = new Date().getTime();
  sum += end - start;
}
avg = sum / repeat;
console.log(`Method 1: ${avg}ms`);

// method 2
for (let i = 0; i < repeat; i++) {
  start = new Date().getTime();
  for (let i = 0; i < operation; i++) {
    let unique = arr.filter((item, index) => {
      return arr.indexOf(item) === index;
    });
  }
  end = new Date().getTime();
  sum += end - start;
}
avg = sum / repeat;
console.log(`Method 2: ${avg}ms`);

// method 3.1
for (let i = 0; i < repeat; i++) {
  start = new Date().getTime();
  for (let i = 0; i < operation; i++) {
    let unique = arr.reduce((acc, curr) => {
      if (acc.indexOf(curr) === -1) {
        acc.push(curr);
      }
      return acc;
    }, []);
  }
  end = new Date().getTime();
  sum += end - start;
}
avg = sum / repeat;
console.log(`Method 3.1: ${avg}ms`);

// method 3.2
for (let i = 0; i < repeat; i++) {
  start = new Date().getTime();
  for (let i = 0; i < operation; i++) {
    let unique = arr.reduce((acc, curr) => {
      return acc.includes(curr) ? acc : [...acc, curr];
    }, []);
  }
  end = new Date().getTime();
  sum += end - start;
}
avg = sum / repeat;
console.log(`Method 3.2: ${avg}ms`);

// method 4
for (let i = 0; i < repeat; i++) {
  start = new Date().getTime();
  for (let i = 0; i < operation; i++) {
    let unique = arr.map((item, index) => {
      return arr.indexOf(item) === index ? item : null;
    }).filter(item => item !== null);
  }
  end = new Date().getTime();
  sum += end - start;
}
avg = sum / repeat;
console.log(`Method 4: ${avg}ms`);

Conclusion

We have seen 5 different methods of how javascript remove duplicates from array. We have also compared the performance of all the methods.

From the Comparison of all the methods, we can see that the Set() method is the best method to remove duplicates from the array. It is the fastest method and it is also the most efficient method.