Morioh
Login
Dylan North

Dylan North

4 years ago

Javascript Tutorial : Remove duplicate values from Javascript array

Recently in one of my projects I had to remove duplicates from an array. Instead of looking up a npm package that could do it for me I chose to implement it myself. Here’s what I learned about this seemingly straightforward task.

Javascript array is a collection of different types of values which are saved at a particular index of an array. Values stored in a javascript array can be duplicate values or unique values.

Javascript has a built-in Object Set which is the best approach to remove duplicates from javascript array because it only keeps unique values. Set object available in ES6.

But there are also other approaches to get a unique or distinct array in js. which we are going to implement in this post.

In this tutorial, you will learn how to remove duplicates from javascript array. There are multiple ways to remove duplicate elements from an array in javascript.

Remove Duplicates From Javascript Array

1. Using ES6 Set

Let’s know first what is set.

Javascript has an built in Set Object, which is introduce in ES6 which only keep unique values when array pass through Set and remove duplicates from an array

Let’s get back to the code and implement the functionality using Set. There are many ways you can use it.

  1. First, we create a new Set by passing an array to Set Object. Which will remove all duplicate values and only keep unique values.
  2. Then, we have two option either use spread operator … or use Array.from to convert set Object to array.

Using Set With …

// javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distinct" ]

// using es6 Set Object to remove duplicates
let removeDuplicates = new Set(jsArray);

// using ... spread operator, pass value to array
let uniqueArray = [ ...removeDuplicates ]

// result => [ "codekila" , 1 , 5 , "unique" , "distinct" ]
console.log(uniqueArray);

You can also optimize the above code by making it short and convert it into reusable functionality. We can also Array.from instead from … spread operator.

Using Set With Array.from

  function removeDuplicatesUsingSet(jsArray){
    // using es6 Set Object to remove duplicates
    // using ... spread operator, pass value to array
    let uniqueArray = Array.from(new Set(jsArray))

    // return array with distinct values
    return uniqueArray;
  }

  // javascript array with some unique and duplicate values
  let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distinct" ]

  // result => [ "codekila" , 1 , 5 , "unique" , "distinct" ]
  console.log(removeDuplicatesUsingSet(jsArray));

As you can see in the above example, we pass the jsArray to removeDuplicatesUsingSet function which converts the array with duplicate values to distinct javascript array with unique values only.

2. Using Filter Method

Let’s know first that what’s Javascript filter method

Javascipt filter is a array method used to filter an array based on condition and it return a new array.

Array.filter(function(elem, index, arr){ })

Array.filter((elem, index, arr)=>{ })

We use a filter method to remove duplicate values from an array. Filter method callback takes three-parameter which are current_element, index_of_current_element an array which we are processing.

But in order to create a distinct array which only contains unique values we also need an indexOf method which searches an array to get an index of value.

So move on to our code, first we iterate an array using filter method.

Then check for an index of every value, if it exists already.

let distinctArray = jsArray.filter(function(elem, index, arr){

 // checking element index is already exist or not
    return arr.indexOf(elem) === index

 })

 // result => [ "codekila" , 1 , 5 , "unique" , "distinct" ]
 console.log("Distinct Array =>",distinctArray);

So, here you can see how we check for unique values using indexOf, if element index does not match with current index then it means it’s a duplicate value.

We can also optimize the above code by converting into reusable functionality and also use arrow function in the filter method.

  // javascript array with some unique and duplicate values
  let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distinct" ]

  // result => [ "codekila" , 1 , 5 , "unique" , "distinct" ]
  console.log(removeDuplicatesUsingFilter(jsArray));

   // function hoisting is working
  function removeDuplicatesUsingFilter(jsArray){

    // we filter it and check for unique values together in one line statement
    let distinctArray = jsArray.filter((elem, index, arr) => arr.indexOf(elem) === index)

    // return array with unique values
    return distinctArray;
  }

So finally, we remove the duplicate values from a javascript array and just retrieve a distinct array. Javascript filter method is a very handy method to filter array with any condition which satisfies it.

Also read : How to empty an array in javascrpit

3. Using Reduce Method

Let’s know first that what’s Javascript reduce method

Javascript reduce is a method which reduce array to single value. Reduce have a four argument in it’s callback function.

Array.reduce(function(accumulator, elem, index, arr){ }, initialValue)

Array.reduce((accumulator, elem, index, arr)=>{ }, initialValue)

Reduce method have a very nice argument which is known as an accumalator variable in reduce callback argument.

So, We use reduce method in javascript to remove duplicate from an array with the help first argument in reduce method callback.

Reduce method iterate the array from left to right and using accumulator variable we collect all unique values and save it into an accumalator variable.

For every value,we check a unique value in the accumalator array with the help of Array.includes and if it exists then we do not insert that value.

So let’s move to a code with example in which we use reduce method.

  // javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distinct" ]

let distinctArray = jsArray.reduce(function(uniqueArray, elem){

 // checking element is already exist or not
 // if not accessed then pass uniqueArray and new Element
     return uniqueArray.includes(elem) ? uniqueArray : [ ...uniqueArray, elem]

 },[])

 // result => [ "codekila" , 1 , 5 , "unique" , "distinct" ]
 console.log("Distinct Array =>",distinctArray);

we can optimize the above code as we do with a filter method, using arrow function and implement it as reusable code.

  // javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distict" ]

function removeDuplicateUsingReduce(jsArray){

 // checking element is already exist or not
 // if not, then pass uniqueArray and new Element

return distinctArray = jsArray.reduce((uniqueArray, elem) =>uniqueArray.includes(elem) ? uniqueArray : [ ...uniqueArray, elem] },[]);

}

 // result => [ "codekila" , 1 , 5 , "unique" , "distict" ]
 console.log("Distinct Array =>",distinctArray);

As we can see how to remove duplicates from an array using the reduce method. Reduce a method is also a good approach to work with non-primitive value means Object in an array.

Array.includes() check value exists in an array on each iteration of reduce function.

4. Using for loop

for loop is a traditional approach to implementing anything which requires the array to be iterate. So we can also use for loop to remove duplicates from array by creating another array.

In another array, we push a unique values only. So when we iterate an array if check value exists in another array. If it not exist then we push it.

Let’s move on to a code section so you can check it.

  // javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distict" ]

function removeDuplicateUsingForLoop(jsArray){
  let unique_array = []

  for(let i = 0; i < jsArray.length; i++)
  unique_array = unique_array.includes(jsArray[i]) ? unique_array : [...unique_array, jsArray[i]]

   return unique_array 
}

 // result => [ "codekila" , 1 , 5 , "unique" , "distict" ]
 console.log("Distinct Array =>",removeDuplicateUsingForLoop(jsArray));

for loop is also a nice and easy approach to remove check for unique value from an array and push that value to a different array.

5. Using Lodash Library

Until now, we are seeing the functionality provide by javascript to remove duplicates from the array but their certain library available which have an inbuilt function to remove duplicates from an array and gives an unique array which contains a distinct values only.

Lodash is a javascript utility library which reduce the time to develop this kind of functionality. Lodash has a function uniq which remove the duplicates from an array.

To implement lodash library, first, we need to include a lodash library in HTML page using script tag.

<head>
    <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/3.10.1/lodash.js"></script>
</head>

Lodash uses underscore as a reference to apply their functionality. Like you have an array with duplicate values then use below code to get a unique array.

  // javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distict" ]

 // result => [ "codekila" , 1 , 5 , "unique" , "distict" ]
 console.log("Unique Array =>",_.uniq(jsArray));

uniq method returns an array which contains a unique value by removing a duplicate value from an array.

6. Using Underscore.js library

So, Underscore.js is also a javascript utility library like lodash. which reduce the time of creating already known problems like removing duplicate values from an array.

Same as lodash, in order to implement in HTML page we need to put an underscore.js library in the head section of an HTML page.

<head>
    <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.8.3/underscore-min.js"></script>
</head>

Underscore.js have uniq method which removes the duplicate from an array and gives an array with unique value.

  // javascript array with some unique and duplicate values
let jsArray = [ "codekila", 1, 5, 1, "unique", "codekila", "distict" ]

 // result => [ "codekila" , 1 , 5 , "unique" , "distict" ]
 console.log("Unique Array =>",_.uniq(jsArray));

As you can see there are many ways you can implement for how to remove duplicate value from array.

We have used only functions and functionality provide by JavaScript only like a set, filter, reduce and more.

And we have also used a library like lodash and underscore to get the same result.

Learn More

#js #javascript

Javascript Tutorial : Remove duplicate values from Javascript array
60.45 GEEK