Java Find duplicate objects in list

Here is different ways to find duplicate objects in list like Find duplicate objects in list using Set ,Find duplicate objects in list using Stream Group by, hash map etc..

Java Find duplicate objects in list using Set

java.util.List<String> duplicates = Arrays.asList("A", "B", "B", "C", "D", "D", "Z", "E", "E");
     java.util.HashSet unique=new HashSet();
     System.out.println("Java Find duplicate objects in list using Set");  
     for (String s:duplicates){
         if(!unique.add(s)){   //  // java.util.Set only unique object so if object will not bee add in Set it will return false so can consider it as Duplicate
             System.out.println(s);
         }
     }

Output:

Java Find duplicate objects in list using Set
B
D
E

Java Find duplicate objects in list using Stream Group by

In Java Stream perform group by operation based on that we can find duplicate object from collection or list.

java.util.List<String> list = Arrays.asList("A", "B", "B", "C", "D", "D", "Z", "E", "E");
      list.stream().collect(Collectors.groupingBy(Function.identity(),     
              Collectors.counting()))                                             // perform group by count
          .entrySet().stream()
          .filter(e -> e.getValue() > 1L)                                        // using take only those element whose count is greater than 1
          .map(e -> e.getKey())                                                  // using map take only value   
          .collect(Collectors.toList())                                        // convert group by result to list      
          .forEach(System.out::println);

Output:

B
D
E

Find duplicate objects in a list using a hash map

Now we are going to find duplicate objects in the list using hashmap/hashtable. This solution is useful when we also want to find the occurrences of duplicate elements.

In below example, we have iterate of all the elements one by one and put elements in hash map with counter 1. If the map already contains key increment to int’s counter by one. Once the hash map is ready, loop through all the elements and check it’s counter, Consider duplicate elements if a counter value greater than.

java.util.List<String> list = Arrays.asList("A", "B", "B", "C", "D", "D", "Z", "E", "E");
        Map<String,Integer> valueCounter = new HashMap<>();
        //Iterate all the elements from list and prepare HashMap, Key is List Elements and value is duplicate element occurences 
        for(String str : list){
            Integer val = valueCounter.get(str);
            if(valueCounter.get(str)==null){
                valueCounter.put(str,1); //if map does not contains key, put element with value 1
            }else{
                val = val+1;//val++, ++val
                valueCounter.put(str,val); //increment counter if map already exists element
            }
        }
        Set<Map.Entry<String, Integer>> entrySet = valueCounter.entrySet();
        for(Map.Entry<String,Integer> entry : entrySet){
            if(entry.getValue()>1){
                System.out.println(entry.getKey()+"=>"+entry.getValue());
            }
        }
        /*valueCounter.entrySet().stream() //we can also use stream api to achieve the same
                .filter(map -> map.getValue()>1)
                .forEach(map -> System.out.println(map.getKey()+"=>"+map.getValue()));*/

Output:

Here as you can see each duplicate elements with its occurrences.

B=>2
D=>2
E=>2

Find duplicate objects in a list using loops

This solution is very straight forward. Here we have used loops instead of any Java collections. Here we have used two loops. Iterate over all the elements and compare with each other.

For some optimization purpose inner loop starts with i+1to reduce the complexity of an algorithm.

java.util.List<String> list = Arrays.asList("A", "B", "B", "C", "D", "D", "Z", "E", "E");
        for (int i = 0; i < list.size(); i++) {
            for (int j = i+1; j <list.size() ; j++) {
                if(list.get(i).equals(list.get(j))){
                    System.out.println(list.get(i));
                }
            }
        }

Output:

B
D
E

Find duplicate user-defined objects in a list

In real life use cases, we always come to a situation where we want to deal with user-defined objects. Two  methods equals and hashcode we need to override when we worked with user-defined objects and want to store in the collection such as HashMap, HashSet etc.. Collections use a hashcode value of an object to determine how it should be stored inside a collection, and the hashcode is used again in order to locate the object in its collection.

equals() method is used to compare the equality of two Objects. Equality can be compared in two ways.

1. shallow comparison
2. deep comparison

hashcode()method must be overridden in every class which override equals() method.

Example

Employee.java

package com.javadeveloperzone;

import java.util.Objects;

public class Employee {
    int empId;
    String empName;
    String empAddress;

    public int getEmpId() {
        return empId;
    }

    public void setEmpId(int empId) {
        this.empId = empId;
    }

    public String getEmpName() {
        return empName;
    }

    public void setEmpName(String empName) {
        this.empName = empName;
    }

    public Employee(int empId, String empName, String empAddress) {
        this.empId = empId;
        this.empName = empName;
        this.empAddress = empAddress;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Employee employee = (Employee) o;
        return empId == employee.empId &&
                empName.equals(employee.empName) &&
                empAddress.equals(employee.empAddress);
    }

    @Override
    public int hashCode() {
        return Objects.hash(empId, empName, empAddress);
    }
}

Find Duplicate User defind object code snipptes

List<Employee> employees = new ArrayList<>();
        employees.add(new Employee(1,"John","street 1,"));
        employees.add(new Employee(2,"Frank","Nr. Cosmos,"));
        employees.add(new Employee(3,"Danyy","street 101,Washington DC,"));
        employees.add(new Employee(1,"John","street 1,"));
        employees.add(new Employee(2,"Frank","Nr. Cosmos,"));

        employees.stream().collect(Collectors.groupingBy(Function.identity(),
                Collectors.counting()))
                .entrySet().stream()
                .filter(e -> e.getValue() > 1L)
                .map(e -> e.getKey())
                .collect(Collectors.toList())
                .forEach(employee -> System.out.println(employee.getEmpId()+"==>"+employee.getEmpName()));

Output

2==>Frank
1==>John

References

Java java.util.Set document

Java Stream Group by document

Java 8 Blogs

Source Code

You can download complete example from our git repositoty.