Code optimization : Divide and process records in parallel

1. Divide and process the records received

• Usecase

  • We are getting 6.5k records from the database and then we are traversing each item , creating hashmap from it and then getting the values from the hashmap
  • storing it in a object and then adding all the objects in a list and then adding that list to another list and then returing.

• Problem

  • If we divide the work of traversing 6.5k records to 3 threads and each thread will process the number 3.25 records and then create object out of it and then add that to the final object
  • here we need to use threads to process this as well as use some kind of datastrucuture on which we can write parallely.

Simple Example of code --> without multitreading : taking 15540

import java.util.*;
public class index {
        public static void main(String args[]) throws InterruptedException {
            List<Employee> FinalObject = new ArrayList<>();
            //list of size 10k
            List<Integer> parentList= new ArrayList<>();

            for(int i=0;i<10000;i++) {
                parentList.add(i+1);
            }

            // traverse the parent list and then creating object for each element and then adding each object to new list

            int parentListSize=parentList.size();
            long starttime=System.currentTimeMillis();
            //TODO : this piece i need to divide into multiple threads so that each thread is processing part of it
            for(int i=0;i<parentListSize;i++) {

                Employee e1=new Employee();
                e1.setId(i);
                e1.setName(i+ "test name");
                e1.setDep(i+ "test department");
                e1.setSeatNumber(i*2);
                Thread.sleep(1);
                FinalObject.add(e1);
            }

            System.out.println("Time Taken to complete the traversing and addition is  "+(System.currentTimeMillis()-starttime));
        }
    }
    class Employee {

        private int id;
        private String name;
        private String dep;
        private int seatNumber;

        Employee() {
        }

        public int getId() {
            return id;
        }
        public void setId(int id) {
            this.id=id;
        }
        public void setName(String name){
            this.name=name;
        }
        public void setDep(String dep) {
            this.dep=dep;
        }

        public void setSeatNumber(int seatNumber) {
            this.seatNumber=seatNumber;
        }
    }

Same code with single thread

• here we wants to divide the 10k records on 5 different threads and then execute them in parallel and write the data in the same thread.

• Time taken is almost same : 15581

import java.util.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class index {

    List<Employee> FinalObject = new ArrayList<>();

        public static void main(String args[]) throws InterruptedException {

            //list of size 10k
            List<Integer> parentList= new ArrayList<>();

            for(int i=0;i<10000;i++) {
                parentList.add(i+1);
            }

            // traverse the parent list and then creating object for each element and then adding each object to new list


            int parentListSize=parentList.size();
            long starttime=System.currentTimeMillis();
            //TODO : this piece i need to divide into multiple threads so that each thread is processing part of it
//            for(int i=0;i<parentListSize;i++) {
//
//                Employee e1=new Employee();
//                e1.setId(i);
//                e1.setName(i+ "test name");
//                e1.setDep(i+ "test department");
//                e1.setSeatNumber(i*2);
//                Thread.sleep(1);
//                FinalObject.add(e1);
//
//            }

            index s1=new index();
            s1.createdNewObjectsAndSave(parentListSize);
            //create a single thread and then pass this task to a single thread
            ExecutorService executorService= Executors.newSingleThreadExecutor();


            //task is created and now send this task to thread
            Runnable r1= () -> {
                try {
                    s1.createdNewObjectsAndSave(parentListSize);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            };

            executorService.submit(r1);
            System.out.println("Time Taken to complete the traversing and addition is  "+(System.currentTimeMillis()-starttime));

            executorService.shutdown();
        }

        public void createdNewObjectsAndSave(int parentListSize) throws InterruptedException {
            for(int i=0;i<parentListSize;i++) {

                Employee e1=new Employee();
                e1.setId(i);
                e1.setName(i+ "test name");
                e1.setDep(i+ "test department");
                e1.setSeatNumber(i*2);
                Thread.sleep(1);
                FinalObject.add(e1);

            }
        }
    }
    class Employee {

        private int id;
        private String name;
        private String dep;
        private int seatNumber;

        Employee() {
        }

        public int getId() {
            return id;
        }
        public void setId(int id) {
            this.id=id;
        }
        public void setName(String name){
            this.name=name;
        }
        public void setDep(String dep) {
            this.dep=dep;
        }

        public void setSeatNumber(int seatNumber) {
            this.seatNumber=seatNumber;
        }
    }

Same code with two threads

How to write concurrently in an arraylist

• we can use --> vector,synchronised list , copuOnWriteArrayList
• we can wrap the arraylist with collections.synchronzed list or use thread safe collection or just put the add calls in a synchronized block
• copyOnWriteArrayList --> is for sequential write and concurrent read
  • we can write a multi-threaded program that allows one thread to add elements to the list while other threads are traversing the list’s elements at the same time, and no worry about ConcurrentModificationException as per the case of a synchronized list.