CSC 103 Lecture Notes Week 8

CSC 103 Lecture Notes Week 8
Sorting

Simple O(N²) sorts -- bubble and insertion.

/****
 *
 * Class ListSortAndSearch contains sorting and searching algorithms for
 * array-based lists.  The sorting algorithms are O(N<sup>2</sup>) bubble sort
 * and insertion, plus O(N log N) heap sort and quicksort.  Two searching
 * algorithms are provided -- an O(N) linear search and and O(log N) binary
 * search.
 *
 * @author Gene Fisher
 * @version 22may01
 *
 */

public class ListSortAndSearch {

    /**
     * Perform a bubble sort on the given list.
     */
    public void bubbleSort(Object[] list) {

        int i, j;                       // Traversal indices
        int k;                          // Trace loop index
        Object temp;                    // Temp variable for swapping

        /*
         * Outta here if list is empty.
         */
        if ((list == null) || (list.length == 0)) {
            return;
        }

        /*
         * Use a basic bubble sort algorithm.
         */
        for (i = 0; i < list.length-1; i++) {
            for (j = list.length - 2; i <= j; j--) {

                printTraceLine("Pass " + i + "," + j + ": ", list);

                /*
                 * Swap the jth and j+1th elements if jth is > j+1th
                 */
                if (((Comparable)(list[j])).compareTo(list[j+1]) > 0) {
                    temp = list[j];
                    list[j] = list[j+1];
                    list[j+1] = temp;
                }

            }
        }
    }

    /**
     * Perform an insertion sort on the given list.
     */
    public void insertionSort(Comparable[] list) {

        int i, j;                       // Traversal indices
        int k;                          // Trace loop index
        Comparable temp;                // Temp variable for swapping

        for (i = 1; i < list.length; i++) {

            /*
             * Use a basic insertion sort algorithm.
             */
            temp = list[i];
            for (j = i; j > 0 && temp.compareTo(list[j-1]) < 0; j--) {

                printTraceLine("Pass " + i + "," + j + ": ", list);

                /*
                 * Move a smaller element down a position.
                 */
                list[j] = list[j-1];

            }
            list[j] = temp;

        }

    }


    /**
     * Perform an iterative linear search of the given list for the given
     * element.  Assume the list is not null.  Return the index position of the
     * first occurrence of the element if found, -1 otherwise.
     */
    public int linearSearch(Object[] list, Object element) {

        int i;                          // Traversal index

        /*
         * Search this list.
         */
        for (i = 0; i < list.length; i++) {
            if (element.equals(list[i])) {
                return i;
            }
        }

        /*
         * Return failure if not found.
         */
        return -1;
    }

    /**
     * Perform an iterative binary search of the given list for the given
     * element.  Assume the list is not null.  Return the index position of the
     * first occurrence of the element if found, -1 otherwise.
     */
    public int binarySearch(Object[] list, Object element) {

        int startPos = 0;               // Initial start position
        int endPos = list.length - 1;   // Initial end position

        /*
         * Iterate through the list while the length of the startPos/endPos
         * interval is >= 1 and we haven't yet found the searched-for element.
         * For each loop iteration, compute the interval midpoint and then do
         * one of the following:
         *
         *     (a) If the searched-for element is at the midpoint, return
         *         successfully.
         *
         *     (b) If the searched-for element is less than the midpoint
         *         element, search the lower half of the list.
         *
         *     (c) If the searched-for element is greater than the midpoint
         *         element, search the upper half of the list.
         */
        while (startPos <= endPos) {
            int midpoint = (startPos + endPos) / 2;
            if (element.equals(list[midpoint]))
                return midpoint;
            else if (((Comparable)element).compareTo(list[midpoint]) < 0 )
                endPos = midpoint - 1;
            else
                startPos = midpoint + 1;
        }

        /*
         * Fail if we never find the element.
         */
        return -1;
    }

    /**
     * Make a space-delimited string out of the elements of an array-based
     * list.
     */
    public String toString(Object[] list) {

        String result = "";                     // Return result
        int i;                                  // Loop index

        for (i = 0; i < list.length; i++) {
            result = result + list[i].toString() + " ";
        }

        return result;

    }

    /**
     * Turn tracing on or off.
     */
    public void setTracingOn(boolean on) {
        tracingOn = on;
    }


    /**
     * If this.tracingOn is true, print a line of tracing information to
     * stdout.  The inforation consists of the given string message followed by
     * a space-delimited list of the elements in the give array.  This method
     * is called from within the sorting methods to help trace the progress of
     * the sort.

     */
    protected void printTraceLine(String message, Object[] array) {

        if (tracingOn) {
            System.out.print(message + toString(array));
        }

    }

    /** Method tracing is on if tracingOn == true. */
    protected boolean tracingOn = false;

}

public class ListSortTrace {

    public static void main(String[] args) {

        ListSortAndSearch sortAndSearch =       // Class with the methods
            new ListSortAndSearch();
        Integer[] listRandom =                   // Testing list random order
            {new Integer(34), new Integer(8), new Integer(64),
             new Integer(51), new Integer(32), new Integer(21)};
        Integer[] listUnsorted =                 // Testing list fully unsorted
            {new Integer(64), new Integer(51), new Integer(34),
             new Integer(32), new Integer(21), new Integer(8)};
        Integer[] listRandom2 =
            {new Integer(34), new Integer(8), new Integer(64),
             new Integer(51), new Integer(32), new Integer(21)};
        Integer[] listUnsorted2 =
            {new Integer(64), new Integer(51), new Integer(34),
             new Integer(32), new Integer(21), new Integer(8)};

        int i,j;                                // Loop indices
        final int SIZE = 15;                    // Size of test list
        Integer lastValue;                      // Last value in sorted list
        int index;                              // Index of search item

        sortAndSearch.sort(listRandom);
        System.out.println();
        sortAndSearch.sort(listUnsorted);

        System.out.println();

        sortAndSearch.insertionSort(listRandom2);
        System.out.println();
        sortAndSearch.insertionSort(listUnsorted2);

    }

}

Analysis of simple sorting (from book on insertion sort)

Average, worse cases for bubble and insertion sort.

Bubble sorting pass 0,4: 34 8 64 51 32
Bubble sorting pass 0,3: 34 8 64 51 21
Bubble sorting pass 0,2: 34 8 64 21 51
Bubble sorting pass 0,1: 34 8 21 64 51
Bubble sorting pass 0,0: 34 8 21 64 51
Bubble sorting pass 1,4: 8 34 21 64 51
Bubble sorting pass 1,3: 8 34 21 64 32
Bubble sorting pass 1,2: 8 34 21 32 64
Bubble sorting pass 1,1: 8 34 21 32 64
Bubble sorting pass 2,4: 8 21 34 32 64
Bubble sorting pass 2,3: 8 21 34 32 51
Bubble sorting pass 2,2: 8 21 34 32 51
Bubble sorting pass 3,4: 8 21 32 34 51
Bubble sorting pass 3,3: 8 21 32 34 51
Bubble sorting pass 4,4: 8 21 32 34 51

Bubble sorting pass 0,4: 64 51 34 32 21
Bubble sorting pass 0,3: 64 51 34 32 8
Bubble sorting pass 0,2: 64 51 34 8 32
Bubble sorting pass 0,1: 64 51 8 34 32
Bubble sorting pass 0,0: 64 8 51 34 32
Bubble sorting pass 1,4: 8 64 51 34 32
Bubble sorting pass 1,3: 8 64 51 34 21
Bubble sorting pass 1,2: 8 64 51 21 34
Bubble sorting pass 1,1: 8 64 21 51 34
Bubble sorting pass 2,4: 8 21 64 51 34
Bubble sorting pass 2,3: 8 21 64 51 32
Bubble sorting pass 2,2: 8 21 64 32 51
Bubble sorting pass 3,4: 8 21 32 64 51
Bubble sorting pass 3,3: 8 21 32 64 34
Bubble sorting pass 4,4: 8 21 32 34 64

Insertion sorting pass 1,1: 34 8 64 51 32
Insertion sorting pass 3,3: 8 34 64 51 32
Insertion sorting pass 4,4: 8 34 51 64 32
Insertion sorting pass 4,3: 8 34 51 64 64
Insertion sorting pass 4,2: 8 34 51 51 64
Insertion sorting pass 5,5: 8 32 34 51 64
Insertion sorting pass 5,4: 8 32 34 51 64
Insertion sorting pass 5,3: 8 32 34 51 51
Insertion sorting pass 5,2: 8 32 34 34 51

Insertion sorting pass 1,1: 64 51 34 32 21
Insertion sorting pass 2,2: 51 64 34 32 21
Insertion sorting pass 2,1: 51 64 64 32 21
Insertion sorting pass 3,3: 34 51 64 32 21
Insertion sorting pass 3,2: 34 51 64 64 21
Insertion sorting pass 3,1: 34 51 51 64 21
Insertion sorting pass 4,4: 32 34 51 64 21
Insertion sorting pass 4,3: 32 34 51 64 64
Insertion sorting pass 4,2: 32 34 51 51 64
Insertion sorting pass 4,1: 32 34 34 51 64
Insertion sorting pass 5,5: 21 32 34 51 64
Insertion sorting pass 5,4: 21 32 34 51 64
Insertion sorting pass 5,3: 21 32 34 51 51
Insertion sorting pass 5,2: 21 32 34 34 51
Insertion sorting pass 5,1: 21 32 32 34 51

Worst case is sum i=2 to N of i = 2 + 3 + ... + N = N * (N-1) / 2 = O(N²), from formula in lecture notes week 1.
The worst and average case analyses are based on the number of inversions in an unsorted list.
1. By some not entirely realistic assumptions, avg case is N(N-1)/4.

Heapsort -- breakthrough to O(N log N) sorting. -- From book.
Analysis of heapsort -- from book.

Mergesort

Very nice illustration of recursive divide and conquer algorithm.
The fundamental sorting is merging two sorted lists, which we do by iteratively copying their elements in order into a third list.
Given this basic operation, the mergesort algorithm on an array of size N is described recursively as follows:
1. If N = 1, we're done.
2. Recursively mergesort the first and second halves of the array, then merge the results into a sorted array.

Here's a detailed trace:

    [8, 5, 3, 1, 0, 9, 7, 4]
  mergesort([8, 5, 3, 1, 0, 9, 7, 4])
  mergesort([8, 5, 3, 1])
  mergesort([8, 5])
  mergesort([8])
  mergesort([5])
    merge([] , [5])
  mergesort([3, 1])
  mergesort([3])
  mergesort([1])
    merge([] , [1])
    merge([5] , [1, 3])
  mergesort([0, 9, 7, 4])
  mergesort([0, 9])
  mergesort([0])
  mergesort([9])
    merge([] , [9])
  mergesort([7, 4])
  mergesort([7])
  mergesort([4])
    merge([] , [4])
    merge([0] , [4, 7])
    merge([1, 3, 5] , [0, 4, 7, 9])
    [0, 1, 3, 4, 5, 7, 8, 9]

Quicksort
1. Like mergesort, it's a recursive divide and conquer.
2. The basic algorithm to sort an array A is:
  1. If number of elems in A is 0 or 1, done.
  2. Pick any element p in A to be the pivot point.
  3. Partition A less p into two disjoint subgroups: A₁ containing all elements less than p; A₂ containing all elements greater than p.
  4. Return quicksort(A₁ followed by p followed by quicksort(A₂.
3. Trace from book.
4. Even though quicksort is O(N²) worst case, it's average behavior is compellingly fast.
  1. It beats heapsort because it doesn't have to do the heapify.
  2. It beats mergesort because it doesn't have to do the temp array copying.
  3. This means it's widely used in practice.
  4. This is similar to hashing, that is widely used in practice because of its good average time, where worst case behavior can be avoided in practice.
5. Picking the pivot.
  1. Bad choice -- the first element.
    1. Leads to O(N²) behavior on sorted list
    2. This is worse even than insertion sort, which is linear on a sorted list.
    3. OK choice -- a random element.
      1. "Safe" but does not guarantee non-quadratic behavior.
    4. Best choice -- the median of all values.
      1. Produces guaranteed O(N log N) behavior.
      2. However, median is an expensive operation to perform up front.
    5. Good choice -- the median of left, right, and center values.
  2. A trace:
```
Quicksort:
    [8, 5, 3, 1, 0, 9, 7, 4, 2, 6]
    before pivot,     low=0,high=9,pivot=0 [8, 5, 3, 1, 0, 9, 7, 4, 2, 6]
    before partition, low=0,high=9,pivot=2 [0, 5, 3, 1, 2, 9, 7, 4, 6, 8]
    after partition,  low=0,high=9,pivot=2 [0, 5, 3, 1, 2, 4, 6, 9, 7, 8]
    before pivot,     low=0,high=5,pivot=3 [0, 5, 3, 1, 2, 4, 6, 9, 7, 8]
    before partition, low=0,high=5,pivot=2 [0, 5, 2, 1, 3, 4, 6, 9, 7, 8]
    after partition,  low=0,high=5,pivot=2 [0, 1, 2, 3, 5, 4, 6, 9, 7, 8]
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
```
6. Proof that in-place sorting can be no faster than O(N log N) -- from book.
index | lectures | labs | handouts | examples | assignments | solutions | doc | grades | help