6. Sorting Algorithms

CS331 - 2021 Spring

Boris Glavic

bglavic@iit.edu

Sorting 101

  • Input:
    • Sequence of elements $S$
    • Total order $\leq$ over the domain of the elements
  • Output:
    • Sequences $S'$ of all elements of $S$ that is sorted
    • $\forall i < j \in {1, \ldots, len(S')}: S'[i] \leq S'[j]$

Example

  • Using the standard order of natural numbers $\mathbb{N}$

  • [1, 3, 10, 15, 16] - is sorted

  • [1, 3, 15, 10] - is not sorted ($15 < 10$)

Stable Sort

  • A sorting algorithm is called stable if it keeps elements that are equivalent according to the sort order in the same order as they appear in the input

  • Example: sorting Persons(Name,Age) on their Age

    • [(Peter,30), (Alice,25), (Bob,30)]

  • Any stable sorting algorithm will produce this result:

    • [(Alice,25), (Peter,30), (Bob,30)]

In-place Sorting

  • A sorting algorithm is in-place if it uses the input collection instead of creating copies of the input (or parts of the input)

Comparison-based Sorting Algorithms

What is comparison-based sorting?

  • How do we detect whether elements in the input are out of order?
  • We can compare two elements $S[i]$ to $S[j]$ to check whether $S[i] < S[j]$

Bubble Sort

Introduction

Implementation in Python

def bubble_sort(lst):
    for i in range(1, len(lst)):
        for j in range(1, len(lst)):
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]

Correctness

  • After the first iteration, the largest elements will be at its final position (the last position of the sequence)
  • Why?
    • In any comparison the largest element will be larger than the other element it is compared against
    • Thus, it will be switched with any element that follows it in the input order
  • generalizing this idea: after the $i^{th}$ iteration the $i$ largest elements will be in their final position
  • after at most $n$ iterations all $n$ elements will be in sort order

Runtime Analysis

def bubble_sort(lst):
    for i in range(1, len(lst)):
        for j in range(1, len(lst)):
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]

Insertion Sort

Introduction

Implementation in Python

def insertion_sort(lst):
    for i in range(1, len(lst)): # after i iterations the first i elements are sorted
        for j in range(i, 0, -1): # trickle the ith element down to its position withing this sorted list
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]
            else:
                break # found final position of element

Runtime Analysis

Quick Sort

Introduction

Implementation in Python

def insertion_sort(lst):
    for i in range(1, len(lst)): # after i iterations the first i elements are sorted
        for j in range(i, 0, -1): # trickle the ith element down to its position withing this sorted list
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]
            else:
                break # found final position of element

Runtime Analysis

Heap Sort

Introduction

Implementation in Python

def insertion_sort(lst):
    for i in range(1, len(lst)): # after i iterations the first i elements are sorted
        for j in range(i, 0, -1): # trickle the ith element down to its position withing this sorted list
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]
            else:
                break # found final position of element

Runtime Analysis

Merge Sort

Introduction

Implementation in Python

def insertion_sort(lst):
    for i in range(1, len(lst)): # after i iterations the first i elements are sorted
        for j in range(i, 0, -1): # trickle the ith element down to its position withing this sorted list
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]
            else:
                break # found final position of element

Runtime Analysis

Distribution-based Sorting

Counting Sort

Introduction

Implementation in Python

def counting_sort(lst):
    for i in range(1, len(lst)): # after i iterations the first i elements are sorted
        for j in range(i, 0, -1): # trickle the ith element down to its position withing this sorted list
            if lst[j] < lst[j-1]:
                lst[j], lst[j-1] = lst[j-1], lst[j]
            else:
                break # found final position of element

Runtime Analysis

Radix Sort

Introduction

Runtime Analysis

Comparison of Sorting Algorithms

Parameters

  • n - number of elements of the sequence to be sorted
  • k - maximum number of “digits” in a key
  • d - size of the domain of elements to be sorted

Runtime Complexity

Algorithm Worst-case Average-case Best-case
Bubble sort $O(n^2)$ $O(n^2)$ $O(n^2)$
Insertion sort $O(n^2)$ $O(n^2)$ $O(n^2)$
Heap sort $O(n \cdot\log n)$ $O(n \cdot\log n)$ $O(n \cdot\log n)$
Merge sort $O(n \cdot\log n)$ $O(n \cdot\log n)$ $O(n \cdot\log n)$
Quick sort $O(n^2)$ $O(n \cdot\log n)$ $O(n \cdot\log n)$
Counting sort $O(n)$ $O(n)$ $O(n)$

Note that, however, counting sort needs $O(m)$ memory where $m$ is the size of the domain of elements, e.g., for 64-bit integers there are $2^{64} = 18,446,744,073,709,551,616$ elements in the domain. Also to address these elements in an array we need $O(\log m)$ time.

Runtime Complexity

Algorithm Worst-case Average-case Best-case
Countingsort $O(n \cdot d)$ $O(n \cdot d)$ $O(n \cdot d)$
Radixsort

Memory Requirements