Concepts

• Recurrence
• Function f(n) defined for n = 0, 1, 2, ... (or some other initial segment of integers)
• Equation or inequality that describes f(n) in terms of f(k), k < n
• Examples

f(n) = f(n-1) + f(n-2) // defines Fibanacci sequence
g(n) = 2g(n/2) + n     // uses integer arithmetic

• Initial conditions
• Required values for first few inputs
• Examples:

f(0) = 0, f(1) = 1 // Fibanacci
may be unspecified

• Exact Solution
• Find (set of all) function(s) f(n) satisfying the recurrence (and the ICs)
• Example: Fibanacci
  f(n) = (φ₁ⁿ - φ₂ⁿ) ⁄ √5
• Asymptotic Solution
• Example:
  g(n) ≤ O( n log n )

Linear Recurrences, Polynomial Method

• Linear recurrence of order k: f(n) = a₁f(n - 1) + a₂f(n - 2) + ... + a_kf(n - k), where the coefficients a_k are constants
• Characteristic polynomial: P(x) = x^k - a₁x^k-1 - a₂x^k-2 - ... - a_k
• Characteristic equation: P(x) = 0 , or x^k = a₁x^k-1 + a₂x^k-2 + ... + a_k
• Roots of characteristic polynomial are solutions to recurrence
• If r₁ and r₂ are roots then f(n) = c₁r₁ⁿ + c₂r₂ⁿ is a solution for any constants c₁ and c₂ .
• Degree two case:
• f(n) = a₁f(n - 1) + a₂f(n - 2)
• Suppose r² - a₁r - a₂ has two distinct roots r₁ and r₂.
  Then the sequence {f_n} is a solution of the recurrence iff
  f_n = c₁r₁ⁿ + c₂r₂ⁿ for n = 0, 1, ..., where c₁ and c₂ are constants.
• Suppose r² - a₁r - a₂ has only one root r₀. Then the sequence {f_n} is a solution of the recurrence iff
  f_n = c₁r₀ⁿ + c₂nr₀ⁿ for n = 0, 1, ..., where c₁ and c₂ are constants.
• General degree n case - mutatis mutandis
• Example: Fibanacci

Techniques for non-linear cases

• Substitution
• Recursion Tree
• Master Theorem.
  • Hypotheses:
    1. a >= 1 and b > 1 are constants
    2. f(n) is a function defined for the non-negative integers
    3. T(n) satisfies the recurrence T(n) = aT(n/b) + f(n) [divide-and-conquer recurrence]
  • Conclusions:
    1. If f(n) <= O(n^{log_ba - ε}) for some constant ε > 0, then T(n) = Θ(n^log_ba).
    2. If f(n) = Θ(n^log_ba), then T(n) = Θ(n^log_ba lg n).
    3. If f(n) >= Ω(n^{log_ba + ε}) for some constant ε > 0, and if af(n/b) <= cf(n) for some constant c < 1 and large n, then T(n) = Θ(f(n)).

The Master Theorem gives information about the relative asymptotic dominance of the "divide" term aT(n/b) versus the "overhead" term f(n). Note that

• p(n) = n^log_ba is a solution to the pure recurrence
           P(n) = aP(n/b)
  as shown by substitution:
           ap(n/b) = a(n/b)^log_ba = a(n^log_ba / b^log_ba) = a(n^log_ba / a) = n^log_ba = p(n)
  
• Thus we can paraphrase the Master Theorem as follows:
  1. When the non-recursive term f(n) is asymptotically dominated by something strictly slower growing than the pure recurrence P(n), then L(n) and p(n) are asymptotically the same.
  2. When f(n) is asymptotically equivalent to P(n), then L(n) is asymptocially equivalent to p(n) multiplied by log n. (This happens because log n is the depth of the recursion tree.)
  3. When f(n) asymptotically dominates something strictly faster growing than P(n), and f(n) is sub-recursive, then L(n) is asymptocially equivalent to f(n).

The master theorem has somewhat intricate hypotheses. Yet it is applicable to a sizeable portion of algorithms of the divide and conquer design, wherein a computation is reduced to a recursive call on a smaller input set plus an overhead term.