Hard Problems: P, NP, and PTAS

It seems impractical at best, and possibly misleading, to offer summary-form notes on this topic.

Pseudo-Polynomial Time

The "hard problem" sub-topic discussed here is the situation with algorithms like:

1. The dynamic programming solution of the Knapsack problem
2. The Sieve of Eratosthenes
3. The linear-runtime Fibonacci number calculator fib_opt

What these have in common is that one of the input arguments is an actual number, as opposed to a size of some structure. We have shown that:

Sieve(n) = Θ(n log log n)

KS(n,W) = Θ(nW)

fib_opt(n) = Θ(n)

These are correct statements. However, a seemingly contradictory statement is the assertion that these problems are NP complete! How are these statements compatible?

Note that the argument n in Sieve and fib_opt, and the argument W in KS, are actual numbers, not sizes of some structure. So if we want to measure the runtime in terms of the size of the input, we have to write the estimate in terms of the size of the numerical input. In the case of KS, the size of W is s = log₂W, and our estimate becomes¹:

KS(n,s) = Θ(n×2^s)

Similarly the size of numerical input to Sieve is s = log₂n and our estimate becomes²

Sieve(s) = Θ(2^s log s)

and

fib_opt(s) = Θ(2^s)

In all cases, our asymptotic runtimes are exponential in the size of their input.

The original estimates are referred to as pseudo polynomial runtime estimates - they are polynomial in the numerical input but are actually exponential as functions of input size.

You can see this in action running fib_opt with very large integers, fib_opt(10n) running in about 100 times the time of fib_opt(n).