In lecture Monday we gave formal definitions of the big-O and big-Omega notations. Here f and g are functions from N to the reals, ε is a positive real, and n and n0 are naturals.
Prove that if f = Ω(g) and g = Ω(h), then f = Ω(h). This is very similar to the proof at the top of page 39.
We are given that
We instantiate a, b, k, and l, noting from the definition that a and b are
positive. Then we define c to be ab and m to be max(k,l). Now we let n be
arbitrary and assume n ≥ m. Since n ≥ k, we know that f(n) ≥ ag(n),
and since n ≥ l, we know that g(n) ≥ bh(n). By arithmetic, f(n) ≥
abh(n) = ch(n). Since n was arbitrary, we have proved
∀n:(n ≥ m)→(f(n) ≥ ch(n)). Note that c is positive. By the Rule
of Existence twice, we have proved f = Ω(h) according to the definition.
Prove that if f1,...,fk each satisfy fi = O(h), then f1 + ... + fk = O(h). Here it is important that k is a constant with respect to n.
One way to prove this is mathematical induction on k, using the fact proved in
lecture (and on the board in Wednesday's discussion) that if f = O(h) and g =
O(h), then f + g = O(h). The base case of k = 1 just says "if f1 =
O(h), then f1 = O(h)", which is clearly true. Assume that for a
given k, whenever f1,..., fk are O(h), their sum is O(h).
We need to prove that for any functions f1,..., fk+1 that
are each O(h), their sum is O(h). Let g be the sum f1 + ...
+ fk. By the inductive hypothesis, g = O(h). Since fk+1
= O(h), we know that g + fk+1 = O(h) by the fact proved in lecture.
Since g + fk+1 is the desired sum of all k+1 functions, we are done.
We can also prove this directly without using induction. Translate each
statement "fi = O(h)" using the definition to get:
We instantiate each of the variables ci and mi. Then
we can m be the maximum of the mi's and let d be the sum of the
ci's. Now let n be arbitrary and assume that n ≥ m. For each
i, we know that n ≥ mi. By specifiying the premise involving
fi to this n and using Modus Ponens, we get that fi(n)
≤ cih(n). Adding these k different inequalities, we get that the
sum f1(n) + ... + fk(n) ≤ (c1 + ... +
cn)h(n) = dh(n). We have thus completed the proof that the sum is
O(h).
Prove that if f = O(h) and g = O(h'), then fg = O(hh').
Apply the definition of big-O to get:
We instantiate a, b, k, and l,
let n be arbitrary, and assume that n is at least m, the maximum of k and l.
Since n ≥ k, we know f(n) ≤ ah(n), and since n ≥ l, we know that
g(n) ≤ bh'(n). Multiplying these two inequalities together, we get that
f(n)g(n) &le' abh(n)h'(n). Letting d = ab, we have proved that there exist
a d and an m such that if n ≥ m, then (fg)(n) ≤ d(hh')(n), which proves
that fg = O(hh').
Prove that if b > 1 and ε > 0, then logbn = O(nε).
Remember that if the limit as n goes to infinity of f(n)/g(n) exists and is less than infinity, we may conclude that f = O(g). So here we want to compute the limit of (logbn)/nε. We can use L'Hopital's Rule to see whether this limit is infinite, calculating f'(n)/g'(n) where the prime means "derivative". This gives us b(logb-1 n)(1/n) over εnε-1, which by canceling a factor of 1/n is equal to (b/ε)(logb-1 n)/nε. This looks like the limit we started to evaluate, except for the b-1 in place of b in the top exponent. If we repeatedly use L'Hopital's Rule, we get several constant factors of b/ε in the front, and the log term eventually becomes 1 (which is log0 n) or something that remains less than 1 as n increases, but the bottom term remains nε which goes to infinity as n increases. Thus the limit is 0, meaning (by L'Hopital's Rule) that the original limit is 0 and thus less than infinity as desired.
Prove that if r > 1 and d > 0, then nd = O(rn).
Again form a limit and use L'Hopital's Rule to prove that it is zero. The calculus is actually simpler: we have the limit of nd/rn, and taking the derivative of top and bottom we get dnd-1 over (ln r)rn. If we do this again, we get polynomials of decreasing degree on top and a constant times rn on the bottom. Once the degree of the polynomial on top goes to 0 or below, the top no longer increases with n, while the bottom still does. At this point the limit as n goes to infinity of the ratio is 0, which means that the original limit is 0, which means that nd = O(rn). (Note that the assumption that r > 1 is important here -- it makes ln r a positive constant).
Put the following functions in order by growth rate, so that if f is before g, then f = O(g). Indicate which functions are also related by big-Omega and thus by big-Theta:
We can first put the three pure polynomials in order by degree, so that
6n+5 = Θ(21n + n1/2) and 21n + n1/2 =
O(n2.1). Then two of the other functions are polynomials multiplied
or divided by log n. 3n2log n + n1.5 is bigger than
n2 but smaller than n2+&epsilon for any positive number
ε, and n/(log n) is smaller than n but bigger than
n1-ε for any ε. This puts five of the seven
functions in order. The function e0.001n grows faster than any
polynomial. Finally, we have to look at 2(log n)1/2.
Its logarithm is just (log n)1/2, which grows more slowly than
εlog n for any positive ε. Thus
2(log n)1/2
grows more slowly than nε
for any ε.
Thus the final order is: 2(log n)1/2, n/(log n),
6n+5, 21n+n1/2, 3n2+n1.5, n2.1,
e0.001. The only Θ relationship is between the two linear
functions as mentioned above.
Last modified 14 September 2006