CMPSCI 311 Discussion #5: Subquadratic Multiplication

David Mix Barrington

16/18 October 2006

Today we will work with a divide-and-conquer algorithm for integer multiplication from section 5.5 of KT. Of course in most cases in the real world, multiplication takes O(1) time, because our machines are built to multiply two 32-bit or 64-bit integers by a primitive operation that can be executed in O(1) clock cycles. But it's worthwhile to examine the way that the time to multiply two numbers depends on their length. An integer multiplication must eventually be implemented by bit operations, so looking at how many such operations we need to multiply n-bit integers can be important for designing the multiplication circuits that operate inside the machine. Also, we may want to multiply very long integers that are stored as sequences of several 32-bit or 64-bit components. Doing this will require implementing the multiplication in terms of additions and multiplications of single components, which is vey similar to implementing integer multiplication in terms of bits. What we'll do today is improve on the standard algorithm that takes O(n²) basic operations.

The basic trick is simple. Given two integers x₁ and x₂, each of at most n bits, we write x₁ = a₁2^n/2 + b₁ and x₂ = a₂2^n/2 + b₂. Then we have:

x₁x₂ = (a₁a₂)2ⁿ + (a₁b₂ + a₂b₁)2^n/2 + b₁b₂.

which suggests a recursive attack -- we can multiply the n-bit numbers using four n/2-bit multiplications. This gives the recurrence T(n) = 4T(n/2) + O(n), but this solves only to T(n) = O(n²).

But we can do better. Note that a₁b₂ = (a₁ + b₁)(a₂b₁) - a₁a₂ - b₁b₂. Since we have to compute a₁a₂ and b₁b₂ anyway, we only need one more multiplication (and two subtractions) to get the middle term of our some. This gives us T(n) = 3T(n/2) + O(n), which solves to T(n) = O(n^1.59) as discussed in section 5.2 of KT.

Use the given algorithm to multiply the eight-bit numbers 10100001 and 11001011. You should use two levels of recursion -- reduce the one eight-bit multiplication to three four-bit multiplications, then to nine two-bit multiplications, which you may do directly. (I'll put a three-bit multiplication table on the board. Because of carries your two-bit operations will sometimes have three-bit numbers.) Do the same multiplication by the ordinary method and compare the answers.
Ordinary method first:
```
              10100001
            * 11001011
            -----------
              10100001
             10100001
            00000000
           10100001
          00000000
         00000000
        10100001
       10100001
      ------------------
       111111110101011    decimal sanity check: 161 * 203 = 32683
```
Now for the fun part. Our three four-bit multiplications are:
- 1010 * 1100: This is 10*11*2⁴ + [(10+10)*(11+00) - (10*11) - (10*00)]2² + (10*00), which is 1100000 + [1100 - 110 - 0]*100 + 0 = 1100000 + 11000 = 1111000. (Decimal check: 10 * 12 = 120.)
- (1010 + 0001) * (1100 + 1011) = 1011 * 10111: This is 10*101*2⁴ + [(10+11)*(101+11) - (10*101) - (11*11)]2² + 11*11 = 10100000 + (101*1000 - 1010 - 1001)*100 + 1001 = 11111101. (Decimal check: 11 * 23 = 253.)
- 0001 * 1011: We know this is 1011, but carrying out the rule we get 00*10*2⁴ + [(00+01)*(10+11) - 00*10 - 01*11)]*2² + 01*11 = 0 + 1000 + 11. (Decimal check: 1 * 11 = 11.)
Now the final result is 111100000000000 + (11111101 - 1111000 - 1011)*10000 + 1011. This is 111100000000000 + 11110100000 + 1011 = 111111110101011, as desired. (Decimal check: 120*256 + (253 - 120 - 11)*16 + 11 = 32683.)
Now we turn to big integers stored in multiple words. Let's assume that adding or subtracting two k-word numbers costs k operations, that multiplying two one-word numbers takes one operation, and that shifting (multiplying by a power of 2^w, where w is the number of bits in a word) takes one operation. Compute the time needed to multiply two n-word numbers as a function of n, for both the ordinary and the new method, for n = 1, 2, 4, ...: Where is the crossing point where the new method becomes cheaper? (Only worry about n's that are powers of two.)
(Note that to multiply two numbers of 2n bits each by the new method, we need two shifts, three multiplications of n bits, two additions of numbers up to 4n bits, and two subtractions of numbers of 2n bits. This gives us a recurrence of T(2n) = 3T(n/2) + 14n + 2, with a base case of T(1) = 1. An easy and only slightly conservative way to describe the old method is that we compute n² individual products, shift each one into place, and add the results together. Actually this would be n² - 1 additions of n-word numbers which would be O(n³), so we need to do better. If we multiply the first number by each component of the other number, taking 2n to do each one, then shift these n n-word numbers into place and add them, we take n*2n + (n-1) + 2n(n-1) = 4n² - n - 1. Use this for your estimate.
The recurrence first:
The ordinary method:
- T(1) = 4 - 1 - 1 = 2
- T(2) = 16 - 2 - 1 = 13
- T(4) = 64 - 4 - 1 = 59
- T(8) = 256 - 8 - 1 = 247
- T(16) = 1024 - 16 - 1 = 1007, close but still less than 1071
- T(32) = 4096 = 32 - 1 = 4063, bigger than 3439

Last modified 19 October 2006