Math and Open Source

Jill Cirasella, Decoding Liberation's in-house librarian, points out a recent piece in Notices of the American Mathematical Society which speaks to the increasing significance of FOSS in mathematical research. (One of the authors, William Stein, is lead developer of SAGE, an open source mathematics software system.) Stein and co-author David Joyner offer this scenario:

Suppose Jane is a well-known mathematician who announces she has proved a theorem. We probably will believe her, but she knows that she will be required to produce a proof if requested. However, suppose now Jane says a theorem is true based partly on the results of software. The closest we can reasonably hope to get to a rigorous proof (without new ideas) is the open inspection and ability to use all the computer code on which the result depends. If the program is proprietary, this is not possible. We have every right to be distrustful, not only due to a vague distrust of computers but because even the best programmers regularly make mistakes.

Indeed, as the authors point out, the makers of proprietary mathematics software, such as Mathematica, are perfectly aware of this limitation, though they deploy some contorted rhetoric in order to cast it as nothing more than an occasional minor inconvenience. Joyner and Stein quote briefly from the Mathematica tutorial document "Why You Do Not Usually Need to Know about Internals," which is worth taking a closer look at. First, according to the tutorial,

You should realize at the outset that while knowing about the internals of Mathematica may be of intellectual interest, it is usually much less important in practice than one might at first suppose.

Why? Oh, because

the vast majority of the computations that Mathematica does are completely specified by the definitions of mathematical or other operations. Thus, for example, 3^40 will always be 12157665459056928801, regardless of how Mathematica internally computes this result.

Fabulous news, really -- Mathematica always produces correct results! Guess we can fire all those guys down in testing (and my friends who do formal methods research will have to retool). One of the keys to this is, apparently, the fact that

Mathematica can usually use its arbitrary-precision numerical computation capabilities to give results where every digit that is generated follows the exact mathematical specification of the operation being performed.

But wait: maybe you're not at all worried about correctness (a secondary consideration for most mathematicians) but would like hack up a more efficient approach for some Mathematical calculation. Might you not need to analyze the implementation of said calculation? Not really, because

most often the analyses will not be worthwhile. For the internals of Mathematica are quite complicated, and even given a basic description of the algorithm used for a particular purpose, it is usually extremely difficult to reach a reliable conclusion about how the detailed implementation of this algorithm will actually behave in particular circumstances.

There you have it: the code of Mathematica is so complicated that analyzing it wouldn't help you get a more efficient implementation, yet it's not so complicated so as to prevent blithe assurances of exact mathematical correctness.