small thoughts on performance engineering

[Eric Drexler's local account <drexler>]

Some probably-obvious thoughts regarding performance engineering:

In talking to Markm and Dean yesterday regarding the astounding
progress in the world of smalltalk, I raised the question of doing
performance engineering in smalltalk. Markm noted that the pattern of
costs in smalltalk will differ from that in X++, but we headed off
onto another topic.

Let us assume that the difference in cost-profile is partly known and
partly mysterious, that code development in smalltalk is twenteen
times faster than in X++, and that there is a lot of performance
engineering to be done, with some changes gaining a speed factor of a
zillion in X++, and some gaining speed factors of 1.05. If one were to
translate first, profiling tools would give an accurate
picture of where the time is going and all of the changes made (and
kept) would be in the right direction, but progress would be slow.  If
one were to translate after doing all performance
engineering, one would find that some changes had been in the wrong
direction (where the difference in cost-profile had been mysterious),
but progress would be fast. Hence the obvious suggestion: do a
lot of performance engineering in smalltalk at twenteen times the
development speed (starting with the clear-win-in-both-languages,
zillion-fold speedups, but perhaps proceeding to the point where one
suspects that 30% or so of the changes made are in the wrong direction
with respect to X++ performance), then translate and do further work
where the real profiles known and the final code is
available. But so long as faster-in-smalltalk correlates with
faster-in-X++, and the mean speedup per change in smalltalk is at
least one-twenteenth the mean speedup expected from better-targeted
changes in X++, stick with smalltalk.

The case for doing performance engineering in smalltalk for
performance engineering involving how data is stored on disk is even
stronger, since the reliability of cost-profile information should be
better. I'm glad to hear that Michael is doing a smalltalk urdi and
moving into this area.

A final obvious remark on Xanadu progress: wow!