> I believe it would be possible to develop a methodology for the Orion
> where you could test all possible or likely combinations of buttons, knobs,
> and menu items, but considering how many ways all those controls
> interact, it'd probably take a couple of hours to conduct such a test.
<snip a lot>
> After each firmware change not only would new or changed features need
> to be tested, but all existing functions would have to be tested to be sure
> none were broken when the firmware was changed.
I don't entirely disagree with your assessment since I found problems with my
first-day-ship Orion that should have been obvious and caught before the radio
left the shop.
But I think you underestimate the scale of the problem. Since sequence of
control operation matters, the number of tests required to test every possible
permutation grows as N!. N on the Orion is a sizable number. Even if you
restrict the number of pushes/rotations/selections to a sequence of say no more
than 5 or 6 operations (or less) and aggregate all of the keypad buttons as one
type, the number gets very big. And if you do that, the control machine will
lose it's state and get lost on the 7th operation (you didn't test), or will
get lost based on the timing interval between operations . And if the latter
impacts the outcome, then you have an untenable state space (at least if you do
it by brute force). And in any case, you're talking about a lot more than a
couple of hours. My experience is that even the most well structured and
architected software can exhibit unanticipated behavior (usually at the worst
possible time, selected by Murphy himself).
If you restrict testing to "common" sequences, somebody with fumble fingers
will test the uncommon sequences for you and blow the radio (I found one
uncommon sequence, it isn't anything anyone might have reasonably anticipated,
and it hasn't been fixed yet .. I leave it as an exercise for the reader to
find the key sequence that leaves you with nothing but a loud whine in both
receivers, and requires a hard reset to fix). So far, I haven't seen anyone
else report the same issue. So is it really problem? Should it have been
found by the factory?
There are interesting ways to limit the search space for testing, or at least
provide a method to the madness, but no matter what you do it's still a large
problem.
I've drawn some conclusions about the structure of the software from it's
behavior. I'd love to get my hands on the source code :-)
I'm not trying to make an excuse for T-T here, but given the scale of the
problem, and the fact that there are currently no known bugs getting in the way
of how I use the radio, I can just enjoy the good stuff.
Grant/NQ5T
|