[TenTec] Re: Continuing Orion evaluation/Dragonball

[nq5t at comcast.net]

> I believe it would be possible to develop a methodology for the Orion
> where you could test all possible or likely combinations of buttons, knobs,
> and menu items, but considering how many ways all those controls
> interact, it'd probably take a couple of hours to conduct such a test.
<snip a lot>
> After each firmware change not only would new or changed features need
> to be tested, but all existing functions would have to be tested to be sure
> none were broken when the firmware was changed.

I don't entirely disagree with your assessment since I found problems with my 
first-day-ship Orion that should have been obvious and caught before the radio 
left the shop.  

But I think you underestimate the scale of the problem.  Since sequence of 
control operation matters, the number of tests required to test every possible 
permutation grows as N!.  N on the Orion is a sizable number.  Even if you 
restrict the number of pushes/rotations/selections to a sequence of say no more 
than 5 or 6 operations (or less) and aggregate all of the keypad buttons as one 
type, the number gets very big.  And if you do that, the control machine will 
lose it's state and get lost on the 7th operation (you didn't test), or will 
get lost based on the timing interval between operations .  And if the latter 
impacts the outcome, then you have an untenable state space (at least if you do 
it by brute force).  And in any case, you're talking about a lot more than a 
couple of hours.  My experience is that even the most well structured and 
architected software can exhibit unanticipated behavior (usually at the worst 
possible time, selected by Murphy himself).

If you restrict testing to "common" sequences, somebody with fumble fingers 
will test the uncommon sequences for you and blow the radio (I found one 
uncommon sequence, it isn't anything anyone might have reasonably anticipated, 
and it hasn't been fixed yet .. I leave it as an exercise for the reader to 
find the key sequence that leaves you with nothing but a loud whine in both 
receivers, and requires a hard reset to fix).  So far, I haven't seen anyone 
else report the same issue.  So is it really problem?  Should it have been 
found by the factory?

There are interesting ways to limit the search space for testing, or at least 
provide a method to the madness, but no matter what you do it's still a large 
problem.

I've drawn some conclusions about the structure of the software from it's 
behavior.  I'd love to get my hands on the source code :-)

I'm not trying to make an excuse for T-T here, but given the scale of the 
problem, and the fact that there are currently no known bugs getting in the way 
of how I use the radio, I can just enjoy the good stuff.

Grant/NQ5T