[TowerTalk] 1 or 2 dB

[Kim Elmore]

This is close to the right idea... Here's a gedanken experiment: have a 
1 dB difference in power on even vs odd minutes over a single contest at 
a major station. Execute a non-parametric permutation test where the 
statistic tested is the difference between the total score for even 
minutes and the total score for odd minutes. The station would have to 
ensure a 1 dB difference between P_out_even_minute and P_out_odd_minute 
on all bands. This will yield a p-value for the difference, where the 
p-value is defined as the probability that the difference occurs by 
random chance. Thus the lower the p-value, the smaller the probability 
that the difference is de to random chance. This way, we can test the 1 
dB hypothesis with all other parameters held constant because there is 
no physical reason for the score of all the even minutes and all the odd 
minutes to be different. The problem could even be made band specific: 
on what band or bands is a 1 dB difference most likely detectable in the 
score?

This is a great gedanken experiment because it holds everything constant 
*except* P_out. It would require a fair bit of effort at a big 
multi-multi contest station to get the best, most definitive data set. 
Alas, no major contester in their right mind would do this because they 
are in it to maximize their score, not to experiment with 1 dB. 
Regardless, this is about the only way the problem can be approached 
while isolating all other variables.

Kim N5OP

On 5/19/2022 12:47 PM, Michael Tope wrote:
> On 5/19/2022 8:43 AM, Kim Elmore wrote:
>> My test worked, so I'll try again...
>>
>> To everyone that has commented: Thank you very much! This discussion 
>> is fabulous!
>>
>> I concede the point that statistically, there is likely to be score 
>> improvements with 1 dB power increase and certainly 2 dB. These are 
>> best described as statistical improvements and I suspect that the 
>> data set has to be moderately large to detect a "significant" 
>> difference, that is for a statistic to have much power. Thus a DXer, 
>> who is not a contester (there are such things) probably won't notice 
>> a significant improvement in how long it takes to break through a 
>> pile-up with a power increase of 1 dB. Over a long run, a contester 
>> will be able to see a score increase at with a 1 dB power increase.  
>> A smaller data set (possibly much smaller) will show statistical 
>> score improvements given a 2 dB increase, significant at some 
>> arbitrary p-value.
> Perhaps this is what you are getting at, Kim, but unless an operator 
> is switching between power levels on some fairly short time interval, 
> so as to create two otherwise identical data sets, I don't see how you 
> can get any meaningful statistics when looking at small differences in 
> power. Comparing previous contests or within a single contest against 
> another nearby station running a slightly different power still leaves 
> you with a lot of uncontrolled variables.
>
>> I tend to skepticism when someone says they can notice a significant 
>> improvement given a *single* contest assuming everything else 
>> (including the operator) is held constant. It will take several 
>> contests to see a *statistical* improvement but I'll now bet it's 
>> there. Fewer contests will be required for a 2 dB increase. I prefer 
>> resampling (i.e. a permutation test) to parametric statistics simply 
>> because parametric test assumptions are almost always violated, 
>> leading to unknown degradations of the test's validity.
>
> You could test this hypothesis by taking some public logs and dividing 
> them in two by making one log out of even minute entries and another 
> log out of odd minute entries and comparing the scores (QSOs, QSO 
> points, mults, total score, etc). That would give you some feel for 
> how sensitive a test would be that interleaved Pout vs Pout + NdB with 
> similar dwell intervals. If the score difference between odd and even 
> minutes in the control logs (i.e. constant power logs) was 
> consistently less than 10%, then the power difference in the 
> interleaved power difference logs would have to show a score 
> difference significantly greater than 10% to be considered 
> statistically significant.
>
> 73, Mike W4EF
>
> _______________________________________________
>
>
>
> _______________________________________________
> TowerTalk mailing list
> TowerTalk at contesting.com
> http://lists.contesting.com/mailman/listinfo/towertalk