[CQ-Contest] Skimmmer vs. Humans at KCDXC

[Doug Grant]

Here's a little more detail on the Skimmer run at the KCDXC suite. It
was the last run, after all the humans were done...especially the guys
running the Skimmer.

We were lucky it ran at all...I bought a cable in the flea market to
connect the audio to my laptop, and it turned out to be OK. At least
we got the audio into the computer. No idea if it was overloading or
underdriving the sound card inputs, but we could see visual dots and
dashes in the display, and it was decoding stuff, so we went with it.

We did not want to take too much of the KCDXC guys' time setting it
up, and only made a couple of adjustments to knock down the background
noise being picked up from the next room. I did not play with the
number of decoders, etc., but N0SS has agreed to get me a copy of the
2008 tape and I will do a little fiddling to find the best Skimmer
setting, and will report back later.

Skimmer requires visual transcription of the calls it decodes in the
KCDXC pileup. In other words, when it is displaying the bandmap, a
callsign only stays on the display until another comes along on the
same frequency. Callsigns get printed in bold when they are heard
twice - this is a problem with the KCDXC tape, since a lot of the
calls are only sent once.

This means an op needs to watch the monitor, and write down a callsign
when it appears, and do it before another displaces it. In an effort
to make sure nothing got missed, there were two pairs of eyes watching
the display (mine and K5ZD's) and transcribing calls to paper. We
merged/purged the list to get the final score of 31. AB0X and KU1CW
from the KCDXC officiated, with W1ZT and W1WEF observing.

I suspect we could have optimized the settings a bit - a more skilled
Skimmer driver like N4ZR certainly would have been able to do better.
Those of you who attended the Contest Forum know first-hand I am a lid
when it comes to driving the audio settings on the laptop - the room
noise and acoustic echos messed up two of the audio-based SDR/Skimmer
demos, but fortunately the 3rd one (the most impressive one anyway)
worked fine, reading a previously-stored wideband digital file
provided by N4ZR.

Running the Skimmer at home on the 2007 KCDXC tape, and playing with
the settings (which I was too dumb to record), the machine ranged from
about 35 to 49 calls copied (the 2007 human winner had 57).

So let me summarize my observations about Skimmer as seen live this
past weekend:

1. The vast majority of the 100+ people who raised their hands at the
Contest Forum to signify that they had posted an opinion on Skimmer to
this reflector admitted by lowering their hands that they had never
actually seen the thing demonstrated before.

2. Skimmer was not really designed for the case of decoding two
simultaneous close-spaced signals in a narrow audio band (the KCDXC
signals are all between about 300 and 800 Hz). I suspect if VE3NEA
chose to play with that specific case, the Skimmer would do better. Or
maybe someone else will want to take up the challenge.

3. I think this is a pretty cool addition to the KCDXC competition. It
is like those competitions between human and computer chess players.
Perhaps we can set aside a run for machines for next year...just like
the 6-human runs, we can have 6 machines plug in.

4. There was a lot of excitement when the Skimmer score was being
announced. Some people were concerned that it might have beaten the
humans...maybe by a lot! I had talked to W9WI and a couple of the
other guys who usually are at the top of the list before the
competition, and told them that they were tasked with preserving the
dignity of the human race. They of course, delivered the goods. Nice
job! There was a very loud cheer when KU1CW posted the number.

5. What Skimmer is intended to do is look at a wide bandwidth and
decode a lot of signals simultaneously. It does that very well. Here
is the data point from the wideband recording of the 40M band during
the 2007 CQWW: in a 1-minute 45-second period, the Skimmer decoded
(and put in bold) 157 callsigns spread out over the band. Rounding a
bit, this is about 100 callsigns per minute. No human tuning a
receiver can do that. I estimate that a skilled human can probably
tune in and identify a new QRM-free station on an active band in about
3-4 seconds. That means 15-20 per minute, compared to Skimmer's
100/minute rate. And if you have several Skimmers on different bands,
the gap widens, since the human does not scale with more signal
inputs.

6. The Skimmer is still in development. I understand that work is
underway to integrate the SCP database and the K1TTT callsign pattern
algorithm to improve the accuracy and filter what gets displayed.

So I guess the moderators will allow the debate to restart, and now
that more people have actually seen the monster, they can post more
informed opinions. Maybe nobody has changed their opinions, but at
least they have a few more facts.

Thanks to N4ZR and N6TV for providing most of what I presented at the
Contest Forum, including all the test files, and to the KCDXC folks
for allowing me to run the Skimmer at their suite.

Hope everyone had a good time at Dayton and got home safely. See you next year!

73,

Doug K1DG

p.s. I'll put the Contest Forum slides on line. QRX a few days