[CQ-Contest] Skimmer (and sprints)

[Leigh S. Jones, KR6X]

I've been telling my closest friends within contesting circles about
the possibilities presented by this kind of technology for at least two
decades.

In about 1977 I had a long conversation with my brother-in-law David
Intersimone (a non-ham), who is now chief evangelist at Code Gear
-- the programming software group breakaway from Borland -- in
which the roadmap for a larger scale version of the skimmer software
was outlined.  I won't bore you with technical details, but while I
was essentially planning a system that did what the skimmer does,
the planned tool puts the skimmer to shame.  The whole conversation
took place in my driveway with David holding his car keys on his
way to his car.

The strength of these kinds of tools is that the operator who is able
to utilize this technology will be able to find new contacts "CQ-ing"
much more quickly with the skimmer than without.  It won't enhance
your ability to "call CQ" yourself -- except perhaps by allowing you
to respond to two or more answering stations in a row without a
"QRZ" in between.

During a DX contest then, a skimmer may allow a very big signal to
increase his rate during the very early going and during the "best of
times", but is of little or no use in helping one to dig down into the
noise level to chase very "weak ones" who are calling CQ themselves.
or answering your CQ's from the more distant continents.

But there are two areas where this took could be extremely effective.

Within a year after David Intersimone and I had roadmapped the
technology needed, I did some test recordings using a betamax
recorder at my home station while I was away operating the SS at
W6HX.  The betamax recorder started itself up perfectly and on
schedule, and produced four and a half hours of high bandwidth
tape of about 20 kHz of the 20M CW band beginning at 2100Z.
These tapes allowed me to process samples of actual contests
on computers through analog to digital converters.  Unfortunately,
I/O ports on computers in those days, including A/D converters
and serial ports, were inadequate to the challenge of real-time
application of this technology.  I could only post-process recorded
audio, but the proof of concept was there nonetheless.  My
first attempt to post-process the tape allowed only the A/D
converter step to be performed, and I was left with several reel-
to-reel tapes with the digitized data on it.  Each one held only
a fraction of the data that was present on the betamax -- being
limited to several one minute long files per reel.

It had taken me about four years since my conversation with my
brother-in-law to carry the technique forward this far, and I had
spent several hundred dollars just to cover the tiny pieces of the
required system that I needed to buy.  First, I'd needed to build
a IF frequency detector with a 20 KHz wide crystal filter to
convert my Drake R4A IF to "baseband".  I needed to attenuate
this audio frequency output to put it into the betamax input stream
at a point after the betamax performed its RF detection.  And, I'd
needed to buy an Analog Devices A/D converter and build up a
circuit that shifted its 16 bit parallel output format to a serial stream
at some incredible baud rate, for its day.   In those days,
baud rates of 300 to 1200 were common, and 9600 was incredibly
avant-guarde. But I had the bit stream formatted into 8 data bits,
one stop bit and no parity by a bunch of shift registers, the audio
was sampled at 44.1 kHz, and there were 16 bits per sample, plus
two stop bits.  I had to wait until computer serial ports caught up
with me to allow this to work, but eventually "non-protocol" serial
ports were available for this serial rate that was actually put into
practice at 1MBPS.

Then the mathematical choices had to be made.  I chose a 1024
sample width to process, used a "Hanning" windowing function in
front of the FFT, progressively overlapped the data by 75% so that
there were 4 FFT output values (24 bits now) per channel for
every 1024 samples of input audio.  The output represented an
input of about 20 kHz in 512 channels -- the other 512 channels
outputted by the FFT were essentially copies of the others.
Believe it or not, I'm still leaving out the technical details to
simplify my story here.

At about that time, home computers started to trickle onto the market,
and I bought an Apple II with floppy disk drives.  I had a great deal
of trouble at one point keeping the project alive, because the computer
which produced the original reel-to-reel tapes described above was
no longer to be available.  It was about 1983-84, and the data on
the tapes was almost lost.  I finally sent a small amount of the data
home to my Apple computer, filling one floppy disk at a time via
300 baud modem.  I had no way to directly connect the two computers,
so I settled for a few minutes of data between about 2130Z and
2135Z.  The data was stored to a couple boxes of 5-1/4 inch floppy
disks, and I started on a computer program to turn this time domain
data into frequency domain.  But, the 64K Bytes of Apple memory
presented me with a tough challenge.  I couldn't fit the 1024 samples
of audio data required to build a frame of FFT data, together with the
values for the windowing function, together with the program to run the
FFT and the windowing function, into the 64K memory together
with the Apple-C programming language and Apple DOS.  I couldn't
even process a single sample window.

I started talking about the project with my friends at that time, and
in a memorable conversation with (now) N6PN and N6TV at N6PN's
shack where Matt coined the term "Big Brother" as the name of the
system.  Frankly, I didn't like the name at first, but later it grew on me.

But that didn't happen until a couple more years had elapsed.  I didn't
buy a new and bigger computer until after my new wife and I rented
Matt's house, and moved in.  This should have been a wonderful time
for me and contesting, but, frankly, I operated little because my
attention was on computers and work for a few years.  The Northridge
earthquake intervened in 1994 and caused some distractions as well.

But I did some contesting from N6UR/W6RU in that decade while
I studied the problems.  I got ahold of a program that allowed me
to read Apple DOS disks under PC DOS and put my decades old
slice (2130Z-2135Z) of SS data onto the hard drive of my new
386 system.  The long planned mathematical computations were
really quite routine.

It started to look quite clear to me that the Sprint contest was quite
perfect for this tool just as soon as I got this close to success.  I
had detected signal strength calculated by channel for my 5 minutes
of data when I realized that a station who is answering CQ's in an
SS contest ends the QSO with a long transmission.  No reception
is possible on the band that this transmission is occurring on, so
at the end of the QSO an interruption occurs in the use of the system.
It's then necessary to listen for a period of time before locating a
fresh QSO.  But in the Sprint, the station abandoning the frequency
is listening for a period before the QSO ends, so he will have the
opportunity to collect data on activity on surrounding channels while
he listens to the exchange being received.  On adjacent channels he
will be able to locate an operator sending <callsign><nr><state><name>
<callsign> -- as opposed to <callsign><callsign><nr><state><name> --
slightly behind the exchange being received on channel, and so time his
move to the frequency of the fresh meat to occur immediately before
the callsign is sent.  In this fashion, only the frequency changes are
aided by the decoded channel occupancy, and all callsigns and
exchanges are manually copied by the operator, not by the machine.
No real value is found in SO2R capability if operating is done with this
technology, because there is no need to listen on a band on which
you are not transmitting to optimize your use of time.

I finally completed the full operation on my 5 minutes of data about
15 years after the contest in which the audio was first recorded.  So
much for "real time" conversion.  I lost the data in a disk crash, together
with the program code that I had created to do the conversion from
signal strength data by channel to letters and numbers.  I can tell you
that I marveled at the data when viewed.  It was clear that the system
could work incredibly well.  Although a few receiving errors were
obviously present, they were not a  frequent as I had imagined.  Most
signals were being copied well.  The software was not complete, from a
contesting standpoint.  I hadn't integrated with logging software, and did
not provide for automatic dupe checking.

I contacted K1EA/K1AR to offer the tool.  They probably laughed off
my effort.  I was too vague about what I was bringing to the table.  A
project.  I intended to contribute my valuable project.  I don't think I
was clear enough.

Just before this, sound cards became available for PC's, and in an odd
coincidence David Intersimone and I were together at Fry's
Electronics in Manhattan Beach and talking about my decade's
old pursuit of this system.  At that point in time, I found a brand of
sound card with a built-in AD2115 DSP chip and memory.  It was
a complete subsystem that would plug into the PC, digitize the audio
stream, and with appropriate software, perform the windowing functions
and FFT conversions needed to produce a stream of signal strength by
channel.  The PC could then display the decoded Morse by channel, do
duping, and connect with the logging software.

By e-mail, I told my friend George, W0UA, to buy one of these sound
cards, thinking that I'd shortly complete the software required to make it
a complete system.  Everything about it was workable, and I'd already
programmed every element of the system once.

Sadly, family pressures prevented me from completing my quest before
the junior "skimmer" project completed.  Frankly, by now the principle of
obviousness would prevent pantenting any such system, so I'm not too
dissappointed over not being the first to put such a system to use.

Obviously, I'd prefer to have used my "invention" from the 1970's for
a few big contests before it was widely known what I was doing.  It
wouldn't have given me DX contest victories, but would have given me
the few percentage points needed to dominate contests like the SS and
especially the Sprint.  The entire time, it has been a real temptation to
tip my hand and tell large numbers of people about the idea, but now
that someone has "gone public" that's impossible.  Family pressures have
kept me from completing the system and putting the system to use
for more than half of my lifetime.  Hopefully, now I will be able to go
out and simply buy this system in order to be able to use it, rather than
keeping the idea secret and fruitlessly working long hours to make
progress toward completion.

But Jim's notion that the "skimmer" is less applicable to the Sprint contest
is entirely wrong -- the Sprint is the perfect application of this
technology due to the factors I've mentioned above...

----- Original Message ----- 
From: "Jim George" <n3bb at mindspring.com>
To: ""José Nunes CT1BOH"" <ct1boh at gmail.com>
Cc: <dave at g4buo.com>; <cq-contest at contesting.com>
Sent: Sunday, February 10, 2008 2:46 PM
Subject: Re: [CQ-Contest] Skimmer and sprints


At 03:03 PM 2/7/2008 +0000, José Nunes CT1BOH wrote:
>G4BUO wrote
> >The only contests that will be immune from the influence of Skimmer are
> >the sprints - already a supreme test of CW skills.
> >
> >Perhaps all contests should have a compulsory QSY rule?
> >
> >Dave G4BUO
>
>Dave I think you are wrong.
>
>If there is a contest where Skimmer can really make a difference is the
>Sprint.
>Being a short contest, any marginal improvement, has a tremendous influence
>in the final score.

In my opinion, Dave is correct. The Sprint is a very quick split-second
competition. Even a half second error in timing will cause a missed
contact. If there ever were a contest that is a hard core test of operator
skill, immune to assistance outside the operator himself, it's the Sprint.

Jim George N3BB



_______________________________________________
CQ-Contest mailing list
CQ-Contest at contesting.com
http://lists.contesting.com/mailman/listinfo/cq-contest