[CQ-Contest] Contest Log Processing

[W0MU Mike Fatchett]

N6TR, Tree was one of the lead Cabrillo guys.  I would email him directly.
I am sure he would be happy to assist.  If not he will shoot me. :) 

-----Original Message-----
From: cq-contest-bounces at contesting.com
[mailto:cq-contest-bounces at contesting.com] On Behalf Of Doug Smith
Sent: Thursday, January 29, 2009 9:30 AM
To: Franki ON5ZO
Cc: cq-contest at contesting.com
Subject: Re: [CQ-Contest] Contest Log Processing

On Thu, 2009-01-29 at 16:01 +0100, Franki ON5ZO wrote:
> This might be a controversial issue but I think today's programming 
> languages provide powerfull tools to process ASCII data under the form 
> of XML.
> If contest logs would come under the XML format with standard tags to 
> ID the fields, parsing logs would be relatively easy.
> The .Net platform has some nice built in routines to write data to or 
> read from XML where minimal code is required.

As the logchecker for the Tennessee QSO Party I'd rather we stick with
Cabrillo.  It's not at all difficult to parse (even with the simplest
languages - such as QBASIC!), is human-readable, and can be easily imported
into spreadsheet applications.  

=========================================================================

To my knowledge W3KM's Cab Evaluator is the only generally-available package
for logchecking.  To be honest I'm a bit surprised there isn't more
information out there about what's used by other contests.  I suppose there
could be a belief release of code would risk making cheating (or
denial-of-service attacks!) easier - but I would think some general concepts
could be described without security risks.

=========================================================================

In the past, for the TNQP each log was converted to Cabrillo (if it wasn't
already) and imported into a spreadsheet.  The spreadsheet would be sorted
in various ways to find dupes, to assign QSO points, and to count up
multipliers.  It would then be exported to a CSV
(tab-delimited) text file and stuffed in a directory with all the other CSV
files.  We'd then go through and search all the text files for the string
"K0WA".  And take a note of any matching records that didn't list the QTH as
Kansas, and mark them as busted QSOs.  

Unfortunately finding busted *calls* in this scheme was nearly impossible.
If a suspicious call (say, K1WA) was seen while importing/marking mults we
could check for a matching QSO from who we thought was really worked - but
I'm sure we missed a lot of busts.

=========================================================================

This year, all QSOs were brought into a MySQL database.  The Perl scripts
that did the importing assigned QSO points and checked for dupes as they
imported.  Further scripts:

- Checked each QSO for a valid QTH. (matching against another table in the
database which contained a list of valid QTHs)  This script also marked a
field in each QSO that was a new multiplier.

- Checked for bonus points - 100 for working K4TCG, 500 for making more than
ten QSOs from a county while mobile.  This script also assigned bonus
multipliers to mobiles.

- Cross-checked each QSO.  If there was a QSO like this:

Band	Mode	MyCall	MyQTH	HisCall	HisQTH
7	SSB	W9WI	ROBE	K0WA	KS

then there should be another QSO like this:

7	SSB	K0WA	KS	W9WI	ROBE

If the second QSO didn't exist but there *were* records specifying K0WA in
the MyCall field, then the first QSO was a bust.

- Prepared a report for each entrant showing the point value and multiplier
status of each QSO and noting if it was a busted QSO.

The time-consuming part, besides writing all the code, was going over each
report and ensuring every QSO that was flagged as "busted" actually was, and
that everyone got credit for everything they should have.  

One of the more vexing issues is dealing with inconsistent QTH
abbreviations.  ON - is it Ontario or Belgium?  OH: Ohio or Finland?  A
human can tell instantly (from the callsign) but it's going to require more
coding to get the computer to do so.  We had to manually edit a few records
where people entered something like "SA" for Saskatchewan or "NB" for
Nebraska instead of New Brunswick.

We have both Hardin and Hardeman Counties and tend to end up with
inconsistent abbreviations because of entrants like myself who don't read
the official abbreviation table until after sending in our logs.
(sorry, Kentucky QSO Party entrants who worked W9WI/m!)

I'm really not prepared to support this code so please don't ask for it!
I would imagine any reasonably competent Perl/SQL programmer (I'm not
one!) could produce something similar fairly quickly.

=========================================================================

On an only vaguely related subject, we received two entries that required
massive manipulation.  (luckily they didn't have many QSOs)  It seems that
the "received QTH" field of the Cabrillo file contained the entire name and
address of the station worked!

Obviously this was NOT transmitted over the air.  It looks as if these
entrants used a general QSO logging program that picked address data from
QRZ.com or some similar database and used it to populate the QTH field.
This obviously raises ethical questions!


_______________________________________________
CQ-Contest mailing list
CQ-Contest at contesting.com
http://lists.contesting.com/mailman/listinfo/cq-contest