[WriteLog] Writelog for general logging?

[Joe Subich, W4TV]

 > I thought LOTW had the capability of simply rejecting a record that
 > matches 100% of what was loaded before. One would think this should
 > not be a major problem for such a complex server?

Yes, LotW has the capability or rejecting duplicate QSOs - but not
until the QSO has been checked to see if it is already in the database!
The most "expensive" part of the entire LotW process is checking to see
if each QSO in the upload is new or has already been entered into the
database.  Thus uploading QSOs that have already been uploaded consume
significant amounts of processor time - which before the new hardware
was responsible for causing LotW to be unable to process all of the new
logs it received.

Even *six years ago* the LotW staff requested that users *not* upload
duplicate QSOs.  That request is still on the LotW User home page:

> Jan 12, 2007: Duplicate QSO Records can Slow the Logbook System --
> Many logs submitted to LoTW contain a high percentage of duplicate
> QSO records. It appears that some folks are just adding a few QSOs to
> logs that have already been submitted, and then submitting the entire
> log file again.
>
> Please consider an alternative, which will save you bandwidth and
> processing time. The TQSL log signing program includes a "date
> window," which allows you to sign only those QSOs between two
> specified dates. Those records outside the specified dates are not
> signed are are not included in the log file you submit. Just enter
> the date of your last submission and TQSL will only sign previously
> unsubmitted records.
>
> Most logging software already accomplishes this in some way. If it
> doesn't we definitely recommend it. Restricting your submission to
> previously unsubmitted records will shorten processing time and
> quicken your uploads.

Unfortunately, WriteLog (and many of the other contest software) does
not use the tQSL "date window" to sign only "new" QSOs.  That is only
logical since there should be no duplication in signing QSOs from a
short duration contest - but the assumption breaks down when singing
the "general DXCC tracking" contest repeatedly.  One would hope that
Writelog would store the date/time of the "last uploaded QSO" in each
log and use that as parameter to limit uploads - or add a hidden
"LotW_QSL_SENT" field, set it to "Y" when uploading a log to LotW,
use it to prevent uploading duplicates and include it in exports so
other software does not upload the QOSs as duplicates but those are
not currently available.

73,

    ... Joe, W4TV


On 2/4/2013 8:54 PM, Andre VanWyk wrote:
> Joe,
>
> I thought LOTW had the capability of simply rejecting a record that matches 100% of what was loaded before. One would think this should not be a major problem for such a complex server?
>
> 73
> NJ0F
>
> Sent from my iPhone
>
> On Feb 4, 2013, at 7:49 PM, "Joe Subich, W4TV" <lists at subich.com> wrote:
>
>>
>>> It loads up to LOTW just fine via Email too.
>>
>> There is one major problem in using Writelog's general tracking mode
>> for everyday operation and then uploading to LotW directly.  WriteLog
>> is not alone in the problem - there are half a dozen other contest
>> programs with the same problem so I'm not picking on WriteLog.
>>
>> A LotW upload from WriteLog uploads *all* QSOs in that log - even
>> those that have been uploaded previously.  Thus, if one uploads to
>> LotW from the General DXCC tracking log every week or every month,
>> one is soon uploading nearly all duplicate QSOs.  Those duplicate
>> QSOs (not just from WriteLog) currently represent between 60 and
>> 75% of all uploads to Logbook of the World and were a significant
>> contributor to the recent backlog at LotW.
>>
>> If you are going to use a contest logging program for every day
>> operation, please export your log to ADIF and use something like
>> ADIF Master to trim that file to only those QSOs made since your
>> last upload *before* signing the log with tQSL and uploading.
>> Please do not simply upload the General DXCC Tracking log with
>> WriteLog's built-in upload and e-mail capability.  The WriteLog LotW
>> support is fine for a specific contest (e.g. CQWW CW, etc.) where a
>> log is uploaded *one time* after the contest is over.  However, the
>> General DXCC tracking "contest" is never over - unless you start a
>> new log each week/month and only upload at the end of that period.
>>
>> The alternative to using something like ADIF Master is to use one of
>> the "everyday" logging programs - DXLab Suite (DXKeeper), DX4Win,
>> Logger32, etc. - to import the ADIF and generate the upload since
>> they keep track of the QSOs uploaded to LotW and only upload QSOs
>> that are new since the last upload.
>>
>> 73,
>>
>>    ... Joe, W4TV
>>
>>
>> On 2/4/2013 6:26 PM, Andre VanWyk wrote:
>>> I have been using the DXpedition mode for general logging for many
>>> years. I think the latest version has a general logging module if my
>>> memory serves me right, but will have to check.
>>>
>>> It loads up to LOTW just fine via Email too.
>>>
>>> 73
>>> NJ0F
>>>
>>> --------------------------------------------------
>>> From: <ke6qr at comcast.net>
>>> Sent: Monday, February 04, 2013 4:59 PM
>>> To: <writelog at contesting.com>
>>> Subject: [WriteLog] Writelog for general logging?
>>>
>>>> Is it possible to use writelog for general day to day casual contacts?
>>>> What would be the best module to use? Thanks! Ke6qr
>>>> _______________________________________________
>>>> WriteLog mailing list
>>>> WriteLog at contesting.com
>>>> http://lists.contesting.com/mailman/listinfo/writelog
>>>> WriteLog on the web:  http://www.writelog.com/
>>>
>>> _______________________________________________
>>> WriteLog mailing list
>>> WriteLog at contesting.com
>>> http://lists.contesting.com/mailman/listinfo/writelog
>>> WriteLog on the web:  http://www.writelog.com/
>> _______________________________________________
>> WriteLog mailing list
>> WriteLog at contesting.com
>> http://lists.contesting.com/mailman/listinfo/writelog
>> WriteLog on the web:  http://www.writelog.com/
>