I was wondering how the popular RTTY programs compare to each other in terms
of decoding accuracy. To my surprise, a search on the Internet proved that
such a comparison had never been performed, and I had no choice but running
a series of tests myself. I downloaded PathSim from the AE4JY web site and
generated a set of messages with AWGN, flat and selective fading, flutter,
and multi-path delay. Then I decoded these messages with MMTTY, MixW and
TrueTTY, and tried to count the errors.
It was immediately obvious why no one did such tests before. Counting errors
was a very time consuming task and, more importantly, was possible only at
low error rates. At CER = 10% or so it was virtually impossible to match the
two sequences of characters and tell the correct letters from the wrong
ones.
To solve this problem, I wrote an RTTY message comparison tool, RttyCompare.
The program converts the transmitted and received text to the Baudot code
and compares the two sequences using the standard dynamic programming
algorithm, counts the missing and incorrect codes, and instantly calculates
CER. It is robust enough at error rates as high as 50% and even higher.
Equipped with RttyCompare, I had no problems completing my tests. The
comparison results are available at http://www.dxatlas.com/rtty with a
description of the testing procedures. RttyCompare can be downloaded from
the same site, with source code.
I am far from drawing any conclusions from the tests, this is just a request
for a discussion. The procedures that I used must be verified by other hams,
and the results must be independently reproduced. Now that the testing
procedure is documented and a complete list of required tools is identified,
everyone can do a similar test - perhaps with longer messages, more
realistic PathSim profiles, and for a wider variety of RTTY programs
(cocoaModem ?).
73 Alex VE3NEA
_______________________________________________
RTTY mailing list
RTTY@contesting.com
http://lists.contesting.com/mailman/listinfo/rtty
|