[RTTY] RttyCompare

[Jerry Flanders]

Hello Alex

Can you distribute the simulated audio files?

I have WSCGen, PathSym, and RttyCompare working fine. I can generate audio 
files and add the distortion effects to them with PathSim. Amazing how 
genuine the flutter effects sound!

I couldn't get Virtual Audio Cable 2.04 installed under XP Pro, so 
downloaded the current V3.something, which appears to work, but I have no 
confidence in it because I can hear a buzzing in the RTTY audio files when 
I route them through it. No buzzing otherwise. I was finally able to patch 
a file through it and into MMTTY and see copy (even with the buzz) , so I 
am encouraged that this is going to be a valid test procedure when I learn 
all the tricks and eliminate the buzz.

Since I have two sound cards in the computer my next step will be to try to 
avoid using VAC (and the buzzing) by just patching externally from one card 
to the other with a hardware cable. A guy with two computers could also 
just patch between them in the same way. (Guys with just one standard 
computer could probably play the files into an external audio recorder and 
then feed it back into the program under test via the sound card .)

In the process of learning how to use the various pieces, it occurred to me 
that if you just packaged and distributed the final path-simulated audio 
files, it would cut out much of the work and uncertainty for the test 
preparation process. It would also cut out many hours of the next guy 
trying to learn to use WSCGen, Pathsim, and VAC and possibly coming up with 
bad data due to a preparation error of some kind. Having standardized audio 
files ready to go should also help to standardize the tests.

Jerry W4UK

At 18:17 1/29/2005, you wrote:
>I was wondering how the popular RTTY programs compare to each other in terms
>of decoding accuracy. To my surprise, a search on the Internet proved that
>such a comparison had never been performed, and I had no choice but running
>a series of tests myself. I downloaded PathSim from the AE4JY web site and
>generated a set of messages with AWGN, flat and selective fading, flutter,
>and multi-path delay. Then I decoded these messages with MMTTY, MixW and
>TrueTTY, and tried to count the errors.
>
>It was immediately obvious why no one did such tests before. Counting errors
>was a very time consuming task and, more importantly, was possible only at
>low error rates. At CER = 10% or so it was virtually impossible to match the
>two sequences of characters and tell the correct letters from the wrong
>ones.
>
>To solve this problem, I wrote an RTTY message comparison tool, RttyCompare.
>The program converts the transmitted and received text to the Baudot code
>and compares the two sequences using the standard dynamic programming
>algorithm, counts the missing and incorrect codes, and instantly calculates
>CER. It is robust enough at error rates as high as 50% and even higher.
>
>Equipped with RttyCompare, I had no problems completing my tests. The
>comparison results are available at http://www.dxatlas.com/rtty with a
>description of the testing procedures. RttyCompare can be downloaded from
>the same site, with source code.
>
>I am far from drawing any conclusions from the tests, this is just a request
>for a discussion. The procedures that I used must be verified by other hams,
>and the results must be independently reproduced. Now that the testing
>procedure is documented and a complete list of required tools is identified,
>everyone can do a similar test - perhaps with longer messages, more
>realistic PathSim profiles, and for a wider variety of RTTY programs
>(cocoaModem ?).
>
>73 Alex VE3NEA
>
>
>_______________________________________________
>RTTY mailing list
>RTTY at contesting.com
>http://lists.contesting.com/mailman/listinfo/rtty