[RTTY] Optimal RTTY Filters

[Kok Chen]

As I mentioned in the earlier posting, a Raised Cosine Filter is an optimal narrow band filter (216 Hz wide at the 6 dB point, for a 45.45 baud 170 Hz shift RTTY signal), but gets beat by the Matched Filter when you can open up the passband really wide.

Both of them are classified as Nyquist Filters, i.e., there is no intersymbol-interference from both of these filters.

For years now, I have been looking for an easy way to come up with some other filter that has a bandwidth in between the two of these filters, still retaining the property of exhibiting no inter-symbol interference, and with a performance this is better than the Raised Cosine.  I finally found the solution to the puzzle late last year, and finally finished writing a white paper on it last week.

http://w7ay.net/site/Technical/Extended%20Nyquist%20Filters/index.html

The paper is at places slightly "technical" (it helps to know about Fourier Transforms, for example) and is meant for people who are designing filters instead of people who are simply using filters.  It can be useful for people who wants to implement a better filter in MMTTY for example.

In spite of the technical nature, there are a couple of charts in the paper that might be of interest to others since the charts are based on 45.45 baud signals.

If you have never seen a comparison between the Rasied Cosine filter and a Matched filter, Figure 3 shows the transfer function (magnitude spectrum) of the two filters.  For a 45.45 baud signal, that first "zero" in the spectrum occurs at 45.45 Hz from the origin.

Figure 4 shows the error rate from these two filters.  The numbers come from an experimental RTTY software platform that I use to explore RTTY algorithms.  (This program has a built in HF Channel Simulator that runs on multiple cores, so I won't have to wait forever for results from my 8-core Nehalem-based Macintosh).  

You can compare this chart with the AWGN chart that Alex VE3NEA had measured for MMTTY, MixW and TrueTTY that we had in the past referenced on this reflector.  I measure with the same 3 kHz noise bandwidth that Alex uses, so his chart and my chart are directly comparable.

>From Figure 4, you can see that a Matched Filter achieves the 0.1% character error rate with about 0.75 dB less SNR than the Raised Cosine Filter. Not a huge amount, but definitely measurable.  And especially if you look at this from an RTTY operator point of view (where you don't have much control over SNR): for the same -6 dB SNR, the Raised Cosine Filter produces about three times more errors than the Matched Filter.

Now, more interesting is what happens when a filter is too narrow (Figure 5) and too wide (Figure 6).

When a filter is too narrow, the main mechanism for error is the the intersymbol interference (a previous data bit smearing into the current data bit) causes errors to increase.  That is why the largest difference come when the SNR is good.

When a filter is too wide, the increased error come from the inclusion of extra noise in the demodulator.  This is why the largest difference is when SNR is poor.

The bandwidths cited in the two figures (0.6xBW, 0.8xBW, 1.5xBW and 3.0xBW) are the bandwidth of the data filter.

I.e., a Raised Cosine is down -6 dB at the frequency that corresponds to half the baud rate.  For a 45.45 baud signal, the filters are down -6 dB at 23 Hz from each RTTY tone.  For a 170 Hz shift signal, the corresponding filter will therefore be down -6 dB at ( 170 Hz + 23 Hz + 23 Hz ), or 216 Hz.

Likewise, a bandpass filter that represents the "3xBW" curve is down -6 dB at ( 170 Hz + 3*23 + 3*23 )Hz, or 308 Hz.  Not really a very wide filter by Amateur RTTY standards, but notice that the error rate has climbed by more than 20 times the error rate of the proper Raised Cosine Filter.

As mentioned earlier, the actual derivation of the impulse response of the new filter (what I am calling the "Extended Nyquist Filter", and a "Extended Raised Cosine" if you start with a Raised Cosine filter) in Equation 1 and its spectrum at Equation 3 can just be skimmed if you are not interested in the EE aspects of filters.

A bit more interesting is the spectrum of each Extended Raised Cosine in Figure 9.  Notice that there are various bandwidths that you can use (as long as you obey Equation 1 :-).

Even more interesting is Figure 10 that shows that practically speaking, a 2nd order extended Raised Cosine filter is virtually as good as a Matched filter.  Going back to Figure 9, you will see that the 2nd order filter is more than 30 dB down at about 80 Hz.  The 2nd order filter has a total filter bandpass that is 216 Hz wide at -6 dB and about 330 Hz wide at -30 dB gives virtually the same performance as a Matched Filter as long as you follow the filter design algorithm that is shown in Equation 1.

Given that this is an open white paper, there is no intellectual property implications for anyone else who wants to include "extended Nyquist filters," or specifically, "Extended Raised Cosine filters" into their software modems.

73
Chen, W7AY