Topband: FT-8 performance

[K4SAV]

I get the feeling that I must be the only person that has ever tested 
FT-8 to the extreme to see what it can do. It seems that everyone else 
just assumes it will do what the published information says.  It will 
not.  Below is a summary of my testing.

First I did a bunch of testing to see if I could figure out how the 
software determined the S/N number.  I measured the strength of a 
station calling CQ and subtracted that from the S meter reading for the 
2500 Hz USB band and that resulted in a number very close to what FT8 
reports.  I repeated the test multiple times.

Official documentation says FT8 can decode signals 24 dB below the noise 
floor.  That's an interesting comment because FT8 has no way of 
determining the level of the noise floor.  Even during the off period 
when no one is transmitting it can't determine the noise floor because 
the noise floor is not represented in the audio signal.  The receiver 
ACG brings the noise back up when no one is transmitting.  Actually for 
FT8 the amount of audio is greater in the off period than it is when 
stations are transmitting.  The only way to measure the noise floor is 
by making the measurement in the RF world, which FT8 can't do.

Actually what the program does is count everything in the 2500 Hz 
bandwidth as noise. (That is comprised mainly of strong signals, not 
noise.) It is limited to decoding signals 24 dB below that level.  From 
that info you can guess that if you narrow the bandwidth to eliminate 
most of the strong signals FT8 will decode weaker signals.  Yes that 
works.  Verified it myself and others have also found this to be true.  
You guys that like to operate FT-8 should take note of that comment.

I thought maybe it could decode stuff below the noise floor if all the 
signals were also below the noise floor, so I tested for that.  I found 
out that the program poops out long before the noise floor is reached.  
With my tests where I decreased the gain of signals from the NE (just 
after sunset in the NE) by pointing my antenna west and increasing the 
gain such that the biggest signals were 15 dB above the noise floor, FT8 
just about quits. There were probably 50 or more stations on the band 
and it was making one decode about every 5 or 10 minutes.

FT8 has a deep search mode where it uses stored data to make guesses at 
call signs some times.  I tested that too.  Before I started WSJTX I set 
up a poor signal to noise ratio test. Started WSJTX in the normal mode 
and it did very poorly.  Then I turned on deep search, increased the S/N 
and let the program look at the band for a little while.  Then I went 
back to the poor S/N condition without turning FT-8 off and turned on 
deep search and it made a lot more decodes.  Nearly all of those decodes 
were reported at -24 dB.  I think those were guesses and it just assigns 
-24 dB for guesses.

I was fooled by the results of a test I did when adding enough noise to 
the audio to cover up the signals and FT8 continued to decode the 
signals.  However I had previously had deep search on and it had already 
memorized the band when I did that.  It was just guessing that the same 
station is on the same frequency as previously.

I keep hearing reports from people that claim they are getting decodes 
without hearing anything in the audio. I set up conditions where that 
should have happened, but it never decoded anything.

In summary. it appears that on an almost dead band, CW (with narrow 
bandwidth) has about a 15 dB advantage at decoding weak signals.  On a 
very crowded band if FT-8 is using a 2500 Hz bandwidth, CW has a huge 
(many dBs) advantage over FT-8 because FT-8 can only decode about 24 dB 
below whatever your S meter is reading.  At 2500 Hz bandwidth in the 
FT-8 band on 160 my radio usually hangs at 20 to 30 dB over S9.

I was using a TS-990S receiver with no audio processing or noise 
limiting or blanking.

If anyone else has run similar tests. I would love to hear about it.

Jerry, K4SAV