2025-05-06 Understanding the FT8 binary protocol ================================================ Paraphrasing `Wikipedia `__: **FT8** (short for Franke-Taylor design, 8-FSK modulation) is a `frequency shift keying `__ digital `mode of radio communication `__ [..] a popular form of digital weak signal communication used primarily by amateur radio operators to communicate on `amateur radio bands `__ You can roughly divide FT8 into three separate parts: 1. A communication protocol defining sequences and states for each endpoint 2. A waveform encoding/decoding system to ensure the data survives transmission and receipt 3. A binary protocol for actually encoding the underlying message data All of these have a bunch of clever stuff in them, but I want to focus on the internal binary protocol. Most of this is taken from `The FT4 and FT8 Communication Protocols by Joe Taylor, K1JT, Steve Franke, K9AN, and Bill Somerville, G4WJS `__, which is well worth a read. The goal here is to be a relatively accessible high level overview that doesn’t require too much pre-existing knowledge or familiarity. Basic Structure --------------- Each FT8 transmission contains exactly 77 bits of user information. The first 3 bits define the type of the message (and associated structure and encoding of the remaining bits), with type 0 using the second 3 bits to define a subtype. Types include standard messages, free text messages and telemetry messages, as well as ones for “special operating activities” like `ARRL Field Day `__ and VHF contests Putting aside the more niche ones (see the protocol details for the full table), they are (with tags explained later): .. csv-table:: :header-rows: 1 "Type", "Subtype", "Description", "Example", "Tagged Content", "Tagged Content Length (bits)" "0", "0", "Free Text", "``TNX BOB 73 GL``", "``f71``", "71" "0", "5", "Telemetry", "``123456789ABCDEF012``", "``t71``", "71" "1", "N/A", "Standard Message", "``K1ABC/R W9XYZ/R R EN37``", "``c28 r1 c28 r1 R1 g15``", "74" "4", "N/A", "Nonstandard Calls", "`` PJ4/K1ABC RRR``", "``h12 c58 h1 r2 c1``", "74" A key step to getting things to fit in the 71-74 bits is using thoughtful encodings for each piece of data. `7-bit ASCII `__ might have worked for the (early) web, but only gives you 10 characters max, just about enough for two five-letter callsigns. But by taking advantage of e.g. most callsigns having a conventional structure (case insensitive, up to three letter suffix, etc.), or there only being 180*180 = 32400 four-digit `Maidenhead Locators `__, which not only fits in 15 bits but gives you a few unused values you can use for other information, FT8 fits an impressive amount of data into each message! Each tag is a specific encoding of information into a bit pattern, with the letter part indicating the type of data (e.g. ``c`` for callsigns, ``r`` for signal reports or `rogers `__) and the number indicating the size in bits. Further details are given in each section about the message type, or again in full in `the Protocols document `__. Telemetry Message Type ---------------------- The Telemetry message type is the simplest - type ``0``, subtype ``5``, then 71 bits of arbitrary data (``t71``). The Protocols document rather generously calls this “18 hexadecimal digits”, with the caveat that one of them is only 3 bits long(!) - 71/4 = 17.75 There’s very little in the documentation about this, but if you’re looking to send arbitrary data blobs, this looks to be the way! Free Text Message Type ---------------------- The Free Text Message type is the second simplest - type ``0``, subtype ``0``, then up to 13 characters encoded into 71 bits (``f71``). The characters are limited to ``[0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ+-./? ]``, which is a 42 character alphabet, and therein lies the “trick” to fitting it all in - log2(42) is just under 5.4, which is to say you can express a character as a 6 bit number, with some “space” left - and 71/5.4 is just over 13. Essentially the encoding algorithm expresses the string as a number in base 42, then converts it to binary - see it in action in ``free_text_to_f71.f90`` in https://www.arrl.org/files/file/QEX%20Binaries/2020/ft4_ft8_protocols.tgz or in `the GitHub WSJT-X mirror in "free_text.f90" `__. Standard Message Type --------------------- The Standard Message (type ``1``) is where things start to get really interesting. An example would be ``K1ABC/R W9XYZ/R EN37`` (that’s W9XYZ/R telling K1ABC/R they’re in EN37). The structure is ``c28 r1 c28 r1 R1 g15``, which is, in brief: - ``c28``: 28 bits encoding either a “standard” callsign, or one of the special message words ``CQ``, ``DE``, or ``QRZ`` (CQ possibly with modifiers, allowing you to e.g. ``CQ GB`` or similar) or a hash code used for “nonstandard” callsigns (see the Nonstandard Callsigns section) - ``r1``: 1 bit encoding the presence of the ``/R`` suffix (which appears to be a US suffix meaning “rover” which seems to mean travelling/portable) - ``R1``: 1 bit encoding the presence of “R” in the message to indicate acknowledgement of a grid / signal report - ``g15``: 15 bits encoding either a 4 character Maidenhead locator, a signal report, or one of the strings ``{RRR|RR73|73|}`` Both ``c28`` and ``g15`` make full (or at least very-near-full!) use of the bit space to encode the necessary data. For example, ``c28`` values: .. csv-table:: :header-rows: 1 "``c28`` value as decimal", "Representation" "0", "``DE``" "1", "``QRZ``" "2", "``CQ``" "3-532443 (with some gaps)", "``CQ`` plus a suffix of either ``[0-9]{3}`` or ``[A-Z]{1,4}``" "2063592 + (0 to 4194303)", "Hash codes representing 'nonstandard' call signs" "6257896 + (0 to 268435455)", "'Standard' call signs" Since a “standard” callsign consists of: a one- or two- character prefix, at least one of which must be a letter, followed by a decimal digit and a suffix of up to three letters. this again narrows down the actual value space significantly, and you can fully encode those in 28 bits with plenty of room to spare for all the other values. Nonstandard Callsign Message Type --------------------------------- The 28 bit encoding is enough for most callsigns, but not all - e.g. special event calls and compound calls. But so long as the special call is 11 or fewer characters, it can be represented. The Nonstandard Call message (type ``4``) has structure ``h12 c58 h1 r2 c1``, which is: - ``h12``: 12 bits representing the hash of a (hopefully) previously-decoded callsign. This saves 16 bits from the standard ``c28`` representation (and more when you factor in ``/R`` etc.) at the cost of the receiver needing to retain some state, and potentially not be able to show a matching callsign. WSJT-X renders this as a callsign (or ellipsis when not found) in angle brackets, e.g. ```` or ``<...>`` - ``c58``: 58 bits representing a “nonstandard” callsign up to 11 characters long. This uses a similar approach to the ``f71`` free text field, but with a smaller character set (``[0-9A-Z /]``) only 38 characters long - ``h1``: 1 bit indicating whether the hashed callsign is the second callsign (sender) or first (recipient) - ``r2``: 2 bits representing the words ``{|RRR|RR73|73}`` - ``c1``: 1 bit indicating if it’s a CQ in which case the ``h12`` is ignored So -- There’s plenty of cleverness in FT8, particularly around the full message encoding and waveform transformations. This just scratches the surface of some of the basic parts of the binary message structure. For more details, definitely read `“The FT4 and FT8 Communication Protocols” `__, from which most of this is taken.