VarAC UTF-8 handling issue

FIX,V 3.6.6

At VarAC ver 3.5. When I send a UTF-8 text in VarAC chat, it is not displayed correctly.

example send "１２３４５６８９０” with UTF-8 3 byte code

Possible Causes

  • Fragmentation of UTF-8 codes that are multi-byte

  • Data changes due to compression process

check VARA modem API

Check the format for passing text to VARA modem. Check who is splitting the sentences into packet sizes.

These commands are sent using the TCP/IP interface through the 8300 port (by default). And the data stream is sent through the 8301 port (by default).

COMPRESSION OFF<cr>

Compression disabled

COMPRESSION TEXT<cr> (by default)

Huffman compression enabled, designed for type text information. Recommended for Winlink.

Who splitting ?

Application to VARA is simple stream TCP port.There is no specification of packet delimitation at this stage.

Tentative solution

I think it is necessary to check whether the tail is a broken UTF-8 before passing it to the display on the receiving side, and if so, delete it, buffer it, and pass it to the next display call.

UTF-8 3 byte format (1110-xxxx : 10xx-xxxx : 10xx-xxxx)

if tail -1 byte mask 0xE0, top of UTF-8 Delete and save for next time.

if tail -1 byte mask 0x80, 2nd or 3rd of UTF-8.

check -2, hec mask 0xe0 for find 1st byte.

UTF-8 2 byte format (110x-xxxx : 10xx-xxxx

UTF-8 4byte format (1111-0xxx : 10xx-xxxx : 10xx-xxxx : 10xx-xxxx)

The fundamental solution

VARA modem must understand UTF-8.

W8DED TNC