Summary: Since Koha switch to mysql_auto_reconnect (#10611), when MySQl connection drop and has to be auto-reconnect, the reconnected connection will use MySQL default character encoding. The original connection is always UTF8 as explicitly set by "set names utf8" in _new_dbh, but the reconnected connection seems to depend on MySQL configuration (in our case latin1). In our case, the reconnected connection will change all non-latin characters into "?". How to reproduce: 1) Use default my.cnf from Debian or Ubuntu. 2) Create a bib/item with non-latin UTF8 characters, let's say barcode "001" has title "ประเทศไทย" 3) Start Koha SIP2 server (since it could hold MySQL connection long enough that it will have to be reconnected) 4) Connect to SIP2 server and send command to get item detail for "001". You will get the correct information (title = "ประเทศไทย") 5) Do not close the connection, wait for about 15-20 mins (so that the MySQL connection drop) 6) Send command to get item detail for "001". You will now get title = "?????????". Possible fix: In our case, we switch back to old code that use DBIx ping(). Changing my.cnf should also work. However the behavior of the first connection and the reconnected one should be the same regardless of MySQL configuration.
Did you try with 3.22?
No, I haven't try with 3.22 yet. But look like the new code in 3.22 should solve the problem. I will try.