freerdp - Cut english, paste Japanese

[grooveman]

Hi, I use Freerdp all the time to remote control windows servers. The other day I was remoted into a 2012R2 server and running SQL Server Management Studio. I was trying to delete some rows in a table when I noticed something very odd. It was a very simple sql command, but I kept getting this error:

[Hu]

This is not an English -> Japanese problem. This is a UTF-8 vs UTF-16 problem. Microsoft systems have an unfortunate habit of trying to use UTF-16 to represent strings, unlike Linux. Mostly, this just causes problems because it gives English-speaking Windows programmers the incorrect belief that they can treat one character (in a wchar_t) as one codepoint.

Somehow you copied a string as UTF-8, then inserted those same bytes into a system that incorrectly thought it was receiving UTF-16. As a result, every pair of characters in the input became a single wide character in the output. You can partially see this by copying your block text into a program that produces hexadecimal dumps, such as xxd. Take care that your tools do not further mangle the text with encoding transformations during this experiment.

I cannot explain why this happens to you or help you solve it, but I hope that knowing your problem is about UTF-8 vs UTF-16 will guide your investigation down the right path.

[grooveman]

Hi Hu,

Thanks for the response.

I figured it was something like that.... but didn't know windows used UTF-16.

So I conducted one more experiment that yields a work-around (because I don't see this getting fixed, unless I can set my linux box to UTF-16).

As I established above, if I cut from Server A and paste into server B, I get Japanese characters (or vice-versa). But, if I cut from Server A, paste into my linux system (from which I'm doing all this remoting), then cut that and paste it into Server B, it comes out using english letters (as was my intent).

So, I'll just have to cut my code from my windows servers, paste it into kwrite (or something similar), then paste it into the destination windows server. I don't like it... but at least it is a way around.

incidentally, is there a way to set my linux system to UTF-16? What complications would that cause?

Thanks.

G
_________________
To look without without looking within is like looking without without looking at all.

[Chiitoo]

This might be somewhat pedantic, and I could be wrong, but that looks more Chinese to me, though Japanese has at least some of those, too. :]

(Not that I can really read it.)


As for the issue, I don't unfortunately have any ideas on what is going on there. I might look into experimenting with locale settings (see for example the Localization Guide at our wikki).
_________________
Kindest of regardses.

[grooveman]

Google translate auto-detected it as Japanese. I defer to them, as I am not an expert.

I will revisit the localization guide.

Thanks.

G
_________________
To look without without looking within is like looking without without looking at all.

[Chiitoo]

Heh. That's interesting.

I gave Google translate a go as well, and to me, it says Chinese. Perhaps it depends on the time of the day, or alignment of things... or both.

OK now it gave me English, Korean, and Japanese, too... haha.
_________________
Kindest of regardses.

[Hu]

The fix is that your RDP client needs to handle transforming the text between UTF-8 / UTF-16 or it needs to properly advise the server about what type of text is being processed, so that the server handles the transformation. I am rather surprised that this ever made it into a release. I suspect you have some unusual configuration that is otherwise untested.

Linux treats strings as a null-terminated sequence of octets. You cannot use UTF-16 in such a system because the first character with its high byte clear would be interpreted as the null terminator for the string. You can use UTF-8 to represent codepoints that are not part of the Latin1 encoding. Your tools should automatically reencode between UTF-16 (Windows) and UTF-8 (everyone else) as needed and transparently to the user.

[Tony0945]

[Chiitoo]