Typing Hindi into Unicode Textbox shows question marks

The place for threads about TimoSoft EditControls.
Post Reply
doublespin
Cadet
Posts: 5
Joined: 04 Aug 2010, 16:16

Typing Hindi into Unicode Textbox shows question marks

Post by doublespin » 04 Aug 2010, 16:57

Hi
I want to use your Unicode Textbox Control in VB6 project for inputing Hindi Characters and bind the control to an access database.
My XP machine is Asian language enabled and the input language (keyboard) is hindi traditional. As font of the textbox I choosed Arial Unicode MS (also tried Mangal, which is an Hindi font).
Typing directly in the textbox control displays questionmarks ??? (same as using textbox control of MS Forms 2.0). In notepad typing Hindi works without any problems (displays the characters correct). Copy and paste from notepad to the text control ist working properly. The Databinding to database is working also, so I'm able to write the pasted string to Access and read it back.
But what about typing?
Do you have any suggestions?
Thanks.
Regards doublespin

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 09 Aug 2010, 23:20

Hi,

this is a known problem. Working on a solution is not easy because I don't know anything about Hindi (except that it's a language :lol:). You would help me alot, if you could post some Hindi text here that works if copy and pasted and doesn't work if entered directly. The information that it works if copy-and-pasted already was very helpful, because this detail was new to me.
It's very strange that this problem seems to exist with Hindi only.

Regards
TiKu

P.S.: Sorry for my late reply, but I have been at the Wacken Open Air festival during the last week.
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

doublespin
Cadet
Posts: 5
Joined: 04 Aug 2010, 16:16

Re: Typing Hindi into Unicode Textbox shows question marks

Post by doublespin » 13 Aug 2010, 10:51

Hi TiKu

Thanks for your reply.

Attached you will find some Hindi strings.

I tried to solve the typing problem writing some code to KeyDown and KeyPress event of the textbox. At the KeyDown event I assign KeyCode and Shift to global variables. At the KeyPress event I use the KeyCode and the Shift variables to set the value of the KeyAscii to the unicode value for the hindi character. Here is my code. But attention, the code may not contain all hindi chars.

Public Declare Function GetKeyboardLayoutName Lib "user32" Alias "GetKeyboardLayoutNameA" (ByVal pwszKLID As String) As Long

Private Sub Textbox1_KeyDown(KeyCode As MSForms.ReturnInteger, Shift As Integer)
i_keyCode = KeyCode
i_shift = Shift
End Sub

Private Sub Textbox1_KeyPress(KeyAscii As MSForms.ReturnInteger)
Dim ascii As Integer
ascii = KeyAscii
KeyAscii = gethindiCode(ascii, i_shift, i_keyCode)
End Sub

Private Function gethindiCode(KeyAscii As Integer, Shift As Integer, KeyCode As Integer) As Integer
Dim tmpStrg As String
Const KL_NAMELENGTH = 9

tmpStrg = String(KL_NAMELENGTH, 0)
GetKeyboardLayoutName tmpStrg 'Keyboard auslesen

If Left(tmpStrg, 8) = "00010439" Then 'hindi keyboard layout

'keyCode bringt den KeyCode für die Grossbuchstaben
If Shift = 0 Then 'Wenn nicht shift gedrückt -> auf die Kleinbuschstaben setzen
If (KeyCode > 64) And (KeyCode < 91) Then 'A bis Z
KeyCode = KeyCode + 32
End If
End If

If KeyAscii = 63 Then 'Fragezeichen --> KeyCode einsetzten
Select Case KeyCode
Case 113
KeyAscii = 2380
Case 119
KeyAscii = 2376
Case 101
KeyAscii = 2366
Case 114
KeyAscii = 2368
Case 116
KeyAscii = 2370
Case 121
KeyAscii = 2348
Case 117
KeyAscii = 2361
Case 105
KeyAscii = 2327
Case 111
KeyAscii = 2342
Case 112
KeyAscii = 2332
Case 97
KeyAscii = 2379
Case 115
KeyAscii = 2375
Case 100
KeyAscii = 2381
Case 102
KeyAscii = 2367
Case 103
KeyAscii = 2369
Case 104
KeyAscii = 2346
Case 106
KeyAscii = 2352
Case 107
KeyAscii = 2325
Case 108
KeyAscii = 2340
Case 122, 120
KeyAscii = 2306
Case 99
KeyAscii = 2350
Case 118
KeyAscii = 2344
Case 98
KeyAscii = 2357
Case 110
KeyAscii = 2354
Case 109
KeyAscii = 2360
Case 81
KeyAscii = 2324
Case 87
KeyAscii = 2320
Case 69
KeyAscii = 2310
Case 82
KeyAscii = 2312
Case 84
KeyAscii = 2314
Case 89
KeyAscii = 2349
Case 85
KeyAscii = 2329
Case 73
KeyAscii = 2328
Case 79
KeyAscii = 2343
Case 80
KeyAscii = 2333
Case 65
KeyAscii = 2323
Case 83
KeyAscii = 2319
Case 68
KeyAscii = 2309
Case 70
KeyAscii = 2311
Case 71
KeyAscii = 2313
Case 72
KeyAscii = 2347
Case 74
KeyAscii = 2353
Case 75
KeyAscii = 2326
Case 76
KeyAscii = 2341
Case 90, 88
KeyAscii = 2305
Case 67
KeyAscii = 2339
Case 86, 66, 78
KeyAscii = 2355
Case 77
KeyAscii = 2358
Case 96:
'`'
Case 49
If Shift = 0 Then
'1'
Else
KeyAscii = 2317
End If
Case 50
'2'
Case 51
If Shift = 0 Then
'3'
Else
'KeyAscii = 2373
End If
Case 52
'4'
Case 53
'5'
Case 54
'6'
Case 55
'7'
Case 56
'8'
Case 57
'9'
Case 48
'0'
Case 45
'-'
Case 61
KeyAscii = 2371
Case 92
KeyAscii = 2377
Case 91
KeyAscii = 2337
Case 93
KeyAscii = 2364
Case 59
KeyAscii = 2330
Case 39
KeyAscii = 2335
Case 44
','
Case 46
'.'
Case 47
KeyAscii = 2351
Case 126
':'
Case 33
KeyAscii = 2317
Case 64
KeyAscii = 2373
Case 35
KeyAscii = 2381
Case 36
KeyAscii = 2352 ';&#2381 '???
Case 37
KeyAscii = 2332 ';&#2381;&#2334 '???
Case 94
KeyAscii = 2340 ';&#2381;&#2352 '???
Case 38
KeyAscii = 2325 ';&#2381;&#2359 '???
Case 42
KeyAscii = 2358 ';&#2381;&#2352 '???
Case 40
','
Case 41
':'
Case 95
KeyAscii = 2307
Case 43
KeyAscii = 2400
Case 124
KeyAscii = 2321
Case 123
KeyAscii = 2338
Case 125
KeyAscii = 2334
Case 58
KeyAscii = 2331
Case 34
KeyAscii = 2336
Case 60
KeyAscii = 2359
Case 62
KeyAscii = 2404
Case 63
KeyAscii = 2399
Case 32
' '
Case 187 '=
If Shift = 0 Then
KeyAscii = 2371
Else
KeyAscii = 2315
End If
Case Else
KeyAscii = KeyCode
End Select
End If
End If
gethindiCode = KeyAscii
End Function

Regards
doublespin
Attachments
hindiStrings.txt
(190 Bytes) Downloaded 522 times

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 13 Aug 2010, 18:43

Many thanks. I'm pretty sure I somehow need to handle the WM_UNICHAR message, but so far I do not know how I should handle it.
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

Rucksacktraeger
Lieutenant
Posts: 23
Joined: 15 Aug 2010, 08:42

Re: Typing Hindi into Unicode Textbox shows question marks

Post by Rucksacktraeger » 15 Aug 2010, 08:54

Hi doublespin,

I am not working with Hindi Language, but with another related language and script: Khmer, the language of Cambodia. They have a similar script with the same Problem to display it on Computer.

There is definitly no Problem to view (Asian) Unicode Characters on Timos EditControls. They displays them even better than the Controls of MS Forms 2.0, because those show Khmer characters sometimes wrong. With Timos Controls I have no Problem.

I am quite sure that you are NOT using Timos Controls. Why? Because in your code is written:

Code: Select all

Private Sub Textbox1_KeyDown(KeyCode As MSForms.ReturnInteger, Shift As Integer)
If Textbox1 would be an Unicode Control, you would get a Problem starting the code because you would have to write:

Code: Select all

Private Sub Textbox1_KeyDown(KeyCode As Integer, ByVal Shift As Integer)
That is why you get ? instead of the correct Hindi Character. Therefore delete this Textbox and insert one of Timos EditControls!

Greetings
Peter

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 15 Aug 2010, 20:11

No, doublespin is right. If you set the keyboard to Hindi Traditional and type into my TextBox control, it shows only question marks while copy & paste works fine. E. g. the very first sign in the txt file that he posted, can be achieved with a German keyboard which is set to Hindi Traditional by pressing j followed by w. Do this in Notepad and it will produce the sign that you see in the txt file. Do this in my TextBox control and it will produce two question marks.

BTW, it's very difficult to input Hindi text if you don't know Hindi. :D
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 15 Aug 2010, 20:27

The first things that I could find out, are not very promising:
  • If I type the रै sign into a unicode text box of the VB6 Events sample, I get 2 question marks.
  • If I type the रै sign into a unicode text box of the WTL8 Events sample, I get the correct रै sign.
Both samples are identical, except that one is a VB6 app and therefore has a main app window which is ANSI and not Unicode, and the other one is a C++ app which is 100% Unicode.

To verify that the problem is the main app window being ANSI, I did more tests using the "Strings" property page (has been introduced with version 1.5.0) of my TextBox control. This property page contains a native(!) Unicode edit control. In VB6 the host window is ANSI, in Visual C++ 2008 it is Unicode. Here are the results:
  • Opened the property page in VB6 and entered the रै sign. Got two question marks.
  • Opened the property page in VC++ 2008 and entered the रै sign. Got the रै sign.
Apparently for some(!) input languages Windows doesn't check the Unicode-ness of the window that will receive the input, but of the application window.

So far I have no idea how I could fix this, but maybe it's enough to subclass the application window and forward all calls to CallWindowProcW (instead of CallWindowProcA). This could have side effects though...
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

Rucksacktraeger
Lieutenant
Posts: 23
Joined: 15 Aug 2010, 08:42

Re: Typing Hindi into Unicode Textbox shows question marks

Post by Rucksacktraeger » 16 Aug 2010, 04:01

Sorry guys, you are right! It doesn't work.
But with Thai or Hebrew Language for example, it works. I found out, that any language that put not ? but some other incorrect symbols to a normal Textbox, works fine with EditControls. Does this information help? Maybe not.

@doublespin
Your code

Code: Select all

Private Sub TextBox1_KeyDown(KeyCode As Integer, ByVal Shift As Integer)
    i_keyCode = KeyCode
    i_shift = Shift
End Sub

Private Sub TextBox1_KeyPress(keyAscii As Integer)
Dim ascii As Integer
    ascii = keyAscii
    keyAscii = gethindiCode(ascii, i_shift, i_keyCode)
End Sub
doesn't work on my computer either. But you could try this:

Code: Select all

Option Explicit

Private Sub TextBox1_KeyDown(keyCode As Integer, ByVal shift As Integer)
Debug.Print keyCode 'um den keycode auszugeben

    Select Case keyCode
    Case 72
        TextBox1.ReplaceSelectedText ChrW$(&H92A)
    Case 73
        TextBox1.ReplaceSelectedText ChrW$(&H917)
    Case 78
        TextBox1.ReplaceSelectedText ChrW$(&H932)
    Case 68
        TextBox1.ReplaceSelectedText ChrW$(&H94D)
    End Select
    
'    keyCode = 0 'does not work with EditControls 1.4 (Unicode)
End Sub

Private Sub TextBox1_KeyPress(keyAscii As Integer)
    Debug.Print keyAscii
    Select Case keyAscii 'exclude the following keys
        Case 32 'space
        Case 8 'back
        Case 9 ' I am not sure what this is and if you need to exclude it
        Case 13 'enter
        Case 27 'esc
        Case Else
            keyAscii = 0
    End Select
End Sub
If you type h i n d i it should display hindi text. With me, it works wonderful!

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 17 Aug 2010, 06:31

Subclassing the application window doesn't help, I tried it.
I could implement some auto-correction like doublespin has done. But doing this for specific input languages would be insane. I need an auto-correction that works for any input language without requiring me to implement each language manually and without requiring me to create dozens of look-up tables. I'll check the Windows NLS API functions, maybe I find something helpful.
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

collarofcantona
Cadet
Posts: 1
Joined: 16 May 2017, 10:45

Re: Typing Hindi into Unicode Textbox shows question marks

Post by collarofcantona » 16 May 2017, 10:52

Hi doublespin,

I have the same issue except that I see question marks while using Chinese Unicode. I have a Double to Hex conversion function. It works fine for values between 0-127. Beyond 127, I get 63 as the output because '?' contains the value 63. How do I resolve this? Here's the code.

Code: Select all

Public Function ConvDoubleToHexString(dVal As Double, ByVal nByteCount As Integer) As String

Dim sHex As String
Dim sTmp As String
Dim n As Integer
Dim m As Long
Dim sRet As String
Dim nByteCountMerk As Integer

m=1
sHex = Hex(dVal)

nByteCountMerk = nByteCount
If nByteCount < Len(sHex) / 2 Then nByteCount = Len(sHex) / 2
If nByteCount < 1 Then nByteCount = 1

sHex = String(nByteCount * 2 - Len(sHex), "0") & sHex 
For n = 0 To nByteCount - 1

sTmp = ChrW(Val("&H" & Mid(sHex, m, 2)))
m = m + 2
sRet = sRet & sTmp
next n

If Not bLH Then
ConvDoubleToHexString = Right(sRet, nByteCountMerk)
Else
For n = Len(sRet) To 1 Step -1
  sLH = sLH & Mid(sRet, n, 1)
Next n
ConvDoubleToHexString = Left(sLH, nByteCountMerk)
End If

End Function
Regards
collarofcantona

doublespin wrote:
13 Aug 2010, 10:51
Hi TiKu

Thanks for your reply.

Attached you will find some Hindi strings.

I tried to solve the typing problem writing some code to KeyDown and KeyPress event of the textbox. At the KeyDown event I assign KeyCode and Shift to global variables. At the KeyPress event I use the KeyCode and the Shift variables to set the value of the KeyAscii to the unicode value for the hindi character. Here is my code. But attention, the code may not contain all hindi chars.

Public Declare Function GetKeyboardLayoutName Lib "user32" Alias "GetKeyboardLayoutNameA" (ByVal pwszKLID As String) As Long

Private Sub Textbox1_KeyDown(KeyCode As MSForms.ReturnInteger, Shift As Integer)
i_keyCode = KeyCode
i_shift = Shift
End Sub

Private Sub Textbox1_KeyPress(KeyAscii As MSForms.ReturnInteger)
Dim ascii As Integer
ascii = KeyAscii
KeyAscii = gethindiCode(ascii, i_shift, i_keyCode)
End Sub

Private Function gethindiCode(KeyAscii As Integer, Shift As Integer, KeyCode As Integer) As Integer
Dim tmpStrg As String
Const KL_NAMELENGTH = 9

tmpStrg = String(KL_NAMELENGTH, 0)
GetKeyboardLayoutName tmpStrg 'Keyboard auslesen

If Left(tmpStrg, 8) = "00010439" Then 'hindi keyboard layout

'keyCode bringt den KeyCode für die Grossbuchstaben
If Shift = 0 Then 'Wenn nicht shift gedrückt -> auf die Kleinbuschstaben setzen
If (KeyCode > 64) And (KeyCode < 91) Then 'A bis Z
KeyCode = KeyCode + 32
End If
End If

If KeyAscii = 63 Then 'Fragezeichen --> KeyCode einsetzten
Select Case KeyCode
Case 113
KeyAscii = 2380
Case 119
KeyAscii = 2376
Case 101
KeyAscii = 2366
Case 114
KeyAscii = 2368
Case 116
KeyAscii = 2370
Case 121
KeyAscii = 2348
Case 117
KeyAscii = 2361
Case 105
KeyAscii = 2327
Case 111
KeyAscii = 2342
Case 112
KeyAscii = 2332
Case 97
KeyAscii = 2379
Case 115
KeyAscii = 2375
Case 100
KeyAscii = 2381
Case 102
KeyAscii = 2367
Case 103
KeyAscii = 2369
Case 104
KeyAscii = 2346
Case 106
KeyAscii = 2352
Case 107
KeyAscii = 2325
Case 108
KeyAscii = 2340
Case 122, 120
KeyAscii = 2306
Case 99
KeyAscii = 2350
Case 118
KeyAscii = 2344
Case 98
KeyAscii = 2357
Case 110
KeyAscii = 2354
Case 109
KeyAscii = 2360
Case 81
KeyAscii = 2324
Case 87
KeyAscii = 2320
Case 69
KeyAscii = 2310
Case 82
KeyAscii = 2312
Case 84
KeyAscii = 2314
Case 89
KeyAscii = 2349
Case 85
KeyAscii = 2329
Case 73
KeyAscii = 2328
Case 79
KeyAscii = 2343
Case 80
KeyAscii = 2333
Case 65
KeyAscii = 2323
Case 83
KeyAscii = 2319
Case 68
KeyAscii = 2309
Case 70
KeyAscii = 2311
Case 71
KeyAscii = 2313
Case 72
KeyAscii = 2347
Case 74
KeyAscii = 2353
Case 75
KeyAscii = 2326
Case 76
KeyAscii = 2341
Case 90, 88
KeyAscii = 2305
Case 67
KeyAscii = 2339
Case 86, 66, 78
KeyAscii = 2355
Case 77
KeyAscii = 2358
Case 96:
'`'
Case 49
If Shift = 0 Then
'1'
Else
KeyAscii = 2317
End If
Case 50
'2'
Case 51
If Shift = 0 Then
'3'
Else
'KeyAscii = 2373
End If
Case 52
'4'
Case 53
'5'
Case 54
'6'
Case 55
'7'
Case 56
'8'
Case 57
'9'
Case 48
'0'
Case 45
'-'
Case 61
KeyAscii = 2371
Case 92
KeyAscii = 2377
Case 91
KeyAscii = 2337
Case 93
KeyAscii = 2364
Case 59
KeyAscii = 2330
Case 39
KeyAscii = 2335
Case 44
','
Case 46
'.'
Case 47
KeyAscii = 2351
Case 126
':'
Case 33
KeyAscii = 2317
Case 64
KeyAscii = 2373
Case 35
KeyAscii = 2381
Case 36
KeyAscii = 2352 ';&#2381 '???
Case 37
KeyAscii = 2332 ';&#2381;&#2334 '???
Case 94
KeyAscii = 2340 ';&#2381;&#2352 '???
Case 38
KeyAscii = 2325 ';&#2381;&#2359 '???
Case 42
KeyAscii = 2358 ';&#2381;&#2352 '???
Case 40
','
Case 41
':'
Case 95
KeyAscii = 2307
Case 43
KeyAscii = 2400
Case 124
KeyAscii = 2321
Case 123
KeyAscii = 2338
Case 125
KeyAscii = 2334
Case 58
KeyAscii = 2331
Case 34
KeyAscii = 2336
Case 60
KeyAscii = 2359
Case 62
KeyAscii = 2404
Case 63
KeyAscii = 2399
Case 32
' '
Case 187 '=
If Shift = 0 Then
KeyAscii = 2371
Else
KeyAscii = 2315
End If
Case Else
KeyAscii = KeyCode
End Select
End If
End If
gethindiCode = KeyAscii
End Function

Regards
doublespin

User avatar
TiKu
Administrator
Administrator
Posts: 832
Joined: 28 Sep 2004, 21:10
Location: Unterföhring
Contact:

Re: Typing Hindi into Unicode Textbox shows question marks

Post by TiKu » 14 Jul 2019, 13:01

This has been fixed with version 1.11.0.
Crunching for Fab36_Folding-Division at Folding@Home. Join Fab36/Fab30! - Folding@Home and BOINC
Boycott DRM! Boycott HDCP!

Post Reply