keyboard emulation problems with Windows 8.1?

[marcusvdt]

I know there is the input.sendkeys and the sendkeys and probably other ways, but I'm interested in which one actually works on Windows 8.1 and hopefully on Windows 10.

I tried some combinations with minimal success, so I suppose I'm either doing something wrong or Windows 8.1 does not like keyboard emulation from VC. I'm able to successfully emulate from AutoIT, so I know Windows 8.1 does support emulation somehow (I don't actually know the code behind AutoIT anyway).

This is what I've tried:

Works and sends text to Notepad

Code Select Expand

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.0.7-->
<command id="669" name="letra, numero {1}" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>InputKeys.Send</cmdType>
    <params>
      <param>{1}</param>
    </params>
    <cmdRepeat>0</cmdRepeat>
  </action>
  <action>
    <cmdType>SendKeys</cmdType>
    <params>
      <param>{1}</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <phrase>letra,número</phrase>
  <payloadList>a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,0,1,2,3,4,5,6,7,8,9</payloadList>
</command>


Does not work and no text appears in Notepad

Code Select Expand

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.0.7-->
<command id="669" name="letra, numero {1}" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>InputKeys.Send</cmdType>
    <params>
      <param>{1}</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>SendKeys</cmdType>
    <params>
      <param>{1}</param>
    </params>
    <cmdRepeat>0</cmdRepeat>
  </action>
  <phrase>letra,número</phrase>
  <payloadList>a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,0,1,2,3,4,5,6,7,8,9</payloadList>
</command>

output of the above:

Code Select Expand

18/08/2015 16:56:32 918 guessed text:digitar ?? (Teclado)
18/08/2015 16:56:33 340 guessed text:digitar número ?? (Teclado)
18/08/2015 16:56:33 589 guessed text:digitar letra ?? (Teclado)
18/08/2015 16:56:33 835 guessed text:digitar letra j ?? (Teclado)
18/08/2015 16:56:34 525 Something was recognized
18/08/2015 16:56:34 525 Rulename:  # Semantics: 3
18/08/2015 16:56:34 525 Group: Teclado Speech: digitar letra j
18/08/2015 16:56:34 527 semanticID: 669
18/08/2015 16:56:34 527 kvp: prefix | digitar
18/08/2015 16:56:34 527 kvp: command | 669
18/08/2015 16:56:34 527 kvp: p669d1 | j
18/08/2015 16:56:34 527 used payload key to get command ID = 669 and payload # 1
18/08/2015 16:56:34 535 semanticID: 669
18/08/2015 16:56:34 535 kvp: prefix | digitar
18/08/2015 16:56:34 535 kvp: command | 669
18/08/2015 16:56:34 535 kvp: p669d1 | j
18/08/2015 16:56:34 535 used payload key to get command ID = 669 and payload # 1
18/08/2015 16:56:34 535 alternate:digitar letra j
18/08/2015 16:56:34 549 A prefix is required to execute this command.
18/08/2015 16:56:34 582 [vcevent] VC.Prefix.End

18/08/2015 16:56:34 605 [info] event in group: Low Confidence

18/08/2015 16:56:34 614 doCommand:Prefix Active
18/08/2015 16:56:34 614 [logic] doing IF: (A)==(B)

18/08/2015 16:56:34 628 action repeat set to: 1
18/08/2015 16:56:34 629 Action:  Results.SetVar - prefixactive&&0
18/08/2015 16:56:34 629 [action] Results.SetVar:prefixactive&&0

18/08/2015 16:56:34 644 Action INFO: Successfully Set: prefixactive = 0
18/08/2015 16:56:34 646 [plugin] Sending json request:
18/08/2015 16:56:34 646 [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"digitar letra j (76,3)","message":"(1 letra, numero {j}) "},"id": "0"}
18/08/2015 16:56:34 647 doCommand:letra, numero {1}
18/08/2015 16:56:34 656 action repeat set to: 1
18/08/2015 16:56:34 656 Action:  InputKeys.Send - j
18/08/2015 16:56:34 656 [action] InputKeys.Send:j
_______________________________________________________






Also, for the ALT+TAB combo or their alternates, neither InputKeys.Send or SendKeys work. A weird thing I can notice from the logs is that the TAB key disappears leading me to believe VC macro is not forwarding it correctly.

Code Select Expand

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.0.7-->
<command id="681" name="alt tab test" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;1</ifParams>
    <then>
      <action>
        <cmdType>InputKeys.Send</cmdType>
        <params>
          <param>{ALT}({TAB})</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;2</ifParams>
    <then>
      <action>
        <cmdType>InputKeys.Send</cmdType>
        <params>
          <param>{CTRL+ALT}({TAB})</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;3</ifParams>
    <then>
      <action>
        <cmdType>InputKeys.Send</cmdType>
        <params>
          <param>{CTRL+ALT+TAB}</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;4</ifParams>
    <then>
      <action>
        <cmdType>SendKeys</cmdType>
        <params>
          <param>^%({TAB})</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;5</ifParams>
    <then>
      <action>
        <cmdType>SendKeys</cmdType>
        <params>
          <param>%{TAB}</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <if ifBlockDisabled="False" ifNot="False">
    <ifType>(A)==(B)</ifType>
    <ifParams>{1}&amp;&amp;6</ifParams>
    <then>
      <action>
        <cmdType>InputKeys.Send</cmdType>
        <params>
          <param>{CONTROL+ALT}({TAB}{TAB})</param>
        </params>
        <cmdRepeat>1</cmdRepeat>
      </action>
    </then>
    <else />
  </if>
  <phrase>switch,next</phrase>
  <phrase>window</phrase>
  <phrase>method</phrase>
  <payloadList>1,2,3,4,5,6</payloadList>
</command>


Results of the above, in sequence:

18/08/2015 17:10:22   142   [info] switch window method 1

18/08/2015 17:10:22   161   emulating recognition on string:switch window method 1
18/08/2015 17:10:22   331   Something was recognized
18/08/2015 17:10:22   331   Rulename:     # Semantics: 2
18/08/2015 17:10:22   331   Group: Programas e Janelas    Speech: switch window method 1
18/08/2015 17:10:22   332   semanticID: 681
18/08/2015 17:10:22   332   kvp: command | 681
18/08/2015 17:10:22   332   kvp: p681d1 | 1
18/08/2015 17:10:22   332   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:10:22   335   semanticID: 681
18/08/2015 17:10:22   335   kvp: command | 681
18/08/2015 17:10:22   335   kvp: p681d1 | 1
18/08/2015 17:10:22   335   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:10:22   335   alternate:switch window method 1
18/08/2015 17:10:22   394   [plugin] Sending json request:
18/08/2015 17:10:22   395   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 1 (100,0)","message":"(1 alt tab test [1]) "},"id": "0"}
18/08/2015 17:10:22   395   doCommand:alt tab test
18/08/2015 17:10:22   395   [logic] doing IF: (A)==(B)

18/08/2015 17:10:22   412   action repeat set to: 1
18/08/2015 17:10:22   413   no match for replacing variables: {ALT}
18/08/2015 17:10:22   413   no match for replacing variables: {ALT}
18/08/2015 17:10:22   413   Action:  InputKeys.Send - {ALT}(   )
18/08/2015 17:10:22   413   [action] InputKeys.Send:{ALT}(   )

18/08/2015 17:10:22   451   Action INFO: keydown: ALT
pressing in sequence:    
textentry[   ]
releasing keys: MENU,

___________________________________________________
18/08/2015 17:12:13   224   received udp command:vc.tellvox&&switch window method 2
18/08/2015 17:12:13   225   action repeat set to: 1
18/08/2015 17:12:13   225   Action:  vc.tellvox - switch window method 2
18/08/2015 17:12:13   225   [action] vc.tellvox:switch window method 2

18/08/2015 17:12:13   240   [info] switch window method 2

18/08/2015 17:12:13   253   emulating recognition on string:switch window method 2
18/08/2015 17:12:13   417   Something was recognized
18/08/2015 17:12:13   417   Rulename:     # Semantics: 2
18/08/2015 17:12:13   417   Group: Programas e Janelas    Speech: switch window method 2
18/08/2015 17:12:13   419   semanticID: 681
18/08/2015 17:12:13   419   kvp: command | 681
18/08/2015 17:12:13   419   kvp: p681d1 | 2
18/08/2015 17:12:13   419   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:12:13   426   semanticID: 681
18/08/2015 17:12:13   427   kvp: command | 681
18/08/2015 17:12:13   427   kvp: p681d1 | 2
18/08/2015 17:12:13   427   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:12:13   427   alternate:switch window method 2
18/08/2015 17:12:13   473   [plugin] Sending json request:
18/08/2015 17:12:13   473   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 2 (100,0)","message":"(1 alt tab test [2]) "},"id": "0"}
18/08/2015 17:12:13   474   doCommand:alt tab test
18/08/2015 17:12:13   474   [logic] doing IF: (A)==(B)

18/08/2015 17:12:13   504   [logic] doing IF: (A)==(B)

18/08/2015 17:12:13   528   action repeat set to: 1
18/08/2015 17:12:13   529   Action:  InputKeys.Send - {CTRL+ALT}(   )
18/08/2015 17:12:13   529   [action] InputKeys.Send:{CTRL+ALT}(   )

18/08/2015 17:12:13   554   Action INFO: keydown: CTRL+ALT
pressing in sequence:    
textentry[   ]
releasing keys: MENU,
18/08/2015 17:12:13   555   [logic] doing IF: (A)==(B)

18/08/2015 17:12:13   568   [logic] doing IF: (A)==(B)

18/08/2015 17:12:13   578   [logic] doing IF: (A)==(B)

___________________________________________________
18/08/2015 17:14:07   629   [info] switch window method 3

18/08/2015 17:14:07   649   emulating recognition on string:switch window method 3
18/08/2015 17:14:07   795   Something was recognized
18/08/2015 17:14:07   795   Rulename:     # Semantics: 2
18/08/2015 17:14:07   795   Group: Programas e Janelas    Speech: switch window method 3
18/08/2015 17:14:07   797   semanticID: 681
18/08/2015 17:14:07   797   kvp: command | 681
18/08/2015 17:14:07   797   kvp: p681d1 | 3
18/08/2015 17:14:07   797   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:14:07   802   semanticID: 681
18/08/2015 17:14:07   802   kvp: command | 681
18/08/2015 17:14:07   802   kvp: p681d1 | 3
18/08/2015 17:14:07   802   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:14:07   802   alternate:switch window method 3
18/08/2015 17:14:07   852   [plugin] Sending json request:
18/08/2015 17:14:07   853   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 3 (100,0)","message":"(1 alt tab test [3]) "},"id": "0"}
18/08/2015 17:14:07   854   doCommand:alt tab test
18/08/2015 17:14:07   855   [logic] doing IF: (A)==(B)

18/08/2015 17:14:07   872   [logic] doing IF: (A)==(B)

18/08/2015 17:14:07   887   [logic] doing IF: (A)==(B)

18/08/2015 17:14:07   909   action repeat set to: 1
18/08/2015 17:14:07   910   Action:  InputKeys.Send - {CTRL+ALT+TAB}
18/08/2015 17:14:07   910   [action] InputKeys.Send:{CTRL+ALT+TAB}

18/08/2015 17:14:07   928   Action INFO: keydown: CTRL+ALT+TAB
pressing in sequence:

releasing keys: TAB,MENU,
_____________________________________________________
18/08/2015 17:16:00   603   [info] switch window method 4

18/08/2015 17:16:00   616   emulating recognition on string:switch window method 4
18/08/2015 17:16:00   742   Something was recognized
18/08/2015 17:16:00   742   Rulename:     # Semantics: 2
18/08/2015 17:16:00   742   Group: Programas e Janelas    Speech: switch window method 4
18/08/2015 17:16:00   745   semanticID: 681
18/08/2015 17:16:00   745   kvp: command | 681
18/08/2015 17:16:00   745   kvp: p681d1 | 4
18/08/2015 17:16:00   745   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:16:00   749   semanticID: 681
18/08/2015 17:16:00   749   kvp: command | 681
18/08/2015 17:16:00   749   kvp: p681d1 | 4
18/08/2015 17:16:00   749   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:16:00   749   alternate:switch window method 4
18/08/2015 17:16:00   806   [plugin] Sending json request:
18/08/2015 17:16:00   806   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 4 (100,0)","message":"(1 alt tab test [4]) "},"id": "0"}
18/08/2015 17:16:00   808   doCommand:alt tab test
18/08/2015 17:16:00   808   [logic] doing IF: (A)==(B)

18/08/2015 17:16:00   825   [logic] doing IF: (A)==(B)

18/08/2015 17:16:00   838   [logic] doing IF: (A)==(B)

18/08/2015 17:16:00   850   [logic] doing IF: (A)==(B)

18/08/2015 17:16:00   864   action repeat set to: 1
18/08/2015 17:16:00   865   Action:  SendKeys - ^%(   )
18/08/2015 17:16:00   865   [action] SendKeys:^%(   )

18/08/2015 17:16:00   960   Action INFO:
18/08/2015 17:16:00   960   [logic] doing IF: (A)==(B)
__________________________________________________________
18/08/2015 17:19:35   45   [info] switch window method 5

18/08/2015 17:19:35   57   emulating recognition on string:switch window method 5
18/08/2015 17:19:35   192   Something was recognized
18/08/2015 17:19:35   192   Rulename:     # Semantics: 2
18/08/2015 17:19:35   192   Group: Programas e Janelas    Speech: switch window method 5
18/08/2015 17:19:35   194   semanticID: 681
18/08/2015 17:19:35   194   kvp: command | 681
18/08/2015 17:19:35   194   kvp: p681d1 | 5
18/08/2015 17:19:35   194   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:19:35   197   semanticID: 681
18/08/2015 17:19:35   197   kvp: command | 681
18/08/2015 17:19:35   197   kvp: p681d1 | 5
18/08/2015 17:19:35   197   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:19:35   197   alternate:switch window method 5
18/08/2015 17:19:35   248   doCommand:alt tab test
18/08/2015 17:19:35   248   [logic] doing IF: (A)==(B)

18/08/2015 17:19:35   249   [plugin] Sending json request:
18/08/2015 17:19:35   249   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 5 (100,0)","message":"(1 alt tab test [5]) "},"id": "0"}
18/08/2015 17:19:35   274   [logic] doing IF: (A)==(B)

18/08/2015 17:19:35   295   [logic] doing IF: (A)==(B)

18/08/2015 17:19:35   310   [logic] doing IF: (A)==(B)

18/08/2015 17:19:35   324   [logic] doing IF: (A)==(B)

18/08/2015 17:19:35   334   action repeat set to: 1
18/08/2015 17:19:35   335   Action:  SendKeys - %   
18/08/2015 17:19:35   335   [action] SendKeys:%   

18/08/2015 17:19:35   427   Action INFO:

________________________________________________________

18/08/2015 17:25:21   532   [info] switch window method 6

18/08/2015 17:25:21   551   emulating recognition on string:switch window method 6
18/08/2015 17:25:21   712   Something was recognized
18/08/2015 17:25:21   712   Rulename:     # Semantics: 2
18/08/2015 17:25:21   712   Group: Programas e Janelas    Speech: switch window method 6
18/08/2015 17:25:21   714   semanticID: 681
18/08/2015 17:25:21   714   kvp: command | 681
18/08/2015 17:25:21   714   kvp: p681d1 | 6
18/08/2015 17:25:21   714   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:25:21   720   semanticID: 681
18/08/2015 17:25:21   720   kvp: command | 681
18/08/2015 17:25:21   720   kvp: p681d1 | 6
18/08/2015 17:25:21   721   used payload key to get command ID = 681 and payload # 1
18/08/2015 17:25:21   721   alternate:switch window method 6
18/08/2015 17:25:21   776   [plugin] Sending json request:
18/08/2015 17:25:21   776   [plugin] {"jsonrpc": "2.0", "method": "GUI.ShowNotification", "params":{"title":"switch window method 6 (100,0)","message":"(1 alt tab test [6]) "},"id": "0"}
18/08/2015 17:25:21   777   doCommand:alt tab test
18/08/2015 17:25:21   777   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   807   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   825   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   839   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   853   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   872   [logic] doing IF: (A)==(B)

18/08/2015 17:25:21   890   action repeat set to: 1
18/08/2015 17:25:21   891   Action:  InputKeys.Send - {CONTROL+ALT}(      )
18/08/2015 17:25:21   891   [action] InputKeys.Send:{CONTROL+ALT}(      )

18/08/2015 17:25:21   921   Action INFO: keydown: CONTROL+ALT
pressing in sequence:       
textentry[   ]textentry[   ]
releasing keys: MENU,CONTROL,

[jitterjames]

Please put your log in a code box.

Inputkeys.Sendkeys works fine for sending letters to notepad, but your syntax is not correct.  This should work, but I don't have access to win8 or win10 at the moment:

Code Select Expand

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.0.6-->
<command id="-1" name="Actions copied from LCB">
  <action>
    <cmdType>InputKeys.Send</cmdType>
    <params>
      <param>{a}</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
</command>

There does appear to be something funny going on with the {tab} key.  I think VC is converting it to an actual tab character, probably because it was requested by someone ...I won't say who ... and I unfortunately did not notice that it was messing up the inputkeys action.  I'll have to see if there is a way to keep both or since it is not really important I can just get rid of the {tab} variable if necessary. http://voxcommando.com/forum/index.php?topic=2223.0

[marcusvdt]

Hahahaha my God! I though you said you were not going to do that... 
In that specific case, I created a python function that pads spaces to achieve a fake tab char...

That is fun isn't it?!?! I suppose you opted not to make "the significant changes to OSD" and tried it the easy way, huh? Gotcha! 

[jitterjames]

Well adding the tab variable seemed like a good idea at the time even if it did not solve your specific problem...