Integrate Java Robot for response emulation

Programming Help
1

Hi, I’m trying to integrate a Java Robot to automatically press the keyboard to try the entire experiment. However, the key pressed is not recognized by my psychtoolbox script.

My psychtoolbox version is: ‘3.0.16’ and I’m using Ubuntu 18.4.

The robot code is:

robot = java.awt.Robot;
robot.keyPress(java.awt.event.KeyEvent.VK_space);
robot.keyRelease(java.awt.event.KeyEvent.VK_space);

And I saved this script as “auto_go.m”

They I’ve tried to use it inside the Peter Scarfe tutorial:

% Clear the workspace and the screen
close all;
clearvars;
sca

% Here we call some default settings for setting up Psychtoolbox
PsychDefaultSetup(2);

% Get the screen numbers
screens = Screen('Screens');

% Select the external screen if it is present, else revert to the native
% screen
screenNumber = max(screens);

% Define black, white and grey
black = BlackIndex(screenNumber);
white = WhiteIndex(screenNumber);
grey = white / 2;

% Open an on screen window and color it grey
[window, windowRect] = PsychImaging('OpenWindow', screenNumber, grey);

% Set the blend funciton for the screen
Screen('BlendFunction', window, 'GL_SRC_ALPHA', 'GL_ONE_MINUS_SRC_ALPHA');

% Get the size of the on screen window in pixels
% For help see: Screen WindowSize?
[screenXpixels, screenYpixels] = Screen('WindowSize', window);

% Get the centre coordinate of the window in pixels
% For help see: help RectCenter
[xCenter, yCenter] = RectCenter(windowRect);

% Draw text in the upper portion of the screen with the default font in red
Screen('TextSize', window, 70);
DrawFormattedText(window, 'Hello World', 'center',...
    screenYpixels * 0.25, [1 0 0]);

% Draw text in the middle of the screen in Courier in white
Screen('TextSize', window, 80);
Screen('TextFont', window, 'Courier');
DrawFormattedText(window, 'Hello World', 'center', 'center', white);

% Draw text in the bottom of the screen in Times in blue
Screen('TextSize', window, 90);
Screen('TextFont', window, 'Times');
DrawFormattedText(window, 'Hello World', 'center',...
    screenYpixels * 0.75, [0 0 1]);

% Flip to the screen
Screen('Flip', window);

% Now we have drawn to the screen we wait for a keyboard button press (any
% key) to terminate the demo
KbStrokeWait;

run auto_go.m

% Clear the screen
sca;

Does someone know how to simulate a keypress?
Thanks

2

Hi,

I don’t know why your script isn’t working. But, fwiw, I do know that the KbQueue* functions can read virtual keypresses. I modified your script and copied it below. I think that it now does what you want.

% Clear the workspace and the screen
close all;
clearvars;
sca

% Here we call some default settings for setting up Psychtoolbox
PsychDefaultSetup(2);

% Get the screen numbers
screens = Screen('Screens');

% Select the external screen if it is present, else revert to the native
% screen
screenNumber = max(screens);

% Define black, white and grey
black = BlackIndex(screenNumber);
white = WhiteIndex(screenNumber);
grey = white / 2;

% Open an on screen window and color it grey
[window, windowRect] = PsychImaging('OpenWindow', screenNumber, grey);

% Set the blend funciton for the screen
Screen('BlendFunction', window, 'GL_SRC_ALPHA', 'GL_ONE_MINUS_SRC_ALPHA');

% Get the size of the on screen window in pixels
% For help see: Screen WindowSize?
[screenXpixels, screenYpixels] = Screen('WindowSize', window);

% Get the centre coordinate of the window in pixels
% For help see: help RectCenter
[xCenter, yCenter] = RectCenter(windowRect);

% Draw text in the upper portion of the screen with the default font in red
Screen('TextSize', window, 70);
DrawFormattedText(window, 'Hello World', 'center',...
    screenYpixels * 0.25, [1 0 0]);

% Draw text in the middle of the screen in Courier in white
Screen('TextSize', window, 80);
Screen('TextFont', window, 'Courier');
DrawFormattedText(window, 'Hello World', 'center', 'center', white);

% Draw text in the bottom of the screen in Times in blue
Screen('TextSize', window, 90);
Screen('TextFont', window, 'Times');
DrawFormattedText(window, 'Hello World', 'center',...
    screenYpixels * 0.75, [0 0 1]);

% Flip to the screen
Screen('Flip', window);

% Now we have drawn to the screen we wait for a keyboard button press (any
% key) to terminate the demo
KbQueueCreate();
KbQueueStart();
KbQueueWait([],1);

robot = java.awt.Robot;
robot.keyPress(java.awt.event.KeyEvent.VK_SPACE);

% Clear the screen
sca;

Again, I’m not sure why KbStrokeWait doesn’t pickup the keypress and release.

Also, note that the virtual key identifier is in all caps. So, it’s not java.awt.event.KeyEvent.VK_space but instead java.awt.event.KeyEvent.VK_SPACE.

Best

3

KbQueues do use X11 event queues, so one would hope they can get virtual key events. KbCheck et al. query the X-Server for the current instant key state, not using queues. That’s a difference.

You probably could also try to do a devIdx = GetKeyboardIndices(‘Virtual core XTEST keyboard’); and use KbCheck(devIdx) etc. to specifically query the virtual keyboard driven by the XTEST extension for key injection. Don’t know if it will work.

4

Thanks for your answer! the code works perfectly!

5

Thanks! I’m not an expert so I don’t understand all your response! Does the ‘devIdx = GetKeyboardIndices(‘Virtual core XTEST keyboard’)’ function belong to psychtoolbox?

6

Yes. GetKeyboardIndices enumerates keyboards and returns a handle to them, which you can pass into the KbXXXX functions to check a specific keyboard. The “XTEST” keyboard is special in that it is a virtual keyboard that is driven by virtual keypresses. If that Java Robot is based on the XTEST extension, it might work.