How to transform coordinates from OpenGL (3D) to Psychtoolbox (2D)

Programming Help
1

Hi, everyone,

The goal of my experimental program is to draw multiple moving cubes with simultaneous rotation and translation, which is adapted from the paradigm of Pylyshyn & Storm (1988) (BJS: Basic MOT Demo).

However, when I use PTB for programming, I encountered some difficult problems.

(1) I don’t know how to accurately detect the collision between cubes；
(2) I don’t know how to transform the 3D coordinates of the cubes in OpenGL to the pixel coordinates of the window (note: My experiment needs to match the spatial position of the cubes to the position of mouse clicks).

This question is similar to this: matlab - OpenGL 2D coordinates to Psychtoolbox (pixel) coordinates - Stack Overflow.

PTB-INFO: Psychtoolbox-3 (Version 3.0.14 - Build date: Oct 3 2017) for Windows 10, under Matlab 2015b (64-Bit).

Demo code:

%% Initialization
sca;close all;clearvars;
PsychDefaultSetup(2);
Screen(‘Preference’, ‘SkipSyncTests’, 2);
InitializeMatlabOpenGL;
screenid = max(Screen(‘Screens’));
[window, windowRect] = PsychImaging(‘OpenWindow’, screenid, 0, [], 32, 2, [], 6, []);
ifi = Screen(‘GetFlipInterval’, window);
%% Start
Screen(‘BeginOpenGL’, window);
ar = windowRect(3) / windowRect(4); % width/height = aspect
screenHeight = 30; % assume 30cm in height
screenWidth = screenHeight * ar;

% Enable lighting
glEnable(GL.LIGHTING); % Enable lighting
glEnable(GL.LIGHT0); % Define a local light source
glEnable(GL.DEPTH_TEST); % Enable proper occlusion handling via depth tests
glMatrixMode(GL.PROJECTION); % set up a projection matrix
glLoadIdentity;

% Calculate the FOV in the y direction assuming a distance to the objects of 100cm
dist = 57;
angle = 2 * atand(screenHeight / dist);
gluPerspective(angle, ar, 0.1, dist*2); % defined FOV. Set up our perspective projection.
glMatrixMode(GL.MODELVIEW); % Setup modelview matrix
glLoadIdentity;
glLightfv(GL.LIGHT0, GL.POSITION, [1 2 3 0]);

% Set camera
cam = [0 0 0]; % Head position
fix = [0 0 -1*dist]; % Object position
up = [0 1 0]; % Up orientation
gluLookAt(cam(1), cam(2), cam(3), fix(1), fix(2), fix(3), up(1), up(2), up(3));

% color
glClearColor(0, 0, 0, 0);
glClear;
glMaterialfv(GL.FRONT_AND_BACK,GL.AMBIENT_AND_DIFFUSE, [0.0 0.0 1.0 1]);
Screen(‘EndOpenGL’, window);

%%
% Setup the positions of the spheres using the mexhgrid command
[cubeX, cubeY] = meshgrid(linspace(-25, 25, 2), linspace(-25, 25, 2));
[s1, s2] = size(cubeX);
cubeX = reshape(cubeX, 1, s1 * s2);
cubeY = reshape(cubeY, 1, s1 * s2);

rotaY = rand(1, length(cubeX)) .* 0;
rotaZ = rand(1, length(cubeX)) .* 0;

degPerSec = 180;
degPerFrame = degPerSec * ifi;
vbl = Screen(‘Flip’, window);
waitframes = 1;
d = 0.1;
ds = [dones(1,length(cubeX)/2), -dones(1,length(cubeX)/2)];
%% Draw cubes
while ~KbCheck
Screen(‘BeginOpenGL’, window);
glClear;
for i = 1:length(cubeX)
glPushMatrix;
glTranslatef(cubeX(i), cubeY(i), -dist);
glRotatef(rotaY(i), 0, 1, 0);
glutSolidCube(2);
glPopMatrix;
HideCursor;
end
Screen(‘EndOpenGL’, window);
vbl = Screen(‘Flip’, window, vbl + (waitframes - 0.5) * ifi);
rotaY = rotaY + degPerFrame;

if max(cubeX) - min(cubeX) <= 2 || max(cubeX) - min(cubeX) > 50
    ds = -ds;
end 
cubeX = cubeX + ds;

end
sca;