GLSL dome shader

1
Hello.
I am trying to display a stimulus inside a dome using a GLSL shader. I use MOGL to display triangles on a 3D ground floor. This works fine. Then I tried to load a vertex and fragment shader, then use Screen('SetOpenGLTexture'... to apply the shader the shader to the active window, then draw the created texture back on the active window (this might be stupid, perhaps I can apply the shaders to the active window directly). But when I do that I just get a blank window. If I only load the shaders and do not try to apply them to a texture I also get a blank window so I suspect the shaders are the issue. I've never used shaders before so I'm having issues understanding how it all works. Could someone take a look or point me to examples on how to use shaders from the PTB? Thanks a lot.
Baptiste


Here is how I load the shaders:
%% create shaders
% vertex shader
fid = fopen('DomeVertexShader.txt', 'r');
vertexShaderSource = fread(fid);
fclose(fid);
vertexShader = glCreateShader(GL.VERTEX_SHADER);
glShaderSource(vertexShader, vertexShaderSource);
glCompileShader(vertexShader);

% fragment shader
fid = fopen('DomeFragmentShader.txt', 'r');
fragmentShaderSource = fread(fid);
fclose(fid);
fragmentShader = glCreateShader(GL.FRAGMENT_SHADER);
glShaderSource(fragmentShader, fragmentShaderSource);
glCompileShader(fragmentShader);

% create shader program
shaderProgram = glCreateProgram;

glAttachShader(shaderProgram, vertexShader);
glAttachShader(shaderProgram, fragmentShader);
glLinkProgram(shaderProgram);

params = glGetProgramiv(shaderProgram, GL.LINK_STATUS);
[infoLength, infoLog] = glGetProgramInfoLog(shaderProgram, 512);

glUseProgram(shaderProgram);


Here is the loop where I draw triangles:
        Screen('BeginOpenGL', window);

        glClear;
        glPushMatrix;
       
        gluLookAt(cameraPosition(1)+(ee-0.5)*pupilaryDistance, cameraPosition(2), cameraPosition(3), fixationPosition(1), fixationPosition(2), fixationPosition(3), cameraUp(1), cameraUp(2), cameraUp(3));     

        % Draw floor triangles
        glBindBuffer(GL.ARRAY_BUFFER, bufferFloorID);
        glEnableClientState(GL.VERTEX_ARRAY);
        glBufferSubData(GL.ARRAY_BUFFER, 0, bufferSize, floorVertLine);
        glNormalPointer(dataType, 0, bufferSize);
        glVertexPointer(3, dataType, 0, 0);
        glMaterialfv(GL.FRONT_AND_BACK, GL.DIFFUSE, trianglesColor);
        glMaterialfv(GL.FRONT_AND_BACK, GL.AMBIENT, trianglesColor);
        glDrawElements(GL.TRIANGLES, numFaces*3, GL.UNSIGNED_INT, int32(floorFacesLine-1));
        glDisableClientState(GL.VERTEX_ARRAY);
        glBindBuffer(GL.ARRAY_BUFFER, 0);

        glPopMatrix;
       
        Screen('EndOpenGL', window);


Here is when I try to apply the shader:
        domedisplay = Screen('SetOpenGLTexture', window, [], 0, GL.TEXTURE_RECTANGLE_EXT, windowRect(3), windowRect(4), 1, shaderProgram);
        Screen('DrawTexture',window,domedisplay,[-0.5*windowRect(3),-0.5*windowRect(4),windowRect(3),windowRect(4)],[]);


Here are the fragment and vertex shaders:

#version 120
void main()
{
  gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
}



#version 120
varying vec4 Vertex_UV;
uniform mat4 gxl3d_ModelViewMatrix;
uniform mat4 gxl3d_ProjectionMatrix;

uniform int do_distorsion;

const float PI = 3.1415926535;

void main()
{
  vec4 P = gxl3d_ModelViewMatrix * gl_Vertex;
  if (do_distorsion == 1)
  { 
    float rxy = length(P.xy);
    if (rxy > 0)
    {
      float phi = atan(rxy, -P.z);
      float lens_radius = phi * 180.0 / PI * 2.0;
      P.xy *= (lens_radius / rxy);
    }
  }
  gl_Position = gxl3d_ProjectionMatrix * P;
  Vertex_UV = gl_MultiTexCoord0 * 10.0;
}



Here is the stimulus if I don't load the shaders:

2
Your code to load shader source and build a shader program can be replaced by a one-liner call to LoadGLSLProgramFromFiles(). Cfe. GLSLDemo.m
Where it goes sideways is when you create your texture and apply the shader. E.g., you specify a texture handle of zero instead of that of an actual texture - what texture? Probably you looked at a demo of ours that only uses shaders for procedural drawing but doesn't take any actual image input. Then a zero would make sense.

I don't understand what you want to achieve, and your code is incomplete. But if the idea is somehow to render your triangle floor into a texture image and then draw that image into the onscreen window for display while applying some kind of shader, then the most easy way is to:

1. Open an Offscreen window via Screen('OpenOffscreenWindow', window, ...
2. Use 'BeginOpenGL' with the handle of that offscreen window.
3. Render your triangle thingy into the offscreen window.
4. 'EndOpenGL'
5. Now the offscreen window contains the rendered image of your triangle floor as a suitable PTB texture.
6. 'DrawTexture' that texture and pass in your shader programs handle to apply your shader to the texture during drawing.
7. Your shader only gets certain input parameters when called via 'DrawTexture', as explained in "help ProceduralShadingAPI" e.g.,

8. You can set those other parameters in your shader, which you currently don't do.

Ultimately you'd first have to explain in detail what you want to achieve here.
-mario


On Thu, Oct 10, 2019 at 9:36 PM baptiste.caziot@... [PSYCHTOOLBOX] <PSYCHTOOLBOX@yahoogroups.com> wrote:

Hello.
I am trying to display a stimulus inside a dome using a GLSL shader. I use MOGL to display triangles on a 3D ground floor. This works fine. Then I tried to load a vertex and fragment shader, then use Screen('SetOpenGLTexture'... to apply the shader the shader to the active window, then draw the created texture back on the active window (this might be stupid, perhaps I can apply the shaders to the active window directly). But when I do that I just get a blank window. If I only load the shaders and do not try to apply them to a texture I also get a blank window so I suspect the shaders are the issue. I've never used shaders before so I'm having issues understanding how it all works. Could someone take a look or point me to examples on how to use shaders from the PTB? Thanks a lot.
Baptiste


Here is how I load the shaders:
%% create shaders
% vertex shader
fid = fopen('DomeVertexShader.txt', 'r');
vertexShaderSource = fread(fid);
fclose(fid);
vertexShader = glCreateShader(GL.VERTEX_SHADER);
glShaderSource(vertexShader, vertexShaderSource);
glCompileShader(vertexShader);

% fragment shader
fid = fopen('DomeFragmentShader.txt', 'r');
fragmentShaderSource = fread(fid);
fclose(fid);
fragmentShader = glCreateShader(GL.FRAGMENT_SHADER);
glShaderSource(fragmentShader, fragmentShaderSource);
glCompileShader(fragmentShader);

% create shader program
shaderProgram = glCreateProgram;

glAttachShader(shaderProgram, vertexShader);
glAttachShader(shaderProgram, fragmentShader);
glLinkProgram(shaderProgram);

params = glGetProgramiv(shaderProgram, GL.LINK_STATUS);
[infoLength, infoLog] = glGetProgramInfoLog(shaderProgram, 512);

glUseProgram(shaderProgram);


Here is the loop where I draw triangles:
Screen('BeginOpenGL', window);

glClear;
glPushMatrix;

gluLookAt(cameraPosition(1)+(ee-0.5)*pupilaryDistance, cameraPosition(2), cameraPosition(3), fixationPosition(1), fixationPosition(2), fixationPosition(3), cameraUp(1), cameraUp(2), cameraUp(3));

% Draw floor triangles
glBindBuffer(GL.ARRAY_BUFFER, bufferFloorID);
glEnableClientState(GL.VERTEX_ARRAY);
glBufferSubData(GL.ARRAY_BUFFER, 0, bufferSize, floorVertLine);
glNormalPointer(dataType, 0, bufferSize);
glVertexPointer(3, dataType, 0, 0);
glMaterialfv(GL.FRONT_AND_BACK, GL.DIFFUSE, trianglesColor);
glMaterialfv(GL.FRONT_AND_BACK, GL.AMBIENT, trianglesColor);
glDrawElements(GL.TRIANGLES, numFaces*3, GL.UNSIGNED_INT, int32(floorFacesLine-1));
glDisableClientState(GL.VERTEX_ARRAY);
glBindBuffer(GL.ARRAY_BUFFER, 0);

glPopMatrix;

Screen('EndOpenGL', window);


Here is when I try to apply the shader:
 domedisplay = Screen('SetOpenGLTexture', window, [], 0, GL.TEXTURE_RECTANGLE_EXT, windowRect(3), windowRect(4), 1, shaderProgram);
Screen('DrawTexture',window,domedisplay,[-0.5*windowRect(3),-0.5*windowRect(4),windowRect(3),windowRect(4)],[]);


Here are the fragment and vertex shaders:

#version 120
void main()
{
gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
}



#version 120
varying vec4 Vertex_UV;
uniform mat4 gxl3d_ModelViewMatrix;
uniform mat4 gxl3d_ProjectionMatrix;

uniform int do_distorsion;

const float PI = 3.1415926535;

void main()
{
vec4 P = gxl3d_ModelViewMatrix * gl_Vertex;
if (do_distorsion == 1)
{
float rxy = length(P.xy);
if (rxy > 0)
{
float phi = atan(rxy, -P.z);
float lens_radius = phi * 180.0 / PI * 2.0;
P.xy *= (lens_radius / rxy);
}
}
gl_Position = gxl3d_ProjectionMatrix * P;
Vertex_UV = gl_MultiTexCoord0 * 10.0;
}



Here is the stimulus if I don't load the shaders:

And if I do (regardless of whether I try to apply the transform or not):




3
Hi Mario.
I attached a picture of the stimulus in my previous email but it disappeared. In short, we use a stimulus that is basically a RDK made of triangles displayed on a 2D ground floor. The observers move on that ground plane to get to a disk target using a joystick. Until now we displayed the stimulus on a regular (flat) display screen, but in the future we would like to display that stimulus inside a dome using a projector. So I would like to distort the openGL window on which I draw the stimulus so that once projected on the spherical inside of the dome it is geometrically correct. After some research it seems shaders would be a good way to apply this geometrical transformation fast. But again, I am new to these things.

I've tried to implement your suggestions. I managed to draw on a offscreen texture but only if I turn off multisampling. If I understand correctly I should use the shader when calling DrawTexture and pass arguments to the shader through auxParameters? Or I could define these parameters inside the shader itself (I assume that would be the projection and view matrices)? I grabbed these shaders somewhere on the internet and I don't fully understand how they work yet.

I attached the script I wrote to generate the stimulus and a modified one with the offscreen window, as well as the shaders. Also, we display the stimulus in stereo (sequential but here I used anaglyph).
Regards,
Baptiste