Getting multiple textures into a single shader

Programming Help
1

Hi I am trying to pass multiple RGB images, say 6 of them, as textures into a shader where I can do some custom image processing. I managed to do this only up to two images using CreateGLOperator.m and Screen('TransformTexture'... but I can’t find a way to expand this to 2+ images.

Any ideas on how I might be able to do this?

Here is the code that works for two images. This is a dummy example of course.

clearvars
close all
clc

% disable timing checks
Screen('Preference', 'SkipSyncTests', 1);

% Check if Psychtoolbox is properly installed:
AssertOpenGL;

% Initiate PTB
PsychDefaultSetup(2);

% Select screen for display of movie:
screenid = max(Screen('Screens'));


try
    PsychImaging('PrepareConfiguration');
    PsychImaging('AddTask', 'General', 'FloatingPoint32Bit');
    PsychImaging('AddTask', 'General', 'EnableNative10BitFramebuffer',false);   

    % Open 'windowrect' sized window on screen, with black [0] background color:
    [win, wrect] = PsychImaging('OpenWindow', screenid, 0.5, []);

    % Define your shader in GLSL. This one performs a 3x3 color transform
    shaderFile = 'demo.frag';
    shader = LoadGLSLProgramFromFiles(which(shaderFile), 1);

    % initialize shader
    glUseProgram(shader);
    glUniform1i(glGetUniformLocation(shader, 'Image1'), 0);
    glUniform1i(glGetUniformLocation(shader, 'Image2'), 1);
    loc1 = glGetUniformLocation(shader, 'size1');
    loc2 = glGetUniformLocation(shader, 'size2');
    destLoc = glGetUniformLocation(shader, 'dest');
    glUseProgram(0);


    % create an operator 
    myOperator = CreateGLOperator(win, kPsychNeed32BPCFloat, shader, 'demo shader.');

    im1 = im2double(imread("cameraman.tif"));
    im2 = im2double(imread("peppers.png"));
    tex1 = Screen('MakeTexture',win, im1 ,[], [], [], [], []);
    tex2 = Screen('MakeTexture',win, im2,[], [], [], [], []);

    
    glUseProgram(shader);
    glUniform2f(loc1, size(im1,2), size(im1,1));
    glUniform2f(loc2, size(im2,2), size(im2,1));
    glUniform2f(destLoc, wrect(3), wrect(4));
    ftex = Screen('TransformTexture', tex1, myOperator, tex2);
    Screen('DrawTexture',win,ftex,[],[],[], [], [], [], shader);
    glUseProgram(0);

    Screen('Flip',win);
    KbStrokeWait(-1);
    sca;

catch
    sca
    psychrethrow(psychlasterror);
end

And here is the shader:

#version 120

uniform sampler2DRect Image1;
uniform sampler2DRect Image2;
uniform vec2 size1;
uniform vec2 size2;
uniform vec2 dest;

void main() {
    vec3 color1 = texture2DRect(Image1, gl_TexCoord[0].xy * size1 / size1).rgb;
    vec3 color2 = texture2DRect(Image2, gl_TexCoord[0].xy * size2 / size1).rgb;
    gl_FragColor = vec4(color1 * 0.5 + color2 * 0.5, 1.0);
}
2

Screen(‘TransformTexture’) can’t do more than 2 source input images. The need for more than 2 has not arisen in the over 16 years of its existence until now.

Our moglmorpher() function, as demonstrated in MorphDemo.m, and MorphTextureDemo.m, does use an iterative multiply-accumulate approach to effectively input more than 2 images though for image and geometry morphing. The demo only shows two shapes, but in production this was used efficiently with at least 27 shapes on graphics hardware from 2009.

What’s the purpose of using 6 images?

Edit: Specifically, skimming your code, I wonder if what you try to do is simply what moglmorpher('morphTexture', ...) implements, for weighted mixing of various images together, as demonstrated in MorphTextureDemo.m.

-mario