Problem with drawtexture

1
Hi,

I have the following piece of code (edited from the real thing for
simplicity)

function testpipeline

screenRect=SCREEN('Rect',0);

window=SCREEN(0, 'OpenWindow', [127 127 127],
screenRect,32,2,0,0,kPsychNeedFastBackingStore);

pixRect=[0 0 300 300];
back_text=repmat(192,pixRect(4),pixRect(3));
backg=SCREEN('MakeTexture',window,back_text);

pic_text=repmat(160,pixRect(4),pixRect(3));

pic=SCREEN('MakeTexture',window,pic_text);
SCREEN('DrawTexture',backg,pic);

SCREEN('DrawTexture',window,backg);
SCREEN('Flip',window);
SCREEN('Flip',window,getsecs+10);
SCREEN('Closeall');
end

When I run this I get the following error

PTB-ERROR: Failed to setup internal framebuffer objects color buffer
attachment for imaging pipeline!
PTB-ERROR: Exact reason for failure is unknown,
glCheckFramebufferStatus() returns code 36054
PTB-ERROR: You may want to retry with the lowest acceptable (for your
study) size and depth of the onscreen window or offscreen window.
INTERNAL PSYCHTOOLBOX ERROR
error: PsychError_internal
general description: Unspecified error, probably a Psychtoolbox bug
specific description: Preparation of drawing into an offscreen window
or texture failed when trying to create associated framebuffer object!
module name: Screen
subfunction call: DrawTexture
file name: Common\Screen\PsychImagingPipelineSupport.c
function name: UNKNOWN
line number: 1075
??? (null)

Error in ==> testpipeline at 14
SCREEN('DrawTexture',backg,pic);


If I change the code to not use the imaging pipeline then the code
executes without error although I'm not sure it does what I am
expecting and I need the imaging pipeline to use dontclear=1 in flip
due to limitations of my graphics card.

I realise that what I'm doing is to draw one texture into another - is
this something that should be possible?

In reality what I'm trying to do to take an image (re-scale it) and
put it into a texture. I tried using putimage but I get an error: Put
Image does not yet support indexed mode which I presume means you
can't putimage into a texture.

Interestingly if instead of using MakeTexture I use
OpenOffScreenWindow (which I thought in PTB the two were virtually
equivalent) to construct backg and then use putimage then the code
works OK.

So what is the difference between an OffScreen Window and a texture in
PTB3 and is there a better way of achieving what I'm trying to do.

Many thanks,

Matthew.
2
--- In psychtoolbox@yahoogroups.com, "empedia" <dexterm@...> wrote:
>
> Hi,
>
> I have the following piece of code (edited from the real thing for
> simplicity)
>
> function testpipeline
>
> screenRect=SCREEN('Rect',0);
>
> window=SCREEN(0, 'OpenWindow', [127 127 127],
> screenRect,32,2,0,0,kPsychNeedFastBackingStore);
>
> pixRect=[0 0 300 300];
> back_text=repmat(192,pixRect(4),pixRect(3));
> backg=SCREEN('MakeTexture',window,back_text);
>
> pic_text=repmat(160,pixRect(4),pixRect(3));
>
> pic=SCREEN('MakeTexture',window,pic_text);
> SCREEN('DrawTexture',backg,pic);
>
> SCREEN('DrawTexture',window,backg);
> SCREEN('Flip',window);
> SCREEN('Flip',window,getsecs+10);
> SCREEN('Closeall');
> end
>
> When I run this I get the following error
>
> PTB-ERROR: Failed to setup internal framebuffer objects color buffer
> attachment for imaging pipeline!
> PTB-ERROR: Exact reason for failure is unknown,
> glCheckFramebufferStatus() returns code 36054
> PTB-ERROR: You may want to retry with the lowest acceptable (for your
> study) size and depth of the onscreen window or offscreen window.
> INTERNAL PSYCHTOOLBOX ERROR
> error: PsychError_internal
> general description: Unspecified error, probably a Psychtoolbox bug
> specific description: Preparation of drawing into an offscreen window
> or texture failed when trying to create associated framebuffer object!
> module name: Screen
> subfunction call: DrawTexture
> file name: Common\Screen\PsychImagingPipelineSupport.c
> function name: UNKNOWN
> line number: 1075
> ??? (null)
>
> Error in ==> testpipeline at 14
> SCREEN('DrawTexture',backg,pic);
>
>
> If I change the code to not use the imaging pipeline then the code
> executes without error although I'm not sure it does what I am
> expecting and I need the imaging pipeline to use dontclear=1 in flip
> due to limitations of my graphics card.
>
> I realise that what I'm doing is to draw one texture into another - is
> this something that should be possible?
>
> In reality what I'm trying to do to take an image (re-scale it) and
> put it into a texture. I tried using putimage but I get an error: Put
> Image does not yet support indexed mode which I presume means you
> can't putimage into a texture.
>
> Interestingly if instead of using MakeTexture I use
> OpenOffScreenWindow (which I thought in PTB the two were virtually
> equivalent) to construct backg and then use putimage then the code
> works OK.
>
> So what is the difference between an OffScreen Window and a texture in
> PTB3 and is there a better way of achieving what I'm trying to do.
>
> Many thanks,
>
> Matthew.
>
3
Hi

PTB-Textures and Offscreen windows are nearly the same. Both are
OpenGL textures internally and treated the same by all texture
related functions, that's why you can pass a texture handle into
'CopyWindow' or an offscreen window handle into 'DrawTexture'. The
only difference is in their creation and initialization. One can
create OpenGL textures in many different formats, optimized for
different purposes. PTB initially creates a texture in a format that
is optimized for the typical purpose of the texture, given the
function you used to create it, trying to maximize speed and minimize
memory consumption for a given purpose. If PTB discovers that the
current format of a OpenGL texture is not suitable for an operation,
it (should) convert it into a suitable format on-the-fly. PTB didn't
do that in your case because you hit a special case whose conversion
routine wasn't implemented in the current beta of the imaging
pipeline. Graphics hardware can't draw into non-RGB(A) textures, you
created a pure luminance texture, PTB therefore initially only
created a luminance texture to save memory, PTB failed to convert it
into a RGB texture when you tried to draw into it -> Error. So the
workaround for your case is to create a 3-layer color texture or use
Offscreen windows until the next beta update is out.

Conversion is (sometimes) an expensive operation, so its best to
create a texture in a way that best matches its later usage to avoid
the conversion step.

Examples:

Screen('MakeTexture'): PTB assumes the texture is usually only used
for drawing it quickly, not drawing into it: The texture is created
as either a luminance, or RGBA texture, depending if the Matlab input
matrix is a grayscale (1 layer, 2D) matrix or something bigger (> 1
layers, 3D). Image data is stored in Matlabs column major order,
reducing the execution time of 'MakeTexture' by up to a factor of 10.
Storage is 8 bits per color or luminance component. There are
optional flags to 'MakeTexture' to create textures of different
formats, e.g., high dynamic range textures with 16 bit or 32bit
floating point precision per color component.

-> Fast creation, minimal memory usage, fast drawing and copying.
-> When you try to draw into it the first time or apply an image
processing operation, PTB will reformat the texture to OpenGL row-
major image format and convert pure luminance textures into RGBA
textures to make them suitable as rendertargets. Expensive, depending
on the size and format of the texture and your gfx-hardware.

Textures from Quicktime movie playback Screen('GetMovieImage') or
video capture engine Screen('GetCapturedImage'): They are created as
luminace, RGB or YUV textures (the latter one is optimized for
typical video data, used by some movie file formats for better
compression). The images are sometimes (some movie formats) upside
down. Resolution always 8 bits per component.

-> Optimized for fast creation, fast drawing copying.
-> When drawing into them, PTB applies conversion (flipping upside if
needed, converting luminance and YUV to RGB if needed).

Screen('OpenOffscreenWindow'): This "textures" are always meant for
drawing into them, therefore they are always created in a suitable
format with RGBA channels of either 8bpc precision or - at request-
16 bpc or 32bpc floating point precision. They are always formatted
in OpenGL's image layout (row major) and filled with their background
color by a routine that is optimized for that.

-> Optimized for fast filling with background color, fast drawing,
fast copy, fast image processing, drawing into them. Never a need to
apply any conversions.
-> Filling them with Matlab image matrices via the slow 'PutImage'
would be drastically slower than using 'MakeTexture'.

Textures created/injected via Screen('SetOpenGLTexture') or Screen
('SetOpenGLTextureFromMempointer'): No conversion or optimization,
PTB assumes the user of these functions to be an expert, wrong usage
will lead to error or crash.

So when only drawing into a texture, start off with an offscreen
window, when only drawing it, start with maketexture, when you need
to do both, try both alternatives, depends on the gfx-hardware and
cpu which method is faster for your case.

Btw. "expensive" on modern graphics hardware means something like 1 -
5 msecs, not 20-100 msecs, so expensive is a pretty relative term.

Reg. your code: You know that you can scale textures very quickly
during drawing by simply setting the optional srcRect and dstRect
parameters of Screen('DrawTexture')?

best,
-mario
4
Hi Mario,

Thanks for that explanation. It is extremely helpful.

For the time being I will use OpenOffScreenWindow as it looks like that creates the kind of texture I neeed anyway since I will be drawing fixations on pretty much every window I create.

Also I did try using DrawTexture with srcRect and dstRect set but it seemed to do the opposite - i.e. when I drew the texture to the screen instead of scaling the source image down it increased the size of the destination texture instead. Not sure if this is a by product of the type of texture again - I will try using OpenOffScreenWindow instead of Maketexture and see if that solves the problem.

Many thanks again,

Regards,

Matthew.




5
srcRect is the part of the texture that's drawn. dstRect is the output area on the screen. If
you set srcRect = [], i.e., its default, then the whole texture is drawn into the screen area
dstRect. Say your texture is 400 x 400 pixels and you set dstRect=[0 0 200 200] it would
draw your texture to the top-left (0,0) corner of the screen, with a size of 200 x 200
pixels, ie scaling it down by a factor of 2. playing with dstRect is the fastest way to up- or
downscale textures, or to stretch/squeeze them. And its high quality via built-in bilinear
filtering.

Btw. i've put the texture explanation into the FAQ section of the wiki.

With fixation you mean drawing a fixation cross? You could do that to the onscreen
window after drawing your (scaled) image, simply with the 2D drawing commands. That
would be more elegant/efficient?

-mario


--- In psychtoolbox@yahoogroups.com, "empedia" <dexterm@...> wrote:
>
> Hi Mario,
>
> Thanks for that explanation. It is extremely helpful.
>
> For the time being I will use OpenOffScreenWindow as it looks like that
> creates the kind of texture I neeed anyway since I will be drawing
> fixations on pretty much every window I create.
>
> Also I did try using DrawTexture with srcRect and dstRect set but it
> seemed to do the opposite - i.e. when I drew the texture to the screen
> instead of scaling the source image down it increased the size of the
> destination texture instead. Not sure if this is a by product of the
> type of texture again - I will try using OpenOffScreenWindow instead of
> Maketexture and see if that solves the problem.
>
> Many thanks again,
>
> Regards,
>
> Matthew.
>