GStreamer + Drawtext trouble on MS-Windows. [was: Movies not loading]

Hello!

I am having difficulties with a script that is supposed to display videos for my studies’ subjects.

I am using Matlab2021b and have both gstreamer-1.0-devel-msvc-x86_64-1.22.4 and gstreamer-1.0-msvc-x86_64-1.22.4 installed, with custom settings where all features are included.

I have the latest version of psychtoolbox downloaded and installed, as far as I know, since I ran the download script today. It went through perfectly, except for the following:

"Trying to trigger an update of the fontconfig cache if that should prove neccessary.
This may take a couple of seconds, or sometimes even minutes. Please be patient...
Warning: Something went wrong with high-quality
text renderer setup. Read 'help DrawTextPlugin'
for troubleshooting. 
> In PsychtoolboxPostInstallRoutine (line 735)
In SetupPsychtoolbox (line 337) 
Press Return key to continue."

I doubt that this is the sole problem but I could be wrong, of course.

The code used to work, so I’m confused. We did have to update GStreamer, so maybe that was problematic.

I saved all the output from matlab to a text file since matlab seems to freeze and must be forced to close when this script is run. The only PTB output that is received is:

PTB-INFO: Multi-display setup in explicit multi-display mode detected. Using the following mapping:
PTB-INFO: Screen 0 corresponds to the full Windows desktop area. Useful for stereo presentations in stereomode=4 ...
PTB-INFO: Screen 1 corresponds to the display area of the monitor with the Windows-internal name \\.\DISPLAY1 ...
PTB-INFO: Screen 2 corresponds to the display area of the monitor with the Windows-internal name \\.\DISPLAY2 ...

PTB-INFO: Your version of Matlab 64-Bit is global system DPI aware. On Windows-8 or later, fullscreen onscreen windows will only work 
PTB-INFO: properly timing-wise when displayed on displays with the same pixel density as your systems primary display monitor.
PTB-INFO: For your multi-display setup the stimulus display monitor must have a DPI of (96, 96), matching that of
PTB-INFO: your primary display monitor. Ideally you will only display on the primary display in the first place.
PTB-INFO: Displaying on anything with a different DPI will cause mysterious visual timing problems, sync failures etc.
PTB-INFO: Read 'help RetinaDisplay' for more info on this topic.


PTB-INFO: This is Psychtoolbox-3 for Microsoft Windows, under Matlab 64-Bit (Version 3.0.19 - Build date: Feb 20 2023).
PTB-INFO: OS support status: Windows 10 (Version 10.0) supported and tested to some limited degree.
PTB-INFO: Type 'PsychtoolboxVersion' for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file 'License.txt' in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid support, support memberships and other commercial services, please type
PTB-INFO: 'PsychPaidSupportAndServices'.

PTB-INFO: The detected endline of the vertical blank interval is equal or lower than the startline. This indicates
PTB-INFO: that i couldn't detect the duration of the vertical blank interval and won't be able to correct timestamps
PTB-INFO: for it. This will introduce a very small and constant offset (typically << 1 msec). Read 'help BeampositionQueries'
PTB-INFO: for how to correct this, should you really require that last few microseconds of precision.
PTB-INFO: Btw. this can also mean that your systems beamposition queries are slightly broken. It may help timing precision to
PTB-INFO: enable the beamposition workaround, as explained in 'help ConserveVRAMSettings', section 'kPsychUseBeampositionQueryWorkaround'.


PTB-INFO: OpenGL-Renderer is NVIDIA Corporation :: NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2 :: 4.6.0 NVIDIA 531.79
PTB-INFO: VBL startline = 600 , VBL Endline = 599
PTB-INFO: Measured monitor refresh interval from beamposition = 16.579788 ms [60.314403 Hz].
PTB-INFO: Will use beamposition query for accurate Flip time stamping.
PTB-INFO: Measured monitor refresh interval from VBLsync = 16.586600 ms [60.289631 Hz]. (50 valid samples taken, stddev=0.158238 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 16.666667 ms [60.000000 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-INFO: Stereo for free fusion or dual-display desktop spanning enabled (2-in-1 stereo).
PTB-INFO: ==============================================================================================================================
PTB-INFO: WINDOWS DWM DESKTOP COMPOSITOR IS ACTIVE. On this Windows-10 or later system, Psychtoolbox can no longer reliably detect if
PTB-INFO: this will cause trouble for timing and integrity of visual stimuli or not. You might be just fine, or you could be in trouble.
PTB-INFO: Use external measurement equipment and independent procedures to verify reliability of timing if you care about proper timing.
PTB-INFO: ==============================================================================================================================

PTB-INFO: Your script requests use of a stereoscopic display mode (stereomode = 4).
PTB-INFO: Stereoscopic stimulus display is usually more flexible, convenient and robust if
PTB-INFO: the Psychtoolbox imaging pipeline is enabled. Your graphics card is capable
PTB-INFO: of using the pipeline but your script doesn't enable use of the pipeline.
PTB-INFO: I recommend you enable use of the pipeline for enhanced stereo stimulus display.
PTB-INFO: Have a look at the demoscript ImagingStereoDemo.m on how to do this.

I wonder if there is any advice that you can provide?

The matlab function used to run the movies is:

function Show_movie(moviename,eye) % eye is where movie is being presented

white = [255 255 255];
gray = [128 128 128];
black = [0 0 0];
red= [255 0 0];
blue=[0 0 255];
fixRadius_inner = 0.15;
white_line_size = 2;    % width of line in the middle of fixation spot
AssertOpenGL;
Screen('Preference', 'SkipSyncTests', 1);


% Wait until user releases keys on keyboard:
KbReleaseWait;

KbName('UnifyKeyNames');

% Setup key mapping:
space=KbName('Space');
shift=KbName('RightShift');

% Parameter Set-Up

screen = min(Screen('Screens'));
% Stereo - mode is 4
[win, windowrect] = Screen(screen, 'OpenWindow', gray, [], [], [], 4); 	% Limit screen to just a window in the top left screen corner. 
HideCursor

% calculate dimentions of the screen
x0      = windowrect(1);
y0      = windowrect(2);
xEnd    = windowrect(3);
yEnd    = windowrect(4);
screenWidth = (xEnd-x0);
screenHeight= (yEnd-y0);
[centerX, centerY] = RectCenter(windowrect);

FOV =30
PixelperDegree = 800/FOV;
size_radius_degree=0.25; %% this is something we might need to change 

Radius  = round(size_radius_degree*PixelperDegree)
fixRadius_inner = round(PixelperDegree * fixRadius_inner);

fixRect = CenterRectOnPointd([0 0 Radius*2 Radius*2],centerX,centerY);
fixRect_inner=CenterRectOnPointd([0 0 fixRadius_inner*2 fixRadius_inner*2],centerX,centerY);

Priority(MaxPriority(win));
%% Wait Screen --> The experiment will start shortly
Screen('SelectStereoDrawBuffer', win, eye);
Screen('FillRect', win,black);
Screen('TextSize',win,20);

Screen('SelectStereoDrawBuffer', win, double(~eye));
Screen('FillRect', win,black);
Screen('TextSize',win,20);

textMessage = 'The experiment will start shortly';
textRect = Screen(win, 'TextBounds', textMessage);
textWidth = textRect(3) - textRect(1);
textHeight = textRect(4) - textRect(2);

% First eye
Screen('SelectStereoDrawBuffer', win, eye);
Screen('DrawText', win, textMessage,...
    centerX-(textWidth/2), centerY-(textHeight/2), white, black);
% Second eye
Screen('SelectStereoDrawBuffer', win, double(~eye));

Screen('DrawText', win, textMessage,...
    centerX-(textWidth/2), centerY-(textHeight/2), white, black);

% Flip info to the real window
Screen('Flip', win);
%% keyboard cheking 
%Let matlab command window ignore incoming keypresses
KbName('UnifyKeyNames');
psychtoolbox_forp_id=0;
%% start after a t has been pressed
try
    % Open movie file:
    Screen('SelectStereoDrawBuffer', win, eye);
    [movie,movieduration,fps,w,h,count] = Screen('OpenMovie', win, moviename);
    
    Screen('SelectStereoDrawBuffer', win, eye);
    % Start playback engine:
    Screen('PlayMovie', movie, 1);
    
    % Playback loop: Runs until end of movie or keypress:
    keysOfInterest=zeros(1,256);
    keysOfInterest(KbName('t'))=1;
    % only look for t as trigger
    KbQueueCreate(psychtoolbox_forp_id, keysOfInterest);	
    KbQueueStart;
    
    [keyPress, keyTime, keyID] = KbCheck(-1);  
    oldKeyID = keyID;
    KbQueueWait
    while 1

      %%%% PAUSE CODE %%%%%%%
      [keyIsDown,secs,keyCode]=KbCheck;
      if (keyIsDown==1 && keyCode(space))
            % Stop playback:
            Priority(0);
            Screen('PlayMovie', movie, 0);
            
            % Close movie:
            Screen('CloseMovie', movie);
            
            % Close Screen, we're done:
            sca;
      end

      % Wait for next movie frame, retrieve texture handle to it
      Screen('SelectStereoDrawBuffer', win, eye);
      tex = Screen('GetMovieImage', win, movie);
          
      % Valid texture returned? A negative value means end of movie reached:
      if tex<=0
        % We're done, break out of loop:
        break;
      end
          
      % Draw the new texture immediately to screen:
      Screen('SelectStereoDrawBuffer', win, eye);
      Screen('DrawTexture', win, tex);
  
      Screen('SelectStereoDrawBuffer', win, eye);
      Screen('FillOval', win, blue, fixRect);
      Screen('FillOval', win, red, fixRect_inner);   
      Screen('DrawLine', win, white, fixRect(1), centerY,fixRect(3), centerY, white_line_size);
      Screen('DrawLine', win, white, centerX, fixRect(2),centerX, fixRect(4), white_line_size);


      % Update display:
      Screen('Flip', win);
          
      % Release texture:
      Screen('Close', tex);
    end
    
    % Stop playback:
    Priority(0);
    Screen('PlayMovie', movie, 0);
    
    % Close movie:
    Screen('CloseMovie', movie);
    
    % Close Screen, we're done:
    sca;
    ShowCursor
    
catch %#ok<CTCH>
    Priority(0);
    sca;
    ShowCursor
    psychrethrow(psychlasterror);
end

Hmm, thats annoying. I ran into this too where the link to the latest
GStreamer version gave me trouble, but an older version worked. Could
you try with the not the latest GStreamer, but the minimum required
version:
https://gstreamer.freedesktop.org/data/pkg/windows/1.20.5/msvc/gstreamer-1.0-msvc-x86_64-1.20.5.msi

Hi there! This solved the problem!!! Thank you :slight_smile:

@Mario: this has come up for me locally, for a collaborator and on the forum. On Windows, the latest version didn’t work, the oldest recommended (1.20.5) does work. I guess for others the latest does work. I have not digged deeper about what causes the drawtextplugin dll to load. Perhaps good to just recommend 1.20.5 instead of latest on windows?

The content of the original bug report is insufficient to find out what was going on, not even how far the script made it before the freeze, or if it only happens with certain movie files, e.g., if our own demos work.

You mean version 1.22.5, the latest as of now, and 1.22.4 didn’t work? Did the 1.22.1 release, tested by myself and recommended in our help text, work? What was the oldest version that worked?

Sticking to old versions is not a proper solution, figuring out what broke when is.

Edit: My testing with the latest GStreamer 1.22.5 with Octave 7.3 and Matlab R2022b on Windows-10 showed no obvious problems with our demos at all. Whatever the problem may be, it is apparently not a general v1.22 problem.

Hi Mario,

I just tested this. I used an up to date PTB and installed exactly the linked 1.22.1 GStreamer version. This is Windows 10. I chose complete when installing gstreamer. DrawTextPlugin fails to load. Using dependencywalker, i see that the issue is that libfreetype-6.dll is not found. Indeed, it is not in gstreamer’s bin directory, where i think it is supposed to be found? So perhaps systems where the install does work happen to have that dll from elsewhere? I think the simple solution is to link to 1.20.5 instead, which does seem to work just fine (and indeed does include that dll).

Cheers,
Dee

Thanks for the diagnosis, confirmed. GStreamer 1.22 apparently changed the name of the file from libfreetype-6.dll to freetype-6.dll. With PTB on Octave this wasn’t a problem as it ships libfreetype-6.dll, and with an Eyelink runtime installed (in my case with Matlab R2022b) it wasn’t a problem either as Eyelink provided the dll.

Anyhow, I solved the problem for the next PTB release by rebuilding the plugin against GStreamer 1.22.5 and now making GStreamer 1.22 the minimum requirement on MS-Windows. Seems to work and link against the correct GStreamer 1.22 dll’s on both Octave and Matlab.

→ This will likely require all MS-Windows users of the next Psychtoolbox 3.0.19.4 to upgrade to the lightly tested/recommended GStreamer 1.22.5, but such is life.

→ Maybe making a copy of libfreetype-6.dll as freetype-6.dll will also work if one wants to stick to the outdated GStreamer 1.20.5+ series, but this is untested and 1.20 is unsupported from my side.

Thanks Mario! That seems like a good, and future-proof, solution. I guess we might get some forum posts about this, but should be easily diagnosed and have a simple solution.

Cheers!

Is anything ever future-proof when it comes to Windows + text rendering? Windows DLL hell at its best. But lets see how long this one sticks, fingers crossed…

Btw. if the new plugin by some weird luck still happens to work with the old GStreamer 1.20.5+, let me know. Very unlikely, but would be nice to let people know if they wouldn’t have to upgrade.

https://github.com/kleinerm/Psychtoolbox-3/raw/master/Psychtoolbox/PsychBasic/PsychPlugins/libptbdrawtext_ftgl64.dll