Text Bounding Boxes

Hi all,

I have been playing around with bounding boxes for text, prompted by a recent forum question. I have made a demo drawing the alphabet and the bounding boxes around the letters. I am getting some behavior that I did not expect.

https://www.peterscarfe.com/alphabet.html

Essentially the corners of the bounding boxes for some of the letters are not aligned with the (1) the requested position of the text on the screen, (2) the report end of the text from DrawFormattedText, or overall (3) the letter. I had expected (1) and (2) to be the case. And (3) to be “approximately” correct.

The bounding boxes nether-the-less look to be the correct size as I can “correct” them to align to (1) and (2).

This behavior is not caused by letters with tails like “y”, which I know have issues in terms of bounding boxes.

Additionally, while I understand that the bounding box is an “approximation”. These misaligned ones seem really very wrong (more than “approximately correct”). Coupled with being visually inconsistent with (1) and (2), makes me think this is a potential bug.

Furthermore, even for letters like “g” and “y”, visually the bounding box sizes “look” correct (I have not measured this), just misaligned with the letter. All be it aligned with (1) and (2).

Note I say “potential bug” as I am absolutely no expert in text rendering.

P

Characters which cover even less of the vertical space than lower-case letters, like ‘=’, ‘-’, ‘_’, ‘+’, also make interesting use cases.

1 Like

Ah, yes, I had not tested those.

The alphabet demo came about because I was having trouble understanding it with words.

P

I should also say I am using the latest PTB (bar the version released yesterday). Running on maxOS 15.1 with Matlab 2024b. Have not yet had a chance to test on Windows.

P

How does this align with the second output of Screen(‘TextBounds’)? thats what DrawFormattedText2 uses for positioning text and seems to work well.

1 Like

Thanks Dee.

I will give it a go and report back.

P

You are hitting a special case in the heuristic for the bounding box calculation, because your “text string” only consists of a single character.

The culprit is likely line 313 of DrawFormattedText.m:
blankbounds = Screen('TextBounds', win, 'X', [], [], 1, righttoleft);

Here, as a heuristic, the bounding box blankbounds of the character capital ‘X’ is used as a prototype for the baseline height of characters, ie. the height of characters which do not have descenders, e.g., xXwWoO etc. but not ygjpqQ. See line 324:

baselineHeight = RectHeight(blankbounds);

baselineHeight is by how much the bounding box must be shifted vertically from computed text bounds. If baselineHeight is wrong for a given text string then the bounding box is shifted. If your text string doesn’t have at least one character that is as tall as a X, the bounding box will be shifted by the difference of the tallest character and X. Therefore the error is small or negligible for typical text strings, but not for strings with only minor characters, or worst case single characters, like in your case. You can see that “small” characters in your example have a large error, but not “tall” ones. You also see that DrawFormattedTextDemo produces less inaccurate bounding boxes for the demo text.

Now the way to get a near-perfect bounding box would be to not use the returned bounding box textbounds, but instead the 4th return arg wordbounds in your case, or for longer text to compute the bounding box over all wordbounds bounding boxes of all words in a text string. That gives as perfect results as the chosen text font allows, iow. any mistake is caused by mistakes the font designers made when creating the font definition file - storing wrong or inaccurate geometry information in the glyph definitions. We could do that internally. The problem with this is that computing per word bounding boxes and then the bounding box over all bounding boxes is much slower than computing one overall bounding box via the chosen heuristic.

And a perfect bounding box can be only computed via image processing, see help TextBounds for that. Of course that is another 1-2 orders of magnitude slower.

Text rendering is a slow operation in itself, the text formatting done by DrawFormattedText() and the more flexible and higher quality DrawFormattedText2() is even much slower. Therefore we always have a speed vs. accuracy vs. quality tradeoff. For a normal typesetting application, a word processor, or a text editor, accuracy is way more important than speed, as long as the human in front of the keyboard can’t type faster than the software can draw. For Psychtoolbox that may have to draw some text and stimuli at > 200 fps, speed is of great importance, so the bounding box heuristic is such a tradeoff. Hence the fuzzy language in the help text that it can be of limited accuracy or an approximation. You are hitting the edge case of really bad here.

Ofc. line 313 could be improved with something more clever for such cases with better heuristics. But one has to be careful not to make things too slow. And especially careful to not make it incompatible/broken for Octave or older Matlab versions which have various limitations wrt. Unicode handling, and operating system differences, one has to work around - see the various casting operations between char() and double() and weird contortions to deal with that. Or with non-western non-alphanumeric character / glyps sets, e.g., chinese, japanese, hebrew, special symbols like emojies etc. After all, unicode has ten-thousands of code points, and western characters only make up < 255 of them.

I think DrawFormattedText2 can handle some of these cases better, but is slower, and I also may misremember.

I don’t remember if DrawFormatterText2 handles this better, but its
worth a try. I think it takes the actual text’s measurements (assuming
they are accurately provided by the font) when positioning, so it may
work better. It also has a mode where you can run all its parsing and
setup once, and not draw but only generate a cache. This cache can
then be used for fast(er) drawing of the resulting text (also with
some subset of operations, such as positioning still possible IIRC),
mitigating some of the slowness.

Thanks very much both for the comments.

I will have a further play and ponder if I could contribute anything useful.

Sorry for the lack of response on this thread. As always, this is just something I got doing in my spare time in the scope of the PTB Demos. And I have not had very much time recently.

Work (and the time I spend on work outside of paid work time) have soaked up everything.

It is great that folk find the demos useful. Only so much time in the day though…

P

Don’t sweat it. I personally find text rendering and formatting one of the absolutely most tedious topics, easy to get wrong, hard to get right, with the highest potential to turn even small changes into endless whack-a-mole games, especially across 3 operating systems and two different scripting environments in many different versions, where “easy” 1 hour projects can easily turn into multi-day (multi week?) odysseys and really bad mood, if and when one comes out of it at the end.

I think if complex text formatting is needed, DrawFormattedText2 is the best choice. Diederick did a masterpiece there in how far one can push the whole thing in M-Code at good quality and still bearable speed. The caching even allows for precomputing and speeding up things sometimes. However, at over 1600 lines of complex code and transformations, I sincerely hope I don’t have to fully understand or touch it ever :exploding_head:.

Wrt. to your specific demo, the most simple solution would be to use the optional wordbounds return arg of DrawFormattedText(). Or simply use Screen('Drawtext', ...) and Screen('Textbounds', ...) directly, as the demo does not actually use any of the formatting provided by DrawFormattedText(). E.g.,
[~, wordbound] = Screen('TextBounds', windows, thisLetter, y(i), x(i), 1) as that is what both DrawFormattedText and DrawFormattedText2 use for usually quite precise per-word (or per letter in your case) bounding boxes. This computes the bounding boxes mathematically, based on geometry info in the font definition files installed on your machine. The fastest method and most accurate, apart from applying slow image processing to find the bounding box. Also, we actually have demos or tests for behavior of this, e.g., TextBoundsTest.m for comparing the fast and slow image processing method. Or TextOffByOneBugTest.m illustrating past trouble with this.

1 Like