TextRenderer and QGLWidget::renderText()

Hello, Benoit!
Since you’ve written TextRenderer, I hope you can help.
There’s a hidden bug in your code which causes missing letters in labels in some cases. I think it isn’t worth to debug, because of discussed limitations of TextRenderer (impossibility to change font, outline color, etc)
I’ve found QGLWidget::renderText working fine in Qt 4.5, however it lacks support for outline rendering. I’ve asked on qt-interest, and was given an answer: "The best approach is to render the text into texture and use the multitexture."
So, we still need TextRender class, but it can be much simplier and powerful. Everything it needs to do is 1) render string into texture; 2) draw outline into another texture; 3) combine them and draw in given point
But (as far as I understood) work with textures in Qt requires some knowledge of OpenGL internals, which I haven’t. Could you make a draft for this rendering, i.e. approximate sequence of function calls?

Regards,
Konstantin

Hi, Konstantin and Avogadro team,

2010/3/1 Konstantin Tokarev annulen@yandex.ru:

Hello, Benoit!
Since you’ve written TextRenderer, I hope you can help.
There’s a hidden bug in your code which causes missing letters in labels in some cases.

OK, I can believe that and actually, let me make it clear that
TextRenderer makes rather naive assumptions about unicode strings (it
was mostly meant as a temporary measure until Qt’s own solutions
matured more).

Indeed, TextRenderer is iterating over the characters (QChar’s) in a
QString, and rendering them successively, next to each other. This
doesn’t really work in full generality: Unicode allows for all sorts
of special cases that break this approach. For example, diacritic
characters.

I think it isn’t worth to debug,

Indeed, debugging this fully would require far more knowledge of
Unicode than I have.

because of discussed limitations of TextRenderer (impossibility to change font, outline color, etc)

These limitations are nothing in comparison, you could fix them in a
few lines of code. The outline color especially is just a matter of
calling glColor3f. Allowing to change fonts would be slightly more
complex, indeed, but still not hard.

I’ve found QGLWidget::renderText working fine in Qt 4.5, however it lacks support for outline rendering. I’ve asked on qt-interest, and was given an answer: “The best approach is to render the text into texture and use the multitexture.”

I’m missing how “using a multitexture” takes care of generating the outline.

Generating the outline is the only hard part, fortunately the code for
that is already present in TextRenderer.

So, we still need TextRender class, but it can be much simplier and powerful. Everything it needs to do is 1) render string into texture; 2) draw outline into another texture; 3) combine them and draw in given point

OK, good idea. Note that 2) and 3) are exactly like what TextRenderer
currently does. The fundamental difference is that in 1) you’re
proposing to render a whole string into a texture, whereas
TextRenderer renders individual characters (glyphs).

Now I fully agree that your idea (to render per-string textures
instead of per-character textures) is both simpler and more powerful.
It should be a great improvement!

But (as far as I understood) work with textures in Qt requires some knowledge of OpenGL internals, which I haven’t. Could you make a draft for this rendering, i.e. approximate sequence of function calls?

OK so, first, let’s explain how TextRenderer currently works.

=======================================
A. How TextRenderer currently works

The user calls:

drawText(position, “hello”);

This takes us to the method TextRendererPrivate::do_draw() which
iterates over all characters, and for each character, asks itself: “do
I have this string already rendered in my cache” ?

Its cache is a QHash declared as follows:

 QHash<QChar, CharRenderer*> charTable;

mapping QChar’s to “pre-rendered characters”, which I (poorly) named
CharRenderer but should have been named RenderedChar. It’s just a
small class storing the two texture ID’s for the character (the
character itself, and its outline).

If the character is found in cache, then actually drawing it is super
simple: just select its texture and draw a OpenGL quad with it. Do
that first for the outline, then for the character itself on top.
These are the methods CharRenderer::drawOutline() and drawGlyph().

If the character is not found in cache, then we must render it and add
it to our cache. This is the big CharRenderer::initialize() method. It
has comments which I hope help you read its code. It works as follows:

  • Step 1: render the character to a QImage
  • Step 2: convert the resulting bitmap to a lightweight 8-bit,
    1-channel bitmap format
  • Step 3: compute the “neighborhood map” which is going to be
    essentially the outline bitmap.
  • Step 4: compute the actual glyph and outline bitmaps that we’ll use
    as textures
  • Step 5: pass these textures to OpenGL, store the texture IDs
  • Step 6: record a small display list to render a quad of the size of
    the character and translate to the right of it. Just a small helper
    for rendering.

Again, I hope that the comments in the body of initialize() will be
helpful to you.

=======================================
B. Action plan for you

You can actually keep much of the existing code, there is only one
fundamental difference: you’ll now be rendering a whole string at
once, instead of rendering only one character at once. You also want
(afaiu) to allow to use different strings. Finally, since you’ll now
be caching each /string/, you could end up with a huge cache, so it’s
probably going to be useful to implement a mechanism to limit the
cache size.

=========
B.1 Adapt TextRenderer to work per-string, not per-character

That’s right, I’m 99% sure that you don’t have to do the work from
scratch, instead it should be rather easy to adapt TextRenderer to
render a whole string at once. The CharRenderer class gets renamed to
StringRenderer, or whatever sounds good in English. The code to
iterate over the chars of a string (where the bug was! very hard to
get right!) simply goes away, which is great news! The rest should be
a trivial adaptation.

With just B.1 you already have something that essentially works.

=========
B.2 Allow to use multiple fonts

In the current TextRenderer, the hash table uses QChar as keys. After
B.1, it will now be QString as keys. But if you want to allow multiple
fonts, you have to let the font be recorded as part of the key!

Unfortunately, Qt doesn’t seem to provide a qHash function for fonts,
but what you can do is to generate a new string putting together both
the string-to-be-rendered and the font, and get a hash value from it.
For example, if you’re rendering “hello” in Helvetica 12 pt, you could
construct the string “hello@@Helvetica@@12” and get a hash value from
it. Here, @@ is just a character sequence that is unlikely to be used
in user strings, nothing more. Of course if you want to be extra safe
you could get a separate hash value for the font and then use
qHash(QPair…), but it would be slower… it’s your call :wink:

=========
B.3 Limit the cache size

Imagine this scenario: you’re running an algorithm in avogadro that
prints a new different string for each frame, at 30 FPS. You let
Avogadro run for 1 hour. You then have rendered 1e+5 different
strings! Do you want them all to be cached ? :stuck_out_tongue:

So some mechanism is needed to limit the cache size… here are a few ideas:

  • everytime you add a string to the cache, record its number of
    pixels (width*height) and add that to some variable cache_pixels. When
    cache_pixels > 1000000, erase the 50% oldest strings from the cache.
    Optimally, you’d be able to erase the strings that haven’t been used
    since the longest amount of time… but that’s not the easiest thing
    to implement…
  • or experiment with no caching at all. After all, what do I know?
    Perhaps text rendering is faster than I think… that will also get
    you rid of B.2, e.g. no problem at all anymore with multiple fonts…

Good luck!
Benoit


Regards,
Konstantin

Hi, Benoit!
Thanks for thorough explanation!
However, AFAIU, QGLWidget::renderText() isn’t used in the scheme of yours. In the code it’s mentioned, that Qt’s text rendering wasn’t used because of Qt’s buggy code, but I believe it’s not about 2D text rendering :slight_smile:
So, I didn’t understand, why character based rendering was used those days instead of string-based. Maybe there are some other problems you didn’t mention?
Also, I’ve found in comments that texture must be square. Why it’s needed? Won’t it be a problem for string based rendering?

2010/3/5 Konstantin Tokarev annulen@yandex.ru:

Hi, Benoit!
Thanks for thorough explanation!
However, AFAIU, QGLWidget::renderText() isn’t used in the scheme of yours. In the code it’s mentioned, that Qt’s text rendering wasn’t used because of Qt’s buggy code, but I believe it’s not about 2D text rendering :slight_smile:
So, I didn’t understand, why character based rendering was used those days instead of string-based. Maybe there are some other problems you didn’t mention?

String-based rendering is definitely the way to go:

  • it leverages Qt’s unicode string handling capabilities
  • it leverages Qt’s font layout (kerning etc) capabilities

It was definitely a poor design decision, that my TextRenderer was
character-based.

For your information, at the time I wrote it, QGLWidget::renderText
was also character-based. Maybe they changed it, I don’t know.

String-based rendering is definitely the way to go.

Also, I’ve found in comments that texture must be square. Why it’s needed? Won’t it be a problem for string based rendering?

In the OpenGL 1 standard, textures are required to be square and of a
size that’s a power of 2. However there is a “standard extension” that
a given implementation of OpenGL may or may not provide, that allows
textures of any rectangular size. If you look at my code, I check for
the presence of that extension and use it when it’s available.

Now we’re in 2010, I’m pretty sure that arbitrary rectangular texture
size are now part of recent revisions of the OpenGL standard, and so
I’m rather confident that you can safely assume support for that
feature.

FYI, alrady 5 years ago, Intel integrated graphics chips (i945)
already supported that feature… so it’s quite reasonable to
require.

Otherwise, yes, it would be a huge problem for string rendering. What
Qt’s renderText does is combine all glyphs into a single huge square
texture of size a power of 2. I don’t recommend taking that approach.
It’s wasting lots of video memory and takes very complex code.

Benoit

On Friday 05 March 2010 14:27:01 Benoit Jacob wrote:

2010/3/5 Konstantin Tokarev annulen@yandex.ru:

Hi, Benoit!
Thanks for thorough explanation!
However, AFAIU, QGLWidget::renderText() isn’t used in the scheme of
yours. In the code it’s mentioned, that Qt’s text rendering wasn’t used
because of Qt’s buggy code, but I believe it’s not about 2D text
rendering :slight_smile: So, I didn’t understand, why character based rendering was
used those days instead of string-based. Maybe there are some other
problems you didn’t mention?

String-based rendering is definitely the way to go:

  • it leverages Qt’s unicode string handling capabilities
  • it leverages Qt’s font layout (kerning etc) capabilities

It was definitely a poor design decision, that my TextRenderer was
character-based.

For your information, at the time I wrote it, QGLWidget::renderText
was also character-based. Maybe they changed it, I don’t know.

String-based rendering is definitely the way to go.

How do you handle caching? One problem I have encountered in some of the work
I have been doing in VTK is caching of string based rendering - it means that
each unique string is cached (rather than each unique glyph). This causes my
application to use more and more memory as more strings are rendered. Do we
cease caching the strings at all, use some kind of stack based approach?

Also, I’ve found in comments that texture must be square. Why it’s
needed? Won’t it be a problem for string based rendering?

In the OpenGL 1 standard, textures are required to be square and of a
size that’s a power of 2. However there is a “standard extension” that
a given implementation of OpenGL may or may not provide, that allows
textures of any rectangular size. If you look at my code, I check for
the presence of that extension and use it when it’s available.

Now we’re in 2010, I’m pretty sure that arbitrary rectangular texture
size are now part of recent revisions of the OpenGL standard, and so
I’m rather confident that you can safely assume support for that
feature.

FYI, alrady 5 years ago, Intel integrated graphics chips (i945)
already supported that feature… so it’s quite reasonable to
require.

Otherwise, yes, it would be a huge problem for string rendering. What
Qt’s renderText does is combine all glyphs into a single huge square
texture of size a power of 2. I don’t recommend taking that approach.
It’s wasting lots of video memory and takes very complex code.

One of the big advantages of using one texture is reducing the number of
OpenGL state changes, which can significantly improve performance (depending on
your use case of course). Unfortunately there are still cards around that do
not support non-power of 2, but I think I am with you on it being reasonable
not to support them.

I don’t have the time to focus on it right now, but string based rendering
comes with its own issues (especially when you might have tens of thousands of
unique strings). I just wanted to throw a little perspective on this from
issues I have hit with my work. Not claiming to have the best solution right
now.

Marcus

One of the big advantages of using one texture is reducing the number of
OpenGL state changes, which can significantly improve performance (depending on
your use case of course). Unfortunately there are still cards around that do
not support non-power of 2, but I think I am with you on it being reasonable
not to support them.
I don’t have the time to focus on it right now, but string based rendering
comes with its own issues (especially when you might have tens of thousands of
unique strings). I just wanted to throw a little perspective on this from
issues I have hit with my work. Not claiming to have the best solution right
now.

May be you know if Qt’s renderText has the same limitations? It doesn’t draw outlines, but it doesn’t skip symbols, so it could be a temporary workaround (e.g., let user to choose style of label’s rendering)


Regards,
Konstantin

2010/3/7 Marcus D. Hanwell marcus@cryos.org:

On Friday 05 March 2010 14:27:01 Benoit Jacob wrote:

2010/3/5 Konstantin Tokarev annulen@yandex.ru:

Hi, Benoit!
Thanks for thorough explanation!
However, AFAIU, QGLWidget::renderText() isn’t used in the scheme of
yours. In the code it’s mentioned, that Qt’s text rendering wasn’t used
because of Qt’s buggy code, but I believe it’s not about 2D text
rendering :slight_smile: So, I didn’t understand, why character based rendering was
used those days instead of string-based. Maybe there are some other
problems you didn’t mention?

String-based rendering is definitely the way to go:

  • it leverages Qt’s unicode string handling capabilities
  • it leverages Qt’s font layout (kerning etc) capabilities

It was definitely a poor design decision, that my TextRenderer was
character-based.

For your information, at the time I wrote it, QGLWidget::renderText
was also character-based. Maybe they changed it, I don’t know.

String-based rendering is definitely the way to go.

How do you handle caching? One problem I have encountered in some of the work
I have been doing in VTK is caching of string based rendering - it means that
each unique string is cached (rather than each unique glyph). This causes my
application to use more and more memory as more strings are rendered. Do we
cease caching the strings at all, use some kind of stack based approach?

I’m not saying it’s easy :slight_smile:

One idea is what I described earlier in this thread: when the number
of pixels in the cache becomes greater than some value N, remove all
the oldest cache entries until the cache size (in pixels) falls under
N/2…

Another idea is to use a special (custom?) kind of Hash map that
automatically forgets the oldest-accessed entries…

Also, I’ve found in comments that texture must be square. Why it’s
needed? Won’t it be a problem for string based rendering?

In the OpenGL 1 standard, textures are required to be square and of a
size that’s a power of 2. However there is a “standard extension” that
a given implementation of OpenGL may or may not provide, that allows
textures of any rectangular size. If you look at my code, I check for
the presence of that extension and use it when it’s available.

Now we’re in 2010, I’m pretty sure that arbitrary rectangular texture
size are now part of recent revisions of the OpenGL standard, and so
I’m rather confident that you can safely assume support for that
feature.

FYI, alrady 5 years ago, Intel integrated graphics chips (i945)
already supported that feature… so it’s quite reasonable to
require.

Otherwise, yes, it would be a huge problem for string rendering. What
Qt’s renderText does is combine all glyphs into a single huge square
texture of size a power of 2. I don’t recommend taking that approach.
It’s wasting lots of video memory and takes very complex code.

One of the big advantages of using one texture is reducing the number of
OpenGL state changes, which can significantly improve performance (depending on
your use case of course). Unfortunately there are still cards around that do
not support non-power of 2, but I think I am with you on it being reasonable
not to support them.

Also keep in mind that if you want to stick to pow-of-2 square
textures, whenever you have to resize your texture to a larger size,
you have to multiply its size by at least 4, which means a large waste
of memory, in addition to the holes that are bound to exist between
the strings rectanges inside of that large square texture…

I can’t comment on how expensive it is to bind to a new Texture Object
id… no clue about that.

I don’t have the time to focus on it right now, but string based rendering
comes with its own issues (especially when you might have tens of thousands of
unique strings).

I have no clue about how expensive it is to have many small
textures… you probably know far better than me :slight_smile:

I just wanted to throw a little perspective on this from
issues I have hit with my work. Not claiming to have the best solution right
now.

Same for me :slight_smile:

Benoit

Marcus

2010/3/7 Benoit Jacob jacob.benoit.1@gmail.com:

2010/3/7 Marcus D. Hanwell marcus@cryos.org:

On Friday 05 March 2010 14:27:01 Benoit Jacob wrote:

2010/3/5 Konstantin Tokarev annulen@yandex.ru:

Hi, Benoit!
Thanks for thorough explanation!
However, AFAIU, QGLWidget::renderText() isn’t used in the scheme of
yours. In the code it’s mentioned, that Qt’s text rendering wasn’t used
because of Qt’s buggy code, but I believe it’s not about 2D text
rendering :slight_smile: So, I didn’t understand, why character based rendering was
used those days instead of string-based. Maybe there are some other
problems you didn’t mention?

String-based rendering is definitely the way to go:

  • it leverages Qt’s unicode string handling capabilities
  • it leverages Qt’s font layout (kerning etc) capabilities

It was definitely a poor design decision, that my TextRenderer was
character-based.

For your information, at the time I wrote it, QGLWidget::renderText
was also character-based. Maybe they changed it, I don’t know.

String-based rendering is definitely the way to go.

How do you handle caching? One problem I have encountered in some of the work
I have been doing in VTK is caching of string based rendering - it means that
each unique string is cached (rather than each unique glyph). This causes my
application to use more and more memory as more strings are rendered. Do we
cease caching the strings at all, use some kind of stack based approach?

I’m not saying it’s easy :slight_smile:

On second thought: per-character rendering does indeed have merits for
printing numbers. Indeed, numbers:

  • are typically where we could end up rendering thousands of different strings
  • expose no unicode subtleties
  • expose little font rendering subtleties (kerning…)

So an idea would be, when rendering a string, to split the string to
make each digit [0-9] be an individual string. No need then to take
any special care for comma/point.

For example the string “Hello 12.3” becomes the following sequence of strings:
“Hello” , “1” , “2” , “.” , “3”

This should make it unnecessary to do anything about limiting the cache size…

Benoit

One idea is what I described earlier in this thread: when the number
of pixels in the cache becomes greater than some value N, remove all
the oldest cache entries until the cache size (in pixels) falls under
N/2…

Another idea is to use a special (custom?) kind of Hash map that
automatically forgets the oldest-accessed entries…

Also, I’ve found in comments that texture must be square. Why it’s
needed? Won’t it be a problem for string based rendering?

In the OpenGL 1 standard, textures are required to be square and of a
size that’s a power of 2. However there is a “standard extension” that
a given implementation of OpenGL may or may not provide, that allows
textures of any rectangular size. If you look at my code, I check for
the presence of that extension and use it when it’s available.

Now we’re in 2010, I’m pretty sure that arbitrary rectangular texture
size are now part of recent revisions of the OpenGL standard, and so
I’m rather confident that you can safely assume support for that
feature.

FYI, alrady 5 years ago, Intel integrated graphics chips (i945)
already supported that feature… so it’s quite reasonable to
require.

Otherwise, yes, it would be a huge problem for string rendering. What
Qt’s renderText does is combine all glyphs into a single huge square
texture of size a power of 2. I don’t recommend taking that approach.
It’s wasting lots of video memory and takes very complex code.

One of the big advantages of using one texture is reducing the number of
OpenGL state changes, which can significantly improve performance (depending on
your use case of course). Unfortunately there are still cards around that do
not support non-power of 2, but I think I am with you on it being reasonable
not to support them.

Also keep in mind that if you want to stick to pow-of-2 square
textures, whenever you have to resize your texture to a larger size,
you have to multiply its size by at least 4, which means a large waste
of memory, in addition to the holes that are bound to exist between
the strings rectanges inside of that large square texture…

I can’t comment on how expensive it is to bind to a new Texture Object
id… no clue about that.

I don’t have the time to focus on it right now, but string based rendering
comes with its own issues (especially when you might have tens of thousands of
unique strings).

I have no clue about how expensive it is to have many small
textures… you probably know far better than me :slight_smile:

I just wanted to throw a little perspective on this from
issues I have hit with my work. Not claiming to have the best solution right
now.

Same for me :slight_smile:

Benoit

Marcus