ATI Radeon 7000 errors in gamma correction

Dear ATI

SUMMARY: The Mac graphics driver for the Radeon 7000 makes an
off-by-one mistake in using the user-supplied gamma table to do gamma
correction, and maps colors 0 and 1 to the same gamma table entry.

I am a vision researcher. 8-bit DAC graphics cards don't have quite
enough resolution to present research-grade visual stimuli. When I
learned that the ATI Radeon 7000 had 10-bit DACs, and that the Mac
driver supported this through a 10-bit gamma table, I ordered one
immediately, from ATI. I've been using it for about a week. I'm
writing to report some problems with the driver.

Please note that I am very pleased by ATI's bold step in providing
10-bit DACs and 10-bit support in its drivers. The Psychophysics
Toolbox, which is used by hundreds of vision scientists has a web
site, and we have been quite excited by ATI's offerings and have
alerted everyone to the possibilities and current glitches. We have
made a significant effort to enhance the 10-bit support of the
Psychtoolbox software, anticipating that many of our users would like
to switch to 10-bit cards, currently only available from ATI. I'm
hoping that ATI might find it easy to fix this simple off-by-one bug
in their driver.
http://psychtoolbox.org/mac.html#drivers

The problem is in the implementation of gamma correction. Apple
defines three relevant graphics driver control calls - cscSetEntries,
cscDirectSetEntries, cscSetGamma - and two relevant status calls -
cscGetEntries and cscGetGamma.

As you know, cscSetEntries/cscDirectSetEntries are used to provide an
8-bit color spec table to the driver, consisting of between 32 and
256 colors (depending on pixelSize), each color being designated as
an rgb triplet of 16-bit numbers, but only the top 8 bits are used.
The driver maintains a gamma table, and transforms all the 8-bit
color components through the gamma table to obtain gamma-corrected
values that it loads into the RAMDAC's CLUT. The gamma table has 256
entries, mapping each 8-bit color value to an 8 to 10-bit value in
the RAMDAC.

The cscSetGamma call allows us to load a custom gamma table. The
gamma table is typically 8-bit (gDataWidth==8), but the Radeon 7000
also supports a 10-bit gamma table. The Radeon 7000 driver accepts a
10-bit gamma table from cscSetGamma, and returns it faithfully to
cscGetGamma. (We've checked, the returned table is correct, identical
to the supplied table.) The Radeon 7000 driver also supports 8-bit
gamma tables.

So far so good.

DRIVER'S GAMMA CORRECTION IS ERRONEOUS

It is difficult to measure the visual image with enough accuracy to
determine whether 10-bit performance is being achieved. But it's easy
to make a cscGetEntries call to the driver to see if the numbers in
the CLUT are indeed the colors supplied with cscSetEntries
transformed by the gamma table supplied with cscSetGamma. We've run
this test on a number of video drivers, including the
.Display_ATImach64_3DR3 version 1.0.1f8, the .Display_Rage128 in my
Ti PowerBook, and the built-in Apple driver in the 7500/8500
PowerMacs. All except the Radeon 7000 faithfully return the correct
values to cscGetEntries. This is a rigorous test. We load random
numbers into both the gamma table and color spec table, and correctly
predict the transformed values that we read with cscGetEntries.

The Radeon 7000 driver fails that test. This driver is
".Display_DualHead" version 1.0f26, installed by the "March 2002 ATI
Retail Update" which I got from the ATI web site. The driver makes
two errors. Firstly, it's off by one in indexing the gamma table, so
that colors of 0 and 1 are both mapped to the gamma value for color
0, and the n+1-th color is mapped to the n-th gamma. Secondly, though
only with the 10-bit gamma table, the output values from the gamma
table lookup are all 1 less than they should be, except that zero is
zero. Having all the DAC output values all off by 1 LSB is of little
practical consequence. However, the wrong indexing of the gamma
table, and the mapping of two different colors to the same gamma
entry creates potentially large artifacts, depending on how different
the successive entries in the gamma table are. In fact, users who
want to make full use of the 10-bit DACs will typically load an
identity (linear) color spec table and do their color programming
through the gamma table, so the driver's gamma-table indexing error
can result in full-scale errors.

I have a test program that runs in the MATLAB environment (with the
free Psychtoolbox extensions) that I'd be happy to share with you,
but I'm guessing that ATI engineers don't use MATLAB. I hope my
description above will be immediately clear to an engineer familiar
with the driver code, and thus may suffice.

If it would help, I could also try to create a standalone
application, in C, to demonstrate the fault, but I'd rather avoid
that effort unless you think it would really help.

Again, this may be a bug report, but I and my colleagues are thrilled
by ATI's bold step in providing 10-bit-DAC support.

Sincerely yours,

Denis Pelli
Professor of Psychology and Neural Science at NYU
http://psych.nyu.edu/pelli/
Apple Developer
http://psychtoolbox.org/

212-614-8772 home
212-998-8338 work
denis.pelli@...

p.s.
Some of your engineers might know the TimeVideo program that has been
widely used over the past decade to check out Mac graphics drivers
before shipping. I wrote it. It's still free.
http://vision.nyu.edu/VideoToolbox/Download.html

p.p.s.
I'm using Mac OS 9.2.2, Radeon 7000 for Mac, March 2002 ATI Retail
Update, on a beige PowerMac G3. A complete ATI "problem report"
accompanies this letter.

=== ATI SOFTWARE ====================

ATI 3D Accelerator Version: 6.3.5
ATI Desktop Video Version: Not Installed.
ATI Desktop Video Extension Version: Not Installed.
ATI Displays Control Panel Version: 2.8.4
ATI Extension Version: 2.8.4
ATI Graphics Accelerator Version: 5.5.1
ATI Mac2TV Monitor Version: 2.8.3
ATI Mac2TV Startup Version: Not Installed.
ATI MPEG Accelerator Version: Not Installed.
ATI Offscreen Mem Manager Version: Not Installed.
ATI Radeon 3D Accelerator: 6.3.5
ATI Rage 128 3D Accelerator 6.3.5
ATI Resource Manager 3.0.5
ATI Shared Library Not Installed.
ATI Sound Catalyst Version: Not Installed.
ATI Tuner Version: Not Installed.
ATI Video Accelerator Version: 4.8.5
ATI Video Digitizer Version: 4.3.7
ATI Video Memory Manager Version: Not Installed.
ATI YUV Accelerator Version: Not Installed.
OpenGLRendererATI Version: 1.2.8