This is not the end of reverse engineering for radeons [LWN.net]

This is not the end of reverse engineering for radeons

Posted Feb 25, 2008 12:54 UTC (Mon) by darktjm (subscriber, #19598)
Parent article: Radeon R5xx 3D programming guide released

This information is a great start.  At least some questions I saw in the r300 source have been
answered here.

Is this it, though?  There are some missing bits.  Some are obvious (ch. 6 & 7, vertex &
fragment programs, seem to provide no information on actual binary instruction format -
correct me if I'm wrong), and some are sneaky (things marked as "reserved", which are marked
with actual useful functionality in the reverse engineered driver).  Some may just not be
there - it's hard to say without spending more time on it than I did.

I suppose they removed all information they considered patent-encumbered.  A quick scan
through the recently released Intel documentation shows some missing keywords that indicate
they may have done the same thing (but I didn't really read those documents yet).  This means
that free drivers can only ever be as good as non-free drivers by reverse engineering them.
Even then, the full functionality of the card may never be used, since the driver writer
doesn't need to actually use everything all the time (in particular, some features may only be
used on platforms that are harder to reverse engineer), and information such as what is in ch.
9, driver notes, is hard to guess by observation.

(Log in to post comments)

This is not the end of reverse engineering for radeons

Posted Feb 25, 2008 15:32 UTC (Mon) by drag (subscriber, #31333) [Link]

The AMD folks are interested in preserving DRM protections and also certain optimizations for
workstation-class hardware.

They are suppose to be providing "Tcore" software that they developed internally for driver
development and debugging purposes. It's going to be a source sanitized version, but it's
essentially what they use to do driver development internally. This is suppose to be available
somewhere, right now. But I am not sure about it. (also there is a chance that they may be
interested in a 'hybrid' model (were the open source driver and fglrx driver share code, I
expect) for driver development were they open source portions of the Fglrx driver. But they
still want to keep certain performance optimizations hidden and preserve DRM requirements so
they will never have it fully open. And the are suppose to be releasing microcode for tcore
and the one from the latest fglrx driver.)

So I figure what your talking about is being hidden to protect DRM or it's going to be
addressed with the Tcore source release, or with the microcode.

This is not the end of reverse engineering for radeons

Posted Feb 25, 2008 19:02 UTC (Mon) by tialaramex (subscriber, #21167) [Link]

It would be pretty silly to have such in-depth discussion of the shader hardware and then omit
mentioning which opcodes are which, wouldn't it?

So it won't surprise anyone that in fact the information is present and you just missed it.
For vertex programs the opcodes (since that's what you mentioned specifically) are listed from
the bottom of page 91 onwards. For fragment programs the "missing" binary instruction format
is described in chapter 10 from page 203 onwards on a register-by-register basis although it
might be covered in chapter 7 as well.

This appears to be much better than the reverse engineered information I'd seen when I last
wondered about trying to fix bugs/ limitations of the r300 driver on my M24. I am certainly
more tempted to try now than I was say, six months ago.

Do you have any examples of useful material that's marked "reserved" in this document? The
word is used quite a few times, but I grew bored before finding any such examples. Most of
what I saw was obviously spare bit fields (there are still 8 bits in a byte and if you can
only think of something useful to do with 7 of them, the remaining bit is reserved) or fields
which might grow in future (reserved for future expansion). There were also quite a lot of
things which were "preserved" and I must thank the Evince authors for not providing a "whole
words only" search function :)

This is not the end of reverse engineering for radeons

Posted Feb 26, 2008 1:35 UTC (Tue) by darktjm (subscriber, #19598) [Link]

Yes, it would be pretty silly. In fact, I suspect(ed) oversight rather than malice. On the
other hand, after I couldn't find the instruction format for vertex shader programs, I didn't
bother searching harder for the format for fragment shaders, so I'm sorry I missed that. I
did think to look for registers for the vertex program instruction format. I still don't see
the instruction format for vertex shaders (op codes aren't enough), but it doesn't really
matter, since it's in the reverse engineered r300_reg.h.

The point is, there is stuff missing, and some of it is not of the "there's a blank here where
there should be something" variety, like with the "see the diagram below" statement w/ no
diagram, or what I thought was a lack of instruction format. As to the "reserved" stuff, the
most obvious and useful things I see missing are the S3TC texture formats (0x44C0, values
15-17). Too bad 3dfx's unencumbered "compression" format never took off. I also looked
specifically for most of the registers marked unknown in the header file, and could not find
some of them, or anything close to them, in the register list (but they wouldn't be in the
header if the closed source driver hadn't written to them).

I believe I already conceded that the information is useful. To be exact, it's probably the
best non-NDA programming information I've seen for a standalone 3D chipset. I just don't like
it when things are missing, especially if the preamble of the document doesn't even mention
that there are things intentionally missing. I also don't like it that everyone seems to have
the impression that AMD/ATI is giving us complete documentation.

And the point of my subject line is not invalidated by my mistakes in researching the
document. There are known things missing, and there are very likely unknown things missing as
well (you don't have to mark it as reserved if you don't list it at all).

Maybe I'm just overly cynical because I was burned by 3dfx when they used flimsy excuses to
delay releasing promised Voodoo4/5 docs (important stuff, like SLI - I could never get it
working based on just the register info) until they could sell their secrets to nVidia.

This is not the end of reverse engineering for radeons

Posted Feb 26, 2008 16:33 UTC (Tue) by AJWM (subscriber, #15888) [Link]

Vendors also use "reserved" for bitfields that may do X in stepping A of the chip, Y in
stepping B, and nothing at all in stepping C -- all while their own drivers actually do
something with that bitfield (perhaps surrounded by "if (stepping == A)" like statements).
Just because the original design intention was that programmers always use some other bitfield
for that particular purpose doesn't mean that some harried developer working from engineers
notes didn't happen to use a register that, while not intended for his desired purpose,
happened to work that way.  Reverse-engineering that driver would turn up the incorrect usage.

Oh, wait, all vendor drivers are bug-free, right?

(Not saying this is what happened -- I have no way of knowing -- but I have seen similar stuff
before, and even been there myself, telling customers using a certain API in effect to "do as
we say, not as we do".  Not for any malicious or secretive reason, just because we learned
stuff as we developed the code and some of the older code doesn't do it the best way.  Also
remembering from my machine-language programming days back on the 6502 where you could do some
cool stuff with undocumented op-codes -- until they came out with a slightly different version
of the chip that didn't do the same things with those undocumented ops.)

This is not the end of reverse engineering for radeons

Posted Feb 28, 2008 5:44 UTC (Thu) by agd5f (guest, #50412) [Link]

The vertex instruction format was an oversight.  I'm already working on getting that
information out.   We didn't have an official R5xx 3D programming guide, so I put this one
together from a variety of internal sources.  There may be things I've missed and we will
release updated information when needed.  Plus, the document isn't the end all and be all, if
you need more information on anything, just ask!  In most cases we can get the information
released.