A couple of factors can inhibit the quality of the capture. I have the AIW 9800 Pro and first off have NO problems with any games, including Doom 3 with this card. My rig is a 3.2 GHZ P4 with 1GB of ram...and no slow downs, pauses or anything.
As for capture...I use MMC 9.03. I have had no capture problems even when I used version 8X of MMC. My processor is powerful enough. I capture in mp2 and go straight to Sonic MyDVD to encode and burn. My videos (Hi 8 and VHS) are all decent quality. Again...the old adage of what goes in comes out...actually applies. My Hi8 captures are better than the VHS...not because of the AIW...but because of the natural resolution of each format. When I convert and burn to DVD...I am quite pleased with the quality of what I see. The idea of looping through firewire is NOT foreign. What you are seeing is a Time Base Shift, which causes wavy lines, etc. in the capture. There are things like Time Base Correctors http://www.bhphotovideo.com/bnh/control ... 891&is=REG
that you can use...or do what you already do TREBOR...which is use your firewire port of your Sony Digital 8 to do the time base correction for you. This issue and more have been discussed at www.rage3d.com
All in all...the AIW 9800 Pro is still a pretty good card if you are using AGP. Currently, the most updated card that ATI makes is the x600 AIW for PCI Express...and even they are hard to find.
Again...I capture 740 x 480 Mp2 straight from the get go with my AIW 9800 Pro and get decent DVD's in the process. I have converted about 30 of my home movies to DVD already with probably another 30 to go...and have no real complaints...Oftentimes...the quality is also effected by the VHS player that I use to convert...if I use my old camera...comes out like garbage. I use a fairly new VHS deck and keep it clean...so far so good.