Could I swap my 03-04 bracket assembly for a softer 04.5-06 bracket assembly and get the same clutch feel with no problems?

 

No clue if that'll work... I believe it wasn't just the bracket assembly... (Can someone expert confirm this?)

before you do anything else, check if your car even has the helper spring (google or search this forum. They have thread with pictures on how to remove them). I drove my friend's 03 Z before (I have 05), and it wasn't stiff to the point of discomfort as you describe... But I can imagine it'd be pain in traffic...

 

The pedal bracket assembly isn't where the difference lies. As I understand it, the earlier 03-04's just have some different clutch components. I know specifically that the pivot ball and release fork are different from looking at factory diagrams. When I replaced the clutch and flywheel on my 04.5, I had to make sure to get the correct one, as the "earlier model" one would not have fit.

Any number of things could cause a heavier pedal. Binding in the MC, a pinch or kink in the clutch line, a problem at the pressure plate (or an aftermarket pressure plate w/ higher clamping force), etc. If it's not a deal breaker for you, just live with it for now. If it's being caused by a failing part, that part will go sooner or later and you'll replace it and fix the issue. If you can't live with is, start testing/disassembling/replacing as needed each part in the system. I'd suggest starting at the pedal and working toward the trans... easy and relatively cheap stuff first.