From my experience I'm not sure if that machine has the resources to run a Win 7 VM with what would be considered good performance. I had no problem running XP SP3 VMs with 1 vCPU and 640M to 1GB of RAM. I was kind of forced to go to Win 7 for work and the minimum I seem to get good performance with is 2 vCPUs and 2 GB of RAM (2.5 is better and that is what I typically run).
Your problem is that your CPU is 2 core and giving the guest more than 1 will likely result in your host starving a bit for CPU which in turn will affect the performance of everything on the machine, include Fusion and the VM.
You may be able to get a little more performance by storing the VM on an external hard drive, but it would have to be a 7200 RPM drive with at least FW800 interface. A faster setup would be better, but that should be reasonable.
When did you start trying to run Win 7? Is the noticeable slowdown related to that or can you definitely notice it when upgrading Fusion to a new major version? I have been upgrading Fusion since 1.1.x, usually waiting for a x.0.1 or x.0.2 before making the jump (especially when it corresponded to a major Mac OS update). In my experience I have not noticed a slowdown in any of those upgrades. In fact the jump from Leopard to Snow Leopard and Fusion 3 (at least I think that was the first release after Snow Leopard) was a decent jump, most likely from Snow Leopard improvements.
Hope this gives you some ideas.