You could try following - if you have a laptop with 1Gbit.
Try to install the veeam on your laptop and then copy something ( ca. 10GB) - you will see then the performance - it should be at least 20 MB/sec.
If so then your server (not esx host) have a problem.
Another thing is the console vmnic - all these is running over it - but maybe the vms backup not - does it run on 1Gb?
I would check the vmkernel log for any warnings.
Reg
Christian