OK I've been watching the traffic to and from the server and client and it seems like lag originates from to much cross traffic i.e traffic waiting for confirmation and traffic pending that along with bandwidth seems to cause lag.

I don't have enough details really to that a guess i'll keep looking into it.