For some time now I have had problems both at work and home with very slow RDP connections making it almost impossible to work with some servers (it was not affecting all servers just "some") and today I tried to find a solution for the problem and found it.

It turns out that all you have to do is run this command as administrator and your will see a very big speed difference,

netsh interface tcp set global autotuninglevel=disabled

I have basically no understanding of what the command dose but it seems to work for me and everything else still seems to work so I guess its not that dangerous.

I found this solution here, Remote Desktop slow problem solved