Another thing about computers is you need to do a cleanup once in a while. All of the cookies, temp inernet, prefetch files need to go. Also it helps to clean up the registery once in a while.
Go here
CCleaner - Download and download CCLEANER. Set it up and run it. After you run it, do a complete computer shut down and then power up your computer again. This could help. CCLEANER is the one that I use. There are more out there and a more literate computer user that me can direct you to other software that they like more.
As far as re-booting your router, Just unplug the power for 30-60 seconds and then plug it back in. As Ron said, let it reboot itself and then see how it works.
Hope this helps,