Hello and welcome to the forum.
I'm not sure, what......
bypassing the router and connecting directly to an ethernet outlet
means exactly.
You take the Ethernet cable that you would generally plug into one of the switch ports on your router and plug that in directly to the port on your ISPs device?
Is that what you mean?
If so then your router is not forwarding DNS
Right click the start button, choose run, type
ncpa.cpl
hit enter
select then right click the Ethernet wired (not wireless as I understand the question) and choose properties.
Uncheck Internet Protocol Version 6 (TCP/IPv6)
Select / highlight Internet Protocol Version 4 (TCP/IPv4)
click the properties button
Near the bottom of the resultant windows select the radio button that says
Use the following DNS server addresses:
and enter
Preferred DNS server:
8.8.8.8
Alternate DNS server:
8.8.4.4
Click OK all the way out.
You might have to reboot, but I don't think so.