Deleting the wiki page 'Phase 1 Network Setup' cannot be undone. Continue?
NOTE: A knowledge ip(6)tables and “dnsmasq” will be essential for success.
The first thing we need to do is capture some data with which to work with. Especially since I like to verify what I’m working with before I write a bunch of code based on false assumptions. And I like to manually perform the task I’m planning on computerizing so I have a clear idea of what I want to automate.
There are two sources of information I want to collect:
A brief note on DNS: web server can host any number of domain names on a single IP address, these are known as “virtual hosts”. In many cases there is no direct way to lookup an IP address and know what domain / host name the request was actually trying to hit. Often times a reverse lookup will return a name that is only useful to the hosting service. The HTTP protocol carries the desired target name, which is how the server knows which site to serve up. But with the proliferation of HTTPS and the fact that other protocols don’t carry the same information, its likely that colating DNS queries to addresses from connection data will give us the most useful information. A lot more can be said on this topic and a proxy server can change things considerably, but its far more intrusive.
A note about DNS over HTTP(S): All of the big name browsers supprt this now, at least I’ve heard rumar to that affect. For maximum visibilty into your network traffic you want to turn this off. Its probable that malware will start to use these DNS-over-HTTPS services to further hide their activity. So access to these services should be blocked to force use of standard DNS resolvers, more specifically the resolver we will setup to record requests.
I’m going to approach this from a generic Linux computer perspective first. This could be a personal notebook or workstation that you are attaching direct to the internet or through someone else’s network, like a hotspot at your local coffee shop. If you want to keep an eye on a whole network there are many options. Probably the simplest solution is a comodity PC or even SBC between the network and internet connection. I recently saw some inexpensive SBCs with multiple Ethernet connections that would make a nice firewall / router.
If you use a firewall appliance or wifi access point to connect your network to the internet its likely you won’t have access to the needed tools or settings, even though most all of them are running Linux. For those users blessed enough to be running appropriate models of Linksys or Netgear appliances (and maybe others) there is probably a custom firmware you can load. I use the Netgear equipment specifically because they support firmware customizations. I’ll cover my Netgear, running “Shibby Tomato”, later.
To setup iptables to log the appropriate information you need rules like these:
iptables -A FORWARD -m state --state NEW -j LOG --log-prefix "ACCEPT "
iptables -A INPUT -m state --state NEW -j LOG --log-prefix "ACCEPT "
iptables -A OUTPUT -m state --state NEW -j LOG --log-prefix "ACCEPT "
If you’re using IPv6 as well then you will need “ip6tables” equivalent commands:
ip6tables -A FORWARD -m state --state NEW -j LOG --log-prefix "ACCEPT "
ip6tables -A INPUT -m state --state NEW -j LOG --log-prefix "ACCEPT "
ip6tables -A OUTPUT -m state --state NEW -j LOG --log-prefix "ACCEPT "
How these rules get applied on your system is going to depend on how the system is used and how the firewall is currently configured. If your system uses a firewall aid like “shorewall” or similar you’ll have to figure that out on your own. I don’t use such tools since IMO they just complicate things. I prefer to write my rules in a standard shell script. The main thing I’m concerned with here is the placement of the rules. They need to come after applicable blocks ("-j DROP”, “-j REJECT”, ...) so we’re not looking at traffic we’ve already done away with and before a “-j ACCEPT” rule or we won’t get anything logged at all. Sometimes its easiest to put these rules along with an “ACCEPT” rule into a chain and point to it with the “-j” switch wherever you have a “-j ACCEPT” currently.
If the computer you’re setting up with firewall rules is not forwarding traffic for anyone else you don’t need to add the “FORWARD” lines above, but it won’t hurt to have them. If the machine is doing only forwarding for others on a net then you might not need the “INPUT” and “OUTPUT” rules above. But since I’m all about finding out what shouldn’t be happening they should be added to catch the unexpected.
The “--log-prefix” prepends the “ACCEPT " to the connection information as it gets written to the logs. This can be changed for your preference but then my tools will likely need to be altered with the new prefix. But how that will work is still somewhat up in the air.
To get “dnsmasq” logging DNS queries and their answers you need to add the log-queries
option to your “dnsmasq.conf”. There is also a “log-facility” option you can use to tailor where your syslogd handles dumping the output. Due to the large number of “sysloggers” out there and the many different ways to set them up that’s something you’ll have to figure out on your own. But as a spoiler: due to the version differences between my Netgear router and my Linux PC based router my logging setup is less than ideal so I have to grep ... | sort ...
my logs before processing.
With those two settings taken care of we now have connection and DNS data getting dumped to syslog and then it should go to disk somewhere, typically in the “/var/log” folder. Most of my systems are setup to log these sorts of messages into “/var/log/messages”. The ip(6)tables messages get logged as the “kernel” facility (kern) usually at a “warning” level but that last part can be changed in the ip(6)tables rule. See: man 8 iptables-extensions
for details of LOG target switches. dnsmasq
usually logs as facility “daemon”. But check man 8 dnsmasq
for details of the log-facility
setting.
Go to Phase 1 - Netgear Router Setup Recipe to see how I configured my WiFi router to contribute to the data pool .
Deleting the wiki page 'Phase 1 Network Setup' cannot be undone. Continue?