Effective reconnaissance with SpiderFoot

Hey, what's going on guys?

Bienvenue dans cette leçon dans laquelle on va voir comment mener une reconnaissance passive et active avec un outil qui s'appelle SpiderFoot.

Comme vous pouvez le voir à l'écran, SpiderFoot est en train d'effectuer un scan sur une cible.

But before showing you how to use SpiderFoot, I would like to introduce you to the different tools available on Kali Linux to carry out the reconnaissance phase.

If you go to the "Information Gathering" menu of Kali Linux, you will find a multitude of tools that allow you to do passive and active reconnaissance on a target network or system.

For example, in the first section "DNS Analysis", you will find tools like dnsenum, dnsrecon or Fierce, which allow you to do specific research on the DNS (Domain Name System) protocol, responsible for translating domain names into IP addresses.

You also have an "IDS/IPS Identification" section.

IDS (Intrusion Detection System) serve to detect attacks on a network or a system, while IPS (Intrusion Prevention System) are designed to block these attacks.

You will find in this section tools to identify if these protection systems or firewalls are protecting your target.

For example, WafW00f allows you to detect if a website uses a WAF (Web Application Firewall).

In the "Live Host Identification" section, tools like Masscan, fping or hping3 allow you to determine if a target or a server is active.

For example, you can try to ping a machine using the `ping` command.

However, on Windows, by default, the firewall blocks ICMP requests (pings).

Thus, if a Windows machine does not respond to a ping, it does not necessarily mean that it is offline.

Let's take an example by opening the command prompt (CMD) on my Windows host machine.

If I type `ipconfig`, I see the IP address of my Windows machine.

If I then open a terminal on Kali Linux (of which I can see the local IP address with `ifconfig`), and I try to ping my Windows machine (at the address `192.168.95.70` for example), the ping does not respond.

This is a default protection from Microsoft to prevent attackers from easily detecting active machines on the network.

This example clearly shows that if a machine does not respond to ping, it can still be active.

Let's try now with another Kali Linux virtual machine.

If I look at its network configuration and put it in "Bridged Adapter" mode instead of "NAT", it finds itself on the same physical network as my host Windows machine, which allows them to communicate directly.

If the virtual machine had remained in NAT mode, it would be isolated on a virtual private network managed by the hypervisor and could not communicate directly with the host on the same subnet.

By configuring the virtual machine in Bridged mode, we assign it an IP address from my local network.

If I then launch a ping from Kali Linux to this other Kali virtual machine, it responds normally because, unlike Windows, it allows ICMP requests by default.

This confirms that the hypervisor handles communication on the local network well.

In the "Network & Port Scanners" section, we find very well-known tools like Nmap and Masscan.

Nmap is the reference tool that we will study in detail in a future lesson.

Masscan is similar, but it is particularly famous for its speed when scanning very large ranges of IP addresses.

We then have the "OSINT Analysis" section.

OSINT (Open Source Intelligence) consists of collecting public information available on the Internet.

Tools like SpiderFoot allow automating this task.

SpiderFoot will query many public sources (search engines, databases, third-party APIs) to collect IP addresses, emails, subdomains, etc., and store all this data.

You will also find sections like "Route Analysis" to map the path of packets on the network (essential for network pentest), "SMB Analysis" with tools like enum4linux, nbtscan and smbmap (very useful because the SMB protocol in Windows environments often has flaws), or even "SMTP Analysis" and "SSL Analysis" to audit mail protocols or analyze the SSL/TLS certificates of a web server (for example to recommend that the client update an old, obsolete version of SSL/TLS).

The more information you collect about your target, the more options you will have for the exploitation phase and relevant recommendations for your report.

Let's now focus on SpiderFoot, which is installed by default on Kali Linux.

To use it, we can launch it via the command line or start its web interface.

To launch the SpiderFoot web server, open a terminal, switch to root with `sudo su`, then run the command `spiderfoot -l 192.168.95.70:80` (indicating the IP address of your attack machine and the desired port, for example port 80 or the default port 5001).

Then open the URL indicated in your browser.

You will arrive at the web interface of SpiderFoot, which I find very well designed and clean.

The terminal continues to run in the background to manage the processes.

To start a new scan, go to the "New Scan" tab, give a name to your scan (for example the name of the client "X-Corp") and enter the target (a domain name, an IP address or a network range like a `/24`).

Scans can take a lot of time depending on the size of the target.

For example, I launched a scan on my site `hackinggeek.com` and it has already been running for 20 minutes.

Be patient, because the larger the target, the longer the process.

SpiderFoot allows you to choose the types of information to collect.

You can choose to perform purely passive reconnaissance (the tool gathers public information without ever sending requests directly to the target or its servers), or to activate all modules (passive and active reconnaissance) to query the target system directly.

You can also select precisely the desired data or the modules to use.

SpiderFoot relies on many third-party services like Google Maps, host.io, Twilio for phone numbers, VirusTotal for security reputation, or even HackerOne for bug bounty.

If you click on the ongoing scan for `hackinggeek.com`, you will access the dashboard.

In the "Browse" tab, you can see the elements found (more than 200 in my case): SSL certificates, IPv6 addresses, country ranges, or even discovered subdomains.

The "Graph" tab offers a very interesting graphic visualization that shows the relationships between your main target and the discovered elements.

For example, SpiderFoot found subdomains like `school.hackinggeek.com` (an old subdomain today deleted), `cours.hackinggeek.com` (used to host my courses and my books), `www.hackinggeek.com`, or associated DNS records and IP addresses.

It also identified technical subdomains like `cpcontacts.hackinggeek.com` or `cpcalendars.hackinggeek.com`, an old domain `dojo.hackinggeek.com`, as well as the mail server `mail.stack`.

It even detected SPF configurations revealing the past use of the Mailkitchen email sending platform.

All of this information is valuable and can be integrated into your report.

The "Browse" tab lists these results as a table, while the graph visually illustrates the topology of the collected information.

SpiderFoot is therefore a fantastic tool to automate the reconnaissance phase.

You generally launch it in the background at the start of your pentest so that it works while you perform other tasks.

We'll meet in the next lesson!