How To Scan AWS's Entire IP Range to Recon SSL Certificates

Tue, May 12, 2020

Naffy (nnwakelam) shared in his recent nahamsec interview a creative approach to extend a target’s attack surface. Scan Amazon AWS’s entire IPv4 network space, and match the SSL certificates back to hostnames of your target.

Many companies and governments, rather than spinning up assets on their assigned IP blocks (ASNs), increasingly rely on cloud services like Amazon AWS and Microsoft Azure to host their production and development assets. With public cloud infrastructure fast becoming the norm, your reconnaissance will miss out on many assets just by scanning an organization’s private network ranges using tools like Nmap or Shodan.

After testing several different open source tools, combining masscan with tls-scan yielded the quickest and cleanest results. Both tools are written in C and optimized to be blazing fast. Masscan claims it can “scan the entire Internet in under 6 minutes, transmitting 10 million packets per second.” That’s sick.

Here’s how to hypothetically scan AWS’s IP range and SSL certificates — chaining together off-the-shelf tooling — to identify your target’s cloud assets.

Collect Amazon AWS’s IP Ranges

First, we begin by gathering all of AWS’s IP addresses. There are two ways to enumerate these. If we are interested in only AWS EC2 (cloud computing) infrastructure, Amazon provides an EC2 Reachability Test status page listing all the IP ranges in a table.

For the most complete data, Amazon also provides all of AWS IP address ranges in JSON format which makes for easier command-line parsing (group by region or service).

Filter Only SSL-Enabled IPs on AWS (masscan)

Once we have gathered all the AWS IPs, we need to filter the list down to only IPs that have SSL enabled, which by default would have port 443 open. Without this step, our SSL certificate checks in the next step will hang or take longer than necessary because we’ll attempt TLS handshakes with dead ports. For deeper scans later, we could expand fuzziness by probing alternate HTTPS ports such as 8443.

There are now 10,130,200 IP addresses in the EC2 range.

For the sake of the rest of this example, let’s target the DoD (U.S. Department of Defense) on AWS GovCloud to discover active hosts. This allows us to significantly narrow down the number of AWS IP addresses upfront — 458,754 versus 10M+ — since we can cherrypick the AWS GovCloud network blocks in regions us-gov-east-1 and us-gov-west-1. Also, we can now play within the scope of the DoD vulnerability disclosure program managed by HackerOne.

Note that only a fraction of these AWS GovCloud IPs are DoD-related since many federal and state-level assets are hosted on AWS GovCloud as well. So don’t get too trigger happy. These other IP addresses will be out-of-scope for further testing. If we find any vulnerabilities on DoD-specific assets, we’ll submit a report at HackerOne.

Optionally, before running masscan, since this list is in IP CIDR notation, it would be a good idea to count how many actual IPs we have ready for input. We can use nmap’s “host discovery” feature -sL with the “skip reverse DNS resolution” flag -n to quickly expand the list of CIDRs to individual IPs.

$ nmap -n -sL -iL aws_ips.in | wc -l

Next, we scan port 443 on all these IPs with masscan:

$ sudo masscan ‐p443 ––rate 100000 -iL aws_ips.in -oL masscan.out

Then clean up the masscan results by trimming fields with awk and sorting with sort -u:

$ cat masscan.out | awk {'print $4'} | awk NF | sort -u > ssl-alive.in

We now have a list of the IPs ready to provide us with their SSL certificates over port 443.

Scan SSL Certificates (tls-scan)

Tls-scan is capable of concurrently scanning thousands of hosts to collect their SSL certificates and outputs that data in json format.

Let’s run tls-scan on all the SSL-enabled IPs. We specify port 443 with a higher concurrency of 150 (I randomly chose that number, so adjust this to your system) and discard standard errors to /dev/null. After downloading the binary package for tls-scan, ca-bundle.crt is included, the location of which should be passed as the --cacert= option for certificate validation.

$ cat ssl-alive.in | tls-scan --port=443 --concurrency=150 \
--cacert=/home/user/tools/tls-scan/ca-bundle.crt \
2>/dev/null -o tls-scan.json

This tls-scan run grabs all the SSL certificates for the list of AWS IPs, so the next step of filtering by hostname is key.

Filter IPs For The Target’s Hostname (jq)

We can quickly filter the JSON output file with jq to match only our target hostname. In our example here, we want all IPs related to .mil, as reflected in test() function.

$ cat results.json | jq --slurp -r '.[] | select(.certificateChain[].subject | test("\\.mil\\W")) | .ip | @text' > tls-scan-filtered.txt

The syntax for jq can be confusing, so here’s a breakdown of the command above:

• --slurp combines all the JSON objects into a single array so we can range over them
• -r gets rid of double quotes as raw output
• select() based on certificateChain’s subject attribute and test() if it matches our regex for .mil, which we could replace with any other root domain for other targets such as tesla.com or airbnb,com
• For jq regex, we need to escape special characters like . and \; learn more about jq’s regex syntax here
• Grab the ip value of what’s filtered by the regex prior
• Pipe to @text for plaintext output instead of default JSON

Suggested Next Steps for Reconnaissance

Now that we have all the in-scope AWS IPs gathered from SSL certificates, we could do a more detailed scan using nmap for top open ports. This may enumerate other interesting, vulnerable services running on those IPs.

sudo nmap -T 4 -iL tls-scan-filtered.txt -Pn --script=http-title --top-ports 100 --open

Depending on the target, this methodology may yield a large number of results. Too large to manually sift through one by one. Screenshot programmatically using a tool like aquatone to organize and visually inspect the results at scale.

This methodology would work with Microsoft Azure IP ranges as well, another popular option for public cloud infrastructure. Similar to AWS GovCloud, US Government Cloud has dedicated IP ranges on Azure.

Alternative to Hunting SSL Certificates On Your Own

Sam Erb (erbbysam) has already done it for us and makes the data available for free through his BufferOver microservice. Sam developed a TLS scanner that runs weekly and can be queried through the q parameter at public endpoint tls.bufferover.run/dns.

Example for finding .mil subdomains and associated IPs:

curl 'https://tls.bufferover.run/dns?q=.mil' 2>/dev/null | jq .Results

Sam outlines his research and methodology at his DEF CON 27 talk titled “Hunting Servers and Certificates” (slides available on GitHub). Fascinating technical details on TLS.