The new open-source tool scans public AWS S3 buckets for secrets

A new open-source ‘S3crets Scanner’ scanner allows researchers and red-teamers to search for ‘secrets’ mistakenly stored in publicly exposed or company’s Amazon AWS S3 storage buckets.

Amazon S3 (Simple Storage Service) is a cloud storage service commonly used by companies to store software, services, and data in containers known as buckets.

Unfortunately, companies sometimes fail to properly secure their S3 buckets and thus publicly expose stored data to the Internet. 

This type of misconfiguration has caused data breaches in the past, with threat actors gaining access to employee or customer details, backups, and other types of data.

In addition to application data, source code or configuration files in the S3 buckets can also contain ‘secrets,’ which are authentication keys, access tokens, and API keys.

If these secrets are improperly exposed and accessed by threat actors, they could allow them far greater access to other services or even the company’s corporate network.

Scanning S3 for secrets

During an exercise examining SEGA’s recent assets exposure, security researcher Eilon Harel discovered that no tools for scanning accidental data leaks exist, so he decided to create his own automated scanner and release it as an open-source tool on GitHub.

To help with the timely discovery of exposed secrets on public S3 buckets, Harel created a Python tool named “S3crets Scanner” that automatically performs the following actions:

• Use CSPM to get a list of public buckets
• List the bucket content via API queries
• Check for exposed textual files
• Download the relevant textual files
• Scan content for secrets
• Forward results to SIEM

The scanner tool will only list S3 buckets that have the following configurations set to ‘False,’ meaning that exposure was likely accidental:

• “BlockPublicAcls”
• “BlockPublicPolicy”
• “IgnorePublicAcls”
• “RestrictPublicBuckets”

Any buckets that were intended to be public are filtered out from the list before the textual files are downloaded for the “secrets scanning” step.

When scanning a bucket, the script will examine the content of text files using the Trufflehog3 tool, an improved Go-based version of the secrets scanner that can check for credentials and private keys on GitHub, GitLab, filesystems, and S3 buckets.

Trufflehog3 scans the files downloaded by S3crets using a set of custom rules designed by Harel, which target personally identifiable information (PII) exposure and internal access tokens.

When used periodically to scan an organization’s assets, the researcher believes that “S3crets Scanner” can help firms minimize the chances of data leaks or network breaches resulting from the exposure of secrets.

Finally, the tool can also be used for white-hat actions, like scanning publicly accessible buckets and notifying the owners of exposed secrets before bad actors find them.