Skip to content

Right verdict, wrong reason

Threat intel pushed a real phish to a raw score of 120 by judging Google Cloud Storage's reputation instead of the link, and the same flaw later turned a classmate's hijacked-account scam from 35 into 90.

Date

This is the second piece about PhishTriage, a command-line phishing analyzer whose every score traces to a rule in a YAML file. The first one is about running my own inboxes through it with the network off. This one is about what happened when I turned threat intel on. As with the rest of the project, I directed and reviewed the calibration and Claude Code wrote the code, including the fix described here.

The 120

The real phish in the corpus that the rules catch cleanly is a “MyChart Medicare kit” lure. It has a spoofed display name, an SPF pass for a seven-label random bounce domain that doesn’t align with From, no DKIM, and a landing page hosted on storage.googleapis.com. Offline it scores 65, LIKELY PHISH, and every point is about the email itself.

With live URLhaus and VirusTotal keys it scored a raw 120 out of 100. The extra 55 points came from two lines:

  • +40, URLhaus. The host storage.googleapis.com is listed with 2,242 malware URLs.
  • +15, VirusTotal. One vendor flags the domain.

The URL in the email was listed nowhere. URLhaus had never seen it and neither had VirusTotal. The verdict was right, and more than a third of the reasoning behind it was about other people’s files.

Whose reputation is it

Google Cloud Storage hosts a vast amount of legitimate content, and some fraction of what’s on it is malware. A host-level listing is true, and it tells you nothing about the one bucket this email links to. The same goes for Dropbox, Firebase, pages.dev and every other free hosting or form service phishers borrow. Their reputation describes the other tenants.

It matters most when the verdict is right, because that’s when nobody looks. A rule that adds 55 points to anything linking to a Google bucket will convict a shared class syllabus the day someone forwards one from an unfamiliar address.

The fix

PhishTriage already kept a list of file-hosting and form services in rules/filehosts.txt, used for a 5-point “link to a file-hosting service” rule. Domains on that list now also carry a shared_hosting flag, and the reputation rules ignore host-level results on anything carrying it. A hit on the exact URL still counts, because that is evidence about this email.

- id: urlhaus_hit
  description: URL or host is listed on URLhaus
  category: enrichment
  weight: 40
  rationale: URLhaus is a curated feed of active malware distribution URLs; a hit means someone already confirmed it is bad. Host-level hits on shared platforms (storage.googleapis.com etc.) are ignored; the specific URL still counts.
  when: {enrichment: {source: urlhaus, status: malicious, unless_flag: shared_hosting}}

Both VirusTotal rules got the same unless_flag. The MyChart phish went back to 65, with the bucket still shown in the report as file_hosting and shared_hosting, and its URLhaus and VirusTotal results still visible, just not scored.

Terminal report for the MyChart lure with enrichment on. The indicators table flags storage.googleapis.com as shared hosting and shows its URLhaus listing of 2,242 malware URLs and one VirusTotal detection, but the why-this-verdict panel scores only the display-name spoof, misaligned Return-Path, missing DKIM and DMARC, file-hosting link, urgency wording and deep subdomain, for LIKELY PHISH 65 out of 100.

Then it happened again

Four of the phish in the corpus came from classmates’ university accounts that had been taken over. The account owners are victims, and the samples replace their names with pseudonyms. Offline, those emails score 25 to 40: the mail never leaves the university’s Microsoft 365 tenant, so there are no authentication results and no external hop to judge, and wording alone is capped at 15 points by design.

One of them, an “Office Administrative Job Opportunity” job scam, has a single link, through shorturl.at. Offline it scores 35. With enrichment on it scored 90, MALICIOUS:

  • +40, URLhaus. shorturl.at is listed with 8 malware URLs.
  • +15, VirusTotal. One vendor out of 91 flags it.
  • The shortened link itself: not listed on URLhaus, never seen by VirusTotal.

Terminal report for a job scam sent from a compromised university account, sender pseudonymised as Student A. Authentication is missing, the hop chain stays inside Microsoft’s servers, and the only link goes through shorturl.at. The verdict panel shows +40 for a URLhaus host listing and +15 for one VirusTotal detection on shorturl.at, plus DKIM missing, no authentication headers, job-scam wording and the shortener itself, for MALICIOUS 90 out of 100.

It’s the same mistake with a different host. A URL shortener is shared infrastructure in the purest sense: its reputation is the average of everything anyone has ever shortened. This time the error made the tool look better, because it turned a documented miss into a catch, and that’s exactly why I don’t count it as one.

The benign side shows the same flaw. An ISC2 webinar invitation scores 25 offline, because the visible link text says isc2.org while every link goes through Salesforce’s click-tracker at cl.s12.exct.net. Online it scores 40, because one VirusTotal vendor flags the tracker domain. Every organisation that sends marketing mail through that platform inherits that detection.

Terminal report for a real ISC2 webinar email. SPF, DKIM and DMARC all pass and align. The verdict panel shows +20 for link text that says isc2.org while the href goes to cl.s12.exct.net, +15 for one VirusTotal detection on that click-tracker domain, and +5 for urgency wording, for SUSPICIOUS 40 out of 100.

Still open

I haven’t fixed the shortener and click-tracker cases yet, and I’d rather publish them as open than tune them quietly. The next steps are clear:

  • Treat shared infrastructure as shared everywhere. Extend the flag from file hosting to URL shorteners and email-marketing click-trackers, kept as a data file like the rest.
  • Judge the destination, not the shortener. Resolve a shortened link’s target through an API, without fetching it, and score the reputation of where it actually goes.

When that lands, the classmate scam goes back to 35. That’s the honest score for what the tool can see.

A verdict that’s right for the wrong reason is a false positive waiting to happen. The reputation lookup was working correctly, and the mistake was in what it was allowed to be evidence of.