Security/Tracking protection

Description

Tracking Protection is a new platform-level technology that blocks HTTP loads at the network level. It is based on the Safe Browsing technology that powers our phishing and malware protection.

This feature was part of the Polaris initiative.

Prefs

• browser.safebrowsing.provider.mozilla.lists: list of tables coming from the Mozilla shavar service
• browser.safebrowsing.provider.mozilla.updateURL: server endpoint for downloading list updates
• browser.safebrowsing.provider.mozilla.gethashURL: server endpoint for completions
• browser.safebrowsing.provider.mozilla.lastupdatetime: timestamp (in ms) of when the last list update happened.
• browser.safebrowsing.provider.mozilla.nextupdatetime: timestamp (in ms) of when the list should next be downloaded.
• privacy.trackingprotection.annotate_channels: flag network channels loading resources on the tracking list (see how that information can be used)
• privacy.trackingprotection.enabled: to enable TP globally
• privacy.trackingprotection.lower_network_priority: lower the priority of channels loading tracking resources
• privacy.trackingprotection.pbmode.enabled: to enable TP in Private Browsing mode (not needed if the global pref is enabled)
• privacy.trackingprotection.ui.enabled: show a checkbox to toggle privacy.trackingprotection.enabled in the Preferences (Nightly only)
• privacy.trackingprotection.introCount
• privacy.trackingprotection.introURL: URL that kicks off the UI tour (target of the "See how this works" button in about:privatebrowsing)
• urlclassifier.disallow_completions: list of tables for which we never call gethash
• urlclassifier.trackingTable: list of tables to use when looking for trackers (they need to be named *-track-*)
• urlclassifier.trackingWhitelistTable: list of tables to use when checking whether or not a tracker is part of the same entity as the page (they need to be named *-trackwhite-*)

Engineering

• Tracking Bug
• Client implementation
• Server implementation
• Telemetry dashboard
• itisatrap.org test page
• Sandboxing trackers (as an alternative to blocking)

Code walkthrough

The classification for tracking protection, separate from the full Safe Browsing classification, is kicked off in nsHttpChannel::BeginConnectContinue() and goes like this:

1. we asynchronously check the blacklist
2. if there's a match, we then check the entity whitelist
3. if it doesn't match the whitelist, we treat it as a tracker
4. we either cancel the channel (for full tracking protection) or set a tracking flag (for tracking annotations only)

Note that only eligible resources are run through the classifier:

• must not be a first-party or top-level load
• addons can request permission to load tracking resources
• we check the permission manager for any manual user overrides (and an in-memory list in Private Browsing)

Tracking annotations

Tracking annotations are used in four different places:

• lower the priority of tracker loads in necko
• enable "tailing" in necko, which delays:
  • async js scripts
  • dynamically added scripts (js/css) and images (static/dynamic)
  • XHR/fetch()/added tags from trackers
  • favicons and <rel icon> resources
• the network predictor has a flag to disable prefetching of tracking resources
• the script loader set a tracking flag which gets propagated to the document to keep track of tracking scripts. There are three users of nsIDocument::IsTrackingScript():
  • the Timeout Manager has an mIsTracking flag to distinguish tracking timeouts from regular ones and enforce a minimum timeout value (dom.min_tracking_timeout_value) for timeouts from a tracking script
  • Fetch uses it to set a flag to honor tailing and lowering networking priority when fetch() is called from a tracking script
  • XHR also uses it for the same tailing and lowering the priority features when the XHR comes from a tracking script

Lists

• Current blacklists (Firefox 50 and later):
  • Base lists:
    • base-track-digest256: non-DNT-compliant trackers
    • baseeff-track-digest256: DNT-compliant trackers (EFF definition)
    • basew3c-track-digest256: DNT-compliant trackers (W3C definition)
    • Upstream source
    • Our copy (i.e. what we ship to clients in binary form)
    • Submit feedback and track changes
    • Excludes the Content category.
  • Content lists:
    • content-track-digest256: non-DNT-compliant content trackers
    • contenteff-track-digest256: DNT-compliant content trackers (EFF definition)
    • contentw3c-track-digest256: DNT-compliant content trackers (W3C definition)
    • Same upstream source as the base list.
    • Only includes the Content category.
• Legacy blacklists (Firefox 42 to 49):
  • Blacklist (mozstd-track-digest256)
    • Same as the union of all of the base lists.
  • "Strict" blacklist (mozfull-track-digest256)
    • Same as the union of all of the base and content lists.
• Entity whitelist (mozstd-trackwhite-digest256)
  • Upstream source
  • Our copy (i.e. what we ship to clients in binary form)
  • Implemented in bug 1141352
• List creation script
  • The script which generates all of the tracking protection lists in binary format.
  • Prod/stage list configuration for the script
• The lists are stored in these files:
  • ~/.cache/mozilla/firefox/XXXX/safebrowsing/mozstd-track* on Linux
  • ~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/mozstd-track* on Mac
  • C:\Users\XXXX\AppData\Local\mozilla\firefox\profiles\XXXX\safebrowsing\mozstd-track* on Windows

QA

• Bugzilla:
  • Firefox::Tracking Protection for UI and general feature requests/bugs
  • Toolkit::Safe Browsing for list updates and the actual blocking in necko
  • Cloud Services::Server: Shavar for server-side bugs
  • Breakage bugs
  • Bug triage

• Test pages
  • Blacklist and whitelist using hardcoded values (start here)
  • Standard blacklist
  • Strict blacklist (includes the Content category)
  • Category-specific lists (used by Focus for example)
    • Advertising
    • Analytics
    • Social
    • Disconnect (entries distributed into the above categories)
• Test URLS
  • itisatrap.org and trackertest.org: blocked by test-track-simple
  • itisatrap.com: blocked by base-track-digest256 (and included in the Disconnect list)
  • itisatrap.org/?resource=itisatracker.org: whitelisted in test-trackwhite-simple
• Test plan for Fx42
• Shavar test plan (includes end-to-end tests)
• Script to dump the contents of mozpub-track-digest256* files

To turn on debugging output, export the following environment variable:

MOZ_LOG_FILE=/tmp/trackingprotection.log
MOZ_LOG="UrlClassifierDbService:5,nsChannelClassifier:5"

To produce the "digest256" hash that sbdbdump -v will contain for example.com:

echo -n "example.com/" | sha256sum 
7fc983ea552f7c8d153fc308d621eb4f52e84aa63ecccf3a735698a11a2a4a8d

Telemetry

Alerts are sent to safebrowsing-telemetry@mozilla.org.

• FENNEC_TRACKING_PROTECTION_STATE: value of the TP setting (0 = disabled, 1 = enabled only in Private Browsing, 2 = fully enabled)
• TRACKING_PROTECTION_ENABLED: true if a session has privacy.trackingprotection.enabled turned on
• TRACKING_PROTECTION_EVENTS: 0 = security state changed, 1 = user clicked the button to disable TP on the current page, 2 = user clicked to re-enable TP on the page
• TRACKING_PROTECTION_PBM_DISABLED: true if a session has privacy.trackingprotection.pbmode.enabled turned off
• TRACKING_PROTECTION_SHIELD: for each pageload, 0 if the shield is not shown (no trackers), 1 if the shield is crossed-out (page is whitelisted) and 2 if the shield is shown normally (trackers blocked)

Notes:

• no telemetry pings are sent while in Private Browsing mode
• we only have telemetry when TP is enabled in the session

Documentation

• SUMO
  • Desktop
  • Android
• MDN page
• Presentation on Air Mozilla
• Announcement blog post
• Paper quantifying the privacy and performance impact of Tracking Protection
• Overview of how TP works in Firefox