Security/Tracking protection

Description

Tracking Protection is a new platform-level technology that blocks HTTP loads at the network level. It is based on the Safe Browsing technology that powers our phishing and malware protection.

This feature was part of the Polaris initiative.

Prefs

• browser.contentblocking.enabled: master switch for all content blocking features (includes tracking protection, but excludes tracking annotations)
• browser.safebrowsing.debug: show debugging info from the JavaScript list update code on the command line as long as browser.dom.window.dump.enabled is also enabled
• browser.safebrowsing.provider.mozilla.lists: list of tables coming from the Mozilla shavar service
• browser.safebrowsing.provider.mozilla.updateURL: server endpoint for downloading list updates
• browser.safebrowsing.provider.mozilla.gethashURL: server endpoint for completions
• browser.safebrowsing.provider.mozilla.lastupdatetime: timestamp (in ms) of when the last list update happened.
• browser.safebrowsing.provider.mozilla.nextupdatetime: timestamp (in ms) of when the list should next be downloaded.
• privacy.trackingprotection.annotate_channels: flag network channels loading resources on the tracking list (see how that information can be used)
• privacy.trackingprotection.enabled: to enable TP globally
• privacy.trackingprotection.lower_network_priority: lower the priority of channels loading tracking resources
• privacy.trackingprotection.pbmode.enabled: to enable TP in Private Browsing mode (not needed if the global pref is enabled)
• privacy.trackingprotection.introCount
• privacy.trackingprotection.introURL: URL that kicks off the UI tour (target of the "See how this works" button in about:privatebrowsing)
• urlclassifier.disallow_completions: list of tables for which we never call gethash
• list of tables to use when looking for trackers (they need to be named *-track-*):
  • urlclassifier.trackingAnnotationTable: for tracking annotations
  • urlclassifier.trackingTable: for tracking protection
• list of tables to use when checking whether or not a tracker is part of the same entity as the page (they need to be named *-trackwhite-*):
  • urlclassifier.trackingAnnotationWhitelistTable: for tracking annotations
  • urlclassifier.trackingWhitelistTable: for tracking protection

Engineering

• Tracking Bug
• Client implementation
• Server implementation
• itisatrap.org test page
• Sandboxing trackers (as an alternative to blocking)

Code walkthrough

The classification for tracking protection, separate from the full Safe Browsing classification, is kicked off in nsHttpChannel::BeginConnectContinue() and goes like this:

1. we asynchronously check the blacklist
2. if there's a match, we then check the entity whitelist
3. if it doesn't match the whitelist, we treat it as a tracker
4. we either cancel the channel (for full tracking protection) or set a tracking flag (for tracking annotations only)

Note that only eligible resources are run through the classifier:

• must not be a first-party or top-level load
• addons can request permission to load tracking resources
• we check the permission manager for any manual user overrides (and an in-memory list in Private Browsing)

Tracking annotations

Tracking annotations are used in a few different places:

• devtools label requests from tracking domain as such
• control center indicates the presence of trackers based on annotations
• lower the priority of tracker loads in necko
• enable throttling of these requests in necko (flag added in bug 1360580)
• enable "tailing" in necko, which delays:
  • async js scripts
  • dynamically added scripts (js/css) and images (static/dynamic)
  • XHR/fetch()/added tags from trackers
  • favicons and <rel icon> resources
• the network predictor has a flag to disable prefetching of tracking resources
• cancel slow trackers in necko ("FastBlock")
• the script loader set a tracking flag which gets propagated to the document to keep track of tracking scripts. There are three users of nsIDocument::IsTrackingScript():
  • the Timeout Manager has an mIsTracking flag to distinguish tracking timeouts from regular ones and enforce a minimum timeout value (dom.min_tracking_timeout_value) for timeouts from a tracking script
  • Fetch uses it to set a flag to honor tailing and lowering networking priority when fetch() is called from a tracking script
  • XHR also uses it for the same tailing and lowering the priority features when the XHR comes from a tracking script

Tests

In addition to the Safe Browsing tests, here are all of the tests which are relevant to tracking protection:

./mach test browser/base/content/test/trackingUI/
./mach test netwerk/test/unit/test_trackingProtection_annotateChannels.js
./mach test netwerk/test/unit_ipc/test_trackingProtection_annotateChannels_wrap1.js
./mach test netwerk/test/unit_ipc/test_trackingProtection_annotateChannels_wrap2.js

Lists

• Current blacklists (Firefox 50 and later):
  • Base lists:
    • base-track-digest256: non-DNT-compliant trackers
    • baseeff-track-digest256: DNT-compliant trackers (EFF definition)
    • basew3c-track-digest256: DNT-compliant trackers (W3C definition)
    • Upstream source
    • Our copy (i.e. what we ship to clients in binary form)
    • Submit feedback and track changes
    • Excludes the Content category.
  • Content lists:
    • content-track-digest256: non-DNT-compliant content trackers
    • contenteff-track-digest256: DNT-compliant content trackers (EFF definition)
    • contentw3c-track-digest256: DNT-compliant content trackers (W3C definition)
    • Same upstream source as the base list.
    • Only includes the Content category.
  • Category-specific lists (currently only used by Focus for Android):
    • ads-track-digest256: trackers in the Advertising category
    • analytics-track-digest256: trackers in the Analytics category
    • social-track-digest256: trackers in the Social category
  • Fingerprinting lists:
    • base-fingerprinting-track-digest256: domains in both the Fingerprinting category AND in one of the tracking categories (Advertising, Analytics, Social, or Content)
    • content-fingerprinting-track-digest256L domains in the Fingerprinting category that are NOT in one of the tracking categories
  • Cryptomining lists:
    • base-cryptomining-track-digest256: domains in the Cryptomining category
    • content-cryptomining-track-digest256: placeholder list, currently empty. Intended to include cryptomining domains that we don't want to block by default (for some reason).
• Legacy blacklists (Firefox 42 to 49):
  • Blacklist (mozstd-track-digest256)
    • Same as the union of all of the base lists.
  • "Strict" blacklist (mozfull-track-digest256)
    • Same as the union of all of the base and content lists.
• Entity whitelist (mozstd-trackwhite-digest256)
  • source list (i.e. what we ship to clients in binary form)
  • Implemented in bug 1141352
• List creation script
  • The script which generates all of the tracking protection lists in binary format.
  • Prod/stage list configuration for the script
• The lists are stored in these files:
  • ~/.cache/mozilla/firefox/XXXX/safebrowsing/mozstd-track* on Linux
  • ~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/mozstd-track* on Mac
  • C:\Users\XXXX\AppData\Local\mozilla\firefox\profiles\XXXX\safebrowsing\mozstd-track* on Windows

QA

• Bugzilla:
  • Firefox::Tracking Protection for UI and general feature requests/bugs
  • Toolkit::Safe Browsing for list updates and the actual blocking in necko
  • Cloud Services::Server: Shavar for server-side bugs
  • Breakage bugs
  • Bug triage

• Test pages
  • Blacklist and whitelist using hardcoded values (start here)
  • Standard blacklist
  • Strict blacklist (includes the Content category)
  • Category-specific lists (used by Focus for example)
    • Advertising
    • Analytics
    • Social
    • Content (same link as "strict blacklist" above)
    • Disconnect (entries distributed into the above categories)
• Test URLS
  • itisatrap.org and trackertest.org: blocked by test-track-simple
  • itisatrap.com: blocked by base-track-digest256 (and included in the Disconnect list)
  • itisatrap.org/?resource=itisatracker.org: whitelisted in test-trackwhite-simple
  • *.dummytracker.org: a test domain added to all lists using the pattern <list_name>.dummytracker.org
    • For example, base-track-digest256.dummytracker.org is on the base-track-digest256 list and base-fingerprinting-track-digest256.dummytracker.org is on the base-fingerprinting-track-digest256 list.
    • Github repo for test page.
• Test plan for Fx42
• Shavar test plan (includes end-to-end tests)
• Script to dump the contents of mozpub-track-digest256* files
• Tools and utility files for analyzing Tracking Protection

To turn on debugging output, export the following environment variable:

MOZ_LOG_FILE=/tmp/trackingprotection.log
MOZ_LOG="UrlClassifierDbService:5,nsChannelClassifier:5"

To produce the "digest256" hash that sbdbdump -v will contain for example.com:

echo -n "example.com/" | sha256sum 
7fc983ea552f7c8d153fc308d621eb4f52e84aa63ecccf3a735698a11a2a4a8d

Telemetry

Alerts are sent to safebrowsing-telemetry@mozilla.org.

• FENNEC_TRACKING_PROTECTION_STATE: value of the TP setting (0 = disabled, 1 = enabled only in Private Browsing, 2 = fully enabled)
• TRACKING_PROTECTION_ENABLED: true if a session has privacy.trackingprotection.enabled turned on
• TRACKING_PROTECTION_EVENTS: 0 = security state changed, 1 = user clicked the button to disable TP on the current page, 2 = user clicked to re-enable TP on the page
• TRACKING_PROTECTION_PBM_DISABLED: true if a session has privacy.trackingprotection.pbmode.enabled turned off
• TRACKING_PROTECTION_SHIELD: for each pageload, 0 if the shield is not shown (no trackers), 1 if the shield is crossed-out (page is whitelisted) and 2 if the shield is shown normally (trackers blocked)

Notes:

• no telemetry pings are sent while in Private Browsing mode
• we only have telemetry when TP is enabled in the session

Documentation

• SUMO
  • Desktop
  • Android
• MDN page
• Presentation on Air Mozilla
• Announcement blog post
• Paper quantifying the privacy and performance impact of Tracking Protection
• Overview of how TP works in Firefox