diff --git a/README.md b/README.md index 8494c6f..84cd0be 100644 --- a/README.md +++ b/README.md @@ -241,6 +241,63 @@ the checker is updated in one pull request: The dpplint image does not change for this; the manifest is applied like any other change. +## Consent banner and conversion + +OwnYourData advertises the DPP Linter with Google Ads. With the visitor's +consent, the start page measures whether a visit that started with an ad leads +to a check. Without configuration the feature is off: no banner, no script, no +"Cookie settings" link and no request to Google. + +| Variable | Value | +|---|---| +| `GOOGLE_ADS_ID` | Conversion ID of the Google Ads account, `AW-` followed by digits. Empty or malformed: off. | +| `GOOGLE_ADS_LABEL_LINTER_CHECK` | Conversion label of a completed check (the part of `send_to` after the slash). Without a valid label the banner still works, but nothing is measured. | + +- With `GOOGLE_ADS_ID` set, the start page embeds `/consent.js` + (`app/javascript/consent.js`, adapted from the DPP Service) and shows + "Cookie settings" in the footer. Without it `/consent.js` is an empty script. +- Before the visitor decides, nothing is loaded from Google. The banner offers + "Decline" and "Accept" in the same style; the choice is kept in + `localStorage` (`oyd-consent`) for twelve months, and "Cookie settings" + opens the banner again. Declining after accepting deletes the `_gcl_` and + `_gac_` cookies on the host and its parent domains and reloads the page + without the tag. +- After "Accept", Google Consent Mode v2 is set (everything denied by default, + then `ad_storage` and `ad_user_data` granted; `ad_personalization` and + `analytics_storage` stay denied), ad personalisation signals are switched off + and `gtag.js` is loaded for the ID. The tag is not configured for the ID (no + `gtag("config")`): no page views, no Google Analytics, no remarketing, no + enhanced conversions. +- A completed check (the API answered with a result, whatever the result) sends + one conversion. A check that cannot be run sends none. +- No user data goes to Google. The conversion reports the address of the start + page with its ad click identifier (`gclid`, `gbraid`, `wbraid`) but without + the product identifier, and only the origin of the referrer + (`app/javascript/click_ids.js`); the result is not sent either. The start + page keeps the click identifier in its address when it adds the product + identifier, so that the conversion can be attributed to the ad without a + configured tag and without help from the server. +- The API documentation loads no web fonts from Google + (`app/views/rswag/ui/home/index.html.erb`, the template of rswag-ui without + them). + +The privacy policy is that of the DPP Service: +. + +`test/integration/consent_test.rb` checks the server side. +`test/browser/consent_check.mjs` checks a running instance in a browser +(Node.js and Playwright); requests to Google are blocked, so no conversion +reaches Google: + +``` +node test/browser/consent_check.mjs https://dpplint.ownyourdata.eu --no-id +node test/browser/consent_check.mjs http://localhost:3000 --screenshots tmp/consent +``` + +The first form is for an instance without `GOOGLE_ADS_ID`. With +`--inspect-tag` the tag itself is loaded from Google (everything else stays +blocked) and the check also reads the conversion request it would send. + ## Tests ``` diff --git a/app/controllers/consent_controller.rb b/app/controllers/consent_controller.rb new file mode 100644 index 0000000..47351a0 --- /dev/null +++ b/app/controllers/consent_controller.rb @@ -0,0 +1,23 @@ +# GET /consent.js: the consent banner and the Google Ads conversion tag of the +# start page (README "Consent banner and conversion"). The start page embeds it +# only when GoogleAds is enabled; without a conversion ID it is an empty +# script that does nothing. +class ConsentController < ActionController::Base + SCRIPT = Rails.root.join("app/javascript/consent.js").read.freeze + + # A script meant to be embedded by the start page; there is no session to + # protect. + skip_forgery_protection + + def show + config = GoogleAds.client_config + body = if config + "(function (config) {\n#{SCRIPT}\n})(#{ERB::Util.json_escape(config.to_json)});\n" + else + "/* Conversion measurement is not configured on this instance. */\n" + end + response.set_header("Cache-Control", "public, max-age=300") + response.set_header("X-Content-Type-Options", "nosniff") + render plain: body, content_type: "text/javascript" + end +end diff --git a/app/controllers/home_controller.rb b/app/controllers/home_controller.rb index 87c2530..fb7b339 100644 --- a/app/controllers/home_controller.rb +++ b/app/controllers/home_controller.rb @@ -3,11 +3,16 @@ class HomeController < ActionController::Base # Runs the check for the page; included in the page, tested on its own with # node (test/javascript). CHECK_CLIENT = Rails.root.join("app/javascript/check_client.js").read.freeze + # Keeps the ad click identifiers in the address of the page (click_ids.js); + # included in the page, tested on its own with node. + CLICK_IDS = Rails.root.join("app/javascript/click_ids.js").read.freeze def show @product_id = params[:productId].to_s @criteria_ref = Rails.configuration.x.dpplint.criteria_ref @check_timeout = Rails.configuration.x.dpplint.check_timeout.round @check_client = CHECK_CLIENT + @click_ids = CLICK_IDS + @google_ads = GoogleAds.enabled? end end diff --git a/app/javascript/click_ids.js b/app/javascript/click_ids.js new file mode 100644 index 0000000..31a5663 --- /dev/null +++ b/app/javascript/click_ids.js @@ -0,0 +1,52 @@ +// Ad click identifiers in the address of the start page. A Google Ads +// conversion is attributed to an ad only when it is sent from the page whose +// address carries the click identifier of the ad (the tag is not configured +// and keeps no identifier across pages). The start page keeps these +// identifiers in its address when it adds the product identifier after a +// check, and the conversion reports an address that has the click +// identifiers but never the product identifier. The start page includes this +// file as DppLintClickIds; consent.js uses it; test/javascript loads it with +// require(). +(function (root, factory) { + const api = factory(); + if (typeof module === "object" && module.exports) module.exports = api; + else root.DppLintClickIds = api; +})(typeof self !== "undefined" ? self : this, function () { + const CLICK_IDS = ["gclid", "gbraid", "wbraid"]; + + // The click identifiers of a query string, in the order of CLICK_IDS. + function clickIds(search) { + const params = new URLSearchParams(search || ""); + return CLICK_IDS.filter((name) => params.get(name)).map((name) => [name, params.get(name)]); + } + + const encode = (pairs) => pairs.map(([name, value]) => name + "=" + encodeURIComponent(value)).join("&"); + + // The query for the address bar after a check is started: the product + // identifier, followed by the click identifiers the address already has. + function queryWithProductId(search, productId) { + return "?" + encode([["productId", productId], ...clickIds(search)]); + } + + // The address reported with a conversion: origin and path of the page and + // its click identifiers. Nothing else of the query (the product identifier) + // and no fragment. + function conversionLocation(href) { + const url = new URL(href); + const query = encode(clickIds(url.search)); + return url.origin + url.pathname + (query ? "?" + query : ""); + } + + // The referrer reported with a conversion: only its origin, so that an + // address with a product identifier never leaves the page. + function conversionReferrer(referrer) { + if (!referrer) return ""; + try { + return new URL(referrer).origin + "/"; + } catch (e) { + return ""; + } + } + + return { CLICK_IDS, clickIds, queryWithProductId, conversionLocation, conversionReferrer }; +}); diff --git a/app/javascript/consent.js b/app/javascript/consent.js new file mode 100644 index 0000000..0ee35ce --- /dev/null +++ b/app/javascript/consent.js @@ -0,0 +1,207 @@ +// Consent banner and Google Ads conversion measurement of the start page +// (README "Consent banner and conversion"). +// +// Adapted from app/views/consent/consent.js of OwnYourData/dpp-service-public +// at commit 4f49703 (Apache License 2.0, like this repository). Changes: one +// conversion, sent by the start page when a check is completed, instead of +// conversions for links; the conversion reports the address of the page with +// its ad click identifiers but without the product identifier (click_ids.js), +// and only the origin of the referrer; banner text for the DPP Linter; the +// privacy policy is that of the DPP Service; colours from the page. +// +// ConsentController wraps this file in a function that receives `config` +// ({ id: "AW-…", conversions: { linter_check: "AW-…/label" } }). It is served +// only when a conversion ID is configured. +// +// Google Consent Mode v2, "basic": nothing is requested from Google before +// the visitor accepts. After acceptance only the Google Ads tag is loaded. It +// is not configured for the conversion ID (no config command), so it reports +// no page views and keeps no click identifier across pages; it sends +// something only when a conversion takes place, without ad personalisation. +// No analytics, no remarketing, no user data: neither the product identifier +// nor the result of a check is passed to Google. The choice is kept in +// localStorage for at most twelve months and can be changed at any time +// through an element with the attribute data-consent-settings (the +// "Cookie settings" link in the footer). +"use strict"; + +var STORAGE_KEY = "oyd-consent"; +var STORAGE_VERSION = 1; +var MAX_AGE_MS = 365 * 24 * 60 * 60 * 1000; +var GOOGLE_COOKIE = /^(_gcl_|_gac_)/; +var TAG_URL = "https://www.googletagmanager.com/gtag/js?id="; +var PRIVACY_URL = "https://dpp-service.ownyourdata.eu/privacy#conversion-tracking"; + +var tagLoaded = false; +var banner = null; + +function readChoice() { + try { + var raw = window.localStorage.getItem(STORAGE_KEY); + if (!raw) return null; + var stored = JSON.parse(raw); + if (!stored || stored.v !== STORAGE_VERSION || typeof stored.ads !== "boolean") return null; + var at = Date.parse(stored.at); + if (!(at > 0) || Date.now() - at > MAX_AGE_MS) return null; + return stored.ads; + } catch (e) { + return null; + } +} + +function storeChoice(ads) { + try { + window.localStorage.setItem(STORAGE_KEY, JSON.stringify({ v: STORAGE_VERSION, ads: ads, at: new Date().toISOString() })); + } catch (e) { + // Without storage the banner simply comes back on the next visit. + } +} + +function gtag() { + window.dataLayer.push(arguments); +} + +function loadTag() { + if (tagLoaded) return; + tagLoaded = true; + window.dataLayer = window.dataLayer || []; + window.gtag = gtag; + gtag("consent", "default", { + ad_storage: "denied", + ad_user_data: "denied", + ad_personalization: "denied", + analytics_storage: "denied" + }); + gtag("consent", "update", { ad_storage: "granted", ad_user_data: "granted" }); + gtag("set", { allow_ad_personalization_signals: false }); + gtag("js", new Date()); + var script = document.createElement("script"); + script.async = true; + script.src = TAG_URL + encodeURIComponent(config.id); + document.head.appendChild(script); +} + +// Sends the named conversion, only after consent and with a configured label. +// The address reported is that of the page with its click identifiers, never +// with the product identifier; the referrer only with its origin. +function convert(name) { + var target = config.conversions[name]; + if (!tagLoaded || !target) return false; + var ids = window.DppLintClickIds; + if (!ids) return false; + gtag("event", "conversion", { + send_to: target, + page_location: ids.conversionLocation(window.location.href), + page_referrer: ids.conversionReferrer(document.referrer), + transport_type: "beacon" + }); + return true; +} + +function deleteGoogleCookies() { + var labels = window.location.hostname.split("."); + document.cookie.split(";").forEach(function (pair) { + var name = pair.split("=")[0].trim(); + if (!GOOGLE_COOKIE.test(name)) return; + document.cookie = name + "=; Max-Age=0; path=/"; + for (var i = 0; i < labels.length - 1; i++) { + document.cookie = name + "=; Max-Age=0; path=/; domain=." + labels.slice(i).join("."); + } + }); +} + +function choose(ads) { + storeChoice(ads); + hideBanner(); + if (ads) { + loadTag(); + return; + } + deleteGoogleCookies(); + if (tagLoaded) { + gtag("consent", "update", { ad_storage: "denied", ad_user_data: "denied", ad_personalization: "denied" }); + window.location.reload(); + } +} + +var STYLE = [ + ".oyd-consent{position:fixed;left:0;right:0;bottom:0;z-index:2147483000;padding:0 12px 12px;pointer-events:none}", + ".oyd-consent[hidden]{display:none}", + ".oyd-consent .oc-box{pointer-events:auto;max-width:920px;margin:0 auto;background:var(--bg,#fff);color:var(--fg,#1d2330);", + "border:1px solid var(--line,#d9dde5);border-radius:12px;box-shadow:0 6px 24px rgba(0,0,0,.18);padding:18px 20px;", + "font:15px/1.5 system-ui,-apple-system,\"Segoe UI\",Roboto,\"Helvetica Neue\",Arial,sans-serif}", + ".oyd-consent h2{font-size:16px;margin:0 0 6px}", + ".oyd-consent p{margin:0 0 12px;color:var(--muted,#5b6475)}", + ".oyd-consent a{color:var(--accent,#2457a6)}", + ".oyd-consent .oc-actions{display:flex;gap:12px;flex-wrap:wrap}", + ".oyd-consent button{flex:1 1 160px;max-width:240px;padding:10px 18px;border-radius:8px;font-family:inherit;font-size:15px;font-weight:600;line-height:1.2;cursor:pointer;", + "border:1px solid var(--accent,#2457a6);background:var(--accent,#2457a6);color:var(--on-accent,#fff)}", + ".oyd-consent button:focus-visible{outline:2px solid var(--fg,#1d2330);outline-offset:2px}", + ".oyd-consent .oc-current{font-size:13px;margin:-4px 0 12px}" +].join(""); + +function hideBanner() { + if (banner) banner.hidden = true; +} + +function showBanner(focus) { + if (!banner) { + var style = document.createElement("style"); + style.textContent = STYLE; + document.head.appendChild(style); + banner = document.createElement("section"); + banner.className = "oyd-consent"; + banner.setAttribute("aria-labelledby", "oyd-consent-title"); + banner.innerHTML = + '
' + + '' + + "

OwnYourData advertises these free tools with Google Ads. With your consent, Google’s conversion tag tells us " + + "when a visit from one of our ads leads to a check in the DPP Linter. The product identifier you enter and the " + + "result of the check are not sent to Google. Google sets cookies and receives your IP address and browser " + + "details, possibly in the USA. No analytics, no remarketing. Without your consent nothing is loaded from Google. " + + "You can change your choice at any time under “Cookie settings” at the bottom of the page. " + + 'Privacy policy

' + + '' + + '
' + + '' + + '' + + "
"; + banner.addEventListener("click", function (event) { + var button = event.target.closest ? event.target.closest("button[data-choice]") : null; + if (button) choose(button.getAttribute("data-choice") === "accept"); + }); + document.body.appendChild(banner); + } + var current = banner.querySelector(".oc-current"); + var choice = readChoice(); + current.hidden = choice === null; + current.textContent = choice === true ? "Your current choice: accepted." : choice === false ? "Your current choice: declined." : ""; + banner.hidden = false; + if (focus) banner.querySelector("button").focus(); +} + +function init() { + var settings = document.querySelectorAll("[data-consent-settings]"); + for (var i = 0; i < settings.length; i++) { + settings[i].hidden = false; + settings[i].addEventListener("click", function (event) { + event.preventDefault(); + showBanner(true); + }); + } + + var choice = readChoice(); + if (choice === true) { + loadTag(); + } else if (choice === null) { + showBanner(false); + } +} + +window.oydConsent = { open: function () { showBanner(true); }, convert: convert }; + +if (document.readyState === "loading") { + document.addEventListener("DOMContentLoaded", init); +} else { + init(); +} diff --git a/app/services/google_ads.rb b/app/services/google_ads.rb new file mode 100644 index 0000000..31c7b70 --- /dev/null +++ b/app/services/google_ads.rb @@ -0,0 +1,47 @@ +# Google Ads conversion measurement on the start page (ConsentController, +# app/javascript/consent.js, README "Consent banner and conversion"). +# +# Everything comes from the environment. Without a valid GOOGLE_ADS_ID the +# feature is off: no consent banner, no script, no "Cookie settings" link and +# no request to Google. Without a valid label the banner still works, but no +# conversion is measured. +# +# GOOGLE_ADS_ID AW-, the conversion ID of the account +# GOOGLE_ADS_LABEL_LINTER_CHECK a check of a product identifier completed on the start page +module GoogleAds + ID_FORMAT = /\AAW-\d{6,15}\z/ + LABEL_FORMAT = /\A[A-Za-z0-9_-]{1,64}\z/ + + CONVERSIONS = { + "linter_check" => "GOOGLE_ADS_LABEL_LINTER_CHECK" + }.freeze + + module_function + + # The conversion ID, or nil when it is missing or malformed. A malformed + # value counts as missing: a typo must not load a tag nobody can account for. + def id + value = ENV["GOOGLE_ADS_ID"].to_s.strip + value if value.match?(ID_FORMAT) + end + + def enabled? = !id.nil? + + # send_to targets ("AW-…/label") by conversion name, only those with a valid + # label. + def conversions + return {} unless enabled? + + CONVERSIONS.each_with_object({}) do |(name, variable), targets| + label = ENV[variable].to_s.strip + targets[name] = "#{id}/#{label}" if label.match?(LABEL_FORMAT) + end + end + + # What the consent script needs, nil when the feature is off. + def client_config + return nil unless enabled? + + { "id" => id, "conversions" => conversions } + end +end diff --git a/app/views/home/show.html.erb b/app/views/home/show.html.erb index 183562c..c63f942 100644 --- a/app/views/home/show.html.erb +++ b/app/views/home/show.html.erb @@ -5,6 +5,9 @@ DPP Linter – Digital Product Passport checks +<% if @google_ads -%> + +<% end -%> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + diff --git a/config/routes.rb b/config/routes.rb index 7d240ef..2f944cb 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -10,6 +10,7 @@ end root "home#show" + get "consent.js", to: "consent#show", format: false, as: :consent_script get "version", to: "application#version" get "ready", to: "application#ready" get "up", to: "rails/health#show", as: :rails_health_check diff --git a/kubernetes/deployment.yaml b/kubernetes/deployment.yaml index 54b382f..3515899 100644 --- a/kubernetes/deployment.yaml +++ b/kubernetes/deployment.yaml @@ -26,7 +26,7 @@ spec: app: dpplint containers: - name: dpplint - image: oydeu/dpplint@sha256:0eecb124de5c44070067d978eb06e09fee3ea4835d53979094236bf2f61d262d + image: oydeu/dpplint@sha256:aa449cae9835fc0c03c29a4956c3de8d5b212eef29b521eecdcdf59b9b336e8f imagePullPolicy: IfNotPresent ports: - containerPort: 3000 @@ -41,6 +41,10 @@ spec: value: "16" - name: NU_CHECKER_URL value: http://127.0.0.1:8888/ + - name: GOOGLE_ADS_ID + value: "" + - name: GOOGLE_ADS_LABEL_LINTER_CHECK + value: VeXiCK3G_K4dEM_nrKoD resources: requests: cpu: 100m diff --git a/test/browser/consent_check.mjs b/test/browser/consent_check.mjs new file mode 100644 index 0000000..b5370b4 --- /dev/null +++ b/test/browser/consent_check.mjs @@ -0,0 +1,219 @@ +// Browser check of the consent banner and the conversion of the start page +// (README "Consent banner and conversion"), with Playwright. +// +// node test/browser/consent_check.mjs [--inspect-tag] [--screenshots ] +// node test/browser/consent_check.mjs --no-id +// +// The instance must have GOOGLE_ADS_ID and GOOGLE_ADS_LABEL_LINTER_CHECK set; +// with --no-id it must have no GOOGLE_ADS_ID, and the check only confirms that +// no page shows a banner or a cookie settings link or requests anything from +// Google, and that the footer links the privacy policy. +// Requests to Google are recorded and blocked, so no conversion reaches +// Google. With --inspect-tag the Google Ads tag itself (gtag.js) is loaded +// from Google, while every other request to Google is still blocked; the +// check then also reads the conversion request the tag would send. With +// --screenshots the banner is saved in light and dark at 375 and 1280 px. +// +// Checks, in a fresh browser profile: +// 1. Before any choice, the start page, a result page, a page whose check +// fails, the API documentation and an unknown path send no request to a +// Google domain, and the start page shows the banner with Decline and +// Accept alike. +// 2. After Decline: still no request to Google, also after a check and a +// reload. +// 3. "Cookie settings" opens the banner again; after Accept the tag is +// requested, with ad personalisation and analytics denied, and is not +// configured for the ID. +// 4. A completed check sends the conversion once, with the configured label, +// and the address it reports keeps the click identifier but not the +// product identifier; the address bar keeps the click identifier. A check +// that cannot be run (the API does not answer with a result) sends none; +// a check that ends with a result, also one saying the passport could not +// be retrieved, sends one. +// 5. Declining again reloads the page without the tag and deletes the _gcl_ +// cookies. +import { createRequire } from "node:module"; +import { mkdirSync } from "node:fs"; +import { join } from "node:path"; + +const require = createRequire(import.meta.url); +const { chromium } = require("playwright"); + +const args = process.argv.slice(2); +const shotsDir = args.includes("--screenshots") ? args[args.indexOf("--screenshots") + 1] : null; +const positional = args.filter((a, i) => !a.startsWith("--") && args[i - 1] !== "--screenshots"); +const base = (positional[0] || "http://localhost:3000").replace(/\/$/, ""); +const inspectTag = args.includes("--inspect-tag"); +const noId = args.includes("--no-id"); +const GOOGLE = /(^|\.)(google(-analytics|adservices|apis|syndication|tagmanager|usercontent)?|gstatic|doubleclick)\.[a-z.]+$/i; +const PRODUCT_ID = "https://dpp.oydapp.eu/01/09520123456788/21/000001"; +const UNREACHABLE = "https://dpplint-consent-check.invalid/01/1"; +// The check of this identifier is answered by the browser itself with HTTP 503, +// so that the page shows "The check could not be run". +const API_FAILURE = "https://dpplint-consent-check-api-failure.invalid/01/1"; +const CLICK_ID = "consent-check-gclid"; + +let failures = 0; +function check(ok, what) { + console.log(`${ok ? "PASS" : "FAIL"} ${what}`); + if (!ok) failures += 1; +} + +// Chromium refuses its sandbox when run as root (e.g. in a container). +const browser = await chromium.launch({ args: process.getuid && process.getuid() === 0 ? ["--no-sandbox"] : [] }); +const context = await browser.newContext(); +const google = []; +await context.route("**/*", (route) => { + const url = new URL(route.request().url()); + if (url.pathname.startsWith("/api/v1/validate/") && decodeURIComponent(url.pathname).includes("api-failure.invalid")) { + return route.fulfill({ status: 503, contentType: "application/json", body: JSON.stringify({ error: "not available (consent check)" }) }); + } + if (GOOGLE.test(url.hostname)) { + google.push(url.href); + if (inspectTag && url.hostname === "www.googletagmanager.com" && url.pathname === "/gtag/js") return route.continue(); + return route.abort(); + } + return route.continue(); +}); +const page = await context.newPage(); +const dataLayer = () => page.evaluate(() => (window.dataLayer || []).map((entry) => Array.from(entry))); +const conversions = async () => (await dataLayer()).filter((e) => e[0] === "event" && e[1] === "conversion").map((e) => e[2]); +const runCheck = async (productId) => { + await page.fill("#productId", productId); + await page.click("#run"); + await page.waitForFunction(() => document.querySelector(".summary") || !document.getElementById("failure").hidden, null, { timeout: 60000 }); +}; +const productIdIn = (text) => text.includes(PRODUCT_ID) || text.includes(encodeURIComponent(PRODUCT_ID)) || text.includes("dpp.oydapp.eu"); + +// 1. Nothing before a choice. +const pages = [ + ["/", "start page"], + ["/?productId=" + encodeURIComponent(PRODUCT_ID), "result page"], + ["/?productId=" + encodeURIComponent(UNREACHABLE), "result page of a passport that cannot be retrieved"], + ["/?productId=" + encodeURIComponent(API_FAILURE), "page whose check cannot be run"], + ["/api-docs/index.html", "API documentation"], + ["/no-such-page", "unknown path"] +]; +for (const [path, what] of pages) { + await page.goto(base + path, { waitUntil: "networkidle" }); + if (path.includes("productId")) { + await page.waitForFunction(() => document.querySelector(".summary") || !document.getElementById("failure").hidden, null, { timeout: 60000 }); + } + check(google.length === 0, `no request to Google before a choice on the ${what}`); +} +if (noId) { + await page.goto(base + "/", { waitUntil: "networkidle" }); + check((await page.locator(".oyd-consent").count()) === 0, "no banner without a conversion ID"); + check((await page.locator("[data-consent-settings]").count()) === 0, "no cookie settings without a conversion ID"); + check((await page.locator('script[src="/consent.js"]').count()) === 0, "the page does not embed /consent.js"); + check(await page.locator('footer a[href="https://dpp-service.ownyourdata.eu/privacy"]').isVisible(), "the footer links the privacy policy of the DPP Service"); + const script = await (await page.request.get(base + "/consent.js")).text(); + check(!/googletagmanager|gtag/.test(script), "/consent.js does nothing"); + await runCheck(PRODUCT_ID); + check(google.length === 0, "no request to Google, also after a check"); + await browser.close(); + console.log(failures === 0 ? "\nAll checks passed." : `\n${failures} check(s) failed.`); + process.exit(failures === 0 ? 0 : 1); +} +await page.goto(base + "/", { waitUntil: "networkidle" }); +const banner = page.locator(".oyd-consent"); +check(await banner.isVisible(), "the banner is shown before a choice"); +const styles = await page.$$eval(".oyd-consent button", (buttons) => + buttons.map((b) => { + const s = getComputedStyle(b); + return { text: b.textContent, look: [s.backgroundColor, s.color, s.fontSize, s.fontWeight, s.borderColor, s.padding].join("|"), width: b.offsetWidth, height: b.offsetHeight }; + }) +); +check(styles.map((s) => s.text).join(",") === "Decline,Accept", "the banner offers Decline and Accept"); +check(styles.length === 2 && styles[0].look === styles[1].look && styles[0].height === styles[1].height && Math.abs(styles[0].width - styles[1].width) <= 2, + "Decline and Accept look alike"); +check(await page.locator('.oyd-consent a[href="https://dpp-service.ownyourdata.eu/privacy#conversion-tracking"]').isVisible(), "the banner links the privacy policy"); +check(await page.locator("footer [data-consent-settings]").isVisible(), "the footer offers the cookie settings"); +check(await page.locator('footer a[href="https://dpp-service.ownyourdata.eu/privacy"]').isVisible(), "the footer links the privacy policy of the DPP Service"); + +if (shotsDir) { + mkdirSync(shotsDir, { recursive: true }); + for (const scheme of ["light", "dark"]) { + for (const width of [375, 1280]) { + const shot = await browser.newPage({ viewport: { width, height: 800 }, colorScheme: scheme }); + await shot.goto(base + "/", { waitUntil: "networkidle" }); + await shot.screenshot({ path: join(shotsDir, `banner-${scheme}-${width}.png`) }); + const overflow = await shot.evaluate(() => document.documentElement.scrollWidth > window.innerWidth); + check(!overflow, `no horizontal scrolling with the banner, ${scheme}, ${width} px`); + await shot.close(); + } + } +} + +// 2. Decline. +await page.click('.oyd-consent button[data-choice="decline"]'); +check(!(await banner.isVisible()), "the banner closes after Decline"); +await runCheck(PRODUCT_ID); +await page.reload({ waitUntil: "networkidle" }); +await page.waitForFunction(() => document.querySelector(".summary") || !document.getElementById("failure").hidden, null, { timeout: 60000 }); +check(!(await banner.isVisible()), "the banner stays closed after a reload"); +check(google.length === 0, "no request to Google after Decline, also after a check"); +check((await conversions()).length === 0, "no conversion after Decline"); + +// 3. Accept through the cookie settings, on an address with a click identifier. +await page.goto(base + "/?gclid=" + CLICK_ID, { waitUntil: "networkidle" }); +await page.click("footer [data-consent-settings]"); +check(await banner.isVisible(), "Cookie settings opens the banner again"); +check((await page.locator(".oyd-consent .oc-current").textContent()) === "Your current choice: declined.", "the banner names the current choice"); +await page.click('.oyd-consent button[data-choice="accept"]'); +await page.waitForTimeout(inspectTag ? 3000 : 1000); +const tag = google.filter((url) => url.startsWith("https://www.googletagmanager.com/gtag/js?id=AW-")); +check(tag.length === 1 && tag[0] === "https://www.googletagmanager.com/gtag/js?id=AW-894120911", + `only the tag for the ID is requested after Accept (${tag[0] || "none"})`); +const layer = await dataLayer(); +const consentDefault = layer.find((e) => e[0] === "consent" && e[1] === "default"); +const consentUpdate = layer.find((e) => e[0] === "consent" && e[1] === "update"); +const config = layer.find((e) => e[0] === "config"); +const set = layer.find((e) => e[0] === "set"); +check(consentDefault && Object.values(consentDefault[2]).every((v) => v === "denied"), "consent default: everything denied"); +check(consentUpdate && JSON.stringify(consentUpdate[2]) === JSON.stringify({ ad_storage: "granted", ad_user_data: "granted" }), + "consent update: only ad_storage and ad_user_data granted"); +check(!config, "the tag is not configured for the ID, so it reports no page views"); +check(set && set[1].allow_ad_personalization_signals === false, "ad personalisation signals are switched off"); +const beforeConversion = google.length; +check(google.length === tag.length, "nothing but the tag is requested from Google before a conversion"); + +// 4. Conversions. +await runCheck(PRODUCT_ID); +await page.waitForTimeout(inspectTag ? 4000 : 500); +const sent = await conversions(); +console.log(` conversion: ${JSON.stringify(sent)}`); +check(sent.length === 1 && sent[0].send_to === "AW-894120911/VeXiCK3G_K4dEM_nrKoD", "a completed check sends one conversion with the label"); +check(sent.length === 1 && new URL(sent[0].page_location).searchParams.get("gclid") === CLICK_ID, "the reported address keeps the click identifier"); +check(sent.length === 1 && !productIdIn(JSON.stringify(sent[0])), "the conversion carries neither the product identifier nor the result"); +const address = new URL(page.url()); +check(address.searchParams.get("gclid") === CLICK_ID && address.searchParams.get("productId") === PRODUCT_ID, + "the address bar keeps the click identifier next to the product identifier"); +if (inspectTag) { + const requests = google.slice(beforeConversion); + console.log(` requests of the tag after the check: ${requests.length}`); + check(requests.length > 0 && requests.every((url) => new URL(url).searchParams.get("label") === "VeXiCK3G_K4dEM_nrKoD"), + "the tag sends the conversion with the label (blocked here)"); + check(requests.length > 0 && requests.every((url) => !productIdIn(decodeURIComponent(url))), "no request of the tag contains the product identifier"); + check(requests.some((url) => new URL(url).searchParams.get("gclaw") === CLICK_ID), "the tag reads the click identifier for the attribution"); +} +await runCheck(API_FAILURE); +await page.waitForTimeout(500); +check(await page.locator("#failure").isVisible() && (await conversions()).length === 1, "a check that cannot be run sends no conversion"); +await runCheck(UNREACHABLE); +await page.waitForTimeout(500); +check((await conversions()).length === 2, "a completed check whose passport cannot be retrieved sends one more conversion"); + +// 5. Withdraw. +await page.evaluate(() => { document.cookie = "_gcl_au=test; path=/"; }); +const before = google.length; +await page.click("footer [data-consent-settings]"); +await Promise.all([page.waitForEvent("load"), page.click('.oyd-consent button[data-choice="decline"]')]); +await page.waitForLoadState("networkidle"); +await page.waitForFunction(() => document.querySelector(".summary") || !document.getElementById("failure").hidden, null, { timeout: 60000 }); +check(google.length === before, "after withdrawing, the reloaded page sends no request to Google"); +check(!(await page.evaluate(() => document.cookie)).includes("_gcl_"), "after withdrawing, the _gcl_ cookies are gone"); + +await browser.close(); +console.log(failures === 0 ? "\nAll checks passed." : `\n${failures} check(s) failed.`); +process.exit(failures === 0 ? 0 : 1); diff --git a/test/integration/api_docs_test.rb b/test/integration/api_docs_test.rb index e544712..6d796c4 100644 --- a/test/integration/api_docs_test.rb +++ b/test/integration/api_docs_test.rb @@ -13,4 +13,11 @@ class ApiDocsTest < ActionDispatch::IntegrationTest assert_includes response.body, "DPP Linter API v1" assert_not_includes response.body, "dpplint API" end + + test "the API documentation page loads nothing from Google, not even web fonts" do + get "/api-docs/index.html" + assert_response :success + assert_includes response.body, "swagger-ui-bundle.js" + assert_no_match(/googleapis|gstatic|googletagmanager|doubleclick/i, response.body) + end end diff --git a/test/integration/check_client_test.rb b/test/integration/check_client_test.rb index 595fa8d..91914ed 100644 --- a/test/integration/check_client_test.rb +++ b/test/integration/check_client_test.rb @@ -12,3 +12,14 @@ class CheckClientTest < ActiveSupport::TestCase assert_match(/^# pass [1-9]\d*$/, out) end end + +# The click identifiers of the start page (app/javascript/click_ids.js) have +# their own tests in JavaScript, run with node as part of the suite. +class ClickIdsTest < ActiveSupport::TestCase + test "the JavaScript tests of the click identifiers pass" do + out, status = Open3.capture2e("node", "--test", "--test-reporter=tap", Rails.root.join("test/javascript/click_ids.test.js").to_s) + assert status.success?, out + assert_match(/^# fail 0$/, out) + assert_match(/^# pass [1-9]\d*$/, out) + end +end diff --git a/test/integration/consent_test.rb b/test/integration/consent_test.rb new file mode 100644 index 0000000..39bdcb7 --- /dev/null +++ b/test/integration/consent_test.rb @@ -0,0 +1,94 @@ +require "test_helper" + +# Consent banner and Google Ads conversion (README "Consent banner and +# conversion"): without a conversion ID nothing of it is on the page; with an +# ID the page embeds /consent.js and the "Cookie settings" link, and still +# names no Google address itself. +class ConsentTest < ActionDispatch::IntegrationTest + GOOGLE = /googletagmanager|google-analytics|googleadservices|googlesyndication|doubleclick|gstatic|gtag/i + ID = "AW-894120911".freeze + LABEL = "VeXiCK3G_K4dEM_nrKoD".freeze + + def with_env(values) + previous = values.keys.to_h { |k| [k, ENV[k]] } + values.each { |k, v| v.nil? ? ENV.delete(k) : ENV[k] = v } + yield + ensure + previous.each { |k, v| v.nil? ? ENV.delete(k) : ENV[k] = v } + end + + def footer = response.body[%r{
.*
}m] + + test "without GOOGLE_ADS_ID: no script, no cookie settings, and /consent.js does nothing" do + with_env("GOOGLE_ADS_ID" => nil, "GOOGLE_ADS_LABEL_LINTER_CHECK" => LABEL) do + get "/" + assert_response :success + assert_not_includes response.body, %() + assert_includes footer, %(Privacy) + assert_includes footer, %() + assert_no_match GOOGLE, response.body, "the page itself loads nothing from Google" + assert_not_includes response.body, ID, "the ID is only in /consent.js" + end + end + + test "/consent.js carries the ID and the conversion, sets no config and passes no user data" do + with_env("GOOGLE_ADS_ID" => ID, "GOOGLE_ADS_LABEL_LINTER_CHECK" => LABEL) do + get "/consent.js" + assert_response :success + assert_equal "text/javascript", response.media_type + assert_equal "nosniff", response.headers["X-Content-Type-Options"] + body = response.body + assert_includes body, %({"id":"#{ID}","conversions":{"linter_check":"#{ID}/#{LABEL}"}}) + assert_no_match(/gtag\("config"/, body, "the tag is not configured for the ID") + assert_includes body, %(gtag("consent", "update", { ad_storage: "granted", ad_user_data: "granted" });) + assert_includes body, "ad_personalization: \"denied\"" + assert_includes body, "analytics_storage: \"denied\"" + assert_includes body, "allow_ad_personalization_signals: false" + assert_includes body, "page_location: ids.conversionLocation(window.location.href)" + assert_no_match(/(? ID, "GOOGLE_ADS_LABEL_LINTER_CHECK" => "not a label") do + get "/consent.js" + assert_includes response.body, %({"id":"#{ID}","conversions":{}}) + end + end + + test "the page sends the conversion once per completed check, through consent.js only" do + get "/" + script = response.body.scan(%r{}m).last.first + assert_equal 1, script.scan("oydConsent.convert(").size + assert_match(/render\(outcome\.data\);\s*\/\/[^\n]*\n\s*\/\/[^\n]*\n\s*if \(window\.oydConsent\) window\.oydConsent\.convert\("linter_check"\);/, script) + assert_includes script, "history.replaceState(null, \"\", DppLintClickIds.queryWithProductId(location.search, productId));" + assert_includes response.body, HomeController::CLICK_IDS + end +end diff --git a/test/integration/home_test.rb b/test/integration/home_test.rb index 5c19a97..ff6e515 100644 --- a/test/integration/home_test.rb +++ b/test/integration/home_test.rb @@ -38,7 +38,7 @@ class HomeTest < ActionDispatch::IntegrationTest assert body.index("") < body.index("