Skip to content

ab-testing-in-dcr.md

Latest commit

 

History

History
72 lines (54 loc) · 3.94 KB

ab-testing-in-dcr.md

File metadata and controls

72 lines (54 loc) · 3.94 KB

AB Testing in DCR

Note

Setting up a client-side A/B test using the A/B Testing Library. The library docs explain the integration and the API.

Quick Start

  1. Create a switch in Frontend
  2. Ensure that you create an A/B test on Frontend using the A/B test API.
  3. Add your test to concurrent tests on Frontend.
  4. Copy the JS file into DCR (and update to TS types) in experiments/tests
  5. Add it to the test array in src/experiments/ab-tests.ts
  6. Force the A/B test (ignoring canRun of A/B test and variant) with the URL opt-in http://local...#ab-YourTest=yourVariant
  7. Set a GU_mvt_id or GU_mvt_id_local cookie with the MVT ID that matches the test Audience and Audience Offset (Use this calculator)
  8. Check the network tab for the Ophan request abTestRegister has your test and variant

Gotchas

  • Your ABTest Switch name is hyphenated, lower case and starts with ab-. The JS AB test ID is in PascalCase. These must match up: ab-my-cool-ab-test (serverside test switch name) === MyCoolAbTest (JS AB test ID).
  • Your ABTest Switch has a sell by date and your abTest has an expiry date. Matching them up avoids confusion.

Use in Components

// Within the components
import { useAB } from '../lib/useAB';

// Example usage of AB Tests
// Used in the e2e tests as smoke test of the AB tests framework integration
const ABTestAPI = useAB()?.api;

// We can check if a user is in a variant, returns a boolean
// ABTestTest being an ab test that was passed in via the ab test array
const abTestDataAttr =
	(ABTestAPI?.isUserInVariant('AbTestTest', 'control') &&
		'ab-test-control') ||
	(ABTestAPI?.isUserInVariant('AbTestTest', 'variant') &&
		'ab-test-variant') ||
	'ab-test-not-in-test';

// We can get the variant straight from a check for
// whether the test is runnable
const runnableTest = ABTestAPI?.runnableTest(abTestTest);
const variantFromRunnable =
	(runnableTest && runnableTest.variantToRun.id) || 'not-runnable';

<div
	data-ab-user-in-variant={abTestDataAttr}
	data-ab-runnable-test={variantFromRunnable}
>
	AB Test
</div>;

Server-side A/B tests

In order to set up a server-side test in DCR, follow steps 1-4 outlined in the frontend documentation.

On the live website, Fastly automatically assigns users to buckets. You can force yourself into a test on your local machine by following these steps:

  1. Ensure you are running frontend locally and your server-side experiment is enabled in the dashboard.
  2. Use the Header Hacker extension to change the HTTP headers as described in the frontend documentation. Please note that this is the only way to opt-in locally. If testing in the CODE environment, use the /opt/in/ link.

You can verify that you have been correctly assigned to the variant by appending .json?dcr to the end of an article link (e.g. http://localhost:9000/world/2021/jan/01/your-article.json?dcr. This will return the document data in JSON format. Your A/B test will be within the config object in camel case, as follows:

"abTests": {
	"yourAbTestVariant": "variant"
}

You can access server-side abTests within DCR wherever the CAPI object is used (CAPIArticle.config.abTests).