# Conclude an A/B test and close it down

Parlowen · Product-management work sample
All company, customer and data in this exercise are original synthetic material.

## Situation and outcome
Parlowen is a fictional language-learning app, free with a Parlowen Plus subscription, on iOS, Android and web in the US, UK, Canada and Australia. EXP-214 tested an annual-first paywall with a 7-day free trial (variant) against the monthly-first paywall with no trial (control). Enrollment ran 7 Sep-4 Oct in four weekly cohorts. You took over as the Monetization PM on 12 Oct (R057). The VP Growth wants the decision record and results for the growth review on Thursday 22 October (R055). Any rollout is a separate staged-release plan that she approves (R010).

Your outcome is a decision record, a short results report and a closedown request. The decision record gives the criteria the test is judged by, what each showed with its record IDs, the verdict, who decides what, and a plain line on what it is not. The report tells the growth review the result in plain words. The closedown request is a request you write, not something you carry out: no one expects you to switch anything off or delete anything. It names what must end or be removed, who should do it and by when. A record that says plainly what the evidence cannot yet show, with a complete closedown request, is a valid result. Your record does not approve a rollout, decide refunds or remediation, give a legal opinion or change the test's rules.

The period is 2026-08-24 through 2026-10-19 inclusive; the snapshot is Tuesday 20 October. Work with 57 records (plans, platform extracts, operations logs, support records and team messages) and an AI first draft of the readout. The draft is a draft, not an authority. Use the supplied record IDs and source context; do not contact the fictional teams or seek private data or live queries.

## Your role
You are joining the team as Product Manager.

Own the verdict under R004 with the data analyst and engineering lead, the results report and the closedown request. You may end allocation under a stopping rule with the engineering lead (R010) and must route breaches to counsel. You may not approve a rollout, decide remediation or refunds, or change the plan's criteria.

## About the team
A consumer subscription team concluding a paywall test for the growth review.

An experimentation agent drafts the readout and decision record from the dashboard, the current plan document and the growth channel; analysts write queries with AI; a person checks every figure against its source before the decision review. Agents cannot change allocation.

The PM records the verdict with the data analyst and engineering lead; counsel decides remediation for any breach of its conditions; the VP Growth approves any rollout through a separate staged-release plan.

## Work to complete
1. Frame the question: what the growth review must decide on 22 October and what you propose. Read sources/coverage-and-access.md before you make any claim about what the records cover.
2. Find the criteria the test is to be judged by, and where they come from. For each criterion say what the records show, which population and cohorts that figure covers, and when it can be read.
3. Count from the source rows. Say which cohorts and users a number covers, count a repeat or a copy once, and say which figures are not yet readable.
4. Audit the AI first draft against the original records. Keep, correct or reject each row and say why. Keep evidence that points the other way visible, and tell apart a measured figure, a person's claim and a gap.
5. Treat harm separately from frequency. Name the owner who can confirm or decide each item without deciding for them.
6. Record the verdict and everything that bears on trusting it. Write the results report for its audience, and a closedown request (a request to write, not an action to take) with owners and dates.
7. Apply the correction note (U001) in your download like the other sources. Carry each justified change through every file, and keep the original reading of any record it changes visible next to the revised one. The revision log is your own account of how you worked.
8. Hand off clearly: verdict, what is not decided, who decides, and owner questions. Say that the record is not rollout approval.

## Time
You have 90 minutes of work time. The clock starts when you press Start.

After that comes a 10-minute break, then a written interview: up to eight typed questions about your own files, for about 15 minutes. If you submit early, your break starts straight away.

## Mid-task correction
There is no mid-task correction in this take-home. Everything you need is in the files you download at Start, including the correction note **updates/U001-replies-and-correction-v1.md**. Read it with the other sources, apply it where it changes your work, and cite it as U001.

## Tools and help
Use any tools and help you like: AI tools and agents, search, your own notes, friends or colleagues. You don't need a paid tool, an API key or an account.

Nothing you do in your own tools is watched. Only your files and your interview answers are reviewed. When you submit, you can say which tools or help you used. That is optional, and leaving it empty is never held against you.

## Submission
Hand in one ZIP with these files at its top level. Start from the copies in the templates folder.
- README.md: list every file you hand in and say how you used the correction note (U001).
- decision-record.md: at most 900 words. The criteria table, the verdict, validity and deviations, what the record is not, the deciders and next step, and the closedown request.
- results-report.md: at most 400 words, for the growth review audience.
- evidence.csv: already has one row for every record, R001–R057, with the record ID and cohort ID filled in. Do not retype them or add or remove rows. For each row, fill in finding (the finding the record bears on most; name your own findings), disposition (supports, contrary, context, excluded or unverified are suggested and free text is accepted) and a rationale of one sentence. If the correction note changes a record, revise its row in place, set evidence_version to U001, and keep the earlier reading visible in its rationale or the revision log.
- revision-log.md: at most 500 words. Which draft rows you kept, changed or rejected, then what the correction note changed. No private prompts or unrelated files.
- questions.md: at most 8 open questions, each with one suggested owner and the decision that depends on it.

Keep every file within these limits: the reviewer reads all of it.

Words are counted in the prose of each document you write; headings, tables, the evidence file and the README do not count.

Use plain text files only: .md, .txt, .csv or .json. Put your work in the files themselves; a link to a document somewhere else does not count. Your ZIP can be up to 5 MB, or 20 MB unzipped, with at most 39 files of up to 2 MB each. Programs, macros, password-protected ZIPs and ZIPs inside your ZIP are not accepted.

Upload your ZIP on your take-home page and wait for **Received and checked**. It normally appears within a few seconds of the upload finishing. If it has not appeared after a minute, upload the ZIP again; if it still does not appear, use **Help and support**. You can replace your ZIP until you submit. Submitting is final and starts your break.

If time runs out before you submit, the last ZIP marked **Received and checked** is handed in for you. If none was received, the team is told that you need support.

If something goes wrong, use **Help and support** on your take-home page. A technical problem is never counted against you. Writing to us does not pause the clock.

## What happens next
The written interview covers these parts of the job:
- Checks sources and counts
- Weighs risk and uncertainty
- Checks and repairs AI work
- Leaves a usable decision record
- Explains their reasoning
- Reads the test by its plan

An AI then drafts a review from your files and your answers. Each point quotes what it rests on. A person at the company reads the review and makes any decision. No one is hired, rejected or ranked automatically.
