Experimentation · variant P&L · verdicts

Which variant
made more money.

Assign the split, join each variant to the payment data, and read the verdict: which arm earned more per acquired customer, and whether the difference is significant. Enter your own numbers below to see one.

your-company.optimiser.pro / experiments / try-a-verdict

Try a verdict · enter the counts from one of your tests

01 · Split
50 / 50
20,000 visitors each · sticky, server-side
Sample ratio ✓ · no mismatch
02 · Join
controlvariant
paid5.00%5.45%
€ / customer€18.90€21.10
€ / 1,000 visitors€945€1,150
03 · Verdict
+€205
Ship
per 1,000 visitors · lift +9.0%
p = 0.0432 · 98% chance variant leads

The variant earns more and the gap is significant. Ship it.

Computed from the numbers entered above. Nothing here is a customer result. In a workspace the payment figures come from the payments database.

The problem

Why teams stop testing

The usual choices are an enterprise contract, a homegrown setup, or no experiments. This is a fourth: the split, the payment data and the verdict in one tool.

01

Enterprise tools are priced for enterprise budgets

Indie products, bootstrapped SaaS and in-house growth teams run on margins where a bad release costs real money. A $40k annual seat is out of reach.

02

Homegrown experiment frameworks stop being maintained

Teams build their own. The person who wrote it leaves, the results sit in a spreadsheet, and the next test starts from zero.

03

Event analytics is not an experimentation layer

Analytics records what happened. Significance tests, holdouts and feature flags need their own layer.

04

Dashboards built for statisticians, read by product managers

A product manager needs one of three answers: ship, stop, or keep running. The significance figures belong next to that answer, not in place of it.

Built by the team that uses it.

optimiser.pro is built by a product-ops team for its own payment-led products. The team had a homegrown setup that stopped being maintained and could not justify an enterprise contract.

The tool puts the split, the payment data and the verdict in one place and answers one question: did the variant earn more than the control, and is the difference significant.

Where it runs today

One workspace runs on it so far: the team's own products. This page describes what the product does. There are no customer quotes because there are no other customers yet.

Results

How a resolved test reads

Each experiment is a hatched column. The one that cleared its control with a significant revenue lift is solid, with the lift on it.

Revenue lift · how a resolved test reads
Six still running. One resolved.
Ship
T1
T2
T3
Test 4
+12.4%
T4
T5
T6
T7

Illustrative. The lift shown is an example of how a resolved test reads, not a result we are claiming.

Four steps · one loop

From a split to a decision in four steps

A dashboard and one HTTP endpoint. Nothing to install and no SDK to version. The significance test runs on every read.

01 · REGISTER
Register the experiment
Name it, give it a key, list the variants and their weights. Each variant points at the slug it serves. Tests already running in VWO import into the archive.
# one row per variant
key: "checkout_copy"
  control 50% → ?theme=classic
  variant 50% → ?theme=direct
02 · ASSIGN
Assign the visitor
Your storefront posts a visitor id and gets back their variant. The assignment is a hash, not a stored record — the same id lands in the same arm on every request, with nothing to keep in sync.
# PostHog-compatible
POST /api/decide
  { distinctId: "user_42" }
→ { themes: ["direct"] }
03 · MEASURE
Join it to the payments
Events land through the same API. The results query then pulls the per-variant P&L: how many authorised, how many rebilled, how much each acquired customer was worth.
# per variant, from payments
auth rate      61.4%
rebill rate    38.9%
rev / acquired €21.07
04 · DECIDE
Read the verdict
A winner per metric, tested against control. Each metric is judged in its own direction: a lower refund rate is a win, a lower auth rate is not. Settled runs move to the archive.
# two-proportion z-test
winner: direct
  rev / acquired p=0.02
  auth rate      not significant

Figures shown are an illustration of the output shape, not a customer result.

Fewer tools · one job

Your stack stays where it is.

The tool decides which slug a visitor gets and reads what the payments database recorded. Your pages, content and analytics stay where they are.

  • No CMS
  • No visual editor
  • No SDK
  • No tag manager
  • No data team
What stays
  • Your storefront and the stack it runs on. It already renders variants from a slug.

  • Your CMS, if you have one keeps owning content. The tool never writes a page.

  • Your analytics keep counting pageviews. The tool counts revenue.

  • Your payment provider is the source of truth we read from.

What goes
  • The testing suite and the script it injects into every page.

  • The flicker from client-side swaps after the page has painted.

  • The results spreadsheet that only one person could read.

  • The contract renewal priced on traffic you never tested.

Three ideas · one method

What is counted, and what is not
claimed.

Three rules decide what appears on the dashboard and what a winner means.

01 · THE VERDICT
Money, not clicks
The verdict is called on the payment data: authorisation, rebill and net revenue per acquired customer, not on conversion rate. A variant can lift sign-ups and still lose once refunds and failed rebills land.
# same test, two readings
conversion   +4.0%
rev / acquired −€2.10
verdict      not a win
02 · THE LEDGER
Every test is evidence
A test that never reaches significance still records what was tried and which way it moved. The ledger keeps every run: the page, the element, the change and the per-arm result, so no idea is run twice by accident.
# nothing wasted
tests run     37
conclusive    2
logged        37
03 · THE SAMPLE
A direction, not a decision
The probability and the sample size sit beside every number. When the sample is too small, the result is labelled a direction to re-test, not a decision to ship. A large lift on a few hundred visitors is a hypothesis, not a winner.
# read it honestly
lift          +48%
probability   66%
call          re-test
Product tour · five screens

Split. Join. Verdict. Ledger.

Four things the tool does, and one overview. Pick a screen.

01 · Every test, one row

Ship, stop or keep running, on one row.

The winner, the money and the p-value sit together. Settled runs get out of the way into the archive; the ones that need a call stay on top.

No dashboard builder. The rows are the dashboard.

your-company / store-a.example·experiments · active
live · updated 4m ago

Active experiments

14 running · 3 ready to ship
Running
14
↗ +3 this week
Ready to ship
3
95% CI reached
Avg. lift · 7d
+4.8%
↗ vs +2.1% last wk
Losing
2
↘ kill candidates
Experiment
Lift
Significance
Status
Note
exp_04 · store_a_cta_copyvariant B · 50/50 split · 8d
+4.8%
p=0.02 · 95%
ship ready
ship fri
exp_07 · store_b_pricing_anchor3 variants · 33/33/33 · 12d
+1.2%
p=0.18 · ns
running
keep
exp_11 · store_c_fast_trackvariant A · 50/50 · 14d
−2.1%
p=0.03 · 95%
running
kill
exp_13 · store_c_upsell_bannerdraft · not started
—
—
draft
mon?
exp_14 · store_a_amount_presetsvariant C · 25/25/25/25 · 3d
+0.3%
p=0.41 · ns
running
—

Illustrative data. Your workspace shows your own experiments and your own payment figures.

Pricing · per workspace, per month

Flat, published, priced on events

Priced on the events you send: the assignment and capture calls processed, not everyone who loaded a page. Two months free on annual billing. Every tier is published.

Start here
Operator
One brand, one product team, one payment funnel.
€149/ month
€1,490 / yr billed annually · 2 months free
500k events / mo · 3 seats
  • ✓Unlimited experiments and feature flags
  • ✓Server-side assignment, no anti-flicker snippet
  • ✓Significance and SRM readouts
  • ✓Per-variant payment P&L and the verdict
  • ✓Assignment API — /decide, /capture, /flags
Start
Studio
Several brands under one roof, tested in parallel.
€449/ month
€4,490 / yr billed annually · 2 months free
5M events / mo · 10 seats
  • ✓Everything in Operator
  • ✓Multiple projects in one workspace
  • ✓Metric registry, each metric with its own direction
  • ✓Priority support
Start
Portfolio
Many brands under one account.
€1,299/ month
€12,990 / yr billed annually · 2 months free
25M events / mo · unlimited seats
  • ✓Everything in Studio
  • ✓Multi-brand, multi-tenant workspaces
  • ✓Onboarding of your existing test history
  • ✓SLA
Start

Priced on events, not traffic. Overage is billed at the same per-event rate. For more than 25M events a month, send a request.

See it first

The preview runs the product on staged data with no form. Read a verdict, walk the assignment API, see how a variant joins to the payment data. Then ask for a workspace.

Existing workspace: Sign in.

Reference

One workspace in production

optimiser.pro runs the experiments on one company's payment funnels today, the team that builds it. That is the whole reference list at present.

Assignment

The engine is PostHog-compatible, so a storefront already calling /decide and /capture points at it without a code change.

Migration

Tests that ran in VWO import into the archive, so the history of what was already tried is kept.

Isolation

Each workspace has its own subdomain, and the workspace is resolved from it on every request. One workspace runs on it today.

FAQ

Questions

If yours is not here, write to reach@optimiser.pro.

No. PostHog is a product analytics suite: analytics, replay, surveys. optimiser.pro does one job: assign the split, join it to the payment data, return a verdict. The assignment API is PostHog-compatible, so a storefront already calling /decide and /capture can point at it without a rewrite.

There is no CMS. optimiser.pro never touches your pages: your storefront already renders variants from a slug, we decide which slug a visitor gets and read what the payments database recorded afterwards. Your content stays in whatever manages it today, your analytics keep counting pageviews, and the only thing you add is one server-side call.

Priced on events — the assignment and capture calls we process, not raw traffic. €149 a month for Operator (500k events, 3 seats), €449 for Studio (5M events, 10 seats), €1,299 for Portfolio (25M events, unlimited seats) — every tier published. Two months free on annual billing, and overage is billed at the same rate. There is no free tier and no trial. The preview runs the product on staged data.

A normal report tells you which variant got more clicks. The verdict joins each variant back to the payment data it produced — authorisation rate, rebill rate, revenue per acquired customer — picks a winner per metric, and tests it against control. Each metric is judged in its own direction, so a lower refund rate reads as a win and a lower auth rate does not.

A two-proportion z-test for rates and Welch's t-test for continuous metrics, with the normal CDF implemented inline. Frequentist, not Bayesian. No statistics dependency; Node builtins only, so the arithmetic can be audited.

No. Every experiment carries a plain-language verdict and a recommendation, and the significance figures sit next to it for anyone who wants to check the working. A product manager can read the dashboard and decide to ship, stop or keep running.

No. Existing VWO experiments import into the archive. The archive is also where settled runs land once a verdict is in, and the roadmap points forward from it to re-runs and candidates.

Colour carries state, not decoration: the accent means a test is running, the brighter green means a win.

Each workspace has its own subdomain, and the workspace is resolved from that subdomain on every request. A signed-in user who is not a member of a workspace sees nothing of it. One workspace runs on it today. The platform runs on Vercel with a Postgres database. There is no self-hosted distribution today.

Three products under the same philosophy. optimiser.pro (this site) is the experimentation tool. optimiser.world is a community marketplace for AI agents and skills. optimiser.art is a creative brand consultancy. Different products and audiences under one name.
Next step

Run it on your own funnels.

From €149 a month per workspace. There is no trial and no signup form. Look at the preview, then tell us what you run and we open a workspace with you.