GuideIntermediate
Shipping And Iterating Products
Teaches that "the A/B won" is not "flip it on for everyone": shipping a product change is its own risk decision, with a reverse gear. It closes the Product Engineering ecosystem by covering feature flags (decoupling deploy from release, kill switches for instant rollback without a redeploy), gradual rollout (canary 1% → 10% → 50% → 100%, with controlled blast radius at every stage), monitoring the launch (watching product guardrails during the ramp, and halting if they break), rollback and incident response (revert vs. fix-forward, the runbook, blameless incidents), blameless postmortems and the ship → measure → learn loop, and the modern layer of shipping AI safely (migrating models with shadow mode and canary, practical data privacy when launching). It picks up exactly where the Metrics guide left off: Mercado's `recommendations` bet won its experiment with real statistical significance, but broke a latency guardrail. All the launch decision logic (advance, halt, rollback) runs as executed Node.js code. It closes with a capstone project that ships that full bet and closes out the entire ecosystem arc.
- 64
- lessons
- 8
- modules
- English · Spanish
- available in
- Yes
- certificate
- Free
- access
Outcomes
What you'll be able to do
- Understand why shipping is its own risk decision, not a switch — "the A/B won" doesn't mean "turn it on for 100%"
- Decouple deploy from release with feature flags, including kill switches for instant rollback without a new deploy (`isEnabled`)
- Design a gradual rollout with explicit stage-advance criteria: canary 1% → 10% → 50% → 100% (`rolloutPlan`)
- Monitor guardrails live during a rollout and know when to halt before scaling further
- Decide between rollback and fix-forward with a runbook and incident response process, run as `rollbackDecision`
- Write blameless postmortems (timeline → contributing factors → action items) and close the ship → measure → learn loop
- Migrate AI models safely with shadow mode and canary, comparing agreement rate before switching (`shadowCompare`)
- Apply flags, rollout, monitoring, rollback, and postmortem end to end to Mercado's recommendations launch in the capstone project
Before you start
What you need to bring
It's for you if...
- Engineers who already measured a statistically significant experiment result and need to take it to 100% of users without breaking anything
- Devs who've watched "the test won" turn into a full launch that broke something nobody was watching for
- Backend or fullstack engineers responsible for on-call, incident response, or owning a feature's rollout
- Teams adopting or evaluating feature flags and progressive delivery as a practice
- Engineers who need a practical, not academic, introduction to migrating an AI model in production safely
Requirements and materials
- Product Thinking for Engineers Guide, Product Discovery and Prototyping Guide, and Product Metrics and Experimentation Guide completed (or equivalent: a prioritized bet, validated and measured with a statistically significant result)
- Comfort running simple Node.js scripts (no dependencies)
- Basic familiarity deploying a web application (this guide covers product-level rollout, not CI/CD infrastructure)
Content
The syllabus, module by module
Open any of them to see its lessons.
- Guide overview: from "the A/B won" to shipping it safely, with rollback, and iterating
- Winning the experiment is not the same as shipping the product
- The launch as a risk decision, not a switch
- Blast radius: how many real people a known error touches
- Deploying is not the same as releasing
- The shape of a careful launch: small, observable, reversible
- When a winner breaks a guardrail: the verdict on `recommendations`
- Project: decide `recommendations`' launch approach
- Module introduction: the feature flag, the switch that separates deploy from release
- Deploy vs. release, solved in code: the feature flag
- What a feature flag (really) is: anatomy and where it lives
- Gradual exposure: the same flag, a percentage of users
- The kill switch: turning off instantly, no redeploy
- Flag types: temporary, experiment, operational, permanent
- Flag debt: when (and how) to remove them
- Project: put Mercado's recommendations behind a feature flag
- Module introduction: from having the flag to climbing a ramp
- The ramp: canary 1% → 10% → 50% → 100%
- The canary: how small is too small
- Advance criteria: what makes a stage "advance," concretely
- Blast radius grows with each stage
- Deployment rings: who sees the feature first
- Dwell time: how long to wait before trusting a stage
- Project: design and simulate `recommendations`' rollout ramp
- Module introduction: watching the ramp, not just climbing it
- What to watch as you climb: signals, and how fast they warn you
- Guardrails in flight: watching every stage of the ramp
- When to stop: the same mechanism, two different results
- The launch dashboard: what to show, and what not to
- Alerts and signal: the symptom isn't the same as the impact
- Error budget: how much room you have before you must stop
- Project: watch `recommendations`' rollout and decide to stop
- Module introduction: you halted the ramp. Now what?
- Rollback is your safety net
- Rollback vs. fix-forward: the full decision
- The runbook: the steps you don't improvise
- Mitigate first, diagnose later
- Severity and escalation: who gets woken up
- MTTR: measuring and closing the incident
- Project: respond end to end to the `recommendations` incident
- Module introduction: you resolved the incident. Now learn from it.
- The blameless postmortem: investigate the system, not the person
- Timeline, contributing factors, and action items: a postmortem's structure
- Blameless culture: language decides whether the check matters
- The ship → measure → learn loop: launching isn't the end, it's the start
- Iterating on the winner: fix, relaunch, measure again
- When a "winner" doesn't hold up in production: the novelty effect
- Project: write `recommendations`'s postmortem and plan its iteration
- Module introduction: shipping a model, not just a feature
- A model isn't static code: the same endpoint, a behavior that drifts
- Shadow mode: running the new model without anyone noticing
- The agreement rate: `shadowCompare()` and when a model is ready for a canary
- AI incident postmortems: when "I couldn't reproduce it" doesn't close the case
- Privacy and data when shipping: what gets logged, and what didn't need to
- Migrating a model end to end: from shadow to production, no shortcuts
- Project: decide whether to migrate recommendations from recs-v1 to recs-v2
- Capstone introduction: the first real flight, with passengers
- The flag turned on and the ramp designed
- Watching the ramp live: the HALT at 10%
- Deciding: revert in minutes, not hours
- The postmortem, blaming no one
- Iterating: migrating the model with shadow mode
- Relaunching, and verifying the lift holds
- Final project: ship (and land safely) Mercado's recommendations
Common questions
What people usually ask
No limit. It's a free guide: come in whenever you like, as often as you like.
No. Modules run from easier to harder, but you can jump to the one you need. Progress is saved per lesson.
Whatever is needed is listed under “What you need to bring”, above. If nothing is listed there, you can start from zero.
In the Club's WhatsApp group, and every two weeks there's a live with an instructor where questions get worked through.
Yes. It's issued automatically once you finish every lesson, with a verifiable code you can share on LinkedIn.
Start whenever you like
What students say
These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.
No approved reviews yet.
Be the first to share your experience!