The gap between 'we have SLOs' and 'we have alerts that actually work' is a real problem with no good tool solving it.
There's a Reddit post in r/devops titled 'Drowning in alerts but critical issues keep slipping through' that got 42 upvotes and 16 comments. The top thread is SREs describing how their pages fire on CPU spikes while actual user-facing latency goes unmonitored. This is embarrassingly common. Teams read the Google SRE Book, define their SLOs, and then write alert rules based on gut feel and cargo-culted thresholds. Nobody validates whether those rules would have caught the last outage.
That validation gap is what you're building into a product.
Most alerting setups fire on causes (CPU at 90%, DB connection pool exhausted) rather than symptoms (p99 latency over 500ms, error rate above 1%). The cause-based alerts are noisy because they don't always translate to user pain. The symptom-based alerts are hard to configure correctly because you need to pick a threshold and a burn rate window without knowing how those settings would have performed historically.
SREs know this. They've read the multi-window burn rate chapter. They just don't have a tool that closes the loop between 'I defined my SLO' and 'I can prove this alert rule actually works before I deploy it.' That proof currently happens in spreadsheets, or not at all.
Seriously, don't open Cursor yet.
Spend the first three days on this: write a Notion doc describing the tool with a few mockup screenshots (use v0.dev, takes 20 minutes). Attach a Typeform asking for their current stack, their worst alert pain point, and whether they'd pay $99/month. Post it in r/devops, r/sre, and the CNCF Slack #slo channel.
The offer that converts lurkers into respondents: 'I'll manually generate a backtested SLO alert rule for your service using your Prometheus data, in exchange for a 30-minute call.' Do this for free for the first 10 people. Your success threshold is 5 SREs or engineering managers agreeing to a paid pilot, or 3 Typeform submissions with 'yes I'd pay' and a specific pain point, within two weeks.
If you can't get 5 people interested enough to take a free thing from you, the idea needs rethinking. If you get 15 responses in 48 hours, that's a different situation.
Four features. Nothing else.
First, a Prometheus `remote_read` integration that ingests 90 days of historical metrics. The user provides their endpoint and credentials, the app pulls the data via passthrough queries. No custom metrics storage, which keeps infrastructure costs near zero.
Second, an SLO definition UI. Service name, SLI type (latency, error rate, or availability), target percentage, time window. That's the whole form. The app generates multi-window burn rate alert rules following the Google SRE Book formulas.
Third, the backtesting engine. This is the whole product. Given a proposed alert rule, run it against the historical data. Show the user: how many false positives per week, how many real incidents it would have missed, and for incidents it did catch, how many minutes before impact it would have fired. When someone sees 'this rule would have fired 18 minutes before your last P1 outage,' the product sells itself.
Fourth, rule export to Prometheus YAML or Datadog monitors. The user copies it out or hits a deploy button.
That's a 4-6 week build solo. Prometheus-only at launch. Datadog comes later once you have paying customers asking for it.
Next.js frontend on Vercel. Supabase for auth and storing SLO definitions and incident history. A Python worker for the PromQL evaluation engine — this is where you'll spend most of your time, and it's the part where having Cursor open while referencing the Prometheus HTTP API docs actually accelerates you meaningfully.
The backtest engine is just: for each 5-minute window in your 90-day history, evaluate whether the proposed alerting expression would have returned a firing result. Aggregate the firings, cross-reference against known incidents the user tags. Compute precision and recall. This is not a novel algorithm. The value is in the UI that makes it accessible.
LLM integration is worth adding early. Feed GPT-4o or Claude Sonnet the backtest results and have it explain in plain English why a threshold is too aggressive, what adjustment to make, and which SRE Book principle applies. SREs love seeing their domain language reflected back at them accurately. It also dramatically reduces the learning curve for engineers who haven't memorized the burn rate formulas.
Three tiers: $99/month Starter (one service, Prometheus only, 90-day backtest window), $299/month Growth (ten services, Prometheus and Datadog, one-year backtest, deploy API), $799/month Team (unlimited services, multi-user, PagerDuty, SSO, on-prem Docker option).
Break-even is 17 customers at the $199 average. That's achievable in month two if validation goes well, not month twelve.
Gross margin is around 88% because the backtest compute is cheap at mid-market query volumes. The main cost is the Python worker running PromQL queries, and those are fast.
The retention mechanism that matters: once a team has imported their incident history and built an alert library inside the product, leaving means re-exporting every rule and re-tuning thresholds from scratch. That's painful enough that people don't do it. The weekly 'alert health digest' email showing false positive trends keeps you top-of-mind without being annoying, because it's actually useful data.
Datadog's Watchdog product is doing ML-powered alert noise reduction, and they have SLO features shipping regularly. They have full metrics access for their customers already, which means backtesting historical data against alert rules is technically trivial for them to add. If they ship it in the next major release, the core value prop evaporates for the 60-70% of mid-market companies running Datadog.
This is a real threat, not a hypothetical one. Your defense is Prometheus-first depth and cross-platform deployment. A team running both Prometheus and Datadog during a migration gets unique value from a tool that speaks both languages simultaneously. Pure-Prometheus shops below 200 engineers are also safer ground because Datadog is expensive enough that plenty of teams never adopt it.
The other real risk is the data access problem. Getting a Prometheus `remote_read` endpoint exposed to an external SaaS is not a self-serve action. It requires security review and often a procurement process. The self-hosted Docker option at the $799 tier handles this, but it changes your deployment complexity. Price accordingly and be honest with yourself that 'self-hosted' means you're now shipping on-prem software to some customers, not pure SaaS.
Target Series A-B companies where the SRE team is 1-3 people. They've read the Google SRE Book, they know their alerting is broken, and they explicitly don't have time to build internal tooling. Avoid companies with dedicated platform engineering teams larger than 10 people — those teams will build this themselves and then open-source it.
Week one: record a 5-minute Loom showing the backtesting UI catching a simulated incident. Post in r/devops and r/sre with the title 'Built a tool that simulates whether your alert rules would have caught your last outage — feedback wanted.' Don't make it a launch post. Make it a 'roast my thing' post. SREs respond to genuine solicitation for criticism much better than product announcements.
Week two: DM 20 SREs in CNCF Slack who have posted in #slo about alert noise in the last 30 days. Offer a free one-hour backtesting session using their Prometheus endpoint. Do the work manually if the product isn't ready yet. The goal is learning what their data looks like and what the aha moment actually feels like for them.
Week three: reach out to DevOps and SRE meetup organizers offering a 15-minute lightning talk. A live demo of backtest results from a real (anonymized) incident is a genuinely good conference talk, not a sales pitch.
The addressable market for Prometheus-only at mid-market scale is around $43M ARR based on CNCF survey data extrapolated down. That's enough to build a real business before worrying about Datadog expansion. The opportunity window before platform consolidation is measured in months, so the time to start is now, not after more planning.