LeetCode is theater now. Here's how to build the replacement in a weekend.
There's a 239-upvote Reddit thread on r/programming where engineering managers are basically admitting they have no idea how to hire anymore. The recurring complaint: candidates submit clean code they can't explain, LeetCode scores mean nothing because Copilot solves those in ten minutes, and nobody has a better option yet.
That last part is the opportunity.
Hiring teams know their current process is broken. What they don't have is a credible replacement. The gap isn't awareness, it's a working tool with a scoring rubric they can trust.
The assessment that matters now looks nothing like a whiteboard problem. It's: here's a PR generated by an LLM, find the bug, explain your reasoning, and tell me what test you'd write to catch it. That's a real task. It correlates with real job performance. And right now, zero platforms are testing it in a structured way.
HackerRank and Codility will eventually ship an 'AI Skills' tab. That's probably 6-12 months away and it'll be generic when it arrives. The window to establish a scoring rubric that hiring managers actually trust is right now, before incumbents ship something mediocre and call it done.
Seriously. This one is worth the pre-work.
Build three assessment scenarios in a Notion page or Google Form. Each scenario has a buggy LLM-generated code snippet, a task description, and a free-text explanation box. That's it. No scoring engine, no dashboard, no auth.
Then DM 20 engineering managers on LinkedIn or in the Rands Leadership Slack `#hiring` channel. Don't pitch software. Offer a free 'AI-era skills audit'. They send the form to two or three candidates they're already interviewing, and you hop on a 15-minute call to debrief the results together.
Your success metric: 5 out of 20 agree to the pilot, and at least 3 say yes to paying $99/month without you having to push hard. If you get that, build it. If you get 1 out of 20, your scenarios aren't good enough yet or you're reaching the wrong people.
Don't skip this. The fatal flaw here isn't the tech, it's whether your scenario content is actually useful. A Notion form tells you that faster than six weeks of building.
Three pieces, nothing more:
The scenarios are the product. The software is delivery infrastructure. Don't get this backwards.
Next.js for the frontend, Supabase for auth and database, OpenAI API for the scoring pipeline, Stripe for billing, Resend for transactional email. This is a boring stack on purpose. Cursor or Lovable can scaffold the scenario player and dashboard fast. The part you can't delegate to AI tooling is writing the actual assessment content, which requires someone who understands how LLMs fail in production code.
For the scoring engine: you're running candidate explanations through a rubric prompt. The rubric asks the model to score root-cause identification, whether the candidate noticed the security implication, and whether their proposed test would actually catch the bug. You store raw scores plus the model's reasoning so employers can see why a candidate scored a certain way. At $0.50-$2 per assessment in API costs, your gross margin stays around 82% which is fine at early volume.
One architectural decision worth making early: build a scenario pool with random assignment. Candidates will leak scenarios to GitHub within a month of launch. Plan for it. If you have 60 scenarios rotating, a leaked answer to scenario 12 doesn't kill you.
$99/month for up to 50 assessments covers the SMB tier (early-stage startups, small engineering teams). $299/month unlocks 200 assessments plus an ATS webhook. $999/month-plus for enterprise with custom scenario modules.
18 paying SMB customers covers your infra costs. That's achievable in month two if your pre-launch validation went well.
Bootcamps are an interesting secondary angle. BloomTech, Codesmith, App Academy all have a graduation certification problem. They need something credible to show employers their grads can work with AI tools. A flat $2,400/year license and a case study in exchange for 60 days free is a reasonable trade. It gets your assessment in front of employers through a trusted referral channel without you doing the selling.
Two things could kill this that are worth naming directly.
First, content rot. LLM tooling evolves fast. A scenario you write today about catching a GPT-4 hallucination pattern might be irrelevant in 18 months if Claude 4 stops making that class of error. You need a quarterly refresh cadence and ideally 3-5 senior engineers contributing scenarios on rev-share. Budget this into your unit economics from day one, not as an afterthought.
Second, scoring validity. If a senior engineer takes your assessment, gets a low score on a debugging task they handled correctly but unconventionally, and decides to make noise about it publicly, that's a real problem. Not just a customer service problem. In some jurisdictions, assessment tools used in hiring face employment discrimination scrutiny. Your rubric needs to be defensible. Keep the model's reasoning visible to employers so scores don't feel like a black box.
The bigger strategic question I genuinely don't know the answer to: is 'debugging LLM output' a durable engineering skill or a transitional one? If AI code review automates itself in 24 months, the platform could be solving a problem that disappears. I'd argue the underlying skill (catching subtle logic errors, thinking about security implications, writing tests that actually validate behavior) is durable even if the framing changes. But that's a bet, not a certainty.
Post a detailed essay on Hacker News titled something like 'We built AI-era coding assessments, here are 3 free scenarios to try' with a link to a live demo. Engineering managers are heavy HN users. The comments on that thread will tell you more about product direction than a month of internal debate.
Simultaneously, DM 30 people from Rands Leadership Slack and LinkedIn who have posted about hiring frustrations in the last 90 days. Not cold outreach. Warm outreach to people who are already complaining about the exact problem you're solving.
The unfair advantage here is timing. Hiring managers are actively searching for a new standard right now, and they've said so publicly. The first credible option with real content quality behind it wins the early adopters. Early content quality beats engineering sophistication at this stage.