It’s 9:58 AM. You join the call, and there’s no small talk about the weather, no recruiter apologizing that the hiring manager is running late. Instead, a repository loads. There’s an AI assistant sitting in the editor next to you, ready to help. And the thing evaluating you is also software. You start reading the code, and two minutes in, it asks why you picked that file first.
That’s roughly the shape of what HackerRank is running with Chakra, its AI interviewer, which has now conducted more than 500,000 interviews. Snowflake, Snorkel, and Capgemini are among the early testers. TechCrunch framed it as a glimpse into what job interviews could become, and I think that’s fair, though the glimpse looks awfully familiar to anyone who has spent time in open source.
Open source has been grading process for years
Here’s what strikes me as someone who spends weekends reviewing pull requests from strangers. The pitch behind Chakra is that it evaluates not just the answers, but critical thinking, judgment, and what HackerRank calls “AI fluency.” Interviews resemble real-world tasks: actual code repositories, AI assistance available, follow-up questions probing how you arrived at your approach.
Swap out the scoring report and that’s just maintainer work. When a first-time contributor opens a PR against a project I help maintain, the diff is maybe a third of what I’m reading. The rest is: did they read CONTRIBUTING.md? Did they scope the change or did they reformat 400 unrelated lines? When I push back on a design choice, do they defend it with reasoning, or do they immediately rewrite it to whatever I hinted at? That last one tells me more about a person than any algorithm question ever has.
Open source never had to invent process-based evaluation because the process was always the artifact. The commit history is right there. The review thread is public. Nobody in this world has ever been impressed by a correct answer with no visible reasoning behind it, because a correct answer with no reasoning is indistinguishable from a copy-paste.
Where the whiteboard broke
Traditional technical interviews got stuck in a weird spot. They tested recall under pressure in an environment nobody works in, then stopped working entirely once AI assistants became standard equipment. You can’t ask someone to invert a binary tree on a whiteboard anymore and pretend it means something about their day job, where they’re orchestrating agents, reviewing generated diffs, and deciding which suggestions to trust.
So putting a candidate in a repo with an AI assistant and watching what they do is, honestly, the more honest test. The interesting signal in agent-assisted development is almost entirely judgment. Which task do you hand off? When do you stop trusting the output and read it yourself? What do you do when the assistant confidently produces something that compiles and is also wrong? Those are skills, they vary wildly between developers, and they’re nearly invisible in a resume.
What I’d worry about
I’m not going to pretend I’m fully comfortable with this. A few things nag at me.
- Legible process beats good process. HackerRank’s April 2026 release notes mention improved reporting, new ATS integrations, and quantitative scoring to speed up structured interview analysis. The moment a number lands in an applicant tracking system, that number becomes the candidate. Scores compress. Judgment doesn’t compress well.
- Candidates will optimize for the observer. They always have. The old game was memorizing problem patterns. The new game will be performing visible reasoning, narrating your thinking in whatever register the model rewards. Some of that is genuinely good practice. Some of it is theater with better production values.
- “AI fluency” is a moving target. The tooling I used six months ago barely resembles what I use now. Testing fluency with a specific class of assistant risks measuring familiarity with this quarter’s tools rather than the underlying adaptability you actually want.
The part I’m optimistic about
Half a million interviews is enough volume that this isn’t a demo anymore. Something is working well enough for companies to keep running candidates through it.
And if the practical outcome is that hiring shifts toward how people work in a repo, with the tools they actually use, under questions about why they made a call — that’s closer to the open source model than the whiteboard ever was. The contributors I trust most aren’t the ones who ship perfect code on the first try. They’re the ones who explain their reasoning, change their mind when the argument is better, and know which parts of a generated patch deserve a second look.
If an AI interviewer can spot that, it’s spotting the right thing. I’d just like to see the score next to the replay, not instead of it.
🕒 Published: