How we built an AI grading system for 800+ schools

This is the story of one of the biggest projects we've worked on. An academic competition program needed a way to grade hundreds of student essays faster and more fairly. They came to us with a problem, and we built something that changed how the whole program works.

Here's what actually happened, including the parts that were hard.

The problem

The program ran a national academic competition. Students from over 800 schools took part. One of the events was essay writing. Hundreds of essays came in after each round, and they all needed to be scored against the same rubric.

The grading was done by hand. Multiple instructors read and scored essays, and each one graded a little differently. Some were strict. Some were lenient. Students from different regions got different levels of feedback, and some got barely any feedback at all.

The turnaround took days. By the time scores came back, the moment had passed. Students couldn't learn from the feedback because it arrived too late to matter.

On top of that, fairness was a constant question. Was a score of 7 from one grader the same as a score of 7 from another? Nobody could be sure.

What we built

We built a custom AI scoring system that reads each essay and grades it against the program's own rubric. Not a generic AI that gives vague feedback. A system trained on the specific criteria that the program already used.

The AI reads the essay. It scores each rubric category. It writes detailed comments explaining why it scored the way it did. And it does all of this in a fraction of the time a human grader would take.

But the human never leaves the loop. Every score, every comment, goes to a faculty dashboard where instructors can review, adjust, or override anything. The AI handles the heavy lifting. The instructor makes the final call.

This was important to the program. They didn't want to hand grading over to a machine. They wanted a machine that did the slow part so humans could focus on the important part: making sure students got fair, helpful feedback.

What was hard

Building the AI wasn't the hardest part. Training it on a specific rubric with consistent results took time and careful work, but the technology was sound. The hard parts were the ones we didn't expect.

Getting the rubric right. The program's rubric had been interpreted differently by different graders for years. Before we could teach the AI to score, we had to work with the program to clarify what each criterion actually meant. This was human work, not tech work, and it was some of the most valuable work in the project.

Earning trust. Instructors were wary. Some worried the AI would replace them. Others worried it would score unfairly. We spent real time showing them how it worked, letting them compare AI scores to their own, and making sure they had full control over the final result. Trust isn't a feature you can build. It's something you earn by being honest about what the system can and can't do.

Scale during peak times. When hundreds of essays come in at the same time, the system needs to handle the load without slowing down. We built it to scale up during competition periods and scale back down when things were quiet. This sounds simple, but getting the timing right so scores come back fast without running up cloud costs took careful planning.

What we learned

A few lessons from this project that apply to almost any AI build:

AI is a tool, not a replacement. The best results came when AI and humans worked together. The AI did what it was good at (consistency, speed, volume). The humans did what they were good at (judgment, context, nuance). Trying to make AI do everything would have made the system worse, not better.

The hardest part is often not the technology. Clarifying the rubric, building trust with instructors, and designing a workflow that people would actually use, those were the challenges that made or broke the project. The AI was the easy part compared to the people part.

Start with a clear problem. This project worked because the problem was specific: grade essays faster and more consistently, without removing human oversight. If the brief had been "use AI to improve education," we would have had nothing to build toward.

The result

Grading turnaround went from days to under two hours. Students got feedback while it was still fresh. Scores were consistent across all essays, regardless of which instructor reviewed them. And instructors got their time back for teaching and coaching instead of repetitive scoring.

The system now serves over 800 schools. It processes hundreds of essays per round. And it gives every student the same quality of feedback, whether they're at a well-funded school or a small rural program that could never have afforded that level of attention before.

That last part is what we're most proud of. The technology is interesting. But the fact that it made something fair that wasn't fair before, that's what made the project worth doing.