Rethinking Knowledge Assessment in Education

By Dr. Manu Kumar, Founder & Chief Firestarter, K9 Ventures

Dr. Manu Kumar, Founder & Chief Firestarter, K9 Ventures

All of us, including you, the reader, probably went through an education system in which you were graded on the basis of either a numerical score or percentage, or a letter grade. If you think about why this system exists in this form, it’s because it was probably the simplest way for a teacher to record the relative performance of a student.

However, the translation from the actual exam/ test or assignment to a letter grade or numerical score, is akin to an incredibly lousy compression routine. Almost all the interesting information about a student’s performance is lost in translation. Grading scales are used because they offer a simple, and almost lazy way to record student performance. They enable professors to place students into buckets, but those buckets actually tell us very little about what the student has learned—or still needs to learn.

"In the future, the assessment of a person’s knowledge will capture the full breadth of the scope of their understanding and mastery of concepts"

As a stand-up comedian once joked, “My friend just got his pilot’s license, and wanted me to fly with him. He told me, ‘I scored an 89 on my exam and am clear for take-off!’ And I thought, ‘How do I know that the 11 points he missed weren’t about landing!?’”

We live in an age where every single step we take is counted. Every single click is being logged/recorded. Why then should our system of assessment still remain a vestige of the middle ages?

My advisor and mentor, (and Turing Award winner) Dr. Raj Reddy, the former Dean of Computer Science at Carnegie Mellon University, once said to me that we will soon be living in an age where we not only have data on academic performance all the way back to pre-school but also have data on social and emotional factors. I believe that he is right, but we’re nowhere close to that reality yet and never will be as long as our system of assessment throws away valuable information simply because human graders get tired, bored, and lazy.

Concept-Level Assessment and Personalized Education

By using computers to augment the process of human-knowledge assessment, we can today gather information not only on the overall performance of a student, but actually maintain data down to each question or concept being tested. This data can then be used to provide personalized feedback to each student on what concepts they have mastered and what concepts they still need to work on. At a class/ instructor level, the data can be used to determine what parts of the instructors teaching is being effective and what part isn’t. We’ve all heard about Personalized Medicine, but I believe it’s also time, we started working towards Personalized Education.

AI Assisted Assessment

Assessment has always been and remains the most disliked part of a teacher’s role. One of my favorite lines in this regard is: “‘I love grading,’ said nobody.”

Computer assisted assessment allows for grading to be done more efficiently and consistently. By leveraging computer vision and machine learning (also known as AI to the lay person) we can cluster the responses for most STEM based disciplines into similar groups. These 30 students have the same answer and these 28 students have the same answer. The human graders (instructor or TA) can grade just one answer in the cluster and it can be reliably applied to all the other students’ responses in the same cluster. This result in a dramatically more efficient (and pleasant) grading experience for instructors, while preserving the highly individualized feedback for students at a concept level.

The Knowledge Network

Today, companies that are trying to recruit talent, go through time-consuming and intensive interview processes to determine what a candidate knows. This happens because our way of communicating the mastery of human knowledge is often boiled down to a single numerical score where even that score varies from institution to institution (What does a 4.3 GPA on a 4.0 scale really mean?)

In the future, the assessment of a person’s knowledge will capture the full breadth of the scope of their understanding and mastery of concepts. This will eventually become the Knowledge Network. In such a world, we would still interview candidates, but we can reliably communicate their skill levels in a way that is way more high fidelity than a GPA, a letter grade or a numerical score.

If you think that all of this is some far-fetched future, well, then you’re mistaken. All of the concepts described above exist today and are being pioneered by the founders of Gradescope (a K9 portfolio company). Gradescope is founded by a UC Berkeley professor, and by three graduate students who spent many hours begrudgingly have to grade assignments and tests. The solution they created is now in use university-wide at some of the top-ranked schools in the United States.

I’m excited by the future that is coming. A future where education becomes more than just institutionalized learning and where personalized education by virtue of concept-level assessment enables a return to a focus on learning rather than education.

EdTech Startups Special