Ranked Skills

Technical resumes are challenging to produce, and challenging to accurately interpret. One of the biggest obvious challenges is that candidates are highly motivated to exaggerate, and interpreters of resumes are highly motivated to make a quick decision erring on the side of rejection so as to waste a minimal amount of their precious time. Another big challenge, however, one which I think might be solved, is the difficulty in communicating exactly what the resume author means by how skilled they are in a particular technology or specialty. Even if one were to assume candidates are 100% truthful and resume readers give 100% effort to read and parse every word of every resume that comes across their desks, what does “Expert at Python” or “5 years experience C++” really mean? Worse yet, someone says (or writes on their resume) “I am 8 / 10 at Ruby” - what does that mean? Eight what? Years? Projects? This person thinks they know 80% of everything there is to know about Ruby? Or are in the 80th percentile for Ruby programmers?

In discussing this problem with others, I had a brilliant suggestion made to me, and I deeply regret not remembering who to credit for it. As an interviewer, they suggested, one could ask “on a zero to ten point scale, rate yourself on X”. This immediately sounds like a complete bullshit softball question - but stay with me. The candidate will hopefully say something between six and nine. There is a strong urge among people to rate themselves at least a five on most things they are even passingly familiar with. Similarly, except for the most insanely confident candidate, pretty much nobody should claim to be a ten - If a candidate claims to be a ten in Python, they had better be Guido Van Rossum. Let’s say a candidate claims to be a nine out of ten in Python. Great, now immediately follow up with “What would you teach someone who is an eight, in order to make them a nine?”.

This is not exactly a trap, but it does force the candidate to say things which give you a very good idea what they meant by nine out of ten. If they answer “recursion”, that sure puts their nine in a strange and informative context - who has even passing familiarity with Python and needs to learn about recursion? That’s not even a Python specific thing!

If, on the other hand, they answer “how function decorators work”, you know that this candidate is familiar with function decorators and considers it a very advanced language feature. Since I would not currently rate myself a nine at Python, if they talked about things which I am not myself comfortable with, I could conclude that they really are probably more advanced at Python than I am myself.

Next, I would follow up with the inverse question: “What would you expect someone who is a ten to teach you, or what would you expect to learn in order to call yourself a ten?”. Like before, if you get a disappointing answer about some standard language feature, you know this candidate is not very advanced, but if you get an answer like “implementation details, familiarity with the codebase and PEP process, experience with implementing new language features”, that is an answer which shows that the candidate understands what it means to be at the very top of the Python game.

Using both answers, it becomes much easier to understand where a candidate is really at, rather than trying to perform mind-reading tricks to turn an arbitrary number into a complex vector in skill-space. Additionally, the number is easy to exaggerate, but the answers to these questions would be difficult to manipulate, especially if you have a brief conversation diving into details around their answer.

Since I have thought about this, I know what these numbers mean to me and I think this is a good standard I would advocate that others develop and use. I also think what I say below is in line with what most people would guess these numbers mean, but thanks to the process above, it hopefully doesn’t matter because you will have a way to explain what they mean to you using a process which both parties can trust is accurate and difficult to manipulate.

When I rate myself on a ten point scale, I use this guide to do so:

• 10: I could have a protracted discussion with the inventors of this technology including deep discussions about implementation details without prior preparation or embarrassment. I honestly believe I have little or nothing more to learn about this technology. I, personally, am unlikely ever to claim this rating.
• 9: I could follow a deep technical discussion among core developers of this technology regarding most implementation details.
• 8: I am very familiar with all user-facing aspects of this technology, as well as some implementation details necessary for mastery.
• 7: I am able to use this technology as a competent power user with little need to seek references.
• 6: I am a competent power user of this technology, though I occasionally seek references or examples.
• 5: I have a firm understanding of this technology but often seek examples or references.
• 3: I am a user of this technology, but not a power user.
• 1: I have a passing familiarity with this technology, having used it a handful of times or interacted with it superficially. I am listing it primarily because I am interested in learning more.
• 0: I refuse to use this technology or acknowledge any ability in it whatsoever. Think PHP.

I personally think of my resume as a method for communicating my skills and fit as accurately as possible. I don’t want to waste anyone’s time, including my own, so accuracy is paramount.

Addendum, 2026: what a coding assistant does to the scale

Written in 2019, before an LLM could write half the code in the interview for you. The rating trick still works. What “a nine” means has shifted underneath it.

The follow-up questions in this post do something a coding assistant cannot fake for the candidate in real time: they probe the shape of what someone knows, not whether they can produce working code. That distinction used to be mostly academic, because being able to produce working code and understanding the technology were tightly correlated. They have come uncoupled. A candidate can now ship a lot of correct Python without being able to tell you a single true thing about how decorators actually work, because the assistant supplied the decorator and they never had to understand it. So the “rate yourself, now teach an eight how to become a nine” move is more useful than it was in 2019, not less. It reaches past the artifact to the model in the person’s head, and the person still has to have that model or they cannot answer.

I would add one thing to the rubric, though, or at least to how I read it. The top of the scale now includes a skill that barely existed when I wrote this: knowing what to delegate to the machine and, more importantly, being able to tell when its confident answer is wrong. That is not a Python skill or a Ruby skill, it is closer to a taste-and-judgment skill, and it does not show up cleanly on a per-technology ranking. If I were interviewing today I would still run the zero-to-ten trick exactly as written, and I would add a question about a time the assistant confidently handed them something broken and how they caught it. A nine who has never noticed the machine being wrong is not actually a nine anymore. They just have not been burned yet.