Social Icons

Pages

Monday 12 December 2016

How is the JLPT scaled?

I address this point because it’s not clear from the JLPT website, and unless you have a degree in statistics their recommendation that you look up Item Response Theory will probably not be all that helpful.



The idea of “scaling” or “grading on a curve” is common, and designed to make tests fairer. In its simplest form, this involves increasing or decreasing someone's mark linearly to compensate for a more difficult/easy test than normal. So a teacher might decide, to get the same average mark as usual, they will multiply each percentage score by 1.5 and take away 20. You got 60% right, but when this scaling is applied you receive 70%. In this system, if someone else got 60% right they will also receive 70%. All well and good.

However, the JLPT scaling is different, because it takes into account the difficulty of each question. Suppose about 40% of people got the first question on the test right, and they nearly all got a high score overall. We conclude that it was a difficult question. But nearly 90% of people got the second question right, and those that didn’t had poor scores overall, so it must have been easier. We make the second question worth less because it was easy, and the few people with high scores who got it wrong might have just made a careless mistake.

Now suppose only 25% of people got the third question right, and they had a random mixture of high and low scores. Is it an extremely difficult question, and people were just guessing? Or is it poorly written and confusing? In this case, we might conclude that the question was not very suitable for one or both of these reasons, and make it worth less than other questions.

The actual mathematics of how final marks might be calculated is too complicated to go into, but the main point is this: some questions are worth more than others. In the case of the JLPT, you and your friend might both get 60% of the questions right, but you could receive different marks because you answered different questions correctly. In general, we can say a more difficult question is worth more, and these appear towards the end of the paper. People doing the questions in order are extra likely to get these wrong because they are getting tired or are in a hurry to finish - if they don't plan their time or forgot to bring a watch, they may not even have time to guess a random answer.

The JLPT site states that the difficulty of the new format tests should be the same as the old format tests. In this case we could expect a pass mark of about 70%, the same as for the old test. But now you have to pass each section separately as well, which may mean that they lowered the overall pass mark slightly to keep the pass rate at each level fairly constant.


Pass rates for July 2014. Man, those are some scarily low numbers.

What does this all mean for us? Well, I would speculate the overall pass mark is usually in the region of 70%, possibly as low as 65% for a very difficult test, with sectional pass marks more like 60%. So if you get 75% or more overall and at least 65% on each section, you could be confident that you can pass. (When you are studying, however, note that many books with practice questions are harder than the real test, while the JEES sample papers and the samples on the JLPT website appear to be easier than the real test, at least for N2.)

More importantly, we should presume that questions appearing later on the test are worth more than earlier questions, and devote more time to them. This is not strictly the case, and of course everyone has their strengths and weaknesses, but it’s a good approximation for planning your time.

-> More on my tips for tackling the JLPT.

No comments:

Post a Comment