A typical task in adaptive testing is to select, out of a precalibrated item pool, the most appropriate item to ask, given an interim estimate of ability, $$\theta_0$$. A popular approach is to select the item having the largest value of the item information function (IIF) at $$\theta_0$$.

When the IRT model is the Rasch model, the item response function (IRF) is $P(X=1|\theta,b)=\frac{1}{1+\exp(b-\theta)}$ where $$b$$ is the item difficulty, and the IIF can be computed as $$P(X=1|\theta,b)[1-P(X=1|\theta,b)]$$. For all items, this function reaches a maximum of 0.25, encountered exactly where $$\theta=b$$ and $$P(X=1|\theta,b)=0.5$$. So picking the item with the maximum IIF is the same as picking the item for which a person of ability of $$\theta_0$$ has a probability of 0.5 to produce the correct answer.

If we use the two-parameter logistic (2PL) model instead, the IRF is $P(X=1|\theta,a,b)=\frac{1}{1+\exp[a(b-\theta)]}$ where the new parameter $$a$$ is called the discrimination parameter, and the IIF can be computed as $$a^2P(X=1|\theta,a,b)[1-P(X=1|\theta,a,b)]$$. Note that while the contribution of the product, $$P(1-P)$$, remains bounded between 0 and 0.25, the influence of $$a^2$$ can become quite large – for example, if $$a=5$$, $$P(1-P)$$ gets multiplied by 25, ten times the maximum value under the Rasch model. Selecting such an informative item gives us the happy feeling that our error of measurement will decrease a lot, but what happens to the person’s probability to give the correct response?

In our game, we have one person and two items. The person’s ability is 0, represented with a vertical gray line. One of the item is fixed to be a Rasch item with $$b=0$$; its IRF is shown as a solid black curve, and its IIF as a dotted black curve. The second item, shown in red, is 2PL, and you can control its two parameters with the sliders. Initially, $$a=1$$ and $$b=1$$.

The purpose of the game is to find particularly awkward situations where the more informative item is the most inappropriate in the sense of:

• having a difficulty as remote from $$\theta=0$$ as possible;

• having a probability of a correct response at $$\theta=0$$ as low as possible.

Can you find the sweet spots? I have just computed the answer to the second question but I am not telling (in fact, I can tell you the value of $$P$$: 0.0022, quite a bit off 0.5).