An Expanded Derivation of the Threshold Structure of the Polytomous Rasch Model That Dispels Any “Threshold Disorder Controversy”

Abstract

Responses to items with formats in more than two ordered categories are ubiquitous in education and the social sciences. Because the putative ordering of the categories reflects an understanding of what it means to have more of the variable, it seems mandatory that the ordering of the categories is an empirical property of the assessments and not merely a property of the model used to analyze them. To provide an unequivocal interpretation of category ordering in rating formats, this article expands the original derivation of the polytomous Rasch model for ordered categories. To do so, it integrates a complex of mathematical relationships among response spaces from which a space of experimentally independent Bernoulli variables, characterized by Rasch’s simple logistic model, can be inferred. From this inference, the article establishes the necessary and sufficient evidence to test the hypothesis that the required ordering of the categories is an empirical property of the assessments. This expanded derivation, which exposes how Adams, Wu, and Wilson (2012) misconstrue the model and its implications, is intended to dispel the so-called disordered threshold controversy they claim exists.

Keywords

Rasch model rating scale model partial credit model ordered category formats polytomous Rasch model

Introduction

The immediate motivation for this article is a response to Adams, Wu, and Wilson (AWW; 2012) who consider that the threshold order in the polytomous Rasch model (PRM), the special cases of which are the rating and partial credit parameterizations, is controversial. Because the focus is on the response of a single person to a single item, and the threshold structure within an item is identical in the two cases, this article uses the general term PRM rather than rating scale or partial credit model.

Many of the topics AWW canvas seem irrelevant to their title. However, with respect to the topic in their title, AWW assert that (a) the Guttman structure with a hypothesized threshold order is not a formal part of the PRM and that category ordering is independent of the empirical threshold parameter estimates, (b) the reason for reversed thresholds is that a particular category has a low frequency, and (c) claim that item fit to the model adds to their case that reversed threshold estimates imply no anomaly in the empirical ordering of the categories.

AWW’s first two assertions are rebutted in this article, and the test of fit on which they base their third assertion is shown to be irrelevant. As a result, the conclusion of this article, exactly the opposite of AWW’s, is that every item that has disordered threshold estimates shows an anomaly. Moreover, the implication is that a substantive, not a statistical, explanation needs to be sought to understand and correct the anomaly. The article uses an example provided by AWW to illustrate this conclusion and implication.

However, the purpose of the article is broader than simply dispelling AWW’s misunderstandings. The broader purpose is to provide a full appreciation of the set of coherent mathematical properties of the model that enable the empirical investigation of the ordering of categories. Therefore, the emphasis is on explaining the model from first principles, and within this explanation, to reply to AWW’s assertions. It is noted that the various algebraic forms of the model in AWW are not an issue, and the one chosen in this article is used for purposes of efficient exposition.

Assessments in ordered category formats, especially rating formats, are ubiquitous in educational and other social sciences and are used by analogy to measuring instruments of the natural sciences. Essentially, a response in a higher category is intended to reflect a higher location of the respondent on the variable of assessment, and symmetrically, it is intended that the higher the location on the variable, the more likely a response in a higher category.

Order among categories is a substantively severe constraint on the response structure. However, the order may not work empirically as intended. If it does not, then not only is the understanding of what it means to have more of the variable called into question, but also it may lead to incorrect decisions being made, ranging from areas of individual diagnoses, through policy decisions, to research interpretations. Therefore, following Fisher (1958), it seems mandatory that the ordering is a verifiable, empirical property of the categories, and not merely assumed. Thus, the PRM, which can be used to identify whether or not the ordering of categories is working as intended, is an important tool for analyzing responses in ordered categories and, in particular, rating formats.

Two distinct components are integrated in this article. The first specifies, a priori to data collection, category order as a hypothesis to be assessed; the second is the mathematical structure of the PRM that permits assessing this hypothesis. Following first Kuhn (1961/1977) that the function of measurement is to disclose anomalies, and second Rasch (1961) that his models are models for measurement, the term anomaly is used for evidence of any incompatibility between the data and category order in the PRM.

This article establishes that the empirical ordering of categories is equivalent to the empirical ordering of successive thresholds that define adjacent categories on a continuum. If dichotomous assessments at each of the thresholds were experimentally independent, then evidence of threshold order would be available readily. However, because there is just one response in more than two categories, dichotomous assessments at the thresholds are neither observable nor independent. Instead, if they can be considered at all, they must be latent and therefore inferred. This article explicates a complex of mathematical relationships of the PRM to show that inferences from ordered category formats can be made as if responses at the thresholds were experimentally independent.

As shown in the article, deriving the above result in the PRM requires a subtle distinction between the structure and the values of thresholds in the PRM, a distinction that AWW conflate.

The rest of the article is presented as follows. First I explain in detail, including with a simulated example, the role of the Guttman structure in the threshold and category order in the PRM. Then I demonstrate that AWW’s contention that the determinant of disordered thresholds is a low frequency in a related category in the whole sample is incomplete. Following this I explain the distribution of frequencies that leads to reversed thresholds and demonstrate that the test of fit that AWW use is irrelevant to implications of reversed thresholds. All derivations are consistent with the original identification of thresholds in the Rasch model for ordered categories (Andrich, 1978). Then I provide a discussion. Finally, I provide a conclusion anticipating further consideration of the role of tests of fit in the application of models in a measurement context.

Standard Ordered Category Formats and a Resolved Design

Although AWW refer to accounting for local dependence between items using the PRM, an application considered in Andrich (1983, 1985a), that case is not considered in this article. Instead, because AWW, as did the original explication of the thresholds in the Rasch model (Andrich, 1978), focus on a standard, rating format, this article does the same. However, AWW provide an example of partial credit, and their example is reexamined from the perspective of this article.

Andrich (1978) and AWW use only three ordered categories illustratively, and the latter use those of Fail, Pass, and Distinction in the assessment of proficiency. These same categories are used in this article. To make the interpretations concrete, Table 1 shows a specific example from Harris (1991) in the assessment of essay writing with respect to the criterion of setting, which is elaborated with descriptors. It is clear that the successive descriptors imply order of achievement.

Table 1.

Operational Definition for Judging Essays With Respect to the Property of Setting

Fail (F)	Pass (P)	Distinction (D)
Inadequate	Discrete	Integrated and manipulated

Source: Adapted, with permission, from Harris (1991), p. 49. The labels of Fail, Pass, and Distinction have been added for this article.

Resolving a Rating Design Into an Experimentally Independent Design

The inferences from the PRM rest on the relationships among three probability spaces of Bernoulli variables introduced in Andrich (1978) and expanded on here. Unfortunately, AWW pay no attention to response spaces.

To motivate the role of responses spaces, a design with experimentally independent responses based on the format in Table 1 is constructed. This design is considered a thought experiment as in Andrich (1978). As part of their misunderstanding, AWW appear to take such a design literally.

As we have argued above, this is not the case; parameter disorder does not imply category disorder, unless a particular latent process is assumed; a process that can only hold when judges are involved in some judgment processes and even then the process is merely conjecture. (AWW, 2012, p. 24)

Although the design imagines judges making independent decisions, it is proved that the mathematical structure of the PRM is independent of any actual judging process. Because of continuing misunderstandings in AWW, this motivation, which was introduced in Andrich (1978) and elaborated in Andrich (2005, 2010), is expanded further.

Consider a design in which a single response in one of the $three$ ordered categories in Table 1 is resolved into two independent dichotomous decisions. The response might be a self-report or made by a judge. For efficiency of exposition, we use the latter case. Specifically, instead of one judge assigning each essay into one of three categories, two different judges work independently—one judge decides if each essay meets the standard at Pass, the other decides if the same essays meet the standard at Distinction. Figure 1 summarizes the design.

The first two rows locate the thresholds in the resolved design. Each threshold is defined as the point on the real number line at which an essay will have an equal probability of being declared successful and unsuccessful in meeting the standard. In this example, the two threshold values are notated as $δ_{\tilde{P} / P}$ and $δ_{\tilde{D} / D}$ , respectively. In typical achievement assessment, these thresholds are referred to as item difficulties.

As a direct consequence of the intended category order, the a priori specification is that the threshold at Distinction is more difficult than the threshold at Pass, that is, $δ_{\tilde{P} / P} < δ_{\tilde{D} / D} .$

It is stressed that in this design, it is the continuum that is dichotomized at two places (thresholds) with independent decisions made at each threshold. This specification is reinforced in Figure 1 by interpolating descriptors by “~” to indicate that the descriptors lead from one to the other on the continuum and by the incomplete ovals that envelop the descriptors on either side of each threshold. The independence of the decisions is reinforced by the resolution of the continuum for each decision. Rows 1 and 2 of Figure 1 are referred to in this article as a resolved design. This dichotomization of the continuum contrasts with the dichotomization of a single probability distribution in the graded responses model (Samejima, 1969).

Row 3 in Figure 1 reintegrates the location of the thresholds on a single continuum. This depiction is exactly that of locations of two independent dichotomous items when responses to them are characterized by Rasch’s simple logistic model (SLM; Rasch, 1960/1980).

Row 4 in Figure 1 repeats the design of an ordered category format of Table 1 in relation to the resolved design. This row is referred to in this article as the standard design. To reflect this design, the threshold values $δ_{F / P}$ and $δ_{P / D}$ between the adjacent categories in the last row in Figure 2 are presented with alternate subscripts. To anticipate a consequence of the derivations, it is proved that in the PRM, $δ_{\tilde{P} / P} = δ_{F / P},$ $δ_{\tilde{D} / D} = δ_{P / D} .$ Perhaps because they misconstrue the construction of the resolved design in Andrich (1978), AWW do not see this identity and its implications.

Indeed, in relation to Andrich (1978), AWW state,

It appears from Figure 3 that Andrich sees the tau parameters as thresholds in the sense of Thurstone (1928). Furthermore, he says that “ $τ_{k}, k = 1, 2, 3, \dots, m,$ are m thresholds which divide the continuum into m+1 categories” (Andrich, 2005, p. 3). Note however there is nothing in his derivation from which this follows nor does he formalize this property for use in his derivation.

The gross misrepresentation and misunderstanding of Andrich (1978, 2005) is puzzling. First, the division of the continuum in Andrich (1978), as in this article, is central to the identification of thresholds within the general form of the polytomous Rasch model then known in the form of Andersen (1977). This is the only form that retains sufficient statistics characteristic of Rasch models. Second, the 1978 article explicitly contrasts the newly defined thresholds from Thurstone’s in which a single probability distribution is dichotomized. Thurstone’s dichotomization leads to the graded response model (Samejima, 1969) in which no sufficient statistics exist. Thurstone’s conception was shown in Figure 1, mine in Figure 3, which is the only one AWW reproduce. The conceptions were again explicitly contrasted for other implications in Andrich (1995).

Bernoulli Random Variables in the Resolved Design and Order of Success Rates

We now introduce the probabilistic element in the decisions at the thresholds. Let

Y_{nk} = y \in {0, 1}, P_{nk} = Pr {Y_{nk} = 1}; Q_{nk} = 1 - P_{nk}

(1)

be a Bernoulli random variable for the response of (hypothetical) Judge k to essay n at threshold $δ_{k}$ in the resolved design:

δ_{k}, k \in {1, 2} \leftrightarrow {\tilde{P} / P, \tilde{D} / D}

(2)

where the response $Y_{nk} = 1$ designates the successful response. With m+1 ordered categories, there are m Bernoulli variables. In the resolved design, no artificial constraint forces their relative success rates to take a particular order. However, to be consistent with the a priori ordering of $δ_{\tilde{P} / P} < δ_{\tilde{D} / D},$ for any essay or any subset of essays, it is required that data satisfy

\hat{P} r {Y_{\tilde{P} / P} = 1} > \hat{P} r {Y_{\tilde{D} / D} = 1} .

(3)

It is stressed that although an empirical, resolved design could be constructed, it is hypothetical and used to motivate the development. The motivation establishes a required threshold order and probabilistic responses at each threshold. In addition, it is noted that AWW provide more or less complicated definitions of order. Here, the definition is embedded in the intended relative difficulty of the thresholds in the resolved design, where the intended order is clear. All other interpretations follow from this definition.

The Case for Rasch’s Simple Logistic Model at the Thresholds

Not only must Equation (3) hold in data, but to approach the analogy to measurement, distances between thresholds should be invariant with respect to differently proficient essays. For both conditions to hold, the responses need to be characterized by the SLM (Rasch, 1961; Wright, 1997). It is emphasized that justifying the application of the SLM in this way, as with the specification of threshold order, is a priori to any collection of data and that data may not follow the SLM. The model is taken as a hypothesis of the data, not merely their description.

The SLM takes the form

Pr {Y_{nk} = 1} = \exp (β_{n} - δ_{k}) / γ_{nk},

(4)

where $γ_{nk} = 1 + \exp (β_{n} - δ_{k})$ is the normalizing factor, and β_n is the location of essay n on the same continuum as the difficulty of threshold δ_k.

To explicate connections to the PRM, the two standard psychometrics response functions, $Pr {Y_{\tilde{P} / P} = 1}, Pr {Y_{\tilde{D} / D} = 1}$ , are displayed in Figure 2. Such functions, generally referred to as item characteristic curves, are referred to in this article as threshold characteristic curves (TCCs).

If there were a resolved design, then the SLM could be applied, a test of fit conducted, the estimates ${\hat{δ}}_{\tilde{P} / P}, {\hat{δ}}_{\tilde{D} / D}$ obtained, and their invariance and required order confirmed. This article establishes that even when there is no such actual design, an inference can be made using the PRM as if there were such a design. It is recognized that this might appear surprising, and is the inference that AWW implicitly reject. However, that is exactly what is established and is the unique feature of the PRM. Then it is because it is possible to reason as if there is a resolved design with experimentally independent responses that there is a powerful set up for testing the empirical ordering of the categories.

An immediate implication of Figure 2 is that the relative order of response probabilities at the thresholds is specified. It consolidates that at every essay location, the probability of success at $δ_{1} = δ_{\tilde{P} / P}$ is greater than at $δ_{2} = δ_{\tilde{D} / D} .$ For example, if an essay is located at $β_{n} = δ_{2} = δ_{\tilde{D} / D}$ and has .5 probability of being declared a Distinction, it has an even greater probability than .5 of being declared a Pass.

Construction and Analyses of Response Spaces

The resolved design of the first two rows of Figure 1 is now bridged mathematically to the standard design of the last row in the figure.

The Experimentally Independent Response Space Ω

By definition $Y_{nk} = y \in {0, 1}, k = 1, 2, \dots, m$ are experimentally independent. Let $Ω \equiv {(y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{nm})}$ be the space of all 2^m responses. Ω is referred to as an experimentally independent space. Then, Equation (1) can be elaborated to

P_{nk} = Pr {Y_{nk} = 1 | Ω}; Q_{nk} = 1 - P_{nk} .

(5)

The joint probability of all response vectors is

Pr {(y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{n m}) | Ω} = Π_{k = 1}^{m} P_{nk}^{y_{nk}} Q_{nk}^{1 - y_{nk}}

(6)

and

\sum_{Ω} Pr {(y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{nm}) | Ω} = \sum_{Ω} Π_{k = 1}^{m} P_{nk}^{y_{nk}} Q_{nk}^{1 - y_{nk}} = 1 .

(7)

The space $Ω \equiv {(0, 0), (1, 0), (0, 1), (1, 1)$ for m = 2, and where the first and second member of each ordered pair refers to a decision at the threshold $δ_{\tilde{P} / P}$ and $δ_{\tilde{D} / D},$ respectively, are shown in Table 2. This table also shows the Guttman space $Ω^{G}$ (Guttman, 1950) to be defined shortly. Because AWW consider $Ω^{G}$ with a threshold order is not established in the PRM, the way it permeates its every aspect is explicated carefully. The explication is consistent with the exposition in Andrich (1978) and more recent expositions in Luo (2005) and Andrich (2010), but more detailed with elaborated inferences.

Table 2.

The space Ω and subspace Ω^G for m = 2

y _n1	y _n2
0 (Q_n1)	0 (Q_n2)
1 (P_n1)	0 (Q_n2)		Ω^G
1 (P_n1)	1 (P_n2)
0 (Q_n1)	1 (P_n2)	Ω

$\sum_{Ω} Pr {(y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{nm}) \| Ω} = \sum_{Ω} Π_{x = 1}^{m} P_{nk}^{y_{nx}} Q_{nk}^{1 - y_{nx}} = 1$

Implications of the Required Threshold Order

Consider the intersection of the two dichotomizations of Rows 1 and 2 of Figure 1. The intersection of {Fail} versus {Pass or Distinction} and {Fail or Pass } versus {Distinction} is the decision {Pass}. The response pair is ${(1, 0) | Ω} .$ That is, if an essay is deemed to have exceeded $δ_{\tilde{P} / P}$ but not $δ_{\tilde{D} / D},$ then it is a Pass. Figure 3 repeats the Row 3 of Figure 1, in which the two thresholds are placed on the same continuum, and shows the above intersection of the two resolved continuums as the decision Pass. Figure 3 also repeats the TCCs of Figure 2.

In addition to Pass, two more pairs of responses in the resolved design can be placed on the continuum of Figure 3. First, if the decisions at $δ_{\tilde{P} / P}$ and $δ_{\tilde{D} / D}$ are both unsuccessful, ${(0, 0) | Ω},$ the outcome is Fail; second, if both are successful, ${(1, 1) | Ω},$ the outcome is Distinction. In Ω, these three pairs of decisions correspond to the subspace ${(0, 0), (1, 0), (1, 1)}$ and are consistent with the intended threshold order.

In contrast, the response pair ${(0, 1) | Ω},$ unsuccessful at $δ_{\tilde{P} / P}$ and successful at $δ_{\tilde{D} / D},$ defies the intended order—an essay cannot simultaneously be above the Distinction threshold and below the Fail threshold. In a probabilistic framework of the SLM and with independent responses at the thresholds in Ω, this anomalous response will happen with a frequency governed by the differences in distances between the two thresholds. This phenomenon is illustrated where Simulation 1 is introduced. For the present, the focus is on the responses that are consistent with the definition of order. Because of its implications for interpreting reversed threshold estimates, the response ${(0, 1) | Ω}$ is reconsidered later in the article. For the present, and because it defies the required order, it is simply omitted.

The Subspace $Ω$ ^G

Let

\underline{ω_{x}} \equiv {y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{nm}} \equiv \underset{x}{\underset{︸}{(1, 1, 1, \dots 1,}} \underset{m - x}{\underset{︸}{0, 0, 0 \dots, 0)}}

be a vector of responses in which the first x variables, corresponding to the required ordering $δ_{k - 1} < δ_{k}$ of the thresholds in the resolved design are successes, followed by m−x failures. This structure is named after Guttman (1950), who made a case that for dichotomous items located in order of difficulty, the total score should recover the vector of responses. Let $Ω^{G}$ be the subspace of all possible Guttman vectors. The number of vectors is m+ 1. Table 2 shows $Ω^{G}$ for m = 2. For any total score of the specified ordered values of the thresholds, the Guttman vector has the highest probability (Andrich, 1985b). It should be clear, and it is emphasized because of AWW’s misunderstanding in their quote above, that the choice of the subset of vectors in Ω that form $Ω^{G}$ cannot be made without an explicit ordering of the thresholds, an ordering that is immediately a hypothesis to be tested in data.

Bridging the Resolved Design to the Standard Design With the Guttman Space $Ω^{G}$

Within the response subspace $Ω^{G},$ let the sum of the Bernoulli random variables be

X_{n} = \sum_{k = 1}^{m} Y_{nk} with values x_{n} = \sum_{k = 1}^{m} y_{nk}; x_{n} \in {0, 1, 2, \dots, m} .

(8)

Then each sum $x \in {0, 1, 2, \dots, m}$ in $Ω^{G}$ arises from a unique vector. For example, if m = 2, x = 1, arises only from (1,0). All values $x_{n} = \sum_{k = 1}^{2} y_{nk}; x \in {0, 1, 2}$ in $Ω^{G}$ are shown in Table 3, which also highlights the subspace $Ω_{0, 1}^{G} \subset Ω^{G} .$

Table 3.

The Guttman Subspace Ω^G and responses in the standard design

Total Score in Ω^G	Inferred Category
X_ni = x		Y _ni1	Y _ni2
0	Fail (F)	0	0
				$Ω_{0, 1}^{G}$
1	Pass (P)	1	0
2	Distinction (D)	1	1	Ω^G

Because X_n = x determines a unique vector in $Ω^{G},$ it implies a response in one of the categories in the standard design and is consistent with Figure 3. For example, as seen earlier $(0, 0) \equiv x = 0$ is an unsuccessful response at both $\tilde{P} / P$ and at $\tilde{D} / D$ and therefore implies Fail. Likewise $(1, 0) \equiv x = 1$ is a successful response at $\tilde{P} / P$ and an unsuccessful one at $\tilde{D} / D,$ and implies Pass; and $(1, 1) \equiv x = 2$ is a successful response at both thresholds and therefore implies Distinction. Thus, any essay with a Guttman vector from the two independent assessments can be identified with the original categories of the standard design.

The inferred category for each total score in the standard design, and compatible with Figure 3, is also shown in Table 3. It is reemphasized that because a Guttman vector could not be chosen without the specification of threshold order, the inference depends on the a priori specification of threshold order in the resolved design, which in turn arises from the specified category order in the standard design.

The Probability Structure of $Ω$ ^G

The sum of probabilities of the Guttman vectors within Ω is

H_{n} = \sum_{Ω^{G}} Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk} = \sum_{x = 0}^{m} Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk} < 1 .

(9)

Because $H_{n} < 1,$ $Ω^{G} \subset Ω$ does not form a probability space and therefore applications of probability theory to it are not direct. $Ω^{G}$ is made a probability space simply by normalizing the subspace of probabilities within $Ω^{G}$ by H_n to give

Pr {X_{n} = x | Ω^{G}} = (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n},

(10)

ensuring

\sum_{x = 0}^{m} Pr {X_{n} = x | Ω^{G}} = 1 .

The factor H_n does more than ensure that $Ω^{G}$ is a probability space. It is integral to the response structure. Because it involves all thresholds, it causes the probability of a response in any category to be a function of all thresholds. The factor $H_{n}$ creates, and reflects, that the Bernoulli variables within $Ω^{G}$ are not independent as they are within Ω. This distinction of independence and dependence of some of the same variables within different spaces appears in the subsequent development of the article.

The Doubly Conditioned Space $Ω_{x - 1, x}^{G}$

Consider a second-level subspace $Ω_{x - 1, x}^{G} \subset Ω^{G} \subset Ω,$ an example of which, $Ω_{0, 1}^{G},$ is highlighted by dotted lines in Table 3. This space has only two adjacent categories. In Table 3, they are Fail and Pass in the standard design.

Then,

Pr {X_{n} = x | Ω_{x - 1, x}^{G}} = \frac{Pr {X_{n} = x | Ω^{G}}}{Pr {X_{n} = x - 1 | Ω^{G}} + Pr {X_{n} = x | Ω^{G}}} = \frac{(Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n}}{(Π_{k = 1}^{x - 1} P_{nk} Π_{k = x}^{m} Q_{nk}) / H_{n} + (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n}} = \frac{P_{nk}}{Q_{nk} + P_{nk}} = P_{nk},

(11)

where by definition, from Equation (5), $P_{nk} = Pr {Y_{nk} = 1 | Ω} .$

We return to Equations (5) and (11) shortly, but first note that Equation (11) implies a dichotomous response. Let $Z_{nk} = z \in {0, 1}$ be the Bernoulli random variable in which z = 1 indicates a response x, z = 0 indicates the response x− 1, and in which the order continues to be x > x− 1. Thus, z = 1, z = 0 are, respectively, the successful and unsuccessful responses.

Then for $k = x, x > 0,$

P_{nk} = Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}} .

(12)

Just as a threshold on a continuum was specified for each dichotomous response in Ω and characterized the probability of a successful response with the SLM, the same can be specified in the spaces $Ω_{x - 1, x}^{G} .$ Thus, first let $δ_{F / P}$ and $δ_{P / D}$ be thresholds in $Ω_{F / P}^{G}$ and $Ω_{P / D}^{G},$ respectively, where the successful and unsuccessful responses are equally likely, and second, characterize the probability of a successful response with the SLM. From the definition of the SLM in Equation (4),

Pr {Z_{nk} = 1} = \exp (β_{n} - δ_{k}) / γ_{nk},

(13)

where now $δ_{1} = δ_{F / P}, δ_{2} = δ_{P / D} .$

From the construction of the Guttman space, the variables $Z_{nk}, k = 1, 2$ are not independent. This result is consistent with the standard design in which there is only one response and in which, therefore, dichotomous responses at thresholds cannot be independent. However, neither are these dichotomous responses observed. Therefore, the Bernoulli variables $Z_{nk}, k = 1, 2$ in $Ω^{G}$ are implied, inferred, and latent in the standard design. In summary, Equation (12) is the probability of a Bernoulli random variable of a latent, dichotomous response between two adjacent categories in the standard design.

Returning to Equation (11), rather elegantly it is identical to Equation (5). In addition, because Equation (12) is an elaboration of Equation (11), it too is identical to Equation (5). That is,

P_{nk} \equiv Pr {Y_{nk} = 1 | Ω} = Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}} .

(14)

The result in Equation (14) seems remarkable: It is central to the interpretation of reversed thresholds. It shows that for $x = 1, 2, \dots, m,$ the probability value $P_{nk}$ in $Ω_{x - 1, x}^{G} \subset Ω^{G} \subset Ω,$ identified as the probability of a latent, successful, dichotomous response between the two adjacent categories in the standard design of Figure 1, is exactly the same value $P_{nk}$ defined originally as the probability of a successful, observed, dichotomous response at the corresponding threshold in the hypothetical resolved design of Figure 1.

If the probabilities in Equation (14) are both characterized by the SLM, then Equation (4) is identical to Equation (13), and the threshold values from the two sets of variables must also satisfy the following identities:

δ_{1} = δ_{\tilde{P} / P} = δ_{F / P}; δ_{2} = δ_{\tilde{D} / D} = δ_{P / D} .

(15)

It is stressed that the identity in Equation (15) is not merely an interpretation that depends on specific judges making decisions. Instead, it follows from the mathematical structure of the response spaces, including the definition of the Guttman space based on threshold order.

Construction of the PRM Through the Guttman Subspace $Ω^{G}$

Introducing explicitly the SLM of Equation (4) into the probability of a response in $Ω^{G}$ defined by Equation (10), and recognizing that the thresholds δ_k have two interpretations formalized in Equation (150), gives

Pr {X_{n} = x | Ω^{G}} = Π_{k = 1}^{x} \frac{e^{β_{n} - δ_{k}}}{1 + e^{β_{n} - δ_{k}}} Π_{k = x + 1}^{m} \frac{1}{1 + e^{β_{n} - δ_{k}}} / H_{n} .

(16)

Equation (16) simplifies to

Pr {X_{n} = x | Ω^{G}} = \exp (x β_{n} - \sum_{k = 0}^{x} δ_{k}) / γ_{n},

(17)

where (a) $δ_{0} \equiv 0$ is used for notational convenience and (b) after the cancellation of H_n

γ_{n} = \sum_{x = 0}^{m} \exp (x β_{n} - \sum_{k = 0}^{x} δ_{k})

(18)

is the normalizing factor. Equation (17) is a general form of the PRM for the response of a single person to a single polytomous item with m thresholds and m+ 1 categories.

Again elegantly, the coefficient x of β_n is exactly the score $x, x = 0, 1, 2, \dots, m,$ which reflects the category. In particular, it is the count of the number of thresholds exceeded within $Ω^{G},$ complementary to the number of thresholds simultaneously not exceeded. This integer scoring was one of the factors established in Andrich (1978). It results directly from identifying thresholds, with equal discriminations, in the general form of the Rasch model presented in Andersen (1977) and retains the characteristic property of sufficiency in Rasch models.

Furthermore, because γ_n includes explicitly all thresholds, the probability of a response in any category x is a function of all thresholds, and not just of the pair $δ_{x - 1}, δ_{x}$ that bound category x. Thus, if only the last threshold is changed, the probability of a response in the first category changes (Andrich, 1996). Reciprocally, the estimate of any threshold depends on a complex relationship among frequencies of all categories.

The exponent in the numerator in Equation (17) of the PRM, irrespective of how it is motivated, is deceptive in that it appears to involve only a subset of the parameters governed by the total score. If all the parameters are written in full, then the Guttman structure is immediately present. Thus, let $\underline{δ} \equiv {δ_{1}, δ_{2}, \dots, δ_{m}}$ be the vector of thresholds and recall that

\underline{ω_{x}} \equiv {y_{n 1}, y_{n 2}, \dots y_{nk}, \dots y_{nm}} \equiv \underset{x}{\underset{︸}{(1, 1, 1, \dots 1,}} \underset{m - x}{\underset{︸}{0, 0, 0 \dots, 0)}}

is a Guttman vector for the score x compatible with a hypothesized threshold order. Then in vector notation, Equation (17) becomes

Pr {X_{n} = x | Ω^{G}} = \exp (x β_{n} - \underline{{ω'}_{x}} \underline{δ}) / γ_{n} .

(19)

Though less efficient, but making the structure clear, the PRM may be written as

Pr {X_{n} = x | Ω^{G}} = \exp (x β_{n} - 1 δ_{1} - 1 δ_{2} -, \dots, - 1 δ_{x} - 0 δ_{x + 1} - 0 δ_{x + 2} -, \dots, - 0 δ_{m}) / γ_{n}

(20)

showing that the score x implies succeeding on the first x thresholds and simultaneously not succeeding on thresholds x+ 1 and above. This structure is a property of the model.

Alternatively, using matrix notation,

Pr {X_{n} = x | Ω^{G}} = \exp ([Ω^{G}] [\underline{β} - \underline{δ}]) / γ_{n},

(21)

where

\underline{β} \equiv \underset{m}{\underset{︸}{(β_{n}, β_{n}, β_{n}, \dots, β_{n})}} .

In the case of two thresholds, the exponent in Equation (21) is

[Ω^{G}] [\underline{β} - \underline{δ}] = [\begin{matrix} 0 0 \\ 1 0 \\ 1 1 \end{matrix}] [\begin{matrix} β_{n} - δ_{1} \\ β_{n} - δ_{2} \end{matrix}],

(22)

in which the Guttman structure, relative to a hypothesized threshold order, is unmistakable. In addition, the parameters β_n, δ₁, δ₂ are on the same continuum, being the same parameters as those in the resolved design.

Three consequences of the Guttman structure have been established. First, although the responses are in categories, and there is reference to categories, the only parameters in the model are the thresholds. Second, if a threshold is exceeded, then all other thresholds before it in the order are also exceeded, and following the first one failed, all subsequent ones are also failed. Thus, the Guttman structure, as a hypothesis of the threshold order, is integral to the model. However, there is no constraint that forces the threshold values to be ordered in the data. Third, using the PRM, the result in Equation (14) is confirmed. Thus, the conditional probability of a response in one of two adjacent categories in the PRM of Equation (17) simplifies to

Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}} = \frac{Pr {X_{n} = x | Ω^{G}}}{Pr {X_{n} = x - 1 | Ω^{G}} + Pr {X_{n} = x | Ω^{G}}} = \frac{\exp (x β_{n} - \sum_{k = 0}^{x} δ_{k}) / γ_{n}}{\exp ((x - 1) β_{n} - \sum_{k = 0}^{x - 1} δ_{k}) / γ_{n} + \exp (x β_{n} - \sum_{k = 0}^{x} δ_{k}) / γ_{n}} = \exp (β_{n} - δ_{k}) / γ_{nk},

(23)

which is exactly the SLM of Equation (4) of the experimentally independent space.

AWW imply an algebraic understanding Equation (23) as a consequence of Equation (11), but without realizing its implications. In particular, they seem to think that the result in Equation (23), which is linked to the standard design, is no longer relevant in the original resolved design. The identity of Equation (23) and Equation (4) in the two designs and corresponding response spaces is central to the inferences regarding disordered thresholds, which continues to be elaborated in the article.

Other Parameterizations of the PRM

First, however, and for completeness, we note other parameterizations of the PRM also considered by AWW. Equation (17) for a polytomous response in a standard design characterizes the response to a single item. If, as is usual, there is more than one item $j, j = 1, 2, \dots, J$ in the responses of the same person, and the thresholds of each item can take different values, then the PRM can be written as

Pr {X_{nj} = x | Ω^{G}} = \exp (x β_{n} - \sum_{k = 0}^{x} δ_{kj}) / γ_{nj}, x = 0, 1, 2, \dots, m_{j},

(24)

where m_j is the maximum score of item j. This form, where the thresholds can take different values for different items, is the so-called partial credit parameterization of the PRM and is the one used in the simulations shown later in the article.

The following reparameterization, with no change to the model, is used in AWW. Let $τ_{kj} = δ_{kj} - δ_{j},$ where $δ_{j} = (\sum_{k = 1}^{m} δ_{kj}) / m_{j}$ is the mean of the thresholds for an item and can be interpreted as the summary location of item j. This gives

Pr {X_{nj} = x | Ω^{G}} = \exp (x (β_{n} - δ_{j}) - \sum_{k = 0}^{x} τ_{kj}) / γ_{nj} .

(25)

The $τ_{kj}, k = 1, 2, \dots, m_{j}$ are the same thresholds as δ_kj, but with values such that within each item, $\sum_{k = 1}^{m_{j}} τ_{kj} = 0 .$

A further specialization in which all items have the same values of the thresholds $τ_{k} = τ_{kj},$ but with possibly different average locations of the items characterized by δ_j, gives

Pr {X_{nj} = x | Ω^{G}} = \exp (x (β_{n} - δ_{j}) - \sum_{k = 0}^{x} τ_{k}) / γ_{nj} .

(26)

Equation (26) is known as the rating scale parameterization and may be relevant if all items have the same response format as in attitude surveys. This equation was the one first specified in Andrich (1978) in the resolution of the scoring functions and category coefficients in Andersen (1977). For the purpose of this article, these parameterizations are details for noting, as they are in AWW. For convenience, and without loss of generality, Equations (17) and (24) are used when considering a single item and multiple items, respectively.

The Latent Dichotomous Responses at the Thresholds

Figure 4 shows the category characteristic curves (CCCs), the probabilities of the responses of each category, for two ordered thresholds as in Figure 2. Thus, persons of low, medium, and high values on the continuum are, respectively, most likely to obtain scores of 0, 1, and 2. Figure 4 also shows in dotted lines the latent TCCs in the standard design from Equation (23). It is stressed that these dotted TCCs in Figure 4 are theoretically identical to the TCCs in Figure 2 characterizing the responses in the resolved design. To reinforce this point, Figure 5 juxtaposes Figures 2 and 4 where, for further clarity, the CCCs in the latter have been suppressed.

Simulation Study 1: Two Models of Analysis in Two Response Spaces

The key result above is that the values $δ_{\tilde{P} / P}, δ_{\tilde{D} / D},$ in the SLM of the resolved design in the independent space Ω, and the values $δ_{F / P}, δ_{P / D}$ , in the PRM of the standard design in the Guttman subspace $Ω^{G},$ are identical parameters. To illustrate this identity, Simulation 1 is a data set of 20,000 simulated performances and is analyzed in the two response spaces with the corresponding model. The person distribution was normal, $β ~ N (0, 1),$ and the responses were simulated using the SLM according to the resolved design of Figure 1. There were two thresholds in the resolved design corresponding to three categories in the standard design in Figure 1.

Two sets of two independent assessments were simulated, taken as responses from two Expert and two Novice judges, for each person. Clearly, given that the data are simulated, there are no real judges involved; judges are used simply to help explicate the relationships between spaces and implications of threshold order. The two sets of judges are characterized by subscript $j, j = 1, 2 .$ For the two Experts, the successive thresholds have increasing values, $δ_{\tilde{P} / P} < δ_{\tilde{D} / D};$ for the two Novices, however, the values are $δ_{\tilde{P} / P} > δ_{\tilde{D} / D},$ opposite to that required. Specifically, these Novices are simulated to have confused their relative standards.

To emphasize the structure of the data analyzed, Table 4 shows the first 10 persons of the data matrix in which the left part of the table shows the set of dichotomous responses y_nkj of each of the four judges for each person. They are responses from the resolved design. The right part shows the total scores

Table 4.

Excerpt of Responses According to the Bernoulli Responses and the Subset of Guttman Vectors

	Expert, J = 1; k = 1, 2		Novice, J = 2, k = 3, 4		Guttman Vectors
	1	2	3	4	J = 1; k = 1, 2	J = 2, k = 3, 4
Person	y _n1	y _n2	y _n3	y _n4	$x_{n 1} = \sum_{k = 1}^{2} y_{nk 1}$	$x_{n 2} = \sum_{k = 1}^{2} y_{nk 2}$
1	1	1	1	1	2	2
2	1	0	1	0	1	1
3	1	1	1	1	2	2
4	1	1	0	1	2	Missing
5	1	0	1	1	1	2
6	0	0	0	1	0	Missing
7	1	1	0	1	2	Missing
8	1	1	1	0	2	1
9	1	1	0	1	2	Missing
10	0	1	1	0	Missing	1

	All responses in Ω analyzed by the dichotomous SLM (resolved design)				Subset of Guttman vectors of Ω^G analyzed by the PRM (standard design)

X_{nj} = \sum_{k = 1}^{2} Y_{nkj}, j = 1, 2

for the Experts and for the Novices for which the responses conform to the Guttman structure based on the hypothesized order of the thresholds.

In the demonstrations that follow, the Guttman subspace was defined, and therefore the scores constructed, according to the intended ordering of the thresholds. As stressed above, to choose a Guttman subspace, there is no alternative but to hypothesize a threshold order. Thus, for experts, it was based on a compatible ordering; for the Novices on an incompatible ordering. The non-Guttman vectors (which could not be identified with the standard design) were treated as missing data. The number of Guttman vectors in each set is shown in Table 5. As a result of the incompatible ordering, there are many less Guttman vectors for the Novice judges than for the experts, a feature returned to later in the article.

Table 5.

Estimates of Parameters Using the SLM and PRM in Data Simulated According to the SLM

Judges	Generating Threshold Locations	Estimates From the SLM in Ω	Estimates From the PRM in Ω^G
j = 1, Expert 1	δ₁₁ = −1.0	( $\tilde{P} / P$ ) −1.014	(F/P) −1.025
j = 1, Expert 2	δ₁₂ = 1.0	( $\tilde{D} / D$ ) 1.013	(P/D) 1.027

			18,789 Guttman vectors in the data

j = 2, Novice 1	δ₂₁ = 1.0	( $\tilde{P} / P$ ) 0.968	(F/P) 0.969
j = 2, Novice 2	δ₂₂ = −1.0	( $\tilde{D} / D$ ) −0.967	(P/D) −0.971

			11,042 Guttman vectors in the data

Extremes		3,717	3,717
Number used		16,283	6,667

Note. N = 20,000; β~N(0, 1).

Three questions arise from Simulation 1 to which answers will be given. First, will the PRM give correct threshold estimates for the Experts? Second, despite the Guttman vector being incompatible with the a priori threshold order, will the PRM give correct threshold estimates for the Novices? Third, what are some symptoms of reversed thresholds in the PRM? The answers to these three questions provide the understanding of the role of reversed thresholds in the PRM. As will be demonstrated, the answer to the last question is more complex than AWW’s conclusion, noted earlier, that reversed thresholds are simply a function of low frequencies of responses in some categories.

The responses from the two spaces, Ω and Ω^G, were analyzed using the SLM of Equation (4) and the PRM of Equation (24), respectively.

We note that in the PRM, as in the SLM, within any set of responses $j = 1, 2, \dots, J,$ $r_{n} = \sum_{j = 1}^{J} x_{nj}$ is the sufficient statistic for β_n (Andersen, 1977), and that by conditioning on r_n, estimates of the threshold parameters $δ_{kj}, k = 1, 2, \dots m; j = 1, 2, \dots J$ can be obtained independently of $β_{n}, n = 1, 2, \dots, N .$ The estimates in Table 5 were obtained using the program RUMM2030 (Andrich, Sheridan, & Luo, 2012), which employs a pairwise, conditional algorithm described in Andrich and Luo (2003). The estimates are consistent (Zwinderman, 1995). In this algorithm, persons with extreme total scores provide no information and are not used in the estimation of the threshold parameters. The number retained in the estimation is also shown in Table 5.

Table 5 shows the generating thresholds and their estimates from the two analyses. This table also shows that the estimates from the PRM were obtained from a smaller number of persons than from the SLM. As expected, it shows that the generating thresholds were recovered excellently using the SLM with a root mean square deviation (RMSD) of

RMSD = \sqrt{\sum_{k = 1}^{4} {(δ_{ik} - {\hat{δ}}_{ik})}^{2}} / 4 = 0.025 .

More distinctively, and although estimated from a smaller number of responses, the RMSD = 0.028 for estimates using the PRM indicates virtually the same accuracy in recovering the thresholds.

Thus, Simulation 1 illustrates that a PRM analysis of responses in Ω^G estimates identical threshold values as an SLM analysis of responses in the experimentally independent space Ω, including the values where the thresholds are reversed relative to the a priori order specification.

Despite the theoretical derivation above that proves that the PRM in the subset of responses in Ω^G has the same thresholds as the SLM for all responses in the space Ω, it seems remarkable that, despite the incompatible scoring relative to the intended order, the thresholds for the Novices are estimated correctly. This identity in theory and in estimation leads to an understanding of the implications of disordered threshold estimates.

Establishing the Guttman Response Space From the Structure of the PRM

For the inference of this article, the significance of Equation (14), which shows the identity of dichotomous response variables in the experimentally independent space Ω and the corresponding latent dichotomous response variables in the doubly conditioned space $Ω_{x - 1, x}^{G},$ cannot be overstated. To establish its role unequivocally, we proceed now in the reverse direction from that presented above in which, from the required threshold order, the Guttman structure was invoked in the resolved design and then bridged to the standard design. This reverse derivation establishes that the Guttman structure with a hypothesized threshold order is a necessary and sufficient condition for the PRM.

We begin now with the standard design in Row 4 of Figure 1, and bridge it to the resolved design in Rows 1 and 2 of Figure 1. The previous analysis, which proceeded from the resolved design, gives the cue how to proceed. Although for efficiency the same notation as above is used, the derivations are distinct from those in the above sections.

Let $X_{n} = x, x \in {0, 1, 2, \dots, m}$ be an integer-valued random variable for a response in one of m+ 1 successive categories in the standard design. The integer values do not imply any weighting for the categories. Although not a weight, the integer variable does reflect the intended, substantive ordering of the categories. Furthermore, let $Pr {X_{n} = x | Ω'}$ be the probability of the response x, $\sum_{x = 0}^{m} Pr {X_{n} = x | Ω'} = 1,$ in the space $Ω' .$ Because it needs to be inferred from the following definition, $Ω'$ is left general.

Let the sequence of latent, dichotomous, Bernoulli variables $Z_{nk} = z \in {0, 1}, k = 1, 2, \dots, m; k = x > 0$ be defined by

Pr {Z_{nk} = 1 | {Ω'}_{x - 1, x}} = \frac{Pr {X_{n} = x | Ω'}}{Pr {X_{n} = x - 1 | Ω'} + Pr {X_{n} = x | Ω'}} = \frac{\exp (β_{n} - δ_{k})}{1 + \exp (β_{n} - δ_{k})},

(27)

where $Pr {Z_{nk} = 1 | {Ω'}_{x - 1, x}} > 0; Pr {Z_{nk} = 0 | {Ω'}_{x - 1, x}} = 1 - Pr {Z_{nk} = 1 | {Ω'}_{x - 1, x}} > 0,$ and where $Z_{nk} = z \in {0, 1} .$ Clearly Equation (27) implies a dichotomous response relative to adjacent categories x− 1 and x. Equation (27) is the starting point in the derivation of the PRM by Wright and Masters (1982) except that they, as do AWW, ignore a study of the response spaces, resulting in incompleteness of the derivations and a restricted understanding of its implications.

Because of the intended category ordering, x > x− 1, Z_nk = 1, and Z_nk = 0 are, respectively, the latent successful and unsuccessful dichotomous responses between categories x− 1 and x. For example, conditional on the latent dichotomous response of Pass or Distinction, Distinction is the successful one. Therefore, Equation (27) is a starting point that is consistent with the intended ordering of the categories. This starting point is referenced in AWW.

To infer a response space for X_n = x, Equation (27) must be expressed as $Pr {X_{n} = x | Ω'} .$ $Ω'$ needs to be inferred, because, unlike the responses of the resolved design which immediately gives the independent space Ω, the variables Z_nk are not independent, rendering their response space not immediately obvious. Rendering this space explicit is one of the key understandings of the PRM.

From Equation (27), let

P_{nk} = Pr {Z_{nk} = 1 | {Ω'}_{x - 1, x}}; Q_{nk} = 1 - P_{nk} = Pr {Z_{nk} = 0 | {Ω'}_{x - 1, x}} .

(28)

Theorem

Given the definitions of $P_{nk}, Q_{nk}$ in Equations (27) and (28),

Pr {X_{n} = x | Ω^{'}} = (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n},

(29)

where

H_{n} = \sum_{x = 1}^{m} Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk} > 0,

(30)

and where, for convenience, $Π_{k = 1}^{0} P_{nk} \equiv 1 .$ Although somewhat tedious, the above theorem can be proved readily (Andrich, 2010; Luo, 2005).

Equation (29) implies a very clear structure of successes and failures at successive thresholds x for each response $X_{ni} = x .$ Specifically, it implies successes at the first x variables and lack of success at the remaining m−x variables. That is, the sequence of the random variables, $Z_{nk}, k = 1, 2, \dots, m,$ is compatible with

{Z_{n 1}, Z_{n 2}, Z_{n 3}, \dots Z_{nk}, Z_{n (k + 1)}, \dots, Z_{nm}} \equiv {\underset{x}{\underset{︸}{1, 1, 1, \dots 1,}} \underset{m - x}{\underset{︸}{0, 0, 0, \dots, 0}}}

in which a sequence of x 1s is followed by a sequence of (m−x) 0s; moreover, no other vector of responses is compatible with $Z_{nk}, k = 1, 2, \dots m .$

The vector $Z_{nk}, k = 1, 2, \dots m$ is always a Guttman vector beginning with the first threshold in the intended category order. This condition implies that the response space $Ω' \equiv Ω^{G},$ and that ${Ω'}_{x - 1, x} \equiv Ω_{x - 1, x}^{G} .$ However, although the space ${Ω'}_{x - 1, x} \equiv Ω_{x - 1, x}^{G}$ is latent, because the response ${X_{n} = x | Ω^{G}}$ is observed, Ω^G itself is manifest. It is a manifest, observed space whose latent structure is defined in Equation (27).

Thus, P_nk defined in Equation (27) with an unknown space $Ω'$ can be written as

P_{nk} = Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}} = \frac{Pr {X_{n} = x | Ω^{G}}}{Pr {X_{n} = x - 1 | Ω^{G}} + Pr {X_{n} = x | Ω^{G}}}

(31)

and

Pr {X_{n} = x | Ω^{G}} = (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n} .

(32)

Equation (32) is identical to Equation (11). It has a Guttman structure relative to the intended category order and is the structure that was introduced to bridge the resolved design to the standard design. Reintroducing the SLM from Equation (27) into Equation (32) gives the PRM of Equation (17). Therefore, the Theorem, together with the specification of the SLM for the latent dichotomous response at thresholds, establishes that the Guttman structure, relative to the putative ordering of the categories, is an integral part of the PRM.

Inferring an Experimentally Independent Response Space $Ω$ From $Ω^{G}$

Although the ordering of the categories implies a Guttman sequences of successes and failures at the thresholds, at this point there is no structural implication for the ordering of the values of the thresholds. To infer the required ordering of the threshold values, it is necessary to bridge the standard design to the resolved design using the Guttman subspace Ω^G. Specifically, given the space Ω^G and its probability structure, we infer the existence of a hypothetical, complete spaceΩof whichΩ^Gis a subspace.

From Equation (32) we have

\sum_{x = 0}^{m} Pr {X_{n} = x | Ω^{G}} = \sum_{Ω^{G}} (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) / H_{n} = 1,

(33)

from which H_n > 0. Therefore,

\sum_{Ω^{G}}^{m} (Π_{k = 1}^{x} P_{nk} Π_{k = x + 1}^{m} Q_{nk}) = H_{n} \neq 1 .

(34)

Now consider the space $Ω ¹ Ω^{G}$ comprising all 2^m patterns of responses, ${(Z_{n 1}, Z_{n 2}, Z_{n 3}, \dots Z_{nx}, Z_{nx + 1}, Z_{nx + 2}, \dots Z_{nm}) | Ω}$ in which $Z_{nx} = z \in {0, 1}$ includes patterns of responses that are not Guttman vectors, but among which the set of m+ 1 Guttman vectors

{\underset{x}{\underset{︸}{1, 1, 1, \dots 1,}} \underset{m - x}{\underset{︸}{0, 0, 0 \dots, 0}}}

implied in Equation (33) is the subspace $Ω^{G} .$ Then by the definition of Z_nx and Equation (34), the sum of the probabilities of the set of all possible 2^m response patterns is

\sum_{Ω} Pr {(z_{n 1}, z_{n 2}, \dots z_{nk}, \dots z_{n m}) | Ω} = \sum_{Ω} Π_{x = 1}^{m} P_{nk}^{z_{nk}} Q_{nk}^{1 - z_{nk}} = 1 .

(35)

Equation (35) confirms that $Ω ¹ Ω^{G}$ is a probability space. However, in the space Ω, Equation (35) also implies experimental independence of Bernoulli variables. It is exactly the resolved design of the top two rows of Figure 1, whose space is shown in Table 2 for the case of two thresholds. This inference to a hypothetical, experimentally independent response space, which is compatible with the data in the standard ordered design, renders the resolved design in which judges were making decisions and was used to motivate the development completely unnecessary as a concrete design with actual judges. That the design does not have to be concrete, however, does not diminish the power of the inference.

In summary, the inference is as follows: Given there is a Guttman response space Ω^G composed of m latent, but dependent, Bernoulli variables with the probability structure of Equation (32), $Ω^{G}$ can be taken as a subspace of a unique, hypothetical, response space Ω of m experimentally independent Bernoulli response variables compatible with those in Ω^G.

For completeness, let these dichotomous responses in Ω be defined by Bernoulli random variables $Y_{nk} = y \in Ω \equiv {0, 1},$ $k \in {1, 2, \dots m}$ and the response y = 1 taken as the successful response. Then,

P_{nk} \equiv Pr {Y_{nk} = 1 | Ω} = Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}},

(36)

where $Pr {Z_{nk} = 1 | Ω_{x - 1, x}^{G}}$ is defined by Equation (27) with $Ω' = Ω^{G} .$

Equation (36) is the bridge from the standard design to the resolved design and is identical to Equation (14), which was the bridge from the resolved design to the standard design.

It is stressed that if an experiment were conducted in which a set of judges worked independently according to the resolved design of Table 2, and the same or other judges worked within the standard design of Table 1 for the same essays and the same definition of the categories, then there is nothing that guarantees that data collected in two designs would give identical empirical threshold estimates. However, that is not the point of the above inference. The point is to infer a hypothetical, experimentally independent response space Ω with $Ω ¹ Ω^{G} ¹ Ω_{x - 1, x}^{G},$ which is compatible with the data at hand collected according to the standard design. The responses may be by self-report or by judges assessing proficiency, or any other response procedure.

Thus, as anticipated early in the article, the inference is as if the data at hand, collected in the standard design with its probabilities of success at the thresholds, were collected in a resolved design with the same probabilities of success at the same thresholds. This is a mathematical inference from a mathematical structure.

The Required Success Rate Order in the Standard Design

As noted above, Equation (32), which showed that the inferred response space was Ω^G relative to the ordering of the categories, specified nothing about the values of the inferred thresholds. And this lack of specification seems to confuse AWW. However, in the inferred space Ω, the required and intended order of the threshold estimates is readily specified, namely, ${\hat{δ}}_{\tilde{P} / P} < {\hat{δ}}_{\tilde{D} / D} .$ And because from Equation (36) the respective probabilities of successes at the thresholds ${\hat{δ}}_{F / P} < {\hat{δ}}_{P / D}$ are identical, it is straightforward to require that ${\hat{δ}}_{F / P} < {\hat{δ}}_{P / D} .$ If this ordering is violated, then something has gone awry in the empirical ordering of the categories in the responses in the standard design.

Deductions from mathematical models are tautological. However, the deductions can provide genuinely new insights. The derivation that, for categories to demonstrate empirical order, it is necessary that ${\hat{δ}}_{F / P} < {\hat{δ}}_{P / D}$ is one of them. The derivation rests on the identities of Equations (14) and (15) and the a priori requirement in Equation (3) that $\hat{P} r {Y_{\tilde{P} / P} = 1} > \hat{P} r {Y_{\tilde{D} / D} = 1} .$ The new insight is that for categories to be ordered empirically, it is necessary that the probabilities of success of the conditional probabilities of successive categories in the standard design satisfy $\hat{P} r {Y_{F / P} = 1 | Ω^{G}} > \hat{P} r {Y_{P / D} = 1 | Ω^{G}} .$ Specifically, in the above example, it implies that obtaining a Distinction relative to a Pass must demand more proficiency than obtaining a Pass relative to a Fail. Although a new insight, it is intuitively appealing. It is also an insight that AWW seem to miss.

With the above development, misunderstandings by AWW in the following statements can be exposed. For example, they state,

While this statement seems eminently reasonable it raises two questions. Does this statement, and the derivation that corresponds to it, impose an order requirement on the values of the tau parameters? Does the statement clarify what the latent processes are? What Andrich repeatedly states but neither proves nor logically defends is that an ordering of the tau parameters is related to category ordering or is imposed through the Guttman structure. (AWW, 2012, p. 9)

It can now be highlighted from the above development that there is in fact a disjunction between the values and the Guttman structure of the thresholds. Because they assert that I have not shown that the order of the threshold values is a property of the model, it seems that AWW cannot, or will not, make a distinction between the structure of the thresholds in the PRM, which requires a hypothesized order, and values of the thresholds, which may or may not violate the hypothesized order. Indeed, there is nothing in the structure, nor in the estimation process, that forces the values of thresholds estimates to be ordered. If there were such a feature, then the ordering would be a property of the model (as in the graded response model), and not a property of the data, and therefore could not disclose problems with the category ordering. The consistent argument made previously, and repeated here, is for evidence that the thresholds are ordered in the data, as disclosed by the PRM. The PRM can make such a disclosure because the Guttman structure inherent in it characterizes the intended ordering of the thresholds, which in turn reflects the intended ordering of the categories. Then, the freedom of the threshold values in the PRM to reflect the property of the data in Ω^G is identical to freedom of the threshold values in the SLM to reflect the property of the data in the full space Ω. This freedom for the values to reflect a property of the data, including a violation of the hypothesized order of the thresholds, was illustrated in Simulation 1.

Category Characteristic Curves and Disordered Thresholds

Figure 4 showed the CCCs and the latent TCCs of the PRM for three categories with ordered thresholds conceived to be from Expert raters in Simulation 1. Figure 6 shows the CCCs for the PRM for the Novice raters with reversed thresholds, together with their latent TCCs. AWW argue that such a figure provides no evidence of anomaly in the operation of the category order. Their argument is examined here.

Consider a person with a proficiency located exactly at ${\hat{δ}}_{P / D} = - 0.971$ in Figure 6. Inferences must be consistent in all spaces and with the threshold order. First, from the doubly conditioned subspace $Ω_{P, D}^{G},$ the person has an equal probability of a Pass and Distinction, that is, $Pr {Z_{P / D} = 1 | Ω_{P, D}^{G}} = 0.5,$ with the implication the person has relatively high proficiency. Second, from Figure 6, it is evident that in the subspace Ω^G, the person has an even greater probability of the response Fail. Specifically, for $β_{n} = {\hat{δ}}_{P / D} = - 0.971,$ $Pr {X_{n} = P | Ω^{G}} = Pr {X_{n} = D | Ω^{G}},$ but $Pr {X_{n} = F | Ω^{G}} = 0.78 .$ However, having Fail as the highest probability implies low proficiency. This is a contradiction.

AWW see no inconsistency in this case and write,

At the point of equal probability for pass and distinction we are not tossing up if a student should be a pass or distinction: In fact we are more confident that they are a fail (see Figure 5). (AWW, 2012, p. 12)

The inference in the above quote goes beyond anything that can be inferred from the PRM. The model is a static model and has nothing to do with “tossing up” to make a decision, or deciding that in any case “we are more confident that the person is a Fail.”The model only describes the inferred probability distribution of responses of a person at a particular location to the item implied by the data. Although there are no replications of any person to any one item, the PRM estimates the probability distribution of responses to the item as if the person were to respond to the item, independently many times. To expand on this point, Figure 6 shows three vertical lines with dots on the curves, which characterize the distribution of responses for three locations, including one that is at the threshold of Pass and Distinction. At each location on the continuum, there is such a triplet of points characterizing the probability distribution, which sums to 1. And in this distribution, a person who has a 50% chance of a Distinction relative to Pass (and therefore implies substantial proficiency) simultaneously has a 78% chance of a Fail (which therefore implies substantial lack of proficiency). At the other end, a person who has 50% chance of a Fail relative to the Pass has 78% chance of a Distinction. These implications are contradictory. To make the point concrete, it is as if, based on all the evidence, an instrument for measuring height implies that the person on the margin of 180 cm and 181 cm in height (180.5 cm) is simultaneously most likely to be 179 cm in height! It would be understood that the measuring instrument has produced a contradiction and would be examined.

In contrast, there is no such contradiction in Figure 4 with ordered thresholds, where the respective probabilities for $β_{n} = {\hat{δ}}_{P / D} = 1.027$ are $Pr {X_{n} = P | Ω^{G}} = Pr {X_{n} = D | Ω^{G}} = 0.470$ and $Pr {X_{n} = 0 | Ω^{G}} = 0.060$ has the lowest probability. Because the thresholds are ordered, a person with equal probability of a Pass or a Distinction has a lower probability of simultaneously showing a Fail.

Third, in the space Ω in which the inferred responses are independent at the thresholds, the person with location $β_{n} = {\hat{δ}}_{P / D} = - 0.971$ has the probability $Pr {Y_{\tilde{D} / D} = 1 | Ω} = 0.5$ to be given a Distinction by the second Novice judge, but only $Pr {Y_{\tilde{P} / P} = 1 | Ω} = 0.126$ to be given a Pass by the first Novice judge. This contradicts the a priori requirement of ordering in the resolved design, in which success rate at $\tilde{P} / P$ is required to be greater than at $\tilde{D} / D .$ It is stressed that from the mathematical structure of the PRM, thresholds in the standard design and space Ω^G are identical to the thresholds in the inferred, resolved design in the compatible space Ω. And of course, we know that in the simulation, the threshold order was incompatible with the threshold structure for the Novice judges; accordingly, we know that the contradiction disclosed by reversed thresholds in the PRM analysis is correct in negating the putative ordering of the categories.

One might wonder, to paraphrase AWW in the quote above, how a PhD student would interpret similar remarks to theirs—namely, that although as three examiners they had a consensus that the dissertation had the quality to be at the margin of a Distinction relative to a Pass, because of their understanding of conditional probabilities, they were already “more confident” that the dissertation’s quality was a Fail, and so declared it a Fail!

Frequencies of Responses in Categories and Reversed Threshold Estimates

The article now turns to the second implication in AWW, which concerns the role of frequencies and reversed threshold estimates. As noted earlier, AWW indicate that

we examine parameter estimation and show that for any given set of abilities the relative frequency of the number of responses in each category of an item is the only determinant of whether the estimated parameters are ordered in value or not. (AWW, 2012, p. 2)

They subsequently provide an equation, using a joint maximum likelihood method of estimation, from which they write further:

For a given set of students, the parameter estimates for δ_i, τ_i1, and τ_i2 depend solely on the number of observations in each category; they are completely independent of the abilities of the students who respond in each category. (AWW, 2012, p. 15)

That, by definition, the estimation of the item parameters in the class of Rasch models (Andersen, 1977; Rasch, 1961) can be made independent of the person parameters is so well known that it hardly needs reinforcing, even though AWW’s joint maximum likelihood equations, which have person parameters on the right side, do not make this self-evident. Therefore, in their emphasis of frequencies in the equation, AWW must also be saying something else. What they are saying can be gleaned from their next mention of frequencies in conjunction with their Figure 8, which is similar to Figure 6 of this article with reversed thresholds. They state,

The item characteristic curves in Figure 10 show a low curve for score Category 1, reflecting the low frequency of responses for this score category and resulting in disordered thresholds (3.75 and 0.07). (AWW, 2012, p. 28)

(Their reference to “Item characteristic curves” is what are referred to in this article as CCCs.) The implication, consistent with other comments in their article, is that a very small frequency in a category in the sample as a whole leads to reversed threshold estimates. If this is AWW’s interpretation, then it results from a superficial analysis of the role of frequencies in the PRM. In addition, even if their comments needed no qualification, they are merely descriptive. That is, AWW do not go on to explain why such a low frequency of a score of 1 produces the reversed thresholds. Before returning to explain the reason for reversed thresholds as a violation of the hypothesized Guttman structure and threshold order, simulations that demonstrate that reversed thresholds are not simply a result of low frequencies are presented.

Examples of Low Frequency and Correctly Ordered Threshold Estimates and High Frequency and Reversed Threshold Estimates

Simulations 2 and 3 show that categories with very low frequencies (as low as 2 but in principle even 0) need not produce reversed threshold estimates, and Simulation 4 shows that categories with large frequencies can. To show that where the frequencies might be low in the order is irrelevant to the conclusion, one example with low frequencies in the extreme category, and one with low frequencies in the middle category, are included.

Simulation 2

Simulation 2 had small frequencies in the extreme categories. It had 10 items with four categories (three ordered thresholds). The distance between all pairs of successive thresholds within an item was 3 logits and the sum of all thresholds was 0.0, giving an origin of 0.0 for both the generating parameters and their estimates. Two thousand persons, $β ~ N (- 1.25, 1 . 75^{2})$ , were simulated, which relative to a threshold mean of 0.0 indicates some misalignment.

All the category frequencies and threshold values and their estimates are shown in Appendix A. Table 6 shows frequencies for Item 10, in which the frequency of the last category is only 2. With conditional estimation, the person parameter distribution played no role. Taking all thresholds together, the correlation between the parameters and their estimates is 0.999. In particular, for Item 10, in which the last category has a frequency of just 2, and in which the threshold location is 6.00, Table 7 shows that its estimate using RUMM2030 is 6.52. The standard error of the estimate for this threshold is 0.802, giving the standardized residual of z = −0.648, a value well within an acceptable range of no significant difference. Thus, despite the very small frequency, there is no reversed threshold estimate.

Table 6.

Frequencies in Response Categories for Simulation 2

Item	Category Scores
	0	1	2	3
10	1339	575	76	2

Table 7.

Generating Thresholds and Their Estimates in Simulation 2

Item	$δ_{1 j}$	${\hat{δ}}_{1 j}$	$δ_{2 j}$	${\hat{δ}}_{2 j}$	$δ_{3 j}$	${\hat{δ}}_{3 j}$
10	0.00	−0.09	3.00	2.79	6.00	6.52

Consistent with the simulation parameters, Figure 7 shows the person distribution is not well aligned with the distribution of the thresholds. This misalignment results in few responses in the extreme categories of the difficult Item 10. However, because there is no structural problem in the responses, it does not result in reversed thresholds.

Simulation 3

Simulation 3 had small frequencies in a middle category. It had 8 items each with 9 categories (8 thresholds) with a threshold mean of 0. Such large numbers of categories are present in some criteria in national writing assessments in Australia. The 1,000 persons were generated from two subpopulations of 500 persons, $β_{1} ~ N (- 3.5, 0 . 75^{2})$ and $β_{2} ~ N (3.5, 0 . 75^{2}),$ respectively, and then analyzed as one population. The distance between all thresholds within an item was 1.43. Appendix A shows the frequencies and threshold estimates for all items. Table 8 shows the frequencies for Item 5 and Table 9 shows its thresholds and their estimates.

Table 8.

Frequencies in Response Categories for Item 5 for Simulation 3

	Category Scores
Item	0	1	2	3	4	5	6	7	8
5	69	211	156	59	11	75	176	177	66

Table 9.

Thresholds Estimates for Item 5 of Simulation 3

Item	${\hat{δ}}_{1 j}$	${\hat{δ}}_{2 j}$	${\hat{δ}}_{3 j}$	${\hat{δ}}_{4 j}$	${\hat{δ}}_{5 j}$	${\hat{δ}}_{6 j}$	${\hat{δ}}_{7 j}$	${\hat{δ}}_{8 j}$
5	−4.83	−3.10	−1.64	−0.37	0.81	2.01	3.33	4.86

In Table 8, the category with score 4 has a low frequency with only 11 (1.1%) responses. Nevertheless, Table 9 shows that its thresholds are ordered correctly. Figure 8 shows the estimated person distribution is estimated to be clearly bimodal, as generated. Again, because of the lack of persons in the middle of the continuum, there are relatively few responses in the middle category of Item 5. However, also again, because this alignment is the cause of a low frequency, and there are no structural problems in the responses, the threshold estimates are not reversed.

In summary, Simulations 2 and 3 show that both relatively, and absolutely, low frequencies do not necessarily lead to disordered thresholds.

Simulation 4

Simulation 4 shows that large frequencies do not preclude disordered thresholds. It is based on a real example that concerned ratings of muscle tone in five putatively ordered categories according to a clinician-rated scale and is detailed in Andrich (2011). Those details are not reproduced here. However, all items showed that thresholds three and four were reversed, and on the average, the reversal was by 0.34 logits.

To ensure that the data for this article had no misfit features that might be present in real data, and that disordered thresholds are not explained from any particular judging process, they were simulated according to the PRM. The generating parameters were similar to the estimates from the real data. Simulation 4 has 1,000 persons distributed as $N (0.14, 1 . 39^{2})$ with the same number of items and the same number of thresholds as in the real example. The thresholds covered the range of the real data thresholds, and the differences between the thresholds were the same across the items. The last two thresholds are reversed by 0.30 logits. Table 10 shows the frequencies and Table 11 the threshold estimates for all the items. The estimates and the generating values had a correlation of 0.997. Figure 9 shows the distribution of persons and thresholds.

Table 10.

Frequencies in Response Categories for Simulation 4

	Category Scores
Item	0	1	2	3	4
1	45	310	334	129	175
2	39	340	331	132	151
3	44	350	338	120	141
4	56	380	306	112	139
5	74	356	331	106	126
6	74	398	307	121	93
7	52	406	337	101	97
8	80	424	293	102	94

Table 11.

Thresholds and Their Estimates in Simulation 4

Item	${\hat{δ}}_{1 j}$	${\hat{δ}}_{2 j}$	${\hat{δ}}_{3 j}$	${\hat{δ}}_{4 j}$
1	−3.28	−0.39	1.46	0.98
2	−3.48	−0.27	1.54	1.18
3	−3.32	−0.20	1.67	1.23
4	−3.19	0.06	1.73	1.20
5	−2.71	−0.05	1.88	1.25
6	−2.81	0.20	1.79	1.79
7	−3.28	0.11	2.06	1.57
8	−2.73	0.37	1.94	1.68

Table 10 shows that all categories have substantially large frequencies, and in particular, that the frequencies in the last category are greater for each item than those in the first category. In addition, Items 1 to 4 have more responses in the last category than in the second last one, and Items 6 to 8 have it the other way around. This pattern is as in the real example. Nevertheless, Table 11 shows that the threshold estimates are reversed only for Thresholds 3 and 4, and that they are reversed for all items, consistent with the real example. The average size of the reversal in the estimates is 0.4 logits. Again, small frequencies are not the cause of these reversed thresholds. For completeness, Figure 9 shows that close to half of the people are located approximately above 0.5 logits where the threshold estimates are reversed. A structural response anomaly is present in this case.

To summarize the point of the above simulations, estimates of the thresholds are independent of the distribution of persons in the PRM, but the distribution of frequencies themselves is very much dependent on the distribution of the persons relative to the distribution of the thresholds. And depending on the misalignment of persons and thresholds, there might be large or small frequencies in categories. If misalignment is the only source of low frequencies, threshold estimates are not disordered. Disordered thresholds can also appear when there is no low frequency in any category, in which case there are structural problems in the responses that, in real data, need a substantive explanation.

The Distribution Leading to Disordered Thresholds and the Test of Fit

Having demonstrated that a low frequency in a category of the whole sample is not the source of disordered thresholds, the article now explains the sense in which a relatively low frequency produces reversed thresholds, and in the process, it demonstrates that the test of fit used by AWW to justify disordered thresholds is irrelevant.

Simulation 5

To facilitate the above purposes with sound estimates of person locations, Simulation 5 had 26 dichotomous items and two polytomous items with maximum scores of 2. The dichotomous item difficulties were distributed in equal intervals between −3 and 3. The distribution of 20,000 persons was $β ~ N (0, 1)$ and was therefore well aligned to the difficulties of the items. The threshold parameters of Items 27 and 28 with a maximum score of 2 are shown in Table 12. Item 27, with correctly ordered thresholds, was relatively easy; Item 28, with reversed thresholds of magnitude 1.0 logits, was relatively difficult. Table 13 shows the frequencies and proportions of each score for Items 27 and 28, and Table 12 shows the threshold estimates of the two items using RUMM2030.

Table 12.

Estimates of Parameters of Two Polytomous Items in Simulation 5

Items	Generating Thresholds		Estimates from the PRM
27	$δ_{1} = - 3.0$	$δ_{2} = - 1.0$	${\hat{δ}}_{1} = - 2.80$	${\hat{δ}}_{2} = - 0.85$
28	$δ_{1} = 2.5$	$δ_{2} = 1.5$	${\hat{δ}}_{1} = 2.65$	${\hat{δ}}_{2} = 1.59$

17 persons with extreme scores 0 (5) and 30 (12)

Note. N =20,000; $β \tilde{N} (0, 1) .$

Table 13.

Frequencies (and Proportions) of Responses for Items 27 and 28 in Simulation 5

Item	Score
	0	1	2
27	728 (0.04)	5,759 (0.29)	13,496 (0.68)
28	17,126 (0.86)	1,760 (0.09)	1,097 (0.05)

For Item 28, the frequencies are similar to those of AWW’s example in their Figure 7, except that the frequency of the Score 1, although low, is greater than that of Score 2 and therefore is not the lowest frequency as in their example. In addition, the frequency of Score 0 for Item 27 is lower than those of Scores 1 and 2 of Item 28. Despite these relative frequencies, the threshold estimates of Item 27 are ordered and those of Item 28 are not, confirming that low frequencies in the data as a whole cannot explain disordered threshold estimates. The explanation requires a closer examination of the distribution of the responses of each person, and not the data as a whole.

Bimodal Response Distributions for a Single Person

Although it can be shown algebraically, from Figure 4 with ordered thresholds, it is evident that for a location between the thresholds, $δ_{1} < β < δ_{2},$ both $P_{n 1} > P_{n 0}$ and $P_{n 1} > P_{n 2} .$ Therefore, the response distribution has a single mode x_n = 1, and $P_{n 1}^{2} / (P_{n 0} P_{n 2}) > 1 .$ In contrast, from Figure 6 with disordered thresholds, for a location between the thresholds, $δ_{2} < β < δ_{1},$ both $P_{n 0} > P_{n 1}$ and $P_{n 2} > P_{n 1} .$ Therefore, the distribution is bimodal and $P_{n 1}^{2} / (P_{n 0} P_{n 2}) < 1 .$

Expressed in terms of parameters, and therefore generalized, the ratio $P_{n 1}^{2} / (P_{n 0} P_{n 2})$ is fully instructive. Forming the ratio from Equation (24), and after simplification,

P_{n 1}^{2} / (P_{n 0} P_{n_{2}}) = \exp (δ_{2} - δ_{1}),

(37)

which is independent of the person parameter. In general, for m > 2, $P_{nx}^{2} / (P_{n (x - 1)} P_{n (x + 1)}) = \exp (δ_{x + 1} - δ_{x}) .$ Here, we focus on m = 2. Clearly, if and only if $P_{n 1}^{2} / (P_{n 0} P_{n 2}) > 1,$ then $\exp (δ_{2} - δ_{1}) > 1,$ which implies that $δ_{2} - δ_{1} > 0,$ and the thresholds are ordered. Likewise, if and only if $P_{n 1}^{2} / (P_{n 0} P_{n 2}) < 1,$ $\exp (δ_{2} - δ_{1}) < 1,$ which implies that $δ_{2} - δ_{1} < 0,$ and the thresholds are disordered.

Equation (37), which connects a distribution of responses with thresholds of an item explicitly, and which is the only one that seems to do so, characterizes a single person. However, it also characterizes the distribution of every person. This has two immediate implications. First, as noted earlier, that every threshold estimate is affected by all frequencies in a complex way, and that the distribution of Equation (37) must be distinguished from a summary distribution and frequency of responses of all persons in a sample of data. In their justification of low frequencies in the data set as a whole as the cause of reversed thresholds, AWW do not make this distinction.

Second, each person has only one response to an item, and the probability distribution in Equation (37) is inferred from responses to all items. Therefore, to make any comparisons between observed proportions of responses (which are counterparts to the probabilities) and the threshold estimates using Equation (37), it is necessary to take persons with homogeneous estimates rather than the whole sample. In the PRM, classifying persons by the total score, their sufficient statistic is ideal. However, classifying persons into class intervals based on total scores is often more practical. This method is used in tests of fit and is used in particular by AWW in the example they present.

Consider now Item 28 whose parameters are shown in Table 12. According to Equation (37),

{\hat{δ}}_{2} - {\hat{δ}}_{1} = - 1.06 < 0 \leftrightarrow P_{n 1}^{2} / (P_{n 0} P_{n 2}) < 1 .

(38)

Equation (38) holds in general for all persons. In particular, it implies that the inferred response distribution of any person between the two thresholds, $1.59 > β_{n} < 2.65,$ is explicitly bimodal. For example, at the midpoint between the thresholds, $β_{n} = 2.12,$ ${P_{ni 0}, P_{ni 1}, P_{ni 2}} = {0.39, 0.22, 0.39} .$ However, the expected value is $E [X_{ni}] = \sum_{x = 0}^{2} x P_{xni} = 1.0 .$ Thus, although the expected mean response is 1, simultaneously, there is a greater probability of x_n = 0 and x_n = 2 than x_n = 1. Thus, there is a contradiction between the mean and the mode in the distribution and this is analogous to contradictions observed earlier. This contradiction does not arise when the thresholds are ordered. The contradiction is made concrete again by considering an instrument used to measure heights of persons: It is as if on many replications of the measurement of a single person, the mean is 180 cm, but the distribution shows an equally high number of measurements of 179 cm and 181 cm, and relatively few of 180 cm. This deficiency of measurements of 180 cm would not be merely descriptive of the operation of the instrument; it would be understood that it is malfunctioning.

Disordered Threshold Estimates and the Test of Fit

The contradictory implications of a bimodal distribution also provide interpretative problems for the test of fit used by AWW. In particular, it is necessary to invoke the caution drummed into students in elementary statistics classes that it is misleading to characterize a bimodal distribution by a single mean. For example, it would be misleading to characterize the bimodal distribution of persons in Figure 8 with the mean of 0.021. And there is no reason to revoke this principle just because it appears in a less obvious guise in a psychometric distribution of a homogenous group of persons’ responses to a single item. To consolidate how the class interval mean is misleading when thresholds are disordered, Table 14 shows the observed means and expected values in 9 class intervals for Item 28. The table also shows the ratio of Equation (37) for both estimated probabilities and the observed proportions in each class interval.

Table 14.

Nine Class Intervals for Item 28 With Disordered Thresholds in Simulation 5

$(δ_{1} + δ_{2}) / 2 = 2.121,$ $δ_{1} = 2.648; δ_{2} = 1.593$
CI	Frequency	Mean	Scores	0	1	2	$P_{1}^{2} / (P_{0} P_{2})$
1	2,172	−1.758	OBS.P	.99	.01	.00	Undefined
(OM = .01 EV = .01 OM-EV = .00)			EST.P	.99	.01	.00	Undefined
2	1,678	−.986	OBS.P	.97	.03	.00	Undefined
(OM = .03 EV = .03 OM-EV = .00)			EST.P	.97	.03	.00	Undefined
3	2,252	−.608	OBS.P	.96	.04	.00	Undefined
(OM = .05 EV = .05 OM-EV = .00)			EST.P	.96	.04	.00	Undefined
4	2,599	−.241	OBS.P	.95	.05	.00	Undefined
(OM = .05 EV = .07 OM-EV = −.02)			EST.P	.94	.05	.01	0.266
5	2,676	.125	OBS.P	.92	.07	.01	0.533
(OM = .09 EV = .11 OM-EV = −.02)			EST.P	.91	.07	.02	0.269
6	2,630	.494	OBS.P	.88	.09	.02	0.460
(OM = .14 EV = .17 OM-EV = −.03)			EST.P	.87	.10	.03	0.383
7	2,173	.872	OBS.P	.81	.14	.05	0.484
(OM = .24 EV = .27 OM-EV = −.03)			EST.P	.80	.14	.07	0.350
8	2,360	1.359	OBS.P	.70	.18	.12	0.386
(OM = .42 EV = .48 OM-EV = −.06)			EST.P	.67	.18	.15	0.322
9	1,443	2.206	OBS.P	.36	.23	.41	0.358
(OM = 1.05 EV = 1.05 OM-EV = .00)			EST.P	.35	.23	.42	0.360

Note. OM = mean of the class interval; EV = expect value, $\sum_{0}^{2} x P_{x};$ OBS.P = observed proportion; EST.P = estimated probability.

In addition, Figure 10 shows the EVCs and observed means in the class intervals. In Figure 10, the observed means are close to their expected values, and therefore the responses are taken to fit the model. This is the evidence used by AWW to justify that there is no contradiction in reversed threshold estimates. However, as argued below, this evidence is meaningless for their purpose.

First, we confirm that the observed mean in a class interval is meaningless in characterizing a distribution of responses. Table 14 for Item 28 shows that the ratio of Equation (37) is less than 1 for all class intervals in which it can be calculated, and therefore implies reversed thresholds. It is less than 1 for both observed proportions and the estimated probabilities. Because Item 28 is difficult, all but one ratio is undefined for the first four class intervals. For persons within the range of the thresholds, the observed distribution is explicitly bimodal. For example, Class Interval 9 in Table 14 has a mean of 2.206 close to the mean of the two thresholds, (2.65 + 1.59)/2 = 2.121 (from Table 13), and the observed proportions, notated “OBS P” and shaded in Table 14, are bimodal. This bimodality with more scores of both 0 and 2 than of 1 renders the observed mean misleading in characterizing the distribution.

Second, we show that the closeness of observed means to their expected values from threshold estimates is irrelevant to threshold order. Whether disordered or not, threshold estimates reflect a property of the data. Then these threshold estimates are used to obtain the distribution from the model and the expected values. Illustratively, for Class Interval 9, the estimated distribution is ${P_{ni 0}, P_{ni 1}, P_{ni 2}} = {0.35, 0.23, 0.42},$ which is bimodal and similar to the observed distribution. Thus, although the test of fit can be used to diagnose other evidence of the malfunctioning of the item, because the expected value is calculated using the threshold estimates, disordered or not, its closeness to the observed mean has no bearing on whether or not reversed thresholds are contradictory to their intended order.

Thus, for two reasons, first that a mean with a bimodal distribution of responses is misleading, and second that the thresholds estimates themselves are used to calculate the expected value, the evidence of fit displayed in AWW’s Figure 9 (and Figure 10 in this article) has no bearing on the interpretation of reversed threshold estimates. Bimodal distributions do not appear with Item 27, which has ordered thresholds.

Frequencies and Reversed Thresholds

The above analysis of bimodal distributions for each person implied by reversed thresholds also allows a qualification of AWW’s assertion that low frequencies lead to reversed threshold estimates. For this qualification, consider again Figure 6 of Simulation 1, which has reversed threshold estimates. It can be inferred from this figure that reversed thresholds appear when, based on all the evidence from the data, the very people (locations) who should obtain a particular score, do not do so, but instead obtain scores on either side simultaneously with greater frequency. In summary, a relatively low observed frequency of a particular category, and simultaneously inflated frequencies on either side, from those persons who are expected to have the highest frequency causes disordered thresholds. This qualification of comments made by AWW is essential to understanding the role of frequencies on threshold reversals.

Figure 6 reflected the distribution from the simulated Novices in Simulation 1. The disordered thresholds arose because the incompatibility between the threshold order and the hypothesized Guttman vector meant that those persons who were expected to score 1 most frequently were not selected for analysis in with the PRM. Instead, not only less persons but also those for whom the hypothesized Guttman vector was incompatible with the threshold order were selected, resulting in the disordered threshold estimates.

The AWW Example With Reversed Threshold Estimates

Although the emphasis in this article is on the use of the PRM for a rating format, the scoring structure of the example of a partial credit item provided by AWW in their Figures 6 to 9 is analyzed briefly from the perspective of this article. The options for the scoring are reproduced in Table 15, together with the threshold estimates. Students were instructed to show their work.

Table 15.

Scoring Key in Example in AWW

Descriptor	150 with work shown	150 with no work shown	Correct method but with calculation error	30 with calculation leading to 30	Incorrect	Blank
Score	2	1	1	1	0	0
${\hat{δ}}_{1} = 3.75; {\hat{δ}}_{2} = 0.07 ({\hat{δ}}_{2} - {\hat{δ}}_{1} = - 3.68)$

AWW claim that, despite severely disordered thresholds, there are no problems in the ordering. Nevertheless, consider this example from the perspective of the location of a person who is exactly at the threshold $\hat{β} = {\hat{δ}}_{1} = 3.75 .$ Note that it is irrelevant whether or not there are any such persons—the inferences are from the probability distribution of a particular location. This person, who only has sufficient proficiency to have a probability 0.5 of scoring 1 on the item, has simultaneously, sufficient proficiency to have a probability of 0.95 of obtaining a score of 2. That shows extreme dependence of the score of 2 on having achieved marginal proficiency to score 1.

It is not enough to describe the reversed thresholds to be a result of a low frequency of responses in the score of 1 for the whole sample; instead as argued consistently from Andrich (1979) to Andrich (2011), it is necessary to seek a substantive explanation for such an anomaly. For example, the constructors of the item need to be informed of the anomaly, which they may be able to explain by having discussions with markers, and so on.

However, in the AWW example the potential problem in the scoring seems so blatant that there is no need to go to the markers to form a first hypothesis. According to the marking key, a score of 2 is given for the correct answer with work shown, but only a score of 1 for a correct answer with no work shown. On the assumption that cheating was sufficiently controlled for by invigilation during the assessment, a correct answer, however it was obtained, suggests the same proficiency as having a correct answer with work shown.

In assessments of this kind, students are encouraged to show working, which, in case it is essentially correct but there is, say, a calculation error, they may gain partial credit. In fact, the above scoring key does just that with the option “Correct method but with calculation error,” which also is given a score of 1. Therefore, to give only a mark of 1 for a correct answer when no working is shown is giving only partial credit as a punishment for not following instructions. That is, a second variable, irrelevant to mathematical proficiency, is introduced in the marking key. Therefore, it seems not surprising that someone who has 0.5 probability of answering the question correctly without showing working, in fact, has a 0.95 probability of answering it correctly showing the working. In addition, it seems that “Correct with no work shown” is qualitatively different from the other two options that also obtain the partial credit score of 1, that is “Correct method but with calculation error,” and particularly, “30 with calculation leading to 30,” which is in fact incorrect. This raises a complementary problem with the scoring key—students who provide the wrong answer of 30 showing calculations achieve the same score as students who actually provide the right answer without showing their work. On reflection, this seems unfair on the students who answer correctly but do not show their work. If this were a high-stakes test, say entry to university for highly selective courses in which less than one scaled mark can make a difference, it suggests that this scoring key would lead to appeals and be untenable.

AWW use the example to argue that there is no problem demonstrated by reversed thresholds. It seems that they are so wedded to the evidence of the test of fit, shown to be irrelevant, that none of them has studied the marking key itself. Even without the reversed threshold estimates, a close examination of the key would suggest exactly the problems that are manifested by the PRM analysis. This article shows that their example is excellent in demonstrating the power of the PRM to expose, with disordered thresholds, problems in the scoring structure of items. In this case, at least two problems can be hypothesized: first, that one of the response options given a partial credit score of 1 demonstrates the same proficiency as that given the maximum score of 2, and second, that two responses that clearly do not show the proficiency of a correct response are given the same score as one of the correct responses. Of course, the above explanation would need to be tested by rescoring performances.

Discussion

In the applications of the SLM to dichotomously scored items, the difficulties of the items (their thresholds in the nomenclature of this article) are integral to the substantive understanding of increasing values of the variable. Placing persons on the same continuum strengthens the interpretations. Wilson, in particular, makes much of this possibility and has used the term Wright map to characterize such a display (Wilson, 2005). Unfortunately, the item order is generally not spelled out in advance and is used mostly post hoc to confirm that the ordering makes substantive sense. Of course, even if the confirmation is post hoc, the power of the analysis is that the item location estimates may disconfirm a sound substantive order.

In the case of items with more than two ordered categories, there is immediately an a priori condition of order that should reflect an understanding of increasing values of the variable. The unequivocal a priori ordering of categories, together with possible empirical evidence to the contrary, provides the opportunity to use the PRM in more than a statistically descriptive way. As shown in the article, this possibility rests on being able to make inferences from the PRM as if responses at the thresholds were experimentally independent and analyzed with the SLM.

This article has addressed, with a detailed derivation of the PRM and with simulation studies, the three major contentions of AWW indicated at the beginning of the article: (a) that category ordering is independent of the threshold order and that the Guttman structure with order of the thresholds is not part of the PRM, (b) that low frequency is the source of the reversed thresholds, and (c) that evidence of the kind of test of fit justifies reversed thresholds. The article rebuts the first two contentions and demonstrated that the test of fit AWW use is irrelevant to their case.

In challenging the relevance of order in the PRM, AWW seem to want, as with the graded response model, that the order of the values of the thresholds be a property of the model rather than only a property of the data. They seem to think that just because estimates of threshold values can be reversed, the structure of the model is not compatible with the required order of the thresholds. This article shows that the choice of the Guttman vector rests on having a hypothesized order of the thresholds based on the intended ordering of the categories and that reversed threshold estimates implies a rejection of the hypothesized order.

In printing out the thresholds from the graded response model, as they do in Figure 7 of their partial credit item, rather than relying on the thresholds from the PRM with which they analyze the data, AWW must sense that there is something wrong. Like Masters and Wright (1997), who state that it is a disadvantage of the method when no region of the continuum is defined as in Figure 6 of this article, AWW must see reversals as a problem with the model rather than with the data. They effectively abandon the PRM in favor of the graded response model whose thresholds are never reversed, no matter what problems there might be in the empirical operation of the categories. Switching between these two models in particular, which are structurally incompatible, confirms a purely data-descriptive perspective in the application of models. For example, it has been known since 1966 (Rasch, 1966) that summing probabilities of adjacent categories in the PRM destroys the sufficiency of the model (Andrich, 1978, 1995; Roskam & Jansen, 1989), yet that is exactly what AWW do when they calculate the thresholds for the graded response model from the probabilities estimated from the PRM.

AWW make another summary comment that is irrelevant to the mathematics of the model—they write,

We do not dismiss the observation that if certain psychological processes are assumed, then a numerical ordering of the tau (threshold) parameters would be a substantive requirement. (AWW, 2012, p. 24)

They seem to imply that the mathematics of the model, and the derivation that used judges illustratively, depends on psychological processes. It does not—all the examples presented in this article used simulated data to illustrate the mathematical relationships leading to reversed thresholds. Psychological judging processes do not have to be understood to require that a rate of success at Pass be greater than a rate of success at Distinction, conditionally (in the space $Ω_{x - 1, x}^{G}$ ) or unconditionally (in the space Ω)!

Again, recognizing that there is something wrong with reversed thresholds, they write,

We do not dismiss the possibility that disordered estimated parameters may well be an indicator of a problem with an item. (AWW, 2012, p. 24)

And they give particular examples, low frequency of a category, local dependence, and differences in discrimination at the thresholds. This article explains the qualification to the low frequency in detail, and Andrich (1983, 1985a) explained the relationship between local dependence and threshold estimates as well as the case of different discriminations at thresholds. These particular cases were pursued because of a recognition that reversed thresholds presented an anomaly, and which AWW apparently accept as such in these cases. However, from the AWW perspective, there would have been no reason to search for an explanation in these cases. Admitting that reversed thresholds might be an anomaly in an item is inconsistent with AWW’s argument that there is no inherent, structural problem with reversed threshold estimates. Thus, there may be problems that have never been thought of, some may permeate the whole data set as in Simulation 4, and some may simply be idiosyncratic to a particular item, as in the one they parade as having no problems but which seems to have major problems with the partial credit scoring key. The argument from the mathematical structure and implications of the PRM for rating, and ordered categorical data in general, in this article and elsewhere, is that to confirm that the categories are working as intended, the threshold estimates need to be ordered and that any item in which they are not ordered needs to be examined to try to understand the source of the disorder.

Finally, AWW write,

What we are concerned about, however, is proliferation of the view that disordered parameters are indicative of a set of items that are not working because the categories are not ordered. (AWW, 2012, p. 24)

What I am concerned about is that Adams, Wu, and Wilson, who use Rasch models, do not appreciate the elegance, the coherence, and the power of the PRM to disclose that categories intended to be ordered are not operating as required. I am concerned that they use statistical sophistry, exemplified by the incongruous statement that if one understands conditional probabilities, then one understands why a performance of sufficiently high proficiency to be at the margin of a Pass and Distinction is simultaneously of such poor proficiency that it is more likely to be classified as a Fail, to justify ignoring reversed threshold estimates. Unfortunately, this sophistry potentially hides problems, as in the example they present, in every item with putatively ordered categories with reversed threshold estimates.

Conclusion and Further Exposition

As analyzed in the article, AWW use a test of fit that is irrelevant to the implications of threshold order. It is acknowledged, of course, that a satisfactory test of fit confirms that the model describes the data and that there is a sense in which it is better that the data fit the model than when they do not. In depending so much on a descriptive account of the data based on fit, without giving primacy to the substantive understanding of parameters in a context, AWW adopt a common statistical modeling paradigm set in place by the founder of modern statistics, Karl Pearson (MacKenzie, 1981; Norton, 1978). This tradition, importantly, contrasts with the paradigm of R. A. Fisher in which statistical methods are applied to specifically designed experiments in which the substantive meaning of variables is paramount (Box, 1978; MacKenzie, 1981). Although the argument in this article arises from Fisher’s paradigm, there is not the space to consider further the implications of the differences between these two paradigms in a measurement context. These implications have been broached for the case of ordered response categories in Andrich (2011). They are examined in more detail in Andrich (in press).

Footnotes

Appendix A

Table A.4

Thresholds Estimates in Simulation 3

Item	${\hat{δ}}_{1 j}$	${\hat{δ}}_{2 j},$	${\hat{δ}}_{3 j}$	${\hat{δ}}_{4 j}$	${\hat{δ}}_{5 j}$	${\hat{δ}}_{6 j}$	${\hat{δ}}_{7 j}$	${\hat{δ}}_{8 j}$
1	−5.42	−3.77	−2.41	−1.22	−0.07	1.17	2.62	4.41
2	−5.12	−3.64	−2.29	−1.02	0.23	1.49	2.84	4.31
3	−5.09	−3.56	−2.14	−0.79	0.52	1.81	3.10	4.43
4	−4.82	−3.30	−1.99	−0.79	0.40	1.68	3.14	4.88
5	−4.83	−3.10	−1.64	−0.37	0.81	2.01	3.33	4.86
6	−4.78	−2.90	−1.42	−0.19	0.94	2.13	3.54	5.31
7	−4.48	−2.93	−1.52	−0.20	1.07	2.36	3.69	5.13
8	−4.19	−2.83	−1.51	−0.21	1.09	2.40	3.74	5.12

Acknowledgements

Many discussions with Guanzhong Luo and Irene Styles over many years, and Stephen Humphry more recently, have contributed to improving the original demonstration of the threshold structure of the unidimensional, polytomous Rasch model, the special cases of which are known as the rating scale and the partial credit models. Stephen Humphry, Pender Pedler, Tim Dunne, Barry Sheridan, Ida Marais, Irene Styles, and Josh McGrane provided valuable comments on earlier versions of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was in part funded by Australian Research Council Grants and by the Australian Curriculum, Assessment and Reporting Authority and the Curriculum Council of Western Australia as Industry Partners and by Pearson.

References

Adams

R. J.

M. L.

Wilson

(2012). The Rasch rating model and the disordered threshold controversy. Educational and Psychological Measurement. Advance online publication. doi:10.1177/0013164411432166

Andersen

E. B.

(1977). Sufficient statistics and latent trait models. Psychometrika, 42, 69-81.

Andrich

(1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-574.

Andrich

(1979). A model for contingency tables having an ordered response classification. Biometrics, 35, 403-415.

Andrich

(1983). Recognising a violation of the assumption of local independence in the analysis of test data. Educational and Psychological Measurement, 43, 829-833.

Andrich

(1985a). A latent trait model for items with response dependencies: Implications for test construction and analysis. In Embretson

(Ed.), Test design: Contributions from psychology, education and psychometrics (pp. 245-273). New York, NY: Academic Press.

Andrich

(l985b). An elaboration of Guttman scaling with Rasch models for measurement. In Brandon-Tuma

(Ed.), Sociological methodology (pp. 33-80). San Francisco, CA: Jossey-Bass.

Andrich

(1995). Distinctive and incompatible properties of two common classes of IRT models for graded responses. Applied Psychological Measurement, 19, 101-119.

Andrich

(1996). Theoretical and empirical evidence on the dichotomization of graded responses. In Engelhard

Jr. Wilson

(Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 265-288). Norwood, NJ: Ablex.

10.

Andrich

(2005). The Rasch model explained. In Alagumalai

Curtis

D. D.

Hungi

(Eds.), Applied Rasch measurement: A book of exemplars (pp. 308-328). Norwell, MA: Springer-Kluwer.

11.

Andrich

(2010). Understanding the response structure and process in the polytomous Rasch model. In Nering

Ostini

(Eds.), Handbook of polytomous item response theory models: Developments and applications (pp. 123-152). Mahwah, NJ: Erlbaum.

12.

Andrich

(2011). Rating scales and Rasch measurement. Expert Review of Pharmacoeconomics & Outcomes Research, 11, 571-585.

13.

Andrich

(in press). The multiple contributions of the work of R. A. Fisher in the development and interpretation of Rasch’s in the unidimensional polytomous Rasch model for measurement. Educational and Psychological Measurement.

14.

Andrich

Luo

(2003). Conditional estimation in the Rasch model for ordered response categories using principal components. Journal of Applied Measurement, 4, 205-221.

15.

Andrich

Sheridan

B. E.

Luo

(2012). RUMM2030: An MS Windows computer program for the analysis of data according to Rasch unidimensional models for measurement. Perth, Australia: RUMM Laboratory.

16.

Box

J. F.

(1978). R.A. Fisher: The life of a scientist. New York, NY: Wiley.

17.

Fisher

R. A.

(1958). Statistical methods for research workers (13th ed.). New York, NY: Hafner.

18.

Guttman

(1950). The basis for scalogram analysis. In Stouffer

S. A.

Guttman

Suchman

E. A.

Lazarsfeld

P. F.

Star

S. A.

Clausen

J. A.

(Eds.), Measurement and prediction (pp. 60-90). New York, NY: Wiley.

19.

Harris

(1991). Consequences for social measurement of collapsing categories within items with three or more ordered categories (Unpublished Master of Education dissertation). Murdoch University, Murdoch, Western Australia.

20.

Kuhn

T. S.

(1977). The function of measurement in modern physical science. Isis, 52, 161-190. (Original work published 1961)

21.

Luo

(2005). The relationship between the rating scale and partial credit models and the implication of disordered thresholds of the Rasch models for polytomous responses. Journal of Applied Measurement, 6, 443-455.

22.

MacKenzie

D. A.

(1981). Statistics in Britain, 1865-1930. Edinburgh, England: Edinburgh University Press.

23.

Masters

G. N.

Wright

B. D.

(1997). The partial credit model. In van der Linden

W. J.

Hambleton

R. K.

(Eds.), Handbook of item response theory (pp. 101-121). New York, NY: Springer.

24.

Norton

B. J.

(1978). Karl Pearson and statistics: The social origins of scientific innovation. Social Studies of Science, 8, 3-34.

25.

Rasch

(1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. (Expanded edition [1980] with foreword and afterword by Wright

B. D.

. Chicago, IL: University of Chicago Press).

26.

Rasch

(1961). On general laws and the meaning of measurement in psychology. In Neyman

(Ed.), Proceedings of the Fourth Berkeley symposium on Mathematical Statistics and Probability (Vol. IV, pp. 321-334). Berkeley: University of California Press.

27.

Rasch

(1966). An individualistic approach to item analysis. In Lazarsfeld

P. F.

Henry

N. W.

(Eds.), Readings in mathematical social science (pp. 89-108). Chicago, IL: Science Research Associates.

28.

Roskam

E. E.

Jansen

P. G. W.

(1989). Conditions for Rasch dichotomizability of the unidimensional polytomous Rasch model. Psychometrika, 54, 317-332.

29.

Samejima

(1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.

30.

Wilson

(2005). Constructing measures. New York, NY: Psychology Press.

31.

Wright

B. D.

(1997). A history of social science measurement. Educational Measurement: Issues and Practice, 16, 33-45.

32.

Wright

B. D.

Masters

G. N.

(1982). Rating scale analysis: Rasch measurement. Chicago, IL: MESA Press.

33.

Zwinderman

A. H.

(1995). Pairwise parameter estimation in Rasch models. Applied Psychological Measurement, 19, 369-375.

An Expanded Derivation of the Threshold Structure of the Polytomous Rasch Model That Dispels Any “Threshold Disorder Controversy”

Abstract

Keywords

Introduction

Standard Ordered Category Formats and a Resolved Design

Resolving a Rating Design Into an Experimentally Independent Design

Bernoulli Random Variables in the Resolved Design and Order of Success Rates

The Case for Rasch’s Simple Logistic Model at the Thresholds

Construction and Analyses of Response Spaces

The Experimentally Independent Response Space Ω

Implications of the Required Threshold Order

The Subspace Ω G

Bridging the Resolved Design to the Standard Design With the Guttman Space Ω G

The Probability Structure of Ω G

The Doubly Conditioned Space Ω x − 1 , x G

Construction of the PRM Through the Guttman Subspace Ω G

Other Parameterizations of the PRM

The Latent Dichotomous Responses at the Thresholds

Simulation Study 1: Two Models of Analysis in Two Response Spaces

Establishing the Guttman Response Space From the Structure of the PRM

Theorem

Inferring an Experimentally Independent Response Space Ω From Ω G

The Required Success Rate Order in the Standard Design

Category Characteristic Curves and Disordered Thresholds

Frequencies of Responses in Categories and Reversed Threshold Estimates

Examples of Low Frequency and Correctly Ordered Threshold Estimates and High Frequency and Reversed Threshold Estimates

Simulation 2

Simulation 3

Simulation 4

The Distribution Leading to Disordered Thresholds and the Test of Fit

Simulation 5

Bimodal Response Distributions for a Single Person

Disordered Threshold Estimates and the Test of Fit

Frequencies and Reversed Thresholds

The AWW Example With Reversed Threshold Estimates

Discussion

Conclusion and Further Exposition

Footnotes

Appendix A

Acknowledgements

Declaration of Conflicting Interests

Funding

References

The Subspace $Ω$ ^G

Bridging the Resolved Design to the Standard Design With the Guttman Space $Ω^{G}$

The Probability Structure of $Ω$ ^G

The Doubly Conditioned Space $Ω_{x - 1, x}^{G}$

Construction of the PRM Through the Guttman Subspace $Ω^{G}$

Inferring an Experimentally Independent Response Space $Ω$ From $Ω^{G}$