Abstract
The evaluation of sign language proficiency needs to be based on measures with well-established psychometric proprieties. To date, no valid and reliable test is available to assess Polish Sign Language (Polski Język Migowy, PJM) skills in deaf children. Hence, our aim with this study was to adapt the British Sign Language Receptive Skills Test (the first standardized test to determine sign language proficiency in children) into PJM, a less researched sign language. In this paper, we present the first steps in the adaptation process and highlight linguistic and cultural similarities and differences between the British Sign Language Receptive Skills Test and the PJM adaptation. We collected data from 20 deaf children who were native signers (age range: 6 to 12) and 30 deaf children who were late learners of PJM (age range: 6 to 13). Preliminary analyses showed that the PJM Receptive Skills Test has acceptable psychometric characteristics (item analysis, validity, reliability, and sensitivity to age). Our long-term goal with this work was to include younger children (age range: 3 to 6) and to standardize the PJM Receptive Skills Tests, so that it can be used in educational settings and in scientific research.
Keywords
Sign language, the primary form of human language used by deaf communities, is, like spoken language, transferred from one generation to another (Klima & Bellugi, 1979). Sign language is fully accessible and learnable for all deaf children to communicate and discover the world (Humphries et al., 2014). Deaf children typically acquire or learn visual-spatial language in different contexts, inter alia, in the family setting (Bellugi, 1988) or in educational settings (Rinaldi, Caselli, Onofrio, & Volterra, 2014). Children exposed to sign language from infancy go through similar stages of development as children acquiring spoken language (Bellugi, 1988; Mayberry & Squires, 2006; Morgan, 2014; Petitto et al., 2001). Age-appropriate sign language acquisition prevents deaf children from language deprivation and cognitive disorders associated with a lack of spoken language acquisition, as experienced by many (Humphries et al., 2014). However, the process of sign language acquisition can be at risk of delay or disruption (Quinto-Pozos, 2014). This is because although the natural sign language learning environment for a deaf child is the deaf family with deaf signing parents, only a limited percentage of deaf children have early access to sign language and acquire sign language as a first language in the familial milieu. Approximately 5% of deaf children are born to deaf parents and, consequently, are exposed to sign language from birth (Mitchell & Karchmer, 2004). To be raised by deaf signing parents does not guarantee native norms for sign language development in all children (Baker, van den Bogaerde, & Woll, 2005) for a number of reasons. First, despite early and rich sign language input, some native signers exhibit sign language and communication disorders similar to the speech and spoken language disorders found in hearing children; for example, there can be a developmental language disorder in sign language (Mason et al., 2010; Morgan, Herman, & Woll, 2007). Second, some deaf parents can themselves be late learners of sign language, which can influence their sign language competence (Morford & Carlson, 2011). Deaf children who are offered impoverished and inconsistent sign language input by non-native signing deaf parents may differ in sign language production skills from deaf children with deaf native signing parents (Ross & Newport, 1996). Third, deaf children of deaf parents (DCDP) may have interactions with a limited number of signing interlocutors (adults and children) compared to the number of spoken language conversational partners for hearing children, and as a consequence, deaf children can have a reduced amount of input and more limited variability in sign language input compared to hearing children in spoken language environments. Fourth, some deaf parents do not use sign language at home with their children, and their family communication is based on spoken language, gestures, and home signs (Baker et al., 2005).
Unlike the minority of deaf children who are native signers, most deaf children are raised by hearing parents who do not initially know sign language (Humphries et al., 2014); they usually only start to learn sign language as a second language after the birth of their deaf child. Some hearing parents decide to use sign language in daily communication with their deaf child. However, some have problems with accessing appropriate sign language tuition and others struggle with poor language learning abilities, and their skills in sign language can therefore be limited. Consequently, hearing parents of deaf children can struggle to provide rich and well-structured input in sign language (Knoors & Marschark, 2012). Therefore, it is unsurprising that deaf children of hearing parents have been found to generally have lower sign language skills than aged-matched native signers (Herman & Roy, 2006).
Deaf children of hearing parents (DCHP) generally learn sign language at schools that offer bilingual education, either in special schools for deaf children or in integrated settings. However, pupils who have not had rich access to sign language before entering school and whose sign language acquisition is based mainly on school interaction can show atypical language development (Mayberry & Lock, 2003).
Summarizing the evidence, sign language acquisition may be delayed or disordered in a considerable proportion of deaf children. For this reason, the process of a child’s sign language acquisition should be monitored and supported by schools and habilitation centres. In order to measure the development of sign language, and indirectly to assure an adequate language environment, an assessment tool is needed. The first standardized measure for sign language development in the world was the British Sign Language (BSL) Receptive Skills Test (RST) (Herman, Holmes, & Woll, 1999). Currently, in Poland there is a need for a sign language assessment tool, because there are no tests that can be used to monitor the acquisition of Polish Sign Language (Polski Język Migowy, PJM). In kindergarten, primary schools, and habilitation centres, sign language assessment (if it occurs at all) is based on informal procedures similar to those used in other countries without developmental sign language tests (Haug, 2011a, b; Herman, 1998). The lack of an appropriate assessment inspired us to start working on the adaptation of the Receptive Skills Test (RST) from BSL to PJM with respect to the test adaptation guidelines recommended by the authors of the BSL-RST and others (Haug & Mann, 2008). In this paper we describe the first eight steps in the adaptation of the BSL RST into PJM, with the future goal of developing the first standardized assessment of PJM.
The British Sign Language Receptive Skills Test and its adaptations
The BSL RST (Herman et al., 1999) was developed to measure BSL receptive morphosyntactic skills in children from 3 to 11 years of age. The test was normally administered face-to-face with paper booklets, with the administrator conducting the scoring and helping the child with the administration of the video-based online content. The test now has a fully online version (see www.dcalportal.org). The test contains the following: (1) a vocabulary check and (2) a video-based Receptive Skills Test. In the vocabulary check, the child is asked to produce signs used in the main RST using a picture-based elicitation task. The goal of the vocabulary check is to ensure that the child is familiar with the signs utilized in the RST and to verify which version of the test should be used (in the original test, there were two versions of the BSL test, that is, Northern UK and Southern UK versions, each with different regionally varying signs, although there is only a single version in the new online format: again, see www.dcalportal.org). If the child does not produce the elicited signs correctly, the test administrator checks to see if the child recognizes the signs. Based on the vocabulary check scores, the assessor decides if the child knows (can produce or at least recognize) the vocabulary included in the RST, and therefore whether or not to proceed with the test. In the main part of the RST, the child sees signed utterances of increasing difficulty and is required to select one picture from a choice of four that corresponds to the presented phrase. The child is asked to point to the correct picture. Scoring is automatic in the online version; in the paper-based version, the child points to the correct picture in the test booklet and the test administrator notes the child’s responses on the RST score sheet. This part of the test contains three practice items and 40 test items.
The objective of the RST is to evaluate the knowledge of six grammatical structures in BSL: (1) negation, (2) number and distribution (plurals), (3) verb morphology, (4) noun-verb distinction, (5) size and shape specifiers, and (6) handling classifiers (Herman et al., 1999). Examples of each structure are presented in Table 1, and they are explained below. (Table 1 contains PJM equivalents that will be described in Step 3 of the RST adaptation process from BSL into PJM, and this description is in the online supplement that can be found with the online version of this article.)
Linguistic features of BSL included in the Receptive Skills Test with item examples.
In BSL, negation can be expressed by manual negators (signs) and non-manual elements (e.g. headshake) (Sutton-Spence & Woll, 2007). In spoken languages, speakers can optionally add non-manual negation (e.g. headshake) to their speech, whereas in BSL, non-manual negation is obligatory and manual negators are optional in some constructions. Hence, manual negation must be combined in varying ways with non-manual elements (Atkinson, Campbell, Marshall, Thacker, & Woll, 2004). For example, in the BSL RST, item 3 (ICE CREAM NOTHING, English “no ice-cream”) includes both the manual negator NOTHING and the non-manual negator of down-turned lips, headshake, and narrowed eyes.
The grammatical structures for number and distribution are more complex in BSL than the plural in English. Plurals in BSL can be expressed in the following ways: repeating the sign (each repetition placed in a slightly different location representing distribution of referents in space); adding a quantifier (usually before the noun); or using a classifier construction, a predicate expressing inter alia the number and distribution (Sutton-Spence & Woll, 2007). In the BSL RST, item 6 (ONE-TEDDY, English “one teddy”) contains a classifier construction (all fingers extended, spread and loosely curved) and a single location, showing that there is just one teddy.
Three groups of verbs can be identified in BSL: plain verbs, agreement verbs, and spatial verbs (Schembri, Fenlon, & Cormier, 2018). In BSL, verb agreement is expressed by using syntactic space: The subject and the object are indicated by the location of the beginning and end points of signing. For example, in the BSL RST, in item 18 (BOOK-GIVE-TO-CHILD, English “a book is given to the child”), the verb GIVE moves from the initial location where BOOK is signed, to the location where CHILD is signed.
A noun-verb distinction exists in BSL. When both noun and verb are derivationally related, the verb and noun have different movements (a longer movement in the verb compared to the shorter and more abrupt movement in the noun) (Herman et al., 1999). In the BSL RST, item 8 (DRIVE, English “drive”) is a verb because the sign has a long movement with a gradual ending.
In BSL, size and shape specifiers are a type of classifier that describes the shape, patterning or size of a noun (Morgan & Woll, 2007). In the BSL RST, item 22 (WIDE-STRIPES-DOWN-TROUSERS, English “wide, vertical stripes on trousers”) includes a size and shape specifier comparable to a modifier in spoken languages.
Handling classifiers indicates how an object can be manipulated by an animated being (Morgan & Woll, 2007). In the BSL RST, item 37 (EAT-THIN-SANDWICH, English “eating a thin sandwich”) assesses the handling classifier for SANDWICH. This sentence differs from the sentence EAT-THICK-SANDWICH, as the handshape in the verb EAT is different according to the thickness of the object handled.
The psychometric parameters of the BSL RST confirm its value as an assessment that measures sign language skills in deaf children, with evidence available for the reliability (Herman et al., 1999) and the validity (Herman & Roy, 2006) of the test. To measure reliability, test–retest reliability (reported correlation r = 0.87) and split-half reliability analyses (reported correlation r = 0.9) were conducted. To measure validity, the BSL RST scores were correlated with two non-verbal subtests from the Snijders-Oomen Test (Herman et al., 1999), with reading abilities in English, and with BSL skills rated by testers (Herman & Roy, 2006). The BSL RST significantly correlated with the Snijders-Oomen Categories sub-test scores. According to Herman et al. (1999), the non-verbal subtest indirectly requires verbal processing and hence, the Categories subtest may correlate with language skills assessments (such as the BSL RST). The BSL RST did not correlate with the second non-verbal Snijders-Oomen subtest (Mosaic), but the Mosaic is a pure measure of visual-spatial skills (Herman et al., 1999). The BSL RST scores were significantly correlated with Edinburgh Reading Test scores (reported correlation r = 0.70) and with tester ratings of children’s BSL skills (exact correlation not reported) (Herman & Roy, 2006).
The BSL RST has a standardized procedure for conducting the assessment, and its norms are based on empirical data (Herman et al., 1999). The norming sample included 135 participants: 118 deaf children, and 20 hearing children of deaf parents, whose first language was BSL. All the deaf children had had extensive exposure to BSL: the deaf participants included deaf children of deaf parents (DCDP) and deaf children of hearing parents (DCHP), who had been in educational settings with BSL as the language of communication.
The BSL RST has been adapted to a number of other sign languages, including: LSF (French Sign Language; Courtin, Limousin, & Morgenstern, 2010), DSL (Danish Sign Language; Seiler & Larsen, 2005), LIS (Italian Sign Language; Surian & Tedoldi, 2005), DGS (German Sign Language; Haug, 2011a, b), Auslan (Australian Sign Language; Johnston, 2004), LSE (Spanish Sign Language; Valmaseda, Pérez, Herman, & Ramírez, 2013), and ASL (American Sign Language; (Enns & Herman, 2011). Adaptations of the BSL RST into other sign languages has opened discussions (e.g., Haug & Mann, 2008) on the adaptation of sign language tests more generally, emphasizing that adaptations are not simply translations from the source language to the target language. Oakland and Lane (2004) emphasized the distinction between translation, interpreted as a transfer from one language to another one without linguistic and cultural changes in the test construction, and adaptation, which should be managed within a framework recognizing similarities and differences between languages and cultures. “Sign language test adaptation is not a ‘quick and dirty’ approach to designing a test: The result can produce a valid and reliable instrument to be used in schools” (Haug, 2011b, p. 357).
Based on the computerized adaptation of the BSL RST into DGS, Haug (2011a) described procedures for the adaptation of sign language tests that have since become established and accepted in sign language research methodology. The preparation of the PJM RST was influenced by the procedures outlined by Haug (2011a), who described the necessary steps in the adaptation of sign language tests and proposed a model of sign language adaptation that provided guidelines for the present research. The DGS RST was an inspiration for the PJM RST: The layout of the PJM RST was based on that of the computer-based DGS RST, because the interface used in the DGS RST was described as child friendly and easy to navigate (Haug, 2011a). Figure 1 presents the PJM RST layout. A computer-based format was also chosen because this format helps to minimize the possibility of mistakes made by human raters.

The PJM RST screen layout, item number 15 (CAR BEHIND).
The process of test adaptation may involve significant changes in test construction and, therefore, it is good practice for the authors of the source assessment tool to be involved in order to oversee any linguistic and cultural modifications. In the current study, the authors of the BSL RST closely monitored and guided each step taken in the adaptation of the BSL RST into PJM. A similar process of co-operation between researchers took place during the ASL adaptation and was identified by that team as a necessary factor in the process of sign language test adaptation, which in turn may lead to a better understanding of visual-spatial language acquisition in deaf children (Enns & Herman, 2011).
Adapted Polish Sign Language Receptive Skills Test
PJM is used by more than 50,000 deaf signers in Poland, a country with a population of 38.5 million people (signers constitute more than 0.1% of the whole number of inhabitants); the PJM community is one of the largest language minority groups in Poland (Świdziński, 2014). Despite the size of the community, PJM has not yet been recognized as an official state language, although the use of PJM is regulated by the Sign Language and Other Communication Mode law that was established on 19 August 2011 (Rutkowski & Mostowski, 2017) which ensures the right to use different modes of communication, inter alia, PJM. In 2012, Poland ratified the United Nation’s Convention on the Rights of Persons with Disabilities (CRPD); this requires governments within the context of education to promote sign language learning and support to develop the linguistic identity of the deaf community.
In the school year 2018/2019, the number of deaf and hard-of-hearing children in kindergarten, primary, and secondary schools was 12,778 (https://cie.men.gov.pl/, System of Educational Information, 2018) in all types of educational settings, including special schools for deaf children and mainstream schools. Polish law does not guarantee the use of PJM in deaf education. However, there is recent legislation supporting the use of PJM in educational settings. In accordance with the CRPD, the Polish Ministry of Education has followed the decree (28 August 2017) according to which, sign language has a place in individual therapeutic programmes and, hence, deaf and hard-of-hearing pupils are able to attend sign language lessons (Rutkowski & Mostowski, 2017). Sign language courses are supplementary and PJM is not included in education as a subject.
PJM has also become more available to deaf pupils thanks to PJM adaptations of school books that contain signed versions of texts, following the source content and layout; and PJM school programmes for deaf children who are not native signers (Rutkowski & Mostowski, 2018).
Recently, the need to support PJM development in deaf children has become a subject of discussion; however, specialist educators working with deaf children face different issues such as insufficient measures to monitor the development of PJM skills of children using PJM in educational settings. Widely available developmental measures of PJM with good psychometric parameters have been lacking in Poland (Tomaszewski, Niedźwiecka, & Majewska, 2018). The development of a new assessment tool in the absence of limited research on PJM acquisition, or on PJM grammar development in deaf children, has presented challenges. In this context, the adaptation of the BSL RST, a well-established assessment tool, appeared to be a good solution. This standardized test of BSL, constructed with methodological and theoretical precision, was therefore used as a basis for developing the first PJM test for deaf children.
The adapted PJM RST is designed to be an assessment tool that can be administered not only by deaf educators but also by hearing specialists with the appropriate skills in PJM: The minimum PJM level recommended is Level A2 according to the Common European Framework of Reference for Languages (CEFR) (to administer the BSL RST a minimum of BSL Level 2 (comparable to A2) is required). As all instructions and test items in the PJM RST are signed on the test video by a deaf person, persons administering the test are not required to have a high level of PJM skills. However, they need to have sufficient knowledge of PJM to conduct the assessment, especially the Vocabulary Check, where they evaluate whether the child’s sign vocabulary is adequate to undertake the test, and more generally, to be able to sign with the child if she or he wants to ask additional questions or have a more general conversation during the testing. In addition, basic knowledge about language testing is needed to conduct the test; for example, the test administrator should be aware that before starting the test it is necessary to be familiar with the testing procedure including recording responses and providing feedback to the child. The test administrator should also know that his or her behaviour can affect the child’s performance; for example, eye-pointing (i.e., looking at the target item) may influence the child’s choice.
Objectives
The aim of the present study was to adapt the BSL RST with special regard given to linguistic, cultural, and methodological issues (Haug, 2011b), and to collect preliminary evidence of its psychometric properties in terms of the following: (1) item analysis, (2) validity, (3) reliability, and (4) sensitivity to age. Owing to space limitations in this paper, the first seven steps that we took to adapt the BSL RST into PJM are described in full in an online supplement that readers can download from the online location of this paper. In Table 2, we summarize all eight steps that were taken, but again, we refer readers to the online supplement for the full prose description of the first seven steps.
Steps in the adaptation of the BSL RST into PJM (based on Enns & Herman, 2011 and Haug, 2011b).
Main study
Method – participants
In the main study, which was Step 8 in Table 2, the final version of the PJM RST was administrated to 50 prelingually deaf children (gender: ♂ = 23, ♀ = 37), 20 with deaf parents (native signers, DCDP) (M (mean) age (years; months) = 9;11, SD age = 2;0, age range: 6;1 to 12;11), and 30 with hearing parents (late learners of sign language, DCHP) (M age (years; months) = 10;5, SD age = 1;10, age range: 6;7 to 13;8). All the DCDP were rated by teachers as having native PJM skills (for 16 DCDP the evaluation was made by teachers who were deaf signers; for the remaining four, hearing signing teachers made the evaluation). DCDP’s PJM skills were evaluated by a questionnaire in which teachers were asked to assess receptive and production skills on a scale from 1 (no skills) to 7 (excellent skills). The inclusion criterion was above 6 on the 7-point scale. Deaf parents also reported in the questionnaire that PJM was the first and dominant language of all DCDP and they did not declare any sign language impairment in their children’s PJM development. The majority of the DCHP started to communicate in sign language at the age when Polish children enter the school education system (M age of PJM acquisition (years; months) = 6;2, SD age = 2;2). Table 3 shows the characteristics of the sample. Children came from five schools for deaf children, all with a Total Communication approach and all using sign language on a daily basis. Only Total Communication schools ensure PJM access, because none of the Polish schools for deaf children has a bilingual approach with high levels of immersion in Polish and PJM as two different languages. None of the children had additional disabilities or below-average IQ (measured by Raven’s Progressive Matrices).
Demographic information about Main Study sample.
Note: For Pilot 2: All participants N = 12, M age = 10;10, SD age = 1;7. For main study: All participants: N = 50: DCDP – deaf children of deaf parents N = 20, M age = 9;11, SD age = 2;0 and DCHP – deaf children of hearing parents deaf children of deaf parents N = 30, M age = 10;5, SD age = 1;10.
It was not possible to include younger children as the present study was part of a larger project investigating language and cognitive skills in deaf school-aged children.
Materials
The final version of the PJM RST assesses knowledge of five morphosyntactic areas: (1) negation, (2) number and distribution, (3) spatial verb morphology, (4) size and shape specifiers, and (5) handling classifiers. The computer-based PJM RST contains two parts: a vocabulary check (27 signs elicited from pictures) and a video-based RST (three practice items and 47 test sentences: 38 items from the BSL-RST and nine newly developed items). All pre-recorded instructions and test items are signed by a deaf person experienced in preparing educational materials. In the first part, the assessor evaluates whether the child knows the vocabulary used in the PJM RST. This part is managed on the computer by the assessor. The child is asked to name in PJM the pictures that are presented on the computer screen. The assessor judges the appropriateness of the child’s answer: If the child uses the same signs that are in the PJM RST, the child is ready to continue with the PJM RST. If the child produces a different variant of the target sign or a gesture, a short training session is provided in order to familiarize the child with the target sign. In the training session the tester presents video clips of the sign(s) and encourages the child to repeat them. If the child is able to recognize the sign following training, measured by a recognition task (choosing one picture from a choice of four to depict the target sign), then the test can be continued. In the second part of the test, all procedures are managed by the child on the computer with the assistance of the assessor: For each item, after watching a short utterance signed in PJM, the child is asked to choose the most appropriate picture from a choice of three or four on the screen using the computer mouse.
The PJM RST is an attractive assessment approach for children. All instructions and items are included in the computer version, which ensures that all children receive the same instructions and stimuli. The scores are automatically saved.
Procedure
Prior to the study, written parental consent, verbal (spoken/signed) consent from the deaf children and verbal (spoken/signed) consent of the children’s teachers and school head teachers were obtained. All participants were tested individually in a quiet setting in their school. The test was administered in parallel sessions by two hearing signers of PJM (an educational researcher and a PJM interpreter).
Data analysis
All DCDP were native signers, exposed from birth to sign language. For this reason, their scores were taken as a basis for all statistical analyses except for test validity. All DCHP were late learners of sign language (M age of PJM acquisition (years; months) = 6;2, SD age = 2;2) and therefore their results were not incorporated in the calculation of psychometric parameters of the test. DCHP results were only analysed to test validity, defined as the inter group difference between DCDP and DCHP. Data were analysed with SPSS, version 24 (Statistical Package for the Social Sciences).
Results
Item analysis
One approach to item analysis is to measure the difficulty index (p value). The item difficulty index is the proportion of participants who answer an item correctly out of the number of all participants multiplied by 100%. The item difficulty index ranges from 0 to 100%. In other words, a high item difficulty index means that an item is easy and everyone gets it right (Green, 2013). The item difficulty index was calculated for all items of the adapted PJM RST. Five items (number: 27, 31, 37, 42) with a difficulty index lower than 25% (meaning that scores were below chance on a multiple-choice task with four options) were removed from the subsequent analysis (Haug, 2011b). These items may be considered too difficult for this age group. As the children included in the study were from the older age range of the original BSL RST (which was normed on children aged 3–11 years), we did not exclude nine items that had a high difficulty index (in other words, easy items), because they might be sensitive to differentiate PJM skills in younger children in future studies on the PJM RST.
A test should have an average item difficulty index equal to 50% when a choice of answers is not provided. In a multiple choice test, the probability of guessing has to be taken into consideration. To maximize item discrimination, a certain difficulty level is desirable depending on the number of given options. In the PJM RST, 33 items have three distractor pictures (four pictures per item) with a mean difficulty index equal to 85%. For three-response multiple choice items (two distractors + target), the item difficulty index is 89%. Lord (1952) suggested that a desirable item difficulty percentage for four-response multiple choice questions is 74% and for three-response multiple-choice questions, 77%. On this basis, the PJM RST is relatively easy for the deaf children tested. However, this may be because the children were all in the older part of the age range for the test. Taking into account that the RST was designed for children from the age of 3 years, a limitation of the main study is that the youngest child in the sample was 6;1 years. We anticipate that the PJM RST will be more appropriately challenging for participants younger than 6;1 (age range: 3–6 years).
The item discrimination index shows how success on an item corresponds to success on the whole test (Green, 2013). In order to compute the item discrimination index two groups are derived from the sample: the best scoring participants (master’s group) and the lowest scoring participants (no-master’s group). The item discrimination index ranges from −1 to +1, where the score +1 means that all of the best scoring participants got an item correct, while all the lowest scoring participants got it wrong (Green, 2013). Although the discrimination index was calculated, it should be interpreted with caution, because the sample had relatively high scores with low variability (as mentioned above, all were from the older age range for the RST). Only one item was below 0: this item had already been detected by the difficulty index as an outlier (item 37, HOUSE-TOP-RIGHT, English “the house is on the top right side (of the street)”). This item was answered correctly more often by the lower scoring group than by the higher scoring group.
Validity
One method for testing validity is the analysis of group differences when two groups are expected to have different levels of the measured construct (Bachman, 2004). The test should reflect this difference in the scores. We hypothesized that native signers (DCDP), immersed in sign language from birth, would have better scores in the PJM RST than late learners of PJM (DCHP). The Shapiro-Wilk Test demonstrated that the data in both groups were normally distributed (Table 4) and Levene’s Test confirmed the homogeneity of variance (F = 5.372, p > .05). According to the t-test, the difference between DCDP and DCHP was statistically significant (t(46) = 5.81, p < .001, Cohen’s d = 0.158) (Figure 2). This shows that native signers (DCDP) obtained significantly higher scores than deaf children with more limited and later access to sign language (DCHP), and provides evidence supporting the PJM RST. The small effect size (Cohen, 1992) may be the result of the small sample size.
Descriptive statistics for performance on the PJM RST for DCDP and DCHP.

Mean scores on the PJM RST for deaf children of deaf parents (DCDP) and deaf children of hearing parents (DCHP).
Another way to provide validation evidence is to verify the correlation between two tests that are supposed to measure the same construct (Hornowska, 2007). In Poland, there are no other developmental assessments to measure PJM. Therefore, it was not possible to use a comparable test to provide evidence of concurrent validity of the PJM RST. We therefore sought indirect evidence by looking for a lack of correlation between the PJM RST and a test (Raven’s Progressive Matrices) designed to measure another construct (abstract reasoning, fluid intelligence) (Hornowska, 2007). The raw scores in Raven’s Progressive Matrices were translated into 10 scores with a mean (M) of 5.5, a standard deviation (SD) of 2 and maximum of 10. DCDP scores ranged between 5 and 10, and, none of the children scored below 1 SD (M = 7, SD = 2). The Shapiro-Wilk test shows that the scores on Raven’s Progressive Matrices do not have a normal distribution (W(20) = 0.867, p < .05), therefore, the nonparametric Spearman’s rho correlation was used. The PJM RST results do not correlate with scores on Ravens Progressive Matrices (rho = −0.283, p > .05) in the DCDP group, suggesting indirectly the validity of the PJM RST as a measure of language ability, although of course this needs further investigation.
Reliability
Internal consistency was measured by the Kuder-Richardson formula (KR-20 = 0.737). A Kuder-Richardson formula between 0.7 and 0.8 is commonly described as acceptable reliability for a test (Hornowska, 2007). This score is thus a demonstration of the reliability of the PJM RST.
Sensitivity to age
Sign language acquisition is a developmental process and, in consequence, we can hypothesize that the scores of the PJM RST should correlate with chronological age. Two variables, the PJM RST scores (W(20) = 0.953, p > .05) and age (W(20) = 0.949, p > .05) have a normal distribution according to the Shapiro-Wilk Test. For this reason, the parametric Pearson correlation was administered. A strong correlation between the PJM RST scores and age (a large effect size according to Cohen, 1992; r = 0.63, p < .01) (Figure 3) provided further support for the validity of the test.

Scores on the PJM RST by age (in months) for deaf children of deaf parents.
Discussion
The adaptation of the BSL RST into PJM is a long-term process demanding methodological, linguistic, and cultural analyses which cannot be limited to simply translating from one language to another. To adapt a test from one sign language to another one requires knowledge about differences and similarities in sign languages which must begin with cooperation between researchers. Therefore, the first seven steps in the adaptation of the BSL RST into PJM were undertaken in collaboration with the authors of the BSL RST, who shared their experience of sign language acquisition research and test development and also assisted with the revision of the items for the PJM version. Readers are encouraged to read the online supplement for a detailed description of the seven first steps in the eight-step process.
In the online supplement, pilot tests 1, 2, and 3 are described in detail as parts of the first seven steps. Based on the three pilot studies, changes were made and 10 new items developed (seven were included in the final analysis) for the PJM RST. Five linguistic features were assessed in the PJM RST: (1) negation; (2) number and distribution; (3) spatial verb morphology; (4) size and shape specifiers; and (5) handling classifiers, based on comparative analyses of BSL and PJM.
The first linguistic feature from the BSL RST, negation, is present in PJM and is comparable to BSL. Like the BSL version, the PJM RST contains several linguistic negation markers: manual negators, non-manual negation only, and signs with negative incorporation (Kuder, Filipczak, Mostowski, Rutkowski, & Johnston, 2018). However, not all available manual negator signs in PJM were assessed. A similar situation has been described for the BSL RST adaptation into LIS: LIS signers reported a greater variety of negation markers than BSL users (Surian & Tedoldi, 2005). The LSF RST’s authors (Courtin et al., 2010) also faced problems with the linguistic category of negation: LSF seems to contain fewer negation structures than BSL. Courtin et al. (2010) highlighted that in the BSL RST, negation features are assessed in 10 items out of 40 (25%). As negation is quite complex in PJM, a large number of items relating to negation is justified for the PJM RST (10 items out of 43 analysed items, 23%), whereas this may not be the case in other sign languages, such as LSF.
The category of number and distribution was comparable in PJM and in BSL, so the items from this category were adapted from the BSL RST into PJM without major changes, although with some small adaptations in relation to the classifier handshapes expressing number and distribution. Similar changes were required for the ASL RST version (Enns & Herman, 2011), for example where the classifier handshape for vehicles (e.g. CAR) is different from BSL. Another difference was the position of the quantifier “a lot of.” In BSL this usually occurs pre-nominally, whereas in PJM it appears more often post-nominally. In both the DGS RST (Haug, 2011a) and PJM RST, items have post-nominal quantifiers, although this change from the original BSL RST was explicitly planned and discussed by the expert panel for the PJM adaptation; whereas in the DGS RST, the change was made spontaneously during the filming, and only subsequently accepted by other specialists (Haug, 2011a).
The category of spatial verb morphology is also present in both BSL and PJM: items in this category were adapted into PJM without major changes. As far as is known, for the first time complex AB constructions were identified in PJM and confirmed by the expert panel and corpus data.
Size and shape specifiers, as well as handling classifiers, are found in both BSL and PJM; the items from the BSL RST were adapted into the PJM RST with some small changes, for example in handshape of the size and shape specifier for item 16 (CURLY HAIR): In PJM the handshape is “all fingers extended, spread and loosely curved,” whereas for BSL, “the index finger is extended and loosely curved from the fist.” Haug (2011a) has highlighted that the similarity in the categories of size and shape specifiers and handling classifiers in many sign languages explains why the RST may be adapted to other languages, including languages which are less well documented and less researched, such as PJM.
Only one structure from the BSL RST was removed: the verb–noun movement distinction that has not been reported in the PJM literature or observed in the corpus data. The same situation has been described for the DGS RST (Haug, 2011a): The movement difference between noun and verb derivationally related pairs does not exist in DGS. Even if the structure is present in a sign language, it may of course be found in different noun–verb pairs; for example the verb–noun distinction exists in Auslan (Johnston, 2004) and LSE (Valmaseda et al., 2013). However, the pair WRITE and PENCIL from the BSL RST did not occur either in Auslan or LSE, since in both Auslan and LSE the sign WRITE is not derivationally related to the sign PENCIL. For the Auslan RST, item 26 (PENCIL) was removed because Johnston (2004) did not want to change any of the test pictures and also wanted to use the norms for the BSL RST, justified by the close historical relationship between BSL and Auslan. For LSE (Valmaseda et al., 2013), the item was changed twice (first to the pair CHAIR-SIT and then to the pair CLOTHES-DRESS UP).
Based on statistical analyses of the findings from the main study (Step 8 in Table 2), using scores from 20 deaf children who were native signers aged from 6;1 to 12;11 and from 30 deaf children who were late learners of PJM aged from 6;7 to 13;8, the PJM RST was found to be an assessment tool with acceptable psychometric properties.
In the analysis of the PJM RST, we compared DCDP and DCHP scores. It is highly likely that even if being raised by deaf parents does not guarantee native-like sign language acquisition, there is a much greater probability that deaf parents provide a good sign language environment for deaf children. The specific characteristics of deaf families that are crucial for deaf children’s sign language development are still under debate (Haug, 2011a): whether what is most important is parents’ sign language skills, or whether other factors may play an important role in the child’s sign language development, such as the age of first exposure to sign language, or the number of signers in the child’s family. Even if the reasons that make deaf parents good providers of a good linguistic environment are not very well researched, DCDP are usually reported to have better sign language skills than DCHP (Herman & Roy, 2006). DCDP outperformed DCHP on the original BSL RST (Herman & Roy, 2006) as well on the DGS (Haug, 2011a) and Auslan (Johnston, 2004) adaptations.
Another question that still needs to be addressed is which types of education support native-like development of sign language in deaf children and which ones support specific characteristics of hearing parents’ communication. When DCHP met criteria for inclusion (a high degree of exposure to BSL before the age of 5 years, as some hearing parents are able to assure a good sign language environment, and participation in a bilingual education system that may support native-like sign language proficiency in DCHP) they were also included in the normalization sample of the BSL RST (Herman et al., 1999). The BSL RST adaptation into LSE (Valmaseda et al., 2013) was used to investigate the importance of bilingual education for sign language receptive skills in deaf children. When DCHP were immersed in bilingual education before the age of 4 years, the two groups DCDP and DCHP did not differ (Valmaseda et al., 2013). In the present study, inclusion of DCHP immersed early in bilingual education was not possible because there is no Polish educational setting with a strong and early bilingual policy. In their study on the ASL RST, Allen and Enns (2013) found that regular use of sign language at home might be also an important contributor to high RST scores: In their study, DCDP did not differ from DCHP whose parents reported communicating in ASL on a daily basis with their deaf children. Significantly lower scores on the ASL RST were found in DCHP who did not use ASL at home (compared to both DCDP and DCHP with ASL at home). At the time of the Polish data collection, the inclusion of DCHP with regular use of sign language was problematic, because support for hearing parents to learn and use sign language was limited and in need of modification. Hence, a very limited number of hearing parents had started to learn PJM and mostly communicated with their deaf children using spoken language with some isolated signs and fingerspelling.
Investigations of test concurrent validity may aim to correlate the test under development with other measures assessing the same skills (Bachman, 2004). However, no measure to assess deaf children’s knowledge of PJM is available. Analysing RST validity, Johnston (2004) and Courtin et al. (2010) suggested that it is possible that the RST does not measure sign language skills but instead more general visuo-spatial skills since there are a large number of items testing sign language features with high iconicity (Courtin et al., 2010). Hence, in the present study, scores on the PJM RST were compared to an assessment tool that measures non-verbal abilities in logical reasoning in order to check if the PJM RST measures the logical reasoning ability rather than PJM skills. The PJM RST scores did not correlate with Raven’s Progressive Matrices, a measure of non-verbal intelligence (abstract reasoning). Hence, the present data showed that scores on the PJM RST are not correlated with visual non-verbal measures. A similar analysis of validity was undertaken during the standardization of the BSL RST, with BSL RST scores compared to two non-verbal subtests of the Snijders-Oomen Test: one that was supposed to be connected with language skills (Mosaic) and another that was a pure non-verbal measure (Categories subtest) (Herman et al., 1999). The BSL RST scores correlated with the Mosaic subtest and not with the category subtest, indicating that the BSL RST correlates with other language measures, but not with non-verbal intelligence generally.
The developmental sensitivity of the test was explored and the PJM RST scores correlated with the age of participants. Other RST adaptations have also reported to be sensitive to age: DGS (Haug, 2011a), ASL (Enns, 2013), and LSE (Valmaseda et al., 2013). Haug (2011a) stressed that DCDP scores started to approach ceiling values at 6–7 years, with average scores of 44/49, and DCHP approached ceiling values at 7–8 years, indicating that the DGS RST is not sensitive enough for children older than this. The same issue was captured in other adaptation processes (Enns & Herman, 2011; Valmaseda et al., 2013). This highlights the need to test younger deaf children in order to validate the use of the PJM RST. Additionally, it may also suggest that the BSL RST, originally developed 20 years ago (Herman et al., 1999), needs to be revisited and that more difficult items would make the BSL RST more sensitive to assess older children’s understanding of BSL grammar.
The item analyses revealed that for the age group tested, the PJM RST is a relatively easy test. This may be owing to the fact that 3–6-year-old children were not tested. However, Haug (2011a) reported that the DGS RST also contained many easy items, even though he tested children aged 3;9–10;10.
Limitations of this study include small sample size and age range restricted to older children (native signers: age range: 6;1 to 12;11 and late learners of PJM: age range: 6;7 to 13;8). The small number of deaf children of deaf parents is explained by the limited number of them in the deaf children population (Qi & Mitchell, 2012). Recruitment in the present study was via schools, and this determined the age of children because in Poland children enter school at the age of 6 years. As kindergarten is not obligatory in Poland, the recruitment of younger children is more time and resource consuming and is planned as a next step in the adaptation process. The biggest issue that was encountered during the present study was the lack of developmental research on PJM acquisition to provide a strong basis for the PJM RST adaptation. For this reason, different expert panels were included in the research.
In conclusion, the PJM RST, the first assessment tool to measure PJM skills in children, is a pioneering work in Poland that needs to be further developed. To confirm the reliability and validity of the PJM RST, younger deaf children need to participate in future studies, with the final goal to standardize the test on a larger and more representative sample.
Supplemental Material
sj-pdf-1-ltj-10.1177_0265532220924598 - Supplemental material for Adaptation of the British Sign Language Receptive Skills Test into Polish Sign Language
Supplemental material, sj-pdf-1-ltj-10.1177_0265532220924598 for Adaptation of the British Sign Language Receptive Skills Test into Polish Sign Language by Justyna Kotowicz, Bencie Woll and Rosalind Herman in Language Testing
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Polish Ministry of Science and High Education (mini Grant, IP DS 2011).
The manuscript is based on unpublished research used in a doctoral dissertation posted in institutional archives.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
