Abstract
Keywords
Students who are visually impaired encounter challenges in science, technology, engineering, and mathematics (STEM) classes due to the visual nature of the educational content and their need for specialized instruction methods and accessible materials (Bell & Silverman, 2019; Jones et al., 2012; McDonnall et al., 2009; National Science Foundation Division of Science Resource Statistics, 2009; Rosenblum et al., 2018, 2021a, 2021b; Smith, 2017; Wild & Koehler, 2017). To learn and perform at grade level in STEM classes, students who read braille require direct instruction, high-quality braille materials, and multiple opportunities to practice (Bell & Silverman, 2019; Beal & Rosenblum, 2015, 2018; Holbrook & Rosenblum, 2017a, 2017b; Rosenblum et al., 2018). Thus, students who read braille must have support from well-prepared and knowledgeable teachers of students with visual impairments who can provide accurate and tactually clear braille materials in a timely manner. However, accuracy of braille materials varies, depending on training and expertise of the individual who prepares the materials (Bell & Silverman, 2019; Herzberg, 2010; Herzberg & Rosenblum, 2014).
In 1952, the United States adopted the Nemeth Braille Code for Mathematics and Scientific Notation (Nemeth Code) as the official code for mathematics and science materials (Navy, 2009). When the Unified English Braille (UEB) Code was implemented in the United States in 2016, the Braille Authority of North America (BANA) adopted both UEB Technical and Nemeth Code within UEB Contexts. There has been considerable ambiguity and sometimes confusion about how to merge the original Nemeth Code and the literary aspects of the UEB Code to create Nemeth Code within UEB Contexts (Herzberg & Rosenblum, 2022). In addition, BANA (2020) recommended changing the term “Nemeth Code within UEB Contexts” to “UEB with Nemeth”, but the recommended terminology does not appear to be widely used in the field, nor will it be included in the next Nemeth Code book. Not until August 19, 2018, more than 2 years after BANA announced the use of Nemeth Code within UEB Contexts as one of the two options for production of STEM materials did BANA release a guidance document (BANA, 2018). Although these guidelines are valuable to teachers of students with visual impairments and others, they are not comprehensive. In 2022, there are limited training materials available for teachers of students with visual impairments to use to build their skills with this code. Available resources include Cleveland et al. (2017) and a series of on-demand courses developed through Project INSPIRE: Increasing the STEM Potentials of Individuals Who Read Braille.
In Fall 2019, Herzberg and Rosenblum (2022) conducted a national survey and collected responses from 219 teachers of students with visual impairments from 38 states. Just under half (49%) of the respondents reported their reading skills for Nemeth Code within UEB Contexts were at the beginner level and 56% reported their writing skills were at the beginner level. These data were collected as the United States approached the 4-year mark of implementing UEB, yet only 9% of respondents believed their reading and writing skills were at the advanced level. Ninety-seven percent of respondents indicated it would be “very helpful” or “helpful” to offer face-to-face workshops, and more than half (56%) of the respondents said it would be very helpful to receive feedback on materials they produced using Nemeth Code within UEB Contexts.
The current study had two purposes. The first purpose was to examine if a 6-h synchronous professional development opportunity increased the skills of teachers of students with visual impairments in using Nemeth Code within UEB Contexts. The second purpose was to examine the frequency and patterns of errors made by participants as they prepared an assignment containing a title and 12 mathematics problems.
Method
The study was approved by the Institutional Review Board at the University of South Carolina. Participants provided informed consent prior to registering for the online professional development.
Recruitment
In April 2022, the professional development opportunity was advertised on electronic discussion groups, in Facebook groups, and on the social media accounts of both Paths to Literacy and Paths to Technology. Individuals were invited to apply to take part in the professional development if they were teachers of students with visual impairments in the United States and would have a student in grades 6–12 who reads Nemeth Code within UEB Contexts (referred to as Nemeth Code for the remainder of the article) on their caseload during the 2022–2023 school year. Applicants also needed to be able to read and write Nemeth Code symbols used in the early elementary grades.
Data were collected using Qualtrics, an online survey tool on a protected server at the University of South Carolina Upstate. The application included demographic questions and required applicants to answer multiple choice questions about how five mathematics problems (e.g., 8 + (12 ÷ 4) = _____) would be transcribed in Nemeth Code to ensure applicants knew elementary level Nemeth Code symbols. For each problem, they were shown the print problem and asked to select the correct Nemeth Code representation from four provided choices. Applicants who were braille readers were provided an accessible version of the problems and answer choices. Applicants were also provided statements about their knowledge of Nemeth Code, including formatting, and resources for preparing Nemeth Code materials and asked to rate their level of agreement with each statement. A final question asked applicants why they wanted to enroll in the training.
Selection Process
Seven of 35 applicants were not teachers of students with visual impairments or were not currently working with, nor did they anticipate working with, a braille reader in grades 6–12 for the 2022–2023 school year. Thus, 28 applicants were invited to take part in the training.
Topics Covered in the Professional Development
The topics covered in the professional development corresponded to mathematics content covered in the Common Core State Standards (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) in grades 6–12. From the authors’ professional experiences, many teachers of students with visual impairments seek resources to assist them in preparing secondary mathematics instructional materials and teaching Nemeth Code symbols to secondary students. The authors created all materials used in the sessions. Session 1 targeted mathematics symbols used in the middle grades:
grouping symbols (e.g., brackets, braces); negative numbers; absolute value; signs of comparison (e.g., less than or equal to, not equal to); exponents (e.g., superscript indicator, baseline indicator); number line symbols (e.g., open circle for point not included, bold shaded line segment); and Nemeth Code switch indicators. shapes (e.g., shape indicator, triangle, right arrow); signs of comparison (e.g., not congruent to, perpendicular); degrees; Greek letter indicator and the Greek letter pi; and prime and double prime. vertical bar (e.g., magnitude, determinants); infinity; superscript with superscript indicator; enlarged grouping symbols; set theory (e.g., empty set, union); and Greek letters (e.g., beta, sigma).
Session 2 focused on geometry:
Nemeth Code symbols used in Algebra 1, Algebra 2, and advanced mathematics courses were discussed in Session 3:
Procedures, Including Structure of the Professional Development
Prior to the first day of the professional development, participants who met the study requirements completed a 20-item pretest that was presented online in the same format as the five items prospective applicants saw during the application process. In June 2022, participants met online using Zoom video conferencing software for three 2-h instructional sessions. The first four authors were instructors, with two of the authors providing instruction for each session. Prior to the session, participants were provided a PowerPoint of the content and a list of Nemeth Code symbols that would be used.
Following the first two sessions, participants completed an assignment in which they brailled six problems and interlined six problems. This homework provided additional practice with the content covered in the sessions. Participants submitted their completed assignment to the second author who was responsible for tracking the progress of participants. The participants received an answer key to check the accuracy of their work.
At the end of the second and third sessions, the instructors answered participants’ questions about the assignments. Following the third session, the participants were asked to braille a final assignment (See Figure 1) that contained a title and 12 mathematics problems that were similar to problems discussed across the three sessions. Participants submitted their transcription of the final assignment to the second author. After all assignments were received, participants received an answer key. However, they did not receive individualized feedback regarding their braille transcription. Participants also completed a post-training survey and posttest that was identical to the pretest. Seventeen of the participants completed all study requirements and received a $25 Amazon gift card.

Final Assignment Print Worksheet.
Data Analysis Procedure
Pretest, posttest, and survey data were entered into Excel and analyzed by the fifth author using descriptive statistics. The first three authors developed a detailed online form to evaluate the accuracy of 96 items in six categories on the final assignment. We used Nemeth Braille Code for Mathematics and Science Notation (Nemeth, 1972), Guidance for Transcription Using the Nemeth Code within UEB Contexts (BANA, 2018), and Braille Formats: Principles of Print to Braille Transcription (BANA, 2016) to guide the development of the form. The 96 items were categorized according to the following definitions:
Formatting (17 items): Accurate formatting of a centered heading and problems including multiple-choice problems with answer choices. Literary text (13 items): Accurate use of letters and contractions, excluding words used with single-word switch indicators. Punctuation (12 items): Accurate use of UEB or Nemeth Code punctuation, including dot configuration and spacing. Nemeth Code (36 items): Accurate use of Nemeth Code symbols, including dot configuration and spacing of the symbols. Switch indicators (12 items): Accurate use of opening and terminating switch indicators including placement of switch indicators as related to the mathematics, excluding dividing the mathematics. Single-word switch indicator (6 items): Accurate use of the single-word switch indicator, including dot configuration, use of contractions, and spacing. Math placement (up to 10 items): Switch indicators and math content placed on the same braille line when possible. Math divided across lines (up to 11 items): When divided, math content divided according to the BANA guidelines.
Participants were evaluated on items from two additional categories, in which the number of items for each participant varied based on the number of characters per braille line.
The second author selected sample problems from the participants’ final assignment for the first three authors to practice using the online form, determine how errors would be documented, and refine the definitions and form as needed. Next, the first and third authors separately scored two final assignments and met to discuss any differences and reach consensus. This process established interobserver agreement. Subsequently, the first author scored nine submissions and the third author scored six submissions.
Results
Background Information
Twenty-three participants attended Session 1, 19 attended Session 2, and 16 attended Session 3. One participant who had a medical emergency was excused from Session 3 and was permitted to complete the posttest, final assignment, and posttraining survey. Table 1 contains participant demographic information. The 17 participants were from 11 states—the most were from California (n = 4), Texas (n = 3), and Washington (n = 2). They had taught students with visual impairments for an average of 10.8 years.
Background Information of the 17 Teachers.
Changes in Participants’ Nemeth Code Knowledge, Confidence, and Knowledge of Resources
The same 20 problems were used in both the pretest and posttest. Pretest scores ranged from 3 to 18, M = 11.18 (SD = 4.33) and posttest scores ranged from 11 to 20, M = 16.70 (SD = 2.49). The mean difference in scores from the pretest to the posttest was 5.53 (SD = 3.32). The pre-post differences were statistically significant (t = 6.87, p = .00).
As part of the application process, participants were asked to rate their agreement from strongly disagree (1) to strongly agree (4) with the statement “I am confident in my ability to prepare material in Nemeth Code within UEB Contexts for PreK-1st grade students.” They were asked the same question for 2nd–5th grade, 6th–8th grade, and 9th–12th grade students. The same four questions were repeated on the posttraining survey. A participant's confidence ratings for the questions could range from 4 to 16. The mean pre–post difference was 2.24 (SD 1.92) and differences were statistically significant (t = 4.80, p = .00).
Using the same Likert scale, participants also rated their agreement with the statement “When I am preparing Nemeth Code within UEB Context materials for PreK-5th grade students, I have resources I can consult to ensure I am accurately preparing the material.” The participants were asked the same question for 6th–8th grade and 9th–12th grade students. A participant's resource ratings could range from 3 to 12. Pre–post differences were significant (t = 4.56, p = .00). See Table 2 for additional information about test scores, confidence ratings, and resource ratings.
Differences in Teacher'S Pre–Post Test Scores, Confidence in Preparing Nemeth Code Material, and Difference in Knowledge of Resources.
Participants’ Feedback About the Sessions
On the post-training survey participants were asked to rate their agreement from strongly disagree (1) to strongly agree (4) on the statement “The information from the session on Symbols Used in the Middle Grades will help me in my future work with braille readers.” They also rated the other two sessions. Using the same scale, they rated the knowledge of the presenters, the content of the PowerPoint slides, the activities completed during the sessions, and the assignments they completed following the first and second sessions. For all items, the participants were assigned either a rating of agree or strongly agree, indicating that the way in which the sessions were presented and their content was valuable (see Table 3).
Teachers’ Ratings of the Professional Development Components.
Ratings ranged from (1) strongly disagree to (4) strongly agree on the rated items.
When asked to share positive aspects of the training, participant comments included the following comments. I appreciate all of the hard work that the instructors put in to present this class …. I find I benefitted more from this platform than from one that doesn’t have the Zoom meetings. I like to be able to talk with others and be able to find out that my questions are ones others have too. These sessions did help me with understanding the Nemeth within UEB. I did see growth, however not as much as I would like, and that's ok! With time and experience it will be a little more automatic. The homework [assignments after sessions 1 and 2] was very helpful and the resources will be great to use to help me remember things. I feel a lot more confident in my UEB to Nemeth knowledge. An absolutely wonderful learning adventure. The materials and manner of presentation worked very well for me. I enjoyed the comradery between the presenters and the respect shown to the students when responding to questions.
Constructive criticism about the structure of the sessions included the following statements. Additional time to do activities that were embedded into the sessions to enable teachers to practice with the Nemeth Code material. Additional sessions, especially for the content in session 3 that focused on Algebra and more advanced mathematics. Have the slide posted with directions and content for the activities in the Zoom breakout rooms. (Directions were provided in the large group and content was part of the PowerPoint slides provided ahead of time.) Provide answer keys that contain both the print and Simbraille in one document. Written individualized feedback on the assignments provided after the session. Additional time during Sessions 2 and 3 to review in detail the assignment just submitted.
A few participants commented about the Nemeth Code content that they wished had been discussed more in-depth including information about Nemeth Code switch indicators, formatting, and Nemeth Code symbols for advanced chemistry.
Analysis of Participants’ Errors in the Final Assignment
Table 4 reports the percentage of errors on the 96 items separated by category; the highest percentage of errors by category are marked with an asterisk. All participants made at least eight errors, and four participants made errors on half or more items (participants 7, 8, 9, and 13).
Percentage of Errors Teachers Made on the Final Assignment.
The highest percentage of errors by category.
Formatting and Literary Text Errors
Accurate formatting allows individuals who read Braille to easily and quickly navigate documents (BANA, 2016; Herzberg, 2010). Only 3 of the 17 participants formatted all elements of the worksheet correctly, and more than half (n = 9) had an error rate higher than 25%. Common formatting errors included dividing problems between pages, not centering the title, and formatting problems incorrectly.
Literary Text Errors
Six of the 17 participants made no literary errors. In contrast, seven of the participants had literary errors in more than 25% of the problems. Common literary errors included: not using the initial-letter contraction for the word these, using groupsign ea instead of groupsign ar in the word year, not using the groupsign ity in inequality, not using the groupsign ment in supplementary, omitting words, and misconfiguring letters and groupsigns.
Punctuation Errors
Four of the 17 participants had no punctuation errors, and over half (n = 10) of the participants’ punctuation error rate was 25% or less. Common errors included: omitting punctuation signs such as the period, misconfiguring the mathematical comma, and using the punctuation indicator instead of closing Nemeth Code and transcribing the punctuation mark in UEB.
Nemeth Code Errors
Only one participant had no Nemeth Code errors. More than half (n = 11) of the participants had a Nemeth Code error rate of more than 25%. It should be noted that some of the errors changed the problem. For instance, one of the participants transcribed 4 instead of 6 in
Switch Indicators Errors
Only one participant used the opening Nemeth Code indicator and the Nemeth Code terminator correctly throughout their transcription. Eight participants had an error rate of 1% to 25%, and eight had an error rate higher than 25%. Some participants did not close Nemeth Code correctly when there were several words between two mathematics elements, while others misconfigured the opening Nemeth Code indicator or failed to leave a space before and after a switch indicator.
Single-Word Switch Indicator Errors
Seven of the 17 participants had no single-word switch indicator errors. About a third (n = 6) of the participants had a switch-indicator error rate of 1% to 25% while 4 participants had a switch-indicator error rate of more than 25%. Five participants misconfigured the single-word switch indicator, and others did not include a single-word switch indicator before a word within a mathematics element or erroneously used a single-word switch indicator for 2–3 consecutive words between two mathematics elements.
Placement of Math and Switch Indicator Errors
We examined if participants placed the math and switch indicators on the same line when possible. The number of opportunities varied with 15 participants having 9 opportunities and 2 participants having 10 opportunities. Two of the participants placed math and switch indicators on the same line consistently. Seven participants made one or two errors, and the other eight participants made between three and five errors.
Math Divided Across Lines
We counted the number of times each participant did not follow BANA guidelines when dividing math elements across lines. The number of opportunities to divide math elements ranged from 0 to 11 with an average of 6 opportunities. Three participants properly divided all math elements, two participants made only one error, and 11 made two or more errors, with two participants making errors every time they had the opportunity to divide math expressions or problems. The most common errors were dividing between a shape and identifier and dividing a problem after the sign of comparison instead of before the sign of comparison.
Discussion
“The Nemeth Code can be intimidating to learn, so this [professional development] provided the support of knowledgeable instructors and a community of other professionals to share and learn collectively as teachers for students with visual impairments. I loved it!”
All individuals preparing braille materials must ensure the materials they produce are accurate. Seventeen teachers of students with visual impairments participated in a 6-h synchronous professional development training in June 2022. Even after direct instruction and access to the PowerPoint slides containing information and examples, none of the 17 teachers produced an error-free transcription of a worksheet containing a title and 12 math problems typical of content in grades 6 to 12. These data are similar to previous research that examined the accuracy of literary materials (Herzberg, 2010) and math materials (Herzberg & Rosenblum, 2014). For example, frequently occurring errors in this study and both previous studies were correctly formatting numbered problems, omitting words, and misconfiguring braille symbols.
The three categories with the highest percentage of errors in this study were formatting (M = 43%), use of opening and terminating switch indicators (M = 34%), and Nemeth Code (M = 31%). During the synchronous meetings, as the instructors presented the material, they were asked for more clarification on these topics in addition to the single-word switch indicator. Information is sometimes difficult to locate in BANA resources and guidelines, potentially leading to teachers of students with visual impairments “guessing” rather than verifying as they braille for their students. It is clear from this study that teachers of students with visual impairments need continued professional development on topics related to formatting and Nemeth Code within UEB Contexts, but also on literary braille.
BANA did not provide a guidance document on how to prepare braille materials using Nemeth Code within UEB Contexts until 2018 (BANA, 2018). When the document was published the word “may” was used throughout the document and that added to the ambiguity for practitioners in the field. The authors have observed that there were personnel preparation programs, braille transcribers, and teachers of students with visual impairments in the field preparing braille materials who initially were not using the opening Nemeth Code switch indicator, Nemeth Code terminator, and single-word switch indicator properly. For participants with less than 6 years of experience, it is possible that they learned one or both codes, that is they learned English Braille American Edition and the original Nemeth Code; UEB for literary context and Nemeth Code within UEB Contexts for mathematics and science; and/or UEB for all materials.
Limitations
The study was conducted during the summer, a time when some teachers take a much-needed break from their professional responsibilities. The professional development was short, only 6 h, and this amount of time may not have been sufficient to provide comprehensive training. The authors did not completely verify participants’ prior knowledge of the Nemeth Code. Participants were not asked if they learned the Nemeth Code (Nemeth, 1972) or the Nemeth Code within UEB Contexts (BANA, 2018) in their personnel preparation program. A brief multiple-choice instrument was used, but it only included some of the Nemeth Code symbols used when transcribing materials for elementary students. The directions for the assignment did not specifically state that participants were to take the care they would take if preparing the assignment for a braille reader. It is possible that some of the errors would not have been made had the participants known their braille transcription would be used in a classroom by a secondary student. The authors did not evaluate the participants’ ability to interline braille mathematics problems, an important skill for teachers of students with visual impairments. The small sample size of 17 participants does not allow for the generalization of study findings to the population of teachers of students with visual impairments. However, it is the first study to explore patterns of errors since the Nemeth Code was updated.
Future Research
The present study needs to be replicated with a larger sample size. Individuals replicating the study should request that participants prepare the final assignment as if they were preparing an assignment for a student who reads braille. Samples of actual materials participants have prepared for their students should also be collected and analyzed. Comparing both materials participants completed for the study and those used by their students will allow researchers to have a more robust picture of participants’ skills.
An additional line of inquiry should focus on evaluating the impact inaccurate STEM braille materials have on how students learn. Clearly, students cannot be expected to master content if they do not have access to accurate materials. The long-term impact of inaccurate materials on students’ understanding of STEM content and their academic performance needs to be better understood. The authors hypothesize that there are likely two contributing factors as to why few students with visual impairments enter the STEM fields. The first is lack of access to accurately prepared braille materials (Herzberg & Rosenblum, 2014). The second is teachers’ lack of knowledge of Nemeth Code within UEB Contexts and STEM content, especially higher level mathematics knowledge (Bell & Silverman, 2019). Without teachers who pre-teach, teach Nemeth Code, and fully support students with new concepts, students’ ability to learn alongside sighted peers may be diminished.
Implications for Practitioners
The study participants indicated that synchronous professional development provided them with an opportunity to interact with presenters and other teachers. Teachers of students with visual impairments should actively seek opportunities to continue building their knowledge base of the braille code throughout their career. They can informally organize study sessions with other teachers, transcribers, and paraeducators. They can also advocate for professional development that builds their competence with braille STEM codes. Those designing professional development should consider ways in which they can provide “live” training opportunities.
Based on the data collected, it is clear that practicing teachers of students with visual impairments may benefit from refresher courses in braille codes, such as the Nemeth Code within UEB Contexts. Professional development that targets specific topics such as formatting or the use of the single-word switch indicators may be beneficial. Providing individualized feedback on assignments submitted by participants throughout the professional development will allow participants to learn from their mistakes Ongoing professional development should be provided using multiple formats of delivery to reach a geographically dispersed population.
Footnotes
Acknowledgment
The authors thank the teachers of students with visual impairments who participated in the study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the University of South Carolina’s Research Initiative for Summer Engagement competition (internal award/no award number) from the Office of the Vice President for Research at the University of South Carolina.).
