Working to Learn Together: Engaged Scholarship Addressing Long-Term Unemployment

Clair’s Reflection

As the program director of Maryland New Directions (MND), an organization providing job readiness training and employment assistance for long-term unemployed and underemployed Baltimore City residents, my job is to evaluate the program’s effectiveness. Through this process, I noticed a troubling trend—prospective clients/students were not giving us all requested information during initial meetings. In these meetings, we asked potential clients/students to complete an intake form including demographic information and work history/experiences. This information assisted job coaches in understanding the client’s background and motivation for program participation. Missing data impacted evaluation of clients’/students’ program readiness.

Furthermore, I questioned whether clients understood the program’s curriculum, and more importantly, if they applied what they learned after the training ended. At the project’s onset, I wanted to create an intake process to accurately assess clients and identify a way to measure learning gains. I met Leah through a “community partner/engaged researcher speed dating event” at Loyola University. Afterward, we began working together to develop tools to address the program’s initial concerns and gather data about “what works” to share with the larger field of workforce development. Through our relationship, we developed a mutual commitment to learn from each other’s perspectives. Fundamentally, we were/are dedicated to collaborating for program improvement and client success.

Leah’s Reflection

When I first met with Clair and the MND staff, they expressed concern over missing intake data and briefly mentioned concerns with program retention. As an adult literacy researcher with expertise in authentic assessment, my mind immediately jumped to assessing for program readiness. Specifically, were the admitted clients/students academically, socially, and professionally ready to benefit from the program?

MND had a thorough intake system complete with paper application and in-person interview. So, I asked how program administrators decided on an individual’s admission/acceptance. Although MND evaluated social and professional readiness, they did not include any academic skill assessment in the intake process. I began conversations around how potential clients’ current workplace literacy/numeracy skills may (a) affect their ability to adequately complete the application or (b) persist in the classwork/curricular portion of the program. I suggested collecting data on the usability of the paper application with potential clients. From that data, along with programmatic insight, we could extrapolate what authentic workplace literacy/numeracy skill assessments might be necessary.

Clair’s Reflection

Initially, this partnership proved very beneficial. As a small nonprofit, we do not have the funding to hire an independent researcher/evaluator. We also do not have the knowledge of assessment to validly evaluate current tools or develop new tools. Leah brought this understanding of research and practice in adult literacy/learning to the table. MND brought not only access to raw data and implementing and examining the effectiveness of new tools but also content expertise of “real-world” application of learning/literacy skills in the workforce.

Our expertise gave Leah the insight to create assessments specifically tailored to assessing client/student capacity for applying the workforce learning skills in real-world contexts. We collected data using these new assessments and discovered how the new information impacted program admission and retention. The updated intake system also assisted in better referrals for students who were not yet academically, socially, or professionally ready for MND.

Leah’s Reflection

To utilize the initial data, we first revised and piloted a new application. The revision improved the usability for potential students/clients, and the program received more comprehensive data. Next, we assessed the current academic skill level of potential students/clients. As a result of ongoing engagement, I was acutely aware of the time constraints of the intake process versus the reliability/validity of academic assessments. This led me to explore the use of authentic screener probes that could be completed in 5 to 10 min versus using a much longer norm-referenced assessment. A norm-referenced assessment also would not give us the information we sought—whether learners could access the program’s curriculum. With these factors in mind, we created and validated a literacy assessment, and analogous numeracy assessment, that takes less than 5 min to complete and gives a real-world evaluation of current workplace literacy skills. For the first time, MND could assess a learner’s/client’s skills at the beginning of the program.

Clair’s Reflection

One benefit to working with a researcher who values community partnership is jointly developed objectives. We are very flexible and have developed a respect for our individual areas of expertise. We make room for this knowledge and expertise by working closely together to review and understand data. But more than that, we have learned to allow the data to determine “what’s next.” It is a very fluid process and works well for us. If I had to think of one challenge, it would be timely data collection. I heavily depend on my staff to collect the data we need to track and analyze—and sometimes that does not happen. So, ensuring all staff are on the same page and understand the project’s value and the purpose helps ensure efficient data collection and scoring.

Leah’s Reflection

Clair and her staff constantly work to make the program better for clients/students, so I follow their lead in project/research adjustments. Yet, we share the goal of continued improvement for the program and research. For me, commitment to partnership means I secede the direction of the overall purpose and product of the research to MND. This truth was initially frightening for me. I had fairly traditional training in research and evaluation which dictates certain methods and protocols. As an engaged scholar, I find myself constantly challenged to tweak purposes and products for clearer, more valid, or more generalizable outcomes without substantial change. Upon reflection, my main contribution has been guiding some processes/methods to bridge the gap between the organization’s purpose and desired product/outcome—even as these understandably change over time.

Clair’s Reflection

Before this project/partnership, I was not as passionate about adult literacy as I am now. I did not think about the relationship between literacy and employment but have come to realize “employment literacy” is a real issue that can prevent the people we serve from achieving self-sufficiency. Engaged scholarship has made me realize adult education providers and workforce development practitioners should better align work. We both work to assist clients/students toward self-sufficiency, but utilize different processes. However, the two are very intertwined. If clients cannot read the job description or accurately answer application questions, they cannot secure employment.

Leah’s Reflection

As an adult literacy practitioner/researcher, I am well acquainted with andragogical principals including the primacy of prior experience and the learner’s readiness, orientation, and motivation to learn (Knowles, Holton, & Swanson, 2015). Yet, I began to realize my limited understanding of these principals through the lens of workforce development/employment. For the first time, I observed how the goals and purposes for lifelong learning are sometimes under the control of the learner (adult education), and other times, these goals and purposes are dictated by organizational forces outside of the learner (workforce development). Despite this fundamental difference in positioning, I, like Clair, was pleasantly surprised to understand both stances privilege lifelong learning and the learner’s need to know. The overlap is significant and ripe for further exploration.