Abstract
Identifying the competencies required for the nurse manager role is a focus of much of the literature on nursing education and workforce. The unique nature of home health care nursing requires specially designed tools that reflect its distinctive nature. This article describes how one home health care nursing agency developed an assessment tool to measure nurse manager competencies and how the agency uses the tool to provide staff-development programs for their nurse managers.
Keywords
Introduction and Overview of the Project
Identifying the necessary skills and competencies required for the nurse manager role has been the focus of many articles and books on nursing education, workforce issues, and organizational development.1,2 Given its predominance in the market, the majority of literature on nurse manager competencies has focused on hospitals,3,4 though important contributions have been made regarding training and education. 5 To be sure, home health care nurse managers share some of the competencies associated with nurse managers in other settings. However, the unique nature of home health care nursing requires specially designed instruments and tools that reflect the distinctive nature of the delivery system, specifically around capacity and resource management, clinical oversight, and effective communication with a workforce that provides services across wide geographic areas. This article describes how one large not-for-profit home health care nursing agency developed an assessment tool to measure the competencies of its nurse managers and how the agency uses the tool for multiple purposes, including creating and delivering staff-development programs to strengthen the skills and knowledge of nurse managers in home health care.
In short, this article will focus on the development of the home health care nurse manager competency tool, with specific attention paid to the methodological and statistical activities related to testing the instrument.
Literature Review of Nurse Manager Competencies
The literature on nurse manager competencies primarily focuses on (a) exploring the value and benefits of using a competency approach for training, recruitment, and promotions; (b) identifying discrete competencies and associated behaviors and skills for nurse managers; and, to a lesser degree (c) the development of specific tools for assessing nurse manager competencies.
Value and Benefits of the Competency Approach
The term competency may be defined as “the state of having the knowledge, judgment, skill, energy, experience and motivation required to respond adequately to the demands of one’s professional responsibilities.” 6 With few exceptions, the literature on nurse manager competencies focuses on nursing practices in hospitals. The adoption of a competency model is often associated with the development or revision of new nurse manager orientation or training programs.7,8 Identification of competencies may also be a part of efforts to change recruitment or promotion procedures. And, for some organizations, assessments of competency may be substituted for specific education requirements. 9 There is some evidence that use of established competencies, rather than completion of formal education or years of experience, are more reliable for predicting success of managers.10,11 Catano, Darr, and Campbell 12 argue that focus on behaviors (rather than knowledge or education) improves the likelihood that competencies can be measured reliably.
Competency models reflect the current foci on accountability, data-driven decision making and value of lifelong learning, all which aligns well with current emphases on standardized performance processes and outcomes. Rather than concentrating on successful attainment of specific, discrete credentials or technical skills, competency models are more likely to focus on more global leadership concepts such as planning and organizing and being a successful change agent, which require mastery of interpersonal skills and other skills that are difficult to measure. 13
Identifying Discrete Nurse Manager Competencies
Typically, identification of nurse manager competencies reflects organizational standards and expectations, which may differ throughout the health care delivery spectrum. 10 Though a consensus exists that there are common responsibilities for nurse managers regardless of the specific setting, little has been done to distinguish whether competencies in institutional settings, such as hospital settings and nursing homes,3,4 reflect the competencies necessary in home and community-based settings. The very nature of home care—which may limit, for example, the amount of “face time” between nurse managers and field nurses—requires different management approaches.
Different methods have been used to classify nurse manager competencies. 8 However, the most common has been interviews and surveys with experienced nurse managers or their supervisors.2,4,8,14 Hudak, Brooke, and Finstuen 13 employed the Delphi method to survey nurse managers multiple times and Chase 3 surveyed a representative sample of members of the American Organization of Nurse Executive nurse manager council. In these cases, researchers develop a list of skills, behaviors, and abilities that are considered to be associated with the nurse manager role and ask expert informants to rank the relative importance of each skill, behavior, or abilities.
There exists some consensus regarding the core competencies for the nurse manager role—such as planning and organization, building and leading teams, interpersonal effectiveness, and quality clinical oversight—though the exact terminologies might differ.8,9,15,16 Jennings, Scalzi, Rodgers, and Keane 17 note that the literature often conflates management and leadership competencies which illustrates the somewhat imprecise nature of these concepts and can lead to ineffective training and educational for managers.
Tools and Methods for Measuring Nurse Manager Competencies
While interest in identifying competencies has been strong, development of reliable and valid assessment tools has been minimal. Some literature is devoted to sample questions and lists of behaviors that might reflect particular competencies,8,18 but neither have these been formally designed as tools nor have these behaviors been subjected to any testing. The vast majority of research on nurse manager competencies has lacked any proper statistical analysis, and validity and reliability analysis is often not conducted, 4 though Lin, Hsu, Li, Mathers, and Huang 19 is one example of a tool that tested for reliability and validity for their public health nurse competencies tool. To create objective nurse manager competencies, it is necessary to perform thorough statistical analysis to inform decisions on utilization.
In addition, data for such studies are often collected unsystematically or through convenient samples, which limit the generalizability of the findings. Nurse executives or nurse managers typically serve as the primary respondents to these surveys and unstructured interviews. 15 Although regarded as subject matter experts, it may be beneficial to include other data sources and use different methodologies to provide a more complete view of the nurse manager role.
In short, the literature on nurse manager competencies has been primarily devoted to promoting the use of competency models to advance a competent, high-performing nursing workforce. Current interest in standard measures of performance and data-driven outcomes have increased interest in competencies rather than focus on knowledge and credentials. To date, however, the literature has primarily focused on competencies associated with the nurse manager role in hospital settings. Efforts to identify competencies have produced a wide range of desired behaviors and skills, which presumably reflect successful performance. However, systematic efforts to assess the reliability and validity of competency models and tools are less evident in the literature.
Overview of the Project
The development of the home health care nurse manager competency assessment tool was part of a larger initiative within the agency which sought to redesign the nurse manager role. Efforts to redesign the nurse manager role was motivated by a desire to examine the full range of activities being performed by the nurse managers to clearly differentiate activities considered “core processes redesign” (operational concerns such as insurance and referral issues) and those activities involving “clinical quality oversight” of staff nurses and other professionals. Over time, the home health care nurse manager position had become involved in a range of technical, clerical, and administrative functions which impeded their ability to develop management skills such as creating high-functioning teams, being effective change agents, anticipating resource, and staffing demands. The agency’s “job delineation” analysis of the nurse managers resulted in separation of their functions into two distinct spheres: (a) a separate service support unit was created to take on the administrative/clerical tasks formerly performed by the nurse managers and (b) nurse managers were thus freed to concentrate more fully on issues involving clinical oversight, client satisfaction, staff development, and capacity building.
With many of their routine tasks reassigned to the service support unit, incumbent nurse managers could refocus their work in new ways. The agency was committed to developing the competencies and skills of the incumbent nurse managers, offering significant professional development opportunities to bring all nurse managers “up to speed.” However, as a first step, it was necessary to measure the baseline competencies of nurse managers to (a) identify and prioritize areas that required strengthening and (b) assess whether, through intensive professional development, incumbent nurse managers could improve their mastery and competency in desired areas. In the absence of a nurse manager competency measure relevant to home care, staff at the Visiting Nurse Service of New York (VNSNY) Center for Home Care Policy and Research in collaboration with their operations and HR/Organizational Development colleagues set out to
ascertain the competencies relevant to home health care nurse managers,
identify behaviors associated with each competency,
develop a tool to assess the competencies,
pilot and test the instrument to determine validity and reliability, and
integrate the final version of the tool into ongoing professional development and appraisal processes at the agency.
The VNSNY is the largest not-for-profit home health care agency in the United States, providing care to more than 30,000 patients each day throughout the New York metropolitan area. VNSNY employs more than 2,600 registered nurses, more than 1,000 rehabilitation therapists and social workers, and 8,000 home health aides. In 2010, VNSNY served the diverse needs of approximately 144,000 patients of all ages and socioeconomic backgrounds. The organization’s services include acute care, long-term care, hospice and palliative care, HIV/AIDS care, mental health services, and maternity, newborn, and pediatric services.
Designing and Testing a Competency Assessment Tool for Home Health Care Nurse Managers
Phase I: Ascertain the Competencies Relevant to Home Health Care Nurse Managers
The agency’s effort to redesign the nurse manager role necessitated a clearly articulated statement of the responsibilities necessary for the new role. To be sure, incumbent nurse managers had been working successfully in these positions for many years. However, with the redesign a set of new expectations and responsibilities were introduced which required a somewhat different set of skills and practices. It, therefore, became necessary to articulate a clear set of home health care nurse manager competencies from which to assess the current level of competence of the incumbent nurse managers and, equally important, to assess changes over time.
An agency-wide advisory group of executives—including clinical, operational, financial, and research leaders—provided ongoing guidance and support to the nurse manager redesign. In conceptualizing the competencies, clinical oversight was considered the most critical competency to ensure clinical excellence. However, today’s home health care nurse managers must also demonstrate acumen in business, data analysis, outcome management, and other skill sets.2,5,9 Eight broad areas were designated by the advisory group that were best aligned to agency objectives for the role redesign efforts. The eight competencies that were identified as most critical for the home health care nurse manager were (a) aligning performance for success, (b) clinical knowledge and skills, (c) coaching, (d) communication, (e) customer/patient focus, (f) decision making, (g) facilitating change, and (h) planning and organizing work. The next step was to create a competency assessment tool that would “measure” the level of competency demonstrated by each nurse manager at baseline and future time intervals.
The advisory group reviewed a wide range of available competency assessment tools geared toward hospital-based nurse managers. They determined that, given the unique nature of home health care nursing and the agency’s service delivery model, it would be necessary to develop, test, and validate a specifically designed competency assessment instrument for home care nurse managers.
Phase II: Identify Behaviors Associated With Each Home Health Care Nurse Competency
After concluding that an appropriate model for home health care nurse manager competencies did not exist, the next phase involved designing a competency assessment tool that identified specific nursing behaviors for each selected competency. In other words, clear definitions for each competency were needed, as well as identification of specific behaviors that would illustrate those competencies. A comprehensive review of articles in health care, nursing, management, and the social sciences was conducted to examine the core behaviors of competencies among
general managers
nurse managers
other professional groups, such as school teachers, military personnel, and other health care workers
Professional staff in the Center for Home Care Policy and Research and Human Resources/Organizational Development collaborated to assemble, analyze, and synthesize these disparate resources. Not surprisingly, a great deal of overlap existed among the varying resources. In the end, a long list of behaviors thought to reflect of the eight, as applicable to the home health care nurse managers, was compiled. The number of items per competency ranged from four to eight.
An example will illustrate the definitions and associated behaviors of each of the desired competencies. Coaching was defined as “providing timely guidance and feedback to help others strengthen specific knowledge/skill areas needed to accomplish a task or solve a problem.” Four core behaviors reflected this competency: (a) develops team members based on patient needs; (b) recognizes talent and encourages advancement in career path; (c) appreciates and values individual team members; and (d) gives feedback on performance and reinforces effort and progress. In the final analysis, these efforts resulted in a list of 38 individual behaviors which were believed to reflect the designated competencies.
Phase III: Develop a Tool to Assess Home Health Care Nurse Manager Competencies
Once the competencies and associated behaviors were identified, the creation of a behaviorally based performance instrument could proceed. This involved selection of a method to assess and rate each item. The advisory group selected the Behaviorally Anchored Rating Scale (BARS) which presents the rater with a 5-point range on which to assess the performance or behavior of the individual being assessed. The BARS approach has been used widely for a variety of jobs and has been particularly popular within nursing. 20
The BARS approach is visually unique and presented in the form of graphic rating scales arranged vertically. The behavioral anchors are printed beside each horizontal bar at different levels along a vertical line according to their assigned scale position. 20 Each of the 38 behavioral items identified in the earlier phase was arranged into a BARS, which required rephrasing each behavior across a 5-point scale. For example, one of the core behaviors associated with Coaching is “Gives feedback on performance and reinforces effort and progress.”
This item was arranged in the following BARS:
Constructing the individual BARS items was an iterative process. Each individual scale was reviewed and edited numerous times to ensure clarity, consistent use of language, and comprehensiveness. Through this process, it appeared that considerable overlap between domains existed, which suggested that the proposed competencies might actually represent a smaller number of clear-cut domains. Once all items were properly configured into BARS, they were combined into one tool in a randomized manner which gave no indication to which competency they were associated.
As with any assessment tool, limitations do exist with the BARS. First, the appraisers (in this case, supervisors of nurse managers) must be able to recognize the behavior being scrutinized and evaluate their employees as objectively as possible. The appraisers must, therefore, not only have clear knowledge and understanding of all of the important aspects of job performance but also have the ability to discern the degree to which each individual measures up to that particular behavioral standard. 20 Thus, appraisers must be trained prior to performance of employee assessments.
Phase IV: Pilot and Test the Home Health Care Nurse Manager Assessment Tool
It was now necessary to pilot the newly developed BARS and determine its validity in measuring the competencies identified by the agency’s advisory group. In short, could we determine that the 38 items identified in the development phase were in fact measuring the core behaviors of the home health care nurse manager competencies? To accomplish this, data were collected from individuals throughout the agency to test the tool. The VNSNY has a large and diverse workforce of approximately 15,000 employees including registered nurses, physical, occupational and speech therapists, and home health aides, as well as administrative, financial, and other professionals. Among these are individuals who provide direct patient care, frontline managers who oversee the day-to-day clinical operations, and supervisors who provide greater oversight and leadership. Using information from the Human Resources department, a total of 57 individuals were identified at the “supervisor” level that is, overseeing the frontline managers. In other words, these 57 individuals represent the population of “supervisors” within the agency. They were all contacted by email and invited to participate in the study. Each was provided with a list of one to five randomly selected members of his or her direct-report management-level staff and asked to conduct an assessment using the newly designed tool. No “supervisor” was asked to assess more than five managers in an effort to limit the burden and assure broad participation. The “supervisors,” through the use of an online survey software program, voluntarily completed the competency assessment tool to evaluate their designated manager employees. In total, assessment data on 154 managers were collected. Prior to launching the survey, participants were invited to a webinar, which oriented them to the purpose of the survey and provided instructions on completing the surveys online.
“Supervisors” were asked to complete the assessments of their managers during a 1-week data-collection period. Participants received a link to the assessment tool as well as all the instructions necessary for its completion. Participating “supervisors” were instructed to carefully evaluate the designated employee’s work performance in relation to each of the behaviors by assigning a rating from 1 to 5 for each item. It was emphasized that they need not limit themselves to a 1, 3, or 5 rating and that they could assign scores of 2 and 4, if desired. In addition, the participants were asked not to omit any question and to take their best guess, if necessary.
The participants were also asked to complete a brief demographic survey on themselves and the manager they were rating. These questions addressed the employee’s title, program in which the employee worked, region in which the employee worked, number of years the employee had been in the particular position, number of years the employee had been with the agency, and whether the employee was a registered nurse. The “supervisor” completed these questions for both herself or himself and for the manager that was being rated before beginning the evaluation (see Table 1).
Participant Demographics
Validation
Once the data were collected, exploratory factor analysis was used to validate the eight competencies and determine which items were related to which competencies. Analyses included the use of principal components analysis with a Varimax with Kaiser normalization rotation method. The analysis would determine whether the 38 individual scale items could be explained in a smaller number of domains (ie, competencies) than the original number of eight proposed.
Descriptive statistics were calculated for each of the scale items. Results show that “supervisor” participants used the full 5-point scale when evaluating their direct reports. The average rating given for each of the 38 behavioral statements ranged from 3.03 to 4.17. In addition, the majority of scale items had a median rating of 4.0 (n = 29). Table 2 provides the mean and standard deviation for each item.
Item Means, Standard Deviations, and Factor Loadings
The principal components analysis indicated a five domain solution, with eigenvalues all greater than 1.0 and 66.9% of the variance explained by the model. All 38 items of the original BARS were retained and had factor loadings of 0.3 or greater. Table 2 shows the item loadings for each factor.
The internal consistency of each of the domains was measured using Cronbach’s alpha (see Table 3). All of the five factors had extremely high internal consistency. Alpha values ranged from .866 (Scale 4) to .948 (Scale 1). As noted above, the development of the BARS was conceived as involving eight domains nurse management competencies. In the process of constructing specific items for BARS, there were concerns that overlapping existed across the eight identified domains. The findings of the factor analysis revealed that when all 38 BARS items are examined together, the number of distinct domains is actually five, rather than eight. For example, BARS items that were originally designed to reflect “clinical knowledge and skills” did not result in an independent domain but were more appropriately placed in other domains. In the final analysis, five independent domains were identified: (a) leadership, which included some items from communication and facilitating change; (b) problem solving, which included items from clinical knowledge and skill and customer/patient focus; (c) planning and organization; (d) coaching; and (e) aligning performance for success. All 38 items from the original BARS tools were retained in the final version of the tool. These findings confirmed earlier concerns about overlap among behaviors identified with more than one competency domain.
Cronbach’s Alpha and Cohen’s Kappa for Each Scale
Interrater Reliability
Once the validation phase was completed, it was necessary to assess interrater reliability to ensure the objectivity of the BARS tool. The interrater reliability of the BARS tool involved a sample of the initial supervisor participants. Paper versions of the assessment tool were mailed to all participants, as well as all the instructions necessary for its completion. The interrater reliability was assessed using two supervisors who both rated five managers they knew very well (see Figure 1). Cohen’s kappa statistic was used as a measure the agreement between two raters. All scales showed high interrater reliability ranging from .58 to .87. Table 3 shows the weighted kappa values for each scale.

Interrater reliability
Phase V: Integrate Competency Assessment Into Professional Development and Appraisal Processes
The creation of a reliable assessment tool enabled the agency to establish a “baseline” level of competence at the start of the redesign efforts and to measure potential changes and improvements in the future. The inclusion of the BARS tool became a critical component of the nurse manager role redesign efforts. The agency selected one geographic region/county to serve as the pilot region for unfolding the redesign. The competency tool was completed for each nurse manager in the pilot to measure baseline levels of competency and reassess at designated time intervals. The baseline competencies demonstrate large disparities among the pilot home health care nurse managers within specific competency domains. However, all pilot home health care nurse managers were notably weak in coaching, an area particularly germane to the objective of the role redesign. The data provide the opportunity to create professional development programs to address the deficiencies uncovered by the BARS tool. Professional staff in human resources/organizational development designed a range of learning and training modules that addressed the elements of the five competencies including
specially designed webinars,
in-house and externally run workshops and seminars,
one-on-one and conference call coaching sessions, and
selected readings and exercises.
Working with their supervisor, each home health care nurse manager is now expected to select and commit to a personalized set of specific training goals for each year to address the competency they seek to improve. They are not expected to address all the competency areas simultaneously but to identify areas for concentrated efforts over the course of the year. The ultimate goal is to, over time, encourage improvements in all the areas in which the nurse manager requires remediation.
After 1 year, each home health care nurse manager in the pilot region was reassessed using the BARS tool. In all cases, but one, improvements were found in each of the competencies. The agency has now committed to rolling out the home health care competency model and assessment tool to other geographic regions.
Summary and Conclusions
The development of the home health care nurse manager competency tool was motivated by the agency’s mission to ensure a workforce competent to provide the highest quality patient care possible in the home setting. Aware of the unique nature of contemporary home health nursing, the agency committed to redesigning the nurse manager role to respond to the demand for superior clinical oversight, capacity management, and professional development for the nursing and operations staff. It was necessary to identify the desired competencies needed for the redesigned role and to develop an assessment tool to evaluate the baseline competencies of the incumbent nurse managers prior to the implementation of the role redesign. The resultant assessment tool, which has demonstrated validity and reliability, has been effective in establishing a standard measure of management competencies and has provided an opportunity for individual home health care nurse managers to identify areas for improvement. Over time, the assessment tool may be used for ongoing professional development for home health care nurse managers.
As with all research, there are limitations to the use of the home health care nurse management tool. First, the assessment tool was developed for a specific large home health care nursing agency, which has its own unique organizational structure and delivery system. Moreover, the role redesign for home health care nurse managers had specific objectives that were not intended to encompass all the possible management competencies but rather concentrated on those areas that were most pertinent to the goals of the role redesign. In addition, though the competencies of the home health care nurse manager are similar to managers in nonclinical areas, the tool was designed for clinical home health care nurse managers and has not been thoroughly tested with other management groups.
Footnotes
Acknowledgements
The authors would like to thank Cindy Morgan, MSOD, Christopher Grygo, and Nancy Mellos in the organizational development department at the Visiting Nurse Service of New York, New York.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) received no financial support for the research, authorship, and/or publication of this article.
