Client Experiences With Shelter and Community Care Services in the Netherlands

Introduction

In the current economic climate, homelessness continues to pose an important societal challenge (Frazer & Marlier, 2009; Levy & Sidel, 2009). In the Netherlands, the number of clients making use of shelter or community care services (from here referred to as services) for the homeless rose from nearly 50,000 in 2009 to more than 60,000 in 2013 (Federatie Opvang, 2011, 2012, 2014). Around 6,000 of these clients were between the ages of 15 and 23 and are officially defined as “homeless youth” (Federatie Opvang, 2014; Ministerie van Volksgezondheid, Welzijn en Sport, 2011b, 2011c). Adult clients often become homeless due to relationship problems, divorce, and home evictions or end up homeless after leaving prison or a psychiatric hospital. Youth most often become homeless as a result of leaving their home due to conflicts with parents or guardians (Tuynman & Planije, 2013). Homeless persons commonly deal with a multitude of problems, such as substance abuse, addiction, debts, severe mental health problems, and physical problems (Tuynman & Planije, 2013). At the same time, each year over 10,000 women in the Netherlands flee from a violent home situation to women’s shelters, bringing with them more than 5,000 children (Federatie Opvang, 2011). These women often deal with psychosocial and relationship problems as well as physical problems, also as a result of physical, mental, and/or sexual violence (Wolf, Jonker, Meertens, & Te Pas, 2006).

In the past few years, many governments in developed countries focus on offering appropriate shelter services aimed at reintegrating homeless people in society, rather than merely offering short-term physical shelter (American Recovery and Reinvestment Act: Public Law 111-5, 2009; Ministerie van Volksgezondheid, Welzijn en Sport, 2011a). A similar focus is applied to shelter services for women fleeing domestic violence, where the goal is to make a start with building a new, violence-free life (Jonker, Sijbrandij, & Wolf, 2012; Ministerie van Veiligheid en Justitie, 2010; Tuynman & Planije, 2013). In the Netherlands, several types of shelter services are offered, ranging from more short-term and low-threshold crisis shelter services to longer term residential services and supported housing. Depending on the municipality, there can be comparable but separate shelter services for homeless youth (Ministerie van Volksgezondheid, Welzijn en Sport, 2011c). Some municipalities also make use of outreach teams targeting homeless youth (Tuynman & Planije, 2013).

Despite the societal and academic interest in dealing with the problem of homelessness (Lorber, 2009) and intimate partner violence (Waltermaurer, 2006), there are very few studies on how clients themselves experience the services they receive (Spiro, Dekel, & Peled, 2009; Wolf, Luijtelaar, Jansen, & Altena, 2007). The few studies on client satisfaction among homeless people and youth were mostly conducted in the United States, with instruments that were often not specifically designed for or tailored to the target population (Calsyn, Morse, Klinkenberg, Yonker, & Trusty, 2002; Kasprow, Frisman, & Rosenheck, 1999; Klinkenberg, Calsyn, & Morse, 1998). Studies among homeless youth found high levels of overall satisfaction in general, but there were differences between the types of services (Altena, Beijersbergen, & Wolf, 2014). Satisfaction was consistently found to be high for client–worker relationships, which has been significantly associated with overall satisfaction (Altena et al., 2014). Similarly, there is a lack of internationally published studies on the experiences of abused women in shelters (Attala & Warmington, 1996; Bennett, Riger, Schewe, Howard, & Wasco, 2004; Itzhaky & Porat, 2005).

In order for care services to become more client-oriented, it is important to include the client’s own perspective in quality assessments (Damman, Hendriks, & Sixma, 2009). Since 1996, service providers offering services for homeless people, homeless youth, and abused women in the Netherlands are obligated to measure their clients’ satisfaction with the care they receive to facilitate both external accountability and internal quality control (Wet maatschappelijke ondersteuning [Social Support Act], 351, 2006). However, due to the absence of valid, standardized measurement instruments specifically tailored to this particular group and setting, comparisons of outcomes between shelter services providers could not be made (Wolf et al., 2007). In 2010, a unique instrument called the Consumer Quality Index for Shelter and Community Care Services (CQI-SCCS; Beijersbergen, Asmoredjo, Christians, & Wolf, 2015; Beijersbergen, Christians, Asmoredjo, & Wolf, 2010) was developed to fill this gap.

Since 2006, CQI-SCCS questionnaires are developed and used in a standardized way to measure and compare client experiences with health-care and care services in the Netherlands (Damman et al., 2009), following the shift in focus in research on clients’ perspectives on care in the last decade (Davies & Cleary, 2005). Rather than looking at clients’ satisfaction, it focuses on specific experiences clients have with different aspects of (health) care, allowing for more objective information on service providers and more precise quality improvement information (Hopman, de Boer, & Rademakers, 2011). The CQI-SCCS was developed in close cooperation with clients, client advocacy groups, and shelter workers as well as with key stakeholders in shelter and community care management, academia, and local and national government. The instrument measures experiences clients have with the client–worker relationship, the quality of the services received, the results of these services, and the living conditions in the shelters.

The first objective of this study was to gain insight into the experiences of clients with services for homeless people, homeless youth, and abused women offered by service providers throughout the Netherlands. The second objective was to identify priority areas for quality improvement for each of the three target groups, by combining clients’ self-reported experiences with those aspects of care that clients themselves consider most important. This method gives a good impression of which aspects of quality of care are most eligible for quality improvement (Damman et al., 2009). The third aim was to determine whether there were quality differences between service providers and to what extent these differences could be explained by the targeted group and the type of services the service provider offered. The term “service provider” here refers to a specific location or facility providing a specific type of care aimed at one of the three target groups.

Method

Participants

Of the 848 clients who were asked to complete the experience CQI instruments, 762 completed the questionnaire. After data cleaning according to the guidelines for CQI instruments (Sixma, Hendriks, de Boer, & Delnoij, 2008), 744 (88%) respondents were included in the analyses. Descriptive statistics for the background variables of the respondents are presented in Table 1. The 744 respondents received services from 18 service providers for homeless people (37%), 18 service providers for homeless youth (29%), and 24 service providers for abused women (34%). Types of service providers were crisis shelters (19%), residential services (38%), day or night shelters (10%), supported housing (22%), and outreach services for homeless youth (11%).

OPEN IN VIEWER

Table 1.

Descriptive Statistics Background Variables for Total Group and Subgroups.

Variable	Total	Homeless People	Homeless Youth	Abused Women
n	744	275	218	251
Number of service providers (n = 744)	60	18	18	24
Percentage male (n = 738)	50%	81%	68%	0%
Average age (SD); n = 730	33 (12.5)	43 (11.0)	21 (2.3)	31 (8.8)
Educational level (n = 728)
Low	28%	27%	23%	32%
Middle low	45%	46%	50%	40%
Middle high	22%	19%	27%	21%
High	6%	8%	1%	7%
Immigrant parental background (n = 744)	59%	43%	58%	77%
Type of service provider (n = 744)
Day and night shelters	10%	26%	0%	0%
Crisis shelters	19%	23%	0%	32%
Residential services	38%	31%	35%	48%
Supported housing	22%	20%	27%	20%
Outreach services for homeless youth	11%	0%	38%	0%
Rooflessness in the previous montha (n = 743)	24%	28%	33%	13%

^aA person is considered roofless (“Feitelijk dakloos”) in the previous month if he or she mostly spent the night on the streets, night shelters, relatives, friends, or acquaintances, in the past 30 days.

Nonrespondents were 86 clients who did not want to fill in the questionnaire and an additional 18 who were taken out during data cleaning.

The importance questionnaire was completed by 116 respondents (37% homeless people, 28% homeless youth, and 35% abused women). This group was not significantly different from the 744 respondents who filled in the experience questionnaire when it comes to any of the background characteristics presented in Table 1.

Results

Client Experiences

Table 2 shows the average unstandardized scale scores and overall grade for all clients and each subgroup. A higher scale score reflects more positive experiences. The average overall grade (0–10) is 7.4. The client–worker relationship scale receives the highest score (3.32), followed by the services received scale (3.00) and living conditions scale (2.96). The scale scores for results of services are lowest (2.86). In all subgroups, the same pattern can be found when comparing average scale scores. It should be noted that the abused women group has significantly higher scale scores (M = 3.47, SD = .58; M = 3.13, SD = .61; M = 3.03, SD = .71; M = 3.12, SD = .55) than the homeless people group (M = 3.23, SD = .67; M = 2.90, SD = .66; M = 2.76, SD = .76; M = 2.87, SD = .62), t(523) = 4.28, p < .001, d = .38; t(523) = 4.13, p < .001, d = .36; t(517) = 4.12, p < .001, d = .36; t(429) = 4.45, p < .001, d = .43, and homeless youth group (M = 3.28, SD = .63; M = 2.96, SD = .66; M = 2.78, SD = .76; M = 2.87, SD = .57), t(466) = 3.41, p = .001, d = .32; t(466) = 2.90, p = .004, d = .27; t(460) = 3.64, p < .001, d = .34; t(325) = 3.93, p < .001, d = .45. The abused women group also gives a higher overall grade (M = 7.67, SD = 1.85) than the homeless people (M = 7.14, SD = 2.11), t(523) = 3.042, p = .002, d = .27, and homeless youth group (M = 7.27, SD = 1.91), t(466) = 2.309, p = .021, d = .21.

OPEN IN VIEWER

Table 2.

Average Unstandardized Scale Means and Overall Grade.

Scale	n	Mean (1–4)	SD
Client–worker relationship	743	3.32	0.64
Clients of services for homeless people	275	3.23	0.67
Clients of services for homeless youth	218	3.28	0.63
Clients of services for abused women	250	3.47	0.58
Services received	743	3.00	0.65
Clients of services for homeless people	275	2.90	0.66
Clients of services for homeless youth	218	2.96	0.66
Clients of services for abused women	250	3.13	0.61
Results of services	735	2.86	0.75
Clients of services for homeless people	273	2.76	0.76
Clients of services for homeless youth	216	2.78	0.76
Clients of services for abused women	246	3.03	0.71
Living conditions	548	2.96	0.59
Clients of services for homeless people	221	2.87	0.62
Clients of services for homeless youth	117	2.87	0.57
Clients of services for abused women	210	3.12	0.55
Overall grade (0–10)	743	7.35	1.98
Clients of services for homeless people	275	7.14	2.11
Clients of services for homeless youth	218	7.27	1.91
Clients of services for abused women	250	7.67	1.85

Quality Improvement Scores

Table 3 shows average importance scores (1–4), fraction of clients with a negative experience (i.e., who answered “never” or “sometimes”), and average quality improvement scores. In each subgroup, all scales received high average importance scores, ranging from 3.19 to 3.58 (where 3 is important and 4 is extremely important). The client–worker relationship scale receives the relatively highest importance scores, while the results of services scale receive the lowest. When comparing subgroups, the abused women group has significantly higher importance scores on the client–worker relationship (M = 3.58, SD = .49) and living conditions scale (M = 3.54, SD = .34) than the homeless people group (M = 3.32, SD = .52; M = 3.29, SD = .58), t(80) = 2.33, p = .02, d = .52; t(66) = 2.16, p = .03, d = .52. Furthermore, in the abused women group, there were significantly less clients who reported negative experiences on the results of services scale (27%), compared to the homeless people (38%) and homeless youth group (40%), z = 2.64, p = .008, r = .12; z = 2.94, p = .003, r = .14.

OPEN IN VIEWER

Table 3.

Fraction of Clients With Negative Experiences, Average Importance, and Quality Improvement Scores Per Scale and Subgroup.

Scale	Total	Homeless People	Homeless Youth	Abused Women
1. Client–worker relationship
Average importance scores (1–4)	3.43	3.32*	3.40	3.58*
Fraction negative experiences	0.17	0.20†	0.17	0.14†
Quality improvement scores (0–4)	0.58	0.66	0.58	0.50
2. Services received
Average importance scores (1–4)	3.36	3.30	3.35	3.43
Fraction negative experiences	0.30	0.32†	0.31	0.25†
Quality improvement scores (0–4)	1.01	1.06	1.04	0.86
3. Results of services
Average importance scores (1–4)	3.26	3.19	3.26	3.32
Fraction negative experiences	0.35	0.38*	0.40*	0.27*
Quality improvement scores (0–4)	1.14	1.21	1.30	0.90
4. Living conditions
Average importance scores (1–4)	3.41	3.29*	3.35†	3.54*
Fraction negative experiences	0.30	0.33	0.34	0.26
Quality improvement scores (0–4)	1.02	1.09	1.14	0.92

^†marginally significantly different from at least one other subgroup (p < .1).

*Significantly different from at least one other subgroup (p < .05).

For all groups, the client–worker relationship clearly receives the lowest quality improvement score, due to the relatively small fraction of clients with negative experiences with this aspect of care. For the homeless people and homeless youth group, the highest quality improvement scores are found for the results of services scale, followed by the living conditions and the services received scales. For the abused women group, there were no clear differences in quality improvement scores between these three scales. Overall, the quality improvement scores for the abused women group were lower than for the homeless people and youth groups.

Quality Differences Between Service Providers

Table 4 shows the outcomes of the multilevel regression analyses. In the empty model, the variance between scale scores and overall grade is simply divided between the individual level (Level 1) and the service provider level (Level 2). The results of the empty model show that most of the variance (76–88%)—that is, the differences in individual scale scores and overall grades—can be attributed to differences between individuals (Level 1). This means that 12–24% of the variance can be attributed to differences between service providers.

OPEN IN VIEWER

Table 4.

Multilevel Regression Analyses Outcomes: Scale Scores and Overall Grade, With Individuals Nested in Service Providers.

Particulars	Client–Worker Relationship (n = 708)	Services Received (n = 708)	Results of Services (n = 700)	Living Conditions (n = 522)	Overall Grade (n = 708)
Fixed effects Model 1
Intercept	4.14**	3.82**	3.67**	3.63**	9.72**
Level-2 variables: service provider characteristics: target group (reference group = homeless youth)
Homeless people	0.07	0.19	0.25†	0.23	0.57
Abused women	0.23*	0.26*	0.30*	0.30*	0.66†
Type of service provider (reference group = residential services)
Day and night shelter	−0.33*	−0.45*	−0.46*	−0.52*	−0.99*
Crisis shelter	0.07	0.10	0.07	0.05	0.28
Supported housing	0.29**	0.32**	0.26*	0.25	0.60*
Outreach teams	0.58**	0.62**	0.52**	0.53*	1.78**
Level-1 variables: individual background characteristics
Female	0.09	0.08	0.05	0.02	0.21
Age (centralized)	0.01**	0.005†	0.005	0.003	0.01†
Educational level	−0.07*	−0.07*	−0.08*	−0.05†	−0.15†
Immigrant parental background	−0.05	−0.10*	0.09	−0.05	0.08
Rooflessness in the past month	−0.07	−0.12*	−0.11	−0.10	−0.31†
Random effects
Empty model
Total variance	0.40	0.41	0.56	0.34	3.79
Level-1 variance (% of total)	83.2%	78.5%	87.9%	75.6%	83.0%
Level-2 variance (% of total)	16.8%	21.5%	12.1%	24.4%	17.0%
Model 1
Explained variance Level 1  (compared to empty model)	2.5%	1.8%	1.8%	0.3%	0.7%
Explained variance Level 2  (compared to empty model)	68.4%	68.0%	73.8%	57.1%	50.1%
Model fit: −2 restricted log likelihood
Empty model	1,311.90	1,288.07	1,549.10	836.89	2,893.64
Model 1a	1,292.53**	1,268.94**	1,534.18**	840.52**	2,872.95**

^aThe difference between Model 1 and the empty model was tested using a likelihood ratio test.

^† p < .1. *p < .05. **p < .001.

In order to try to explain the variance at the service provider level (Level 2), two characteristics of service providers were included as independent variables: which target group they catered for and what type of services they provided. Looking at the coefficients for the target group variables, the results show that service providers for homeless youth score consistently lower than service providers for both homeless people and abused women. However, only the difference between the homeless youth and abused women group is significant, with the exception of the overall grade. Looking at the coefficients for the type of service provider, another consistent pattern can be observed. Day or night shelter providers score consistently and significantly lower than all the other service providers, followed by residential shelter, crisis shelter, supported housing, and outreach teams. The substantial differences found between target groups and the type of service providers are also reflected in the high percentages of explained variance at the service provider level (Level 2): from as much as 50% for the overall grade to 74% for the results of services scale. This, in turn, also means that 26–50% of the Level-2 variance, that is, the differences between service providers, remains unexplained.

The individual background variables only explain a small part of the considerable variance at the individual level (Level 1): only 0.3–2.5%. Rooflessness in the previous month has a significant negative effect on the services received scores. Education has a slight negative effect (significant for client–worker relationship, services received, and results of services scale), while age has a minimal positive effect on scale scores (only significant for the client–worker relationship). Clients with an immigrant parental background score slightly, but significantly, lower on the services received scale.

Discussion and Applications to Practice

This study is unique as for the first time it gives an overall picture of the quality of services that are offered in the various types of shelters targeting different client groups throughout the Netherlands, from the viewpoint of the clients making use of these services. The results show that clients’ experiences are generally quite positive (mean overall grade 7.4 and average scale scores range from 2.8 to 3.5 on a 4-point scale).

Clients from all three target groups—homeless people, homeless youth, and abused women—report the most positive experiences with the client–worker relationship. This is very encouraging, considering that clients were also found to adhere most importance to this relationship, compared to the other aspects of shelter services. Since clients are already very positive about this area, this aspect of services has the least priority for improvement. The high importance clients adhere to the relationship with the professional is in accordance with findings from other CQI studies (Hopman et al., 2011) and other studies (Johansson, Oleni, & Fridlund, 2002; Kasprow et al., 1999; Roter, Hall, & Katz, 1987; Tamblyn et al., 1994; Wolf et al., 2007) on determinants of client and patient satisfaction, where trust and being treated with respect was one of the most important elements determining good quality of services. In the Netherlands, the client-professional relationship is the central element in the recently implemented strength-based methods in shelter and community care, called “Krachtwerk” (Strength Work; Wolf & Jansen, 2011).

The client–worker relationship and the living conditions in the shelter receive relatively high importance scores in the abused women group, compared to the homeless groups. This may reflect a greater need of abused women for a positive relationship in which they feel taken seriously, after fleeing an abusive situation in which the woman is also often socially isolated from other people (Wolf et al., 2006). The relatively high importance the abused women group attribute to living conditions in the shelter may partly be explained by the fact that many women have relatively higher standards in this area since they bring their children with them (Wolf et al., 2006).

For homeless people and homeless youth, the results of services should get the highest priority for quality improvement, as they have the most negative experiences with this aspect of services, while the importance adhered to this aspect is high. The need for prioritizing this aspect is in line with the shift in focus in the Netherlands toward reintegration and participation in society rather than merely accommodating homeless people (Ministerie van Volksgezondheid, Welzijn en Sport, 2011a). Moreover, a model for monitoring the outcomes of care trajectories for homeless people is currently being developed in the Netherlands (Impuls – Netherlands Center for Social Care Research, 2015). If this model is going to be implemented in the sector, we expect that the results of care will improve, as continuous reflection on outcomes will encourage quality improvement in care delivery. It should also be noted that the items in the results of care scale relate to, for example, having more hope for the future and being better able to handle difficult situations, which may not be expected to be the result of help from day and night shelters. This is also dependent on the responsiveness of communities and the opportunities society provides to these excluded individuals.

For the abused women group, the quality improvement scores do not show a clear priority area for improvement, which can be explained by the fact that there are less pronounced differences in negative experiences between scales. Abused women may not have the most negative experiences with the results of services because of the relatively longer tradition in the abused women service sector to also focus on the transitional phase from support to independent living, as part of the three functions (i.e., asylum, recovery, and transition) of the shelter and community care services (Wolf et al., 2006). The transition may also be easier for women with children, as having children has been found to be a positive factor for women receiving abused women services (Altena, Jonker, & Wolf, 2009; Zugazaga, 2004). For example, they have more contact with and receive more support from their family (Altena et al., 2009; Zugazaga, 2004). They also have a relative advantage when it comes to finding new independent housing, as they receive priority in social housing. For abused women, it may be more valuable to look at quality improvement scores for specific items, rather than the thematic scales, in order to find distinct points for improvement. For example, women’s relatively negative experiences with matters of privacy within the shelter facilities is a specific area where improvements can be made.

To our knowledge, this is the first time that the quality of shelter services could be compared across different shelter service providers targeting different client groups. Multilevel analyses show that 12–24% of the variance in scale scores and overall grade can be attributed to differences between service providers. These percentages are relatively high compared to those found in most other CQI studies (Hammink & Giesen, 2010; ITS, 2010; Koopman & Rademakers, 2008; Reitsma-van Rooijen, Damman, Sixma, Spreeuwenberg, & Rademakers, 2009; Stubbe, Spreeuwenberg, & Asbroek, 2007). Only the CQI for nursing and care homes for the elderly (Wiegers, Stubbe, & Triemstra, 2007) showed comparable large variances between locations (between 6% and 38%). Our study shows that a large part of the differences between service providers (50–74%) could be explained by the characteristics of the service provider. First, there are marked differences in scores between types of service providers. Day or night service providers score consistently and significantly lower than all the other service providers, followed by residential shelter, crisis shelter, supported housing, and outreach teams, who received highest scores. These differences may be due to the different purposes of the type of service providers (Altena et al., 2014). For example, outreach teams for homeless youth mainly focus on achieving a good relationship with the homeless young adults and engaging them in services and support, which has been found to be a successful method (Altena et al., 2014; Connolly & Joly, 2012). The differences may also be accountable to differences in clients’ circumstances. For example, those making use of supported housing services find themselves in a more stable situation than those making use of day or night services and could therefore be more positive in general.

Second, clients receiving services for abused women rated the quality of their services higher than the homeless youth group did, even when controlled for the type of service provider and individual background characteristics, including gender. It may well be that women fleeing domestic violence tend to be more positive about the services they receive, as their primary aim for seeking shelter—that is, the assurance of safety for themselves and their children from an abuser (Wolf et al., 2006)—is already met by making use of the abused women’s shelter services.

While the multilevel analyses show that much of the differences between service providers can be explained by the target group they aimed for and the type of services they offer, still 26–50% of the variance between service providers remains unexplained. This indicates that there are simply some service providers where clients report better experiences than others. It could be helpful to study such “best practice” cases more in depth or for other service providers to get otherwise acquainted with these higher rated facilities, in order to determine whether there are distinct success factors or methods that can account for the more positive client experiences.

The results of this study show that professionals involved in shelter care are already on the right path, with clients being generally positive about the services they receive and, especially, the way they are treated by professionals. The latter being the most important aspect according to the clients themselves and an essential starting point for future interventions. At the same time, areas for improvement such as an increased focus on results of care—for example, being better able to handle difficult situations for the homeless clients—are already acknowledged and integrated in recent approaches aimed at further improving the quality of shelter care. Continually monitoring clients’ experiences with care would give shelter services the opportunity to take the quality of shelter services to the next level.

Key Points