Estimating and Correcting Degree Ratio Bias in the Network Scale-up Method

Abstract

The Network Scale-up Method (NSUM) uses social networks and answers to “How many X’s do you know?” questions to estimate sizes of groups excluded by standard surveys. This paper addresses the bias caused by varying average social network sizes across populations, commonly referred to as the degree ratio bias. This bias likely exists for all populations, resulting in either positively or negatively biased population size estimates. We show how the degree ratio affects size estimates and provide a method to estimate degree ratios without collecting additional data. We demonstrate that our adjustment procedure improves the accuracy of NSUM size estimates using simulations and data from two data sources.

Keywords

size estimation popularity factor degree ratio key populations aggregated relational data

Introduction

The Network Scale-up Method (NSUM) has emerged as a popular and efficient way to estimate the size of hard-to-reach populations such as female sex workers, drug users, and men who have sex with men. These hard-to-reach populations are of critical importance to solving several global health problems, including meeting UNAIDS HIV-related targets (UNAIDS 2021). These populations are at a higher risk of contracting and spreading HIV than the general population while simultaneously suffering from marginalization and negative social stigma.

The NSUM estimates the size of these populations using survey questions of the form “How many X’s do you know,” where X includes both subpopulations with known sizes and subpopulations of interest with unknown sizes, such as female sex workers (Bernard et al. 1989). These survey responses are known as aggregated relational data (ARD). While some research on ARD concerns the estimation of network structures (Breza et al. 2020), we focus on the role ARD play in estimating hard-to-reach subpopulation sizes.

Previous researchers have proposed several modeling improvements to better capture the complexity of the underlying aggregated relational data, including those by Laga et al. (2023), Maltiel et al. (2015), Teo et al. (2019), and Zheng et al. (2006). These approaches aim to either better understand underlying network properties or improve population size estimates from NSUM models by incorporating underlying network properties into the model.

This work focuses on the NSUM subpopulation size estimator proposed in Killworth et al. (1998), which we refer to as the basic scale-up estimator (see McCormick 2020 or Laga et al. 2021 for a comprehensive review). The basic scale-up estimator is currently the most commonly used NSUM estimator. Killworth et al. (1998) assume the ARD come from the following distribution:

y_{i k} \sim Binomial (d_{i}, N_{k} / N),

(1)

where

y_{i k}

denotes the number of people respondent i reports knowing in subpopulation k,

d_{i}

is the degree, or social network size, of respondent i, N is the size of the total population, and

N_{k}

is the size of subpopulation k. For simplicity, we assume there are L “known subpopulations” where

N_{k}

is known, and one “unknown subpopulations” H where

N_{H}

is unknown. In general there may be more than one unknown subpopulation. Assuming that we know the true degrees,

d_{i}

, or are able to consistently estimate them from the ARD, the subpopulation size estimate for H is given by

{\hat{N}}_{H} = N \frac{\sum_{i = 1}^{n} y_{i H}}{\sum_{i = 1}^{n} d_{i}},

which is the maximum likelihood estimator for

N_{H}

from Equation (1) conditional on the

d_{i}

The basic scale-up estimator is subject to a variety of biases, including when respondents are more or less likely to know people from certain populations (barrier effects), do not know everything about their social contacts (transmission error), or cannot accurately recall everyone in their social network (recall error) (Killworth et al. 1998; McCarty et al. 2001). We focus on the degree ratio error introduced by different subpopulations having different average network sizes. Specifically, the degree ratio for subpopulation k is the ratio between the average degree of members of subpopulation k to the average degree of individuals who may be included as respondents in the ARD survey. Feehan and Salganik (2016) propose a generalized scale-up estimator and show that their estimator is equal to the basic scale-up estimator multiplied by three adjustment factors, one of which is the degree ratio. While the authors propose several approaches to correct for these factors, correcting for the degree ratio typically requires collecting additional survey data directly from the subpopulation. For example, Salganik et al. (2011) created the game of contacts, which involves interviewing members of the hard-to-reach population. While their original motivation for the game of contacts was to estimate the transmission error, it may also be used to estimate the degree ratio. Alternatively, Feehan and Salganik (2016) propose collecting additional ARD from the hard-to-reach population to estimate the degree ratio.

Failing to account for the degree ratio can significantly bias NSUM subpopulation size estimates. Shelley et al. (1995) found that HIV positive respondents and respondents who were dialysis patients had networks which were only about 2/3 the size of those of the average respondent in their survey. Therefore, given perfect responses to ARD questions, the basic scale-up estimator would estimate the size of these two subpopulations to be about 2/3 of the true size. The degree ratio may be more influential for even more stigmatized populations like sex workers or for more social populations like priests and doctors.

In this work, we propose a simple approach to estimate and correct for the degree ratio based on the linear relationship between respondents’ social network sizes and the number of people they know in different subpopulations. Our approach conveniently relies on only the original ARD, possibly allowing researchers to obtain more accurate size estimates without collecting additional data like those needed for the game of contacts and the generalized scale-up estimator.

The rest of this article is organized as follows. First, Section “Background” provides additional background information about the degree ratio and presents the bias of the basic scale-up estimator under certain conditions. Then, in Section “Degree Ratio Adjustment”, we introduce our approach to estimate the degree ratio using only the original ARD responses. We apply this approach to both simulated (Section “Simulation Study”) and real (Section “Network Scale-up Method Studies”) ARD surveys. Finally, we close with a discussion in Section “Discussion”.

Background

We first review model properties of the basic scale-up estimator. Feehan and Salganik (2016) show that the basic scale-up estimator is equivalent to their generalized scale-up estimator only when multiplied by three adjustment factors, one of which involves the degree ratio. The degree ratio adjustment factor arises because some populations have larger or smaller social network sizes on average than other populations. The authors define the degree ratio, $δ_{F}$ , as

δ_{F} = \frac{avg #  connections from a member of H to F}{avg #  connections from a member of F to the rest of F} = \frac{{\bar{d}}_{H, F}}{{\bar{d}}_{F, F}},

where F refers to the frame population, the collection of individuals who may be included as respondents in the ARD survey, and H refers to the hidden or unknown subpopulation. Thus, if the degree ratio is 0.5 (i.e., there are only half as many links per member of H from H to F as there are per member of F from F to F), then the basic scale-up estimator is one half the size of the generalized scale-up estimator. Since the basic scale-up estimator implicitly assumes the average degree of the known subpopulation members is equal to the average degree of the hidden subpopulation members, the estimator misattributes the small number of links to a small subpopulation size, rather than to small degrees. In order to estimate these adjustment factors, Feehan and Salganik (2016) propose collecting an additional ARD survey given to members of the hard-to-reach population, to collect what they call enriched ARD.

While we recognize the utility of enriched ARD, there are three significant limitations. First, enriched ARD is often prohibitively expensive to collect. The low cost and easy implementation of the NSUM are two of its key benefits. Collecting enriched ARD therefore undermines this advantage since only well-funded studies will be able to collect the additional data. Second, it is impossible to collect enriched ARD on impossible-to-reach subpopulations such as individuals who died in an earthquake. Finally, it is inconvenient or impossible to collect enriched ARD for previous ARD studies, so the methods proposed in Feehan and Salganik (2016) can only naturally be used for ARD moving forward. To correct for the biases in existing ARD surveys that did not already collect enriched ARD, users must either assume an adjustment factor for the degree ratio and construct confidence intervals using the rescaled bootstrap procedure (as proposed by Feehan and Salganik 2016), or find and survey a similar contemporary population and assume the behavior of the two subpopulations are similar. Instead, we propose the first method to estimate the degree ratio using only the original ARD, allowing researchers to easily correct for bias introduced by the degree ratio.

We present two related findings connecting the bias of the basic scale-up estimator to the degree ratio. For the following results, we assume perfect link reporting (i.e., no transmission error or recall error), that the respondents represent a simple random sample S of size n from the entire population of size N, and that the frame population F is the entire population, where H is included in F. In this case, the inclusion probability for each respondent i is $π_{i} = n / N$ . We consider two estimators, where either (a) the $d_{i}$ are fixed and known, or (b) the $d_{i}$ are estimated using the L known subpopulations. In the first case, we can represent the basic scale-up estimator as

{\hat{N}}_{H, 1} = \frac{\sum_{i \in S} (y_{i H} / π_{i})}{\frac{1}{N} \sum_{i \in S} (d_{i} / π_{i})},

(2)

while the second case includes the estimation of $d_{i}$ , given by

{\hat{N}}_{H, 2} = \frac{\sum_{i \in S} (y_{i H} / π_{i})}{\frac{1}{N} \sum_{i \in S} [(\sum_{k = 1}^{L} y_{i k} / \sum_{k = 1}^{L} N_{k}) / π_{i}]} .

(3)

Using these estimators, we present the following propositions, where Proposition 1 is a special-case result from Feehan and Salganik (2016) adapted here for completeness, and the proof for Proposition 2 is shown in Appendix A in the online supplement.

Proposition 1.

Adapted from Feehan and Salganik (2016). Consider the size estimate ${\hat{N}}_{H, 1}$ in Equation (2), obtained from a survey with perfect link reporting and from a simple random sample of respondents. Then given known degrees $d_{i}$ , the bias of the unknown size estimate is approximately given by

Bias ({\hat{N}}_{H, 1}) \approx N_{H} (\frac{{\bar{d}}_{H}}{{\bar{d}}_{F}} - 1),

where

{\bar{d}}_{H}

denotes the average degree of individuals in the hidden subpopulation and

{\bar{d}}_{F}

denotes the average degree of individuals in the frame population.

Proposition 2.

Consider the size estimate ${\hat{N}}_{H, 2}$ in Equation (3), obtained from a survey with perfect link reporting and from a simple random sample of respondents. Then given that the $d_{i}$ are estimated using the L known subpopulations in the survey, the bias of the unknown size estimate is approximately given by

Bias ({\hat{N}}_{H, 2}) \approx N_{H} (\frac{{\bar{d}}_{H} \sum_{k = 1}^{L} N_{k}}{\sum_{k = 1}^{L} {\bar{d}}_{k} N_{k}} - 1),

where

{\bar{d}}_{H}

denotes the average degree of individuals in the hidden subpopulation and

{\bar{d}}_{k}

denotes the average degree of individuals in subpopulation k.

These results show that when the true degrees are known, the bias depends only on the true subpopulation size of H and the ratio of the average degrees between the unknown subpopulation and the frame population, while the bias of the estimator when the degrees are also estimated additionally depends on the remaining known subpopulation sizes and the average degrees of individuals in each known subpopulation size. Proposition 2 also shows that the accuracy of the unknown size estimate depends on the specific relationship between the average degrees in subpopulations and the sizes of those subpopulations, and relatively large or small subpopulations will introduce more bias when paired with relatively large or small average degrees, respectively.

Degree Ratio Adjustment

Here we propose a method to correct for the bias introduced by the degree ratio in the basic scale-up estimator. It would be sufficient to know ${\bar{d}}_{k}$ for all known and unknown subpopulations. However, these average degrees are unknown, making a direct approach impossible. Furthermore, a primary advantage of the NSUM is avoiding contacting members of hard-to-reach populations. Thus, our goal is to estimate $δ_{k}$ , the degree ratio for subpopulation k, using existing ARD data to produce an adjustment factor for ${\hat{N}}_{k}$ .

For the remainder of this paper, we let $δ_{k} = {\bar{d}}_{k} / {\bar{d}}_{F}$ represent the degree ratio for subpopulation k, where we depart from the original notation from Feehan and Salganik (2016) to emphasize that the degree ratio (i) exists for both the subpopulations with known size and the hard-to-reach subpopulations, and (ii) varies across subpopulation.

The basic assumption of our approach is that the proportion of an individual’s social network that belongs to group k depends on the individual’s degree. To incorporate this assumption, we modify Equation (1) such that

y_{i k} \sim Binomial (d_{i}, \frac{N_{k}}{N} f_{i, k} (d)),

(4)

where

f_{i, k} (d)

is of the form

f_{i, k} (d) = a + g_{i} (d) c_{k}

for each k across all values of

d_{i}

a \neq 0

, and

g_{i} (d)

is any finite-valued function of

d_{i}

that does not depend on k. Additionally,

c_{k}

is a group-specific term that controls how an individual’s degree affects the probability of knowing people from group k. Necessarily,

f_{i, k} (d)

is limited to functions such that

N_{k} f_{i, k} (d) / N

is between 0 and 1. The form of

g_{i} (d)

does not need to be known, but must be the same for all k.

In the context of NSUM, an example of a reasonable $f_{i, k} (d)$ is a mean-one function of the form

f_{i, k} (d) = 1 + (d_{i}^{p} - \frac{1}{n} \sum_{i = 1}^{n} d_{i}^{p}) c_{k} = 1 + (d_{i}^{p} - \bar{d^{p}}) c_{k},

(5)

where

d_{i}^{p}

represents

d_{i}

to the

p^{t h}

power. The form of

f_{i, k} (d)

is general enough to account for many realistic situations. First,

c_{k}

may be positive or negative, leading to respondents with larger degrees having higher binomial probabilities when

c_{k} > 0

, and vice versa when

c_{k} < 0

. Second, the power p controls how quickly departures from

\bar{d^{p}}

affect the probabilities. The difference between biases for varying values of p depends on the specific degree distribution of the respondents. Furthermore, the choice of p also controls what degree corresponds to

f_{i, k} (d) = 1

Given the above, we have the following result.

Proposition 3.

Consider aggregated relational data generated from the likelihood defined by Equation (4) for any $f_{i, k} (d)$ of the form $f_{i, k} (d) = a + g_{i} (d) c_{k}$ , where $a \neq 0$ , $g_{i} (d)$ is any finite-valued function of $d_{i}$ that does not depend on k, and $f_{i, k} (d)$ leads to a valid likelihood for all k. Then given known degrees $d_{i}$ and the estimator in Equation (2), the bias of ${\hat{N}}_{k}$ is given by

Bias ({\hat{N}}_{k}) = N_{k} (\frac{\sum_{i = 1}^{n} d_{i} f_{i, k} (d) / π_{i}}{\sum_{i = 1}^{n} d_{i} / π_{i}} - 1) .

Furthermore, there exists some

γ_{0}

and

γ_{1}

, which are independent of k, such that

E (\frac{N_{k}}{{\hat{N}}_{k}}) \approx E (γ_{0} + γ_{1} (\frac{\sum_{i = 1}^{n} d_{i} y_{i k} / π_{i}}{\frac{1}{n} (\sum_{i = 1}^{n} y_{i k} / π_{i}) (\sum_{i = 1}^{n} d_{i}^{2})})) .

The proof of Proposition 3 is in Appendix A.2 in the online supplement. Proposition 3 provides a specific form of the degree ratio under our assumed binomial likelihood. From Proposition 1, we have $E ({\hat{N}}_{k} / N_{k}) \approx δ_{k}$ and we show in Appendix A.2 in the online supplement that $E (N_{k} / {\hat{N}}_{k}) \approx 1 / δ_{k}$ . Based on Proposition 3, we need only to first estimate $γ_{0}$ and $γ_{1}$ to then estimate $1 / δ_{k}$ . After estimating $1 / δ_{k}$ , we can create an approximately unbiased estimator, $N_{k}^{a d j} = {\hat{N}}_{k} / {\hat{δ}}_{k}$ . We note that while the results hold only for a fixed function $g (\cdot)$ for all k, we show via simulations in Appendix B in the online supplement that if $f_{i, k} (d)$ takes the form in Equation (5), varying p across subpopulations as $p_{k}$ introduces only relatively minor bias.

Our approach to estimate $γ_{0}$ and $γ_{1}$ is detailed in Algorithm 1. We motivate the adjustment procedure here using two regression steps. The first estimates the slope between the scaled responses, $(y_{i k} / π_{i}) / ((1 / n) \sum_{j = 1}^{n} y_{j k} / π_{j})$ , and the estimated degrees, ${\hat{d}}_{i, - k}$ . An illustrative example for a real ARD survey is shown in Figure 1. Larger slopes correspond to subpopulations where respondents with larger degrees report knowing more individuals relative to their own network size than respondents with smaller degrees. Thus, these slopes can be estimated for all subpopulations, including the unknown subpopulation, and measure the relationship between respondent degree and the bias $f_{i, k} (d)$ . We call these slopes “first-stage slopes.”

Next, we treat these first-stage slopes as covariates in another regression model to model $N_{k} / {\hat{N}}_{k}^{LOO}$ for the known subpopulations and estimate $γ_{0}$ and $γ_{1}$ , where ${\hat{N}}_{k}^{LOO}$ is the estimated size of subpopulation k treating $N_{k}$ as unknown. An illustrative example showing the linear relationship between the observed $N_{k} / {\hat{N}}_{k}^{LOO}$ for the known subpopulations and the first-stage slopes is shown in Figure 2 for the two simulation studies in Section “Simulation Study”. Using this relationship, we can then predict $N_{H} / {\hat{N}}_{H}^{LOO}$ for the unknown subpopulation H using the first-stage slope corresponding to H. Finally, we can use our predicted $N_{H} / {\hat{N}}_{H}^{LOO}$ to adjust the unknown subpopulation size estimate.

Given the sociological interest in the degree ratio for different subpopulations, we recommend estimating $δ_{k}$ for all subpopulations included in the ARD survey, including the subpopulations with known sizes. One example of an interesting degree ratio corresponds to the “priest” subpopulation in the Rwanda NSUM study (Rwanda Biomedical Center/Institute of HIV/AIDS, Disease Prevention and Control Department RBC/IHDPC). We find that the leave-one-out size estimate for priests is substantially larger than the known size, likely due to much larger average degrees for priests. This result is consistent with the findings of McCarty et al. (2001), who found that ARD surveys given to clergy yielded larger average network sizes than ARD surveys given to a representative sample. Despite having a known subpopulation size for priests, we are able to observe through the ARD and our proposed approach that priests have substantially larger than average social network sizes.

While we propose the above methodology as a general approach to correct for the degree ratio, the results are based on an assumed form of the data generating process and motivated through empirical results. In practice we recommend using caution when adjusting the size estimates for subpopulations corresponding to names like “Michael” or “Kristina” in the McCarty et al. (2001) ARD survey. While the popularity of certain names may be related to age and similar demographics, we find that this is dataset dependent and empirically the association is often less pronounced. Applying the degree ratio correction in settings where there is only weak correlation between the relative bias and the first-stage slopes risks correcting for spurious relationships in the data rather than for true signals.

Simulation Study

Binomial Model

We simulate ARD from the biased binomial model presented in Equations (4) and (5). We let $p = 2$ , although the choice of p does not substantially change the results. To provide consistent estimates of ${\hat{d}}_{i, - k}$ , we set the number of respondents at 10000 and the number of subpopulations at 20. We set $N = 10^{7}$ and simulate the subpopulation sizes as $N_{k} \sim Unif (10^{3}, 10^{6})$ , which corresponds to subpopulation sizes between $0.01 %$ and $10 %$ of the total population size. We simulate the degrees initially as $d_{i} \sim Unif (10, 1000)$ and then round them to the nearest integer. Finally, we let $c_{k}$ range evenly from the minimum possible value to the maximum possible value, such that the binomial probability is between 0 and 1 for all respondents. For this simulation, $c_{k}$ ranges from $- 1.5 \times 10^{- 6}$ to $1.5 \times 10^{- 6}$ , resulting in ${\hat{N}}_{k} / N_{k}$ ranging from 0.77 to 1.34. We implement Algorithm 1 in a leave-one-out procedure, where we estimate the adjusted subpopulation sizes for each subpopulation sequentially, treating each successive subpopulation as unknown. In Appendix B in the online supplement, we modify this Binomial simulation study to include unequal sampling weights.

We plot the estimated first-stage slopes against $N_{k} / {\hat{N}}_{k}$ for the 20 subpopulations in Figure 2(a), showing that the estimated slopes are almost perfectly correlated with $N_{k} / {\hat{N}}_{k}$ . The point farthest from the best-fit line corresponds to the smallest subpopulation with prevalence $0.02 %$ , which is relatively small for subpopulations included in NSUM surveys. Despite the small size, the departure from the expected value is minimal.

The performance of the adjusted estimator is shown in Figure 3(a) and Table 1. The reduction in mean absolute percent error is 96%, indicating that we almost perfectly recover the true size estimates. We define MAPE for a set of subpopulations as $MAPE = 100 * (1 / K) \sum_{j = 1}^{k} [(N_{k} - {\hat{N}}_{k}) / N_{k}]$ , where ${\hat{N}}_{k}$ refers to the leave-one-out subpopulation size estimates for the corresponding estimation method (e.g., the basic or adjusted estimators). Most of the remaining error is from the smallest subpopulation above. This simulation shows that despite estimating $\hat{d}$ and ${\hat{d}}_{i, - k}$ , we are still able to adjust for most of the subpopulation size error introduced by the degree ratio.

Table 1.

Percent reduction in mean absolute percent error (MAPE) for the adjusted size estimates for the binomial simulation, SBM simulation, McCarty, and Rwanda Meal studies.

Data set	Subpopulations	Adjusted
Binomial simulation	All	96%
SBM simulation	All	92%
McCarty	All	38%
	Non-names	−3%
	All but twin/diabetes	43%
	Names	31%
	Non-names, no twin/diabetes	36%
Rwanda Meal	All	48%
	All but priest	30%
	Non-names, with priest	57%
	Non-names, no priest	34%
	Names	13%

Percent reduction is calculated by $100 * ({MAPE}^{basic} - {MAPE}^{adjusted}) / ({MAPE}^{basic})$ .

Appendix B in the online supplement contains an additional simulation based on the binomial model, but where p is allowed to vary across subpopulations. The results indicate that the correlation between ${\hat{N}}_{k} / N_{k}$ and the first-stage slopes are still fairly linear, although with larger variation. These results indicate that our proposed approach will likely work for real data where the behavior may differ considerably between subpopulations.

Stochastic Block Model

We simulate a network from a stochastic block model (SBM) with 20000 respondents and 20 groups. We set each group size to be 1000. In order to have a range of connectivity, the within-group connectivities (i.e., the diagonal of the connectivity matrix) are given by a sequence from 0.25 to 0.5 in steps of 0.05. All between-group connectivity probabilities are 0.05. These parameters were chosen to provide a sufficient sample size to generate ARD with realistic values and to provide a range of degree ratios across subpopulations of equal sizes. In order to evaluate the model performance, we again implemented Algorithm 1 and performed the same leave-one-out procedure as for the binomial model simulation study.

We plot the estimated first-stage slopes against $N_{k} / {\hat{N}}_{k}$ for the 20 subpopulations in Figure 2(b), again showing that the estimated slopes are highly correlated with $N_{k} / {\hat{N}}_{k}$ , although the relationship is slightly non-linear, unlike for the binomial model.

The results for this simulation study are shown in Figure 3(b), where the original basic scale-up estimator estimates are shown in pink, our adjusted estimates in green, and blue arrows indicate subpopulations where our adjusted estimates have smaller absolute relative error. In this study, we outperform the basic scale-up estimator for all 20 subpopulations. The percent reduction in mean absolute percent error is presented in Table 1. For this simulation study, adjusting the size estimates resulted in an $84 %$ reduction in mean absolute percent error. It is clear in this simulation study that our proposed method is able to consistently correct for the degree ratio and substantially improve the existing basic scale-up estimator estimates despite the data coming from a model different than our assumed binomial likelihood.

Network Scale-up Method Studies

In this section, we apply our adjustment procedure to two real ARD surveys. We show that despite its simplicity, the proposed adjustment substantially improves size estimates of the known subpopulations when we treat them as unknown. We follow the same procedure outlined in the simulation study to evaluate the performance of our proposed methods, where we again estimate ${\hat{d}}_{i, - k}$ when studying subpopulation k. This matches the procedure used when estimating the unknown subpopulations, where only the ARD responses in the known subpopulations are used to estimate the degrees and subpopulation sizes.

McCarty ARD Study

First, we apply our proposed adjustment method to the ARD first collected and presented in McCarty et al. (2001). This dataset contains responses from 574 respondents about 32 subpopulations, 3 of which are unknown (individuals who are homeless, have been raped, or are HIV positive). Twelve of the 29 known subpopulations corresponds to names. We remove 53 respondents for having 1 or more missing responses (47 of those 53 respondents had only 1 missing response), resulting in 521 respondents. As the primary purpose of this work is to evaluate the performance of our proposed adjusted estimator compared to the basic scale-up estimator, we do not study the effect of removing these respondents with missing data.

The percent reduction in mean absolute percent error for different subsets of subpopulations are shown in Table 1. For this dataset, substantial improvements exist when adjusting subpopulations corresponding to names, where based on Figure 4(a), there seems to be a strong linear relationship between $N_{k} / {\hat{N}}_{k}$ and the first-stage slopes for the names, but a weaker relationship with higher variance for the non-name subpopulations. One potential explanation of this results is that the adjustment depends heavily on the bias introduced by the “twin” and “diabetes” subpopulations, which likely have high transmission effect, but relatively low barrier effects. In practice, researchers may choose to remove these two subpopulations from the second-stage regression to improve the performance of the adjustment procedure for the remaining populations and hopefully to improve the estimate for the target populations. However, this decision should not be made lightly. Practitioners should only consider removing populations when they are sufficiently confident that the populations are outliers due to some additional bias outside of the degree ratio which the researchers are aware of but unable to correct for. For example, researchers may be aware from existing research that the diabetes subpopulation exhibits high transmission effect because people with diabetes intentionally hide their status, especially when compared to the other known group traits in the study, like first names, which have very low transmission error. Killworth et al. (1998) argued that they “have anecdotal evidence that a decade or more can pass without a respondent finding out that close coworkers or business associates have a twin or are diabetic.” Using outside knowledge of errors like transmission effects can help researchers make informed decisions about which populations to include in the adjustment procedure.

We compare the final adjusted size estimates for the 17 non-name subpopulations against the original basic scale-up estimator size estimates in Figure 5(a). The adjusted estimator results in a 38% reduction in mean absolute percent error. The results after keeping the name groups but removing the twin and diabetes groups are shown in Figure 5(b) and resulted in a $43 %$ reduction in mean absolute percent error.

Rwanda Meal ARD Study

Next, we consider the Rwanda Meal ARD survey (Feehan et al. 2016; Rwanda Biomedical Center/Institute of HIV/AIDS, Disease Prevention and Control Department RBC/IHDPC). In 2011, researchers collected ARD from 4,669 respondents in Rwanda in order to estimate the size of four key populations: female sex workers (FSW), male clients of sex workers (MCSW), men who have sex with men (MSM), and people who inject drugs (IDU). Thirteen of the 22 known subpopulations correspond to names. Rwanda Biomedical Center/Institute of HIV/AIDS, Disease Prevention and Control (RBC/IHDPC) and their partners require accurate size estimates of these unknown subpopulations in order to plan and implement efficient HIV prevention strategies for current HIV cases and understand the trend of HIV cases across time.

One of the primary motivations of the survey was to compare the results of NSUM size estimates between two definitions of whether a respondent “knows” someone (Feehan et al. 2016). The first definition, called the acquaintance definition, quantifies the “people the respondent has had some contact with—either in person, over the phone, or on the computer in the previous 12 months.” The meal definition restricts the acquaintance definition, quantifying the “people the respondent has shared a meal or drink with in the past 12 months, including family members, friends, coworkers, or neighbors, as well as meals or drinks taken at any location, such as at home, at work, or in a restaurant.” Feehan et al. (2016) were able to show that estimates from the meal definition were consistently closer to the known sizes than estimates from the acquaintance definition. While the authors were unable to confidently extend this finding to subpopulations with unknown size (e.g. FSW), it is not unlikely that these estimates for unknown subpopulations would also be more accurate.

In order to use the dataset least prone to errors, for our analysis, we consider only the dataset collected from the meal definition. Given that the meal definition implies a stronger relationship between the respondent and their social connections, it is reasonable to assume that the respondent knows more about each person they recalled, reducing the transmission error. Furthermore, given that the pool of potential connections is smaller, respondents should have an easier time recalling everyone in a given subpopulation, also reducing recall error. In order to show that our proposed method accurately accounts for the bias introduced by differences in average network sizes between groups, it is helpful to use a dataset that faces smaller biases from other sources.

In this study, we analyzed responses from 2405 respondents about 22 known subpopulations. Only one respondent was removed for a missing response to how many people they know who are Muslims. We incorporated the provided sampling weights as described in Algorithm 1.

The percent reduction in mean absolute percent error for different subsets of subpopulations are again shown in Table 1. We compare the relative error of the basic scale-up estimator and our adjusted estimates in Figure 6(a) for all known subpopulations and in Figure 6(b) for non-name and non-priest known subpopulations. When considering all subpopulations, the adjusted estimator has a 48% reduction in mean absolute percent error. While our adjusted estimator performs best when including the priest group, we remove this group from Figure 6(b) to show that even after removing highly influential groups like priest, our adjusted estimator still outperforms the basic scale-up estimator. Visually, adjusting the estimate via our approach substantially improves the overall performance of the basic scale-up estimator. Numerically, our adjusted estimates perform better in six of the nine non-name and non-priest known subpopulations. However, the adjusted estimators perform significantly better than the basic scale-up estimator for those six subpopulations, while only performing slightly worse for the remaining subpopulations. Our adjusted estimator reduced the mean absolute percent error 34% for the non-name and non-priest groups. With the priest subpopulation included, the percent reduction in mean absolute percent error is reduced by 57%. The adjusted size estimates are substantially better when including the priest subpopulation because priests have relatively large social networks and have significantly larger social networks than other members of the population, emphasizing that our proposed methods works especially well when there are clear differences in social network sizes across subpopulations (McCarty et al. 2001).

Unlike for the McCarty et al. (2001) dataset, the proposed adjustment does not work very well for name-based groups. Adjusting size estimates for only the name-based groups results in only a 13% decrease in mean absolute percent error. Based on Figure 4(b), we see that for the name groups, $N_{k} / {\hat{N}}_{k}$ is not highly correlated with the first-stage slopes.

Discussion

We have demonstrated through both simulations and through two data examples that our proposed degree ratio adjustment can substantially reduce the bias of the basic scale-up estimator. McPherson et al. (2011) found that homophily of social networks exists for a variety of groups, including those characterized by behaviors, attitudes, and occupations. This observations lends some credibility towards the assumed form of the bias term $f_{i, k} (d)$ for both the known and unknown subpopulations since different groups may form social networks in similar ways. We rely on the performance of our proposed approach with respect to the known subpopulations out of necessity since we are unable to access the quality of NSUM estimators for the unknown subpopulations.

The key novelty of this paper is that our proposed method handles the very difficult problem of varying average network sizes across different subpopulations without using auxiliary data. Methods that use auxiliary data may intuitively perform better than our combined procedure and we encourage researchers to use additional data when available. However, collecting additional data is often impossible, necessitating an approach that recycles the available data.

An interesting direction for future work is to consider how the proposed degree ratio adjustment affects the choice of known subpopulations in the NSUM survey. Previous researchers have proposed relying heavily on known groups corresponding to names, since these groups may be subject to fewer and smaller biases. However, if the adjustment procedure relies on estimating the relationship between the first-stage slopes and the estimator bias, it may actually be advantageous to instead include groups in the survey that provide a wider range of estimator bias than name-based groups since this may lead to more accurate adjustments.

Furthermore, as we presented in this work, NSUM models should be evaluated using performance metrics that do not favor large subpopulations. Metrics like root mean squared error are dominated by these large populations like “people who have diabetes” or “people who are twins” so that the accuracy of size estimates corresponding to populations like “people who were murdered” or “people who committed suicide” are not influential.

McCormick and Zheng (2007) proposed a calibration curve to correct for bias in subpopulation size estimates due to under-reporting of large subpopulations and over-reporting of small subpopulations. Our proposed methodology here can be thought of as an analogous calibration curve, but to correct for bias introduced by the degree ratio. In both cases, the NSUM size estimates are calibrated by reusing the known subpopulation sizes, hopefully leading to more accurate estimates for target subpopulation sizes.

As with all methods used to estimate the size of hard-to-reach subpopulations, it is difficult to understand, model, and account for all sources of bias. In some cases, accounting for one source of bias may result in worse estimates if the other sources of bias are ignored. Continued research is needed to understand how the different NSUM biases interact together and whether it suffices to account for each form of bias independently. We believe the NSUM holds an important role in providing accurate, quick, and affordable size estimates and urge future researchers to continue developing this promising method.

Supplemental Material

sj-pdf-1-smr-10.1177_00491241251364233 - Supplemental material for Estimating and Correcting Degree Ratio Bias in the Network Scale-up Method

Supplemental material, sj-pdf-1-smr-10.1177_00491241251364233 for Estimating and Correcting Degree Ratio Bias in the Network Scale-up Method by Ian Laga, Jessica P. Kunke, Tyler H. McCormick and Xiaoyue Niu in Sociological Methods & Research

Footnotes

Declaration of Conflicting of Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute of Mental Health of the National Institutes of Health under award DP2MH122405, by the National Science Foundation under grant NSF SES-2215369, and by National Institute of Allergy and Infectious Diseases/National Institutes of Health grant R01-AI136664.

Supplemental Material

Supplemental material for this article is available online.

ORCID iDs

Ian Laga

Jessica P. Kunke

Tyler H. McCormick

Data and Code Availability Statement

The datasets analyzed during the current study are publicly available, but we do not have permission to distribute them. All code used to create the results presented in this manuscript and instructions for requesting and downloading the datasets (Laga, 2025).

Author Biographies

Ian Laga is an assistant professor in the Department of Mathematical Sciences at Montana State University. His research interests include the modeling of hard-to-reach populations and Bayesian small area estimation.

Jessica P. Kunke is a PhD candidate in statistics at the University of Washington. Her research interests include survey methodology and spatiotemporal statistics with environmental and social applications.

Tyler H. McCormick is a professor of statistics and sociology at the University of Washington, and a core faculty member in the Center for Statistics and the Social Sciences.

Xiaoyue Niu is an associate professor of statistics at Pennsylvania State University. Her research focuses on network modeling with applications in social sciences and global health.

References

Bernard

H. Russell

Johnsen

Eugene C.

Killworth

Peter D.

Robinson

Scott

. 1989. “Estimating the Size of an Average Personal Network and of an Event Subpopulation.” in Pp. 159–175 The Small World, Ablex Press.

Breza

Emily

Chandrasekhar

Arun G.

McCormick

Tyler H.

Pan

Mengjie

. 2020. “Using Aggregated Relational Data to Feasibly Identify Network Structure Without Network Data.” American Economic Review 110: 2454–84.

Feehan

Dennis M.

Salganik

Matthew J.

. 2016. “Generalizing the Network Scale-up Method: a New Estimator for the Size of Hidden Populations.” Sociological Methodology 46: 153–86.

Feehan

Dennis M.

Umubyeyi

Aline

Mahy

Mary

Hladik

Wolfgang

Salganik

Matthew J.

. 2016. “Quantity Versus Quality: A Survey Experiment to Improve the Network Scale-up Method.” American Journal of Epidemiology 183: 747–757.

Killworth

Peter D.

McCarty

Christopher

Bernard

H Russell

Shelley

Gene Ann

Johnsen

Eugene C.

. 1998. “Estimation of Seroprevalence, Rape, and Homelessness in the United States Using a Social Network Approach.” Evaluation Review 22: 289–308.

Laga

Ian

. 2025. “ilaga/NSUM_Degree_Ratio: Code for Estimating and Correcting Degree Ratio Bias in the Network Scale-up Method.” Zenodo. https://doi.org/10.5281/zenodo.15724790.

Laga

Ian

Bao

Niu

Xiaoyue

. 2021. “Thirty Years of The Network Scale-up Method.” Journal of the American Statistical Association 116: 1548–1559.

Laga

Ian

Bao

Niu

Xiaoyue

. 2023. “A Correlated Network Scale-up Model: Finding the Connection Between Subpopulations.” Journal of the American Statistical Association 118: 1515–1524.

Maltiel

Rachael

Raftery

Adrian E.

McCormick

Tyler H.

Baraff

Aaron J.

. 2015. “Estimating Population Size Using the Network Scale Up Method.” The Annals of Applied Statistics 9: 1247.

10.

McCarty

Christopher

Killworth

Peter D.

Bernard

H Russell

Johnsen

Eugene C.

Shelley

Gene A.

. 2001. “Comparing Two Methods for Estimating Network Size.” Human Organization 60: 28–39.

11.

McCormick

Tyler H.

2020. “The Network Scale-Up Method.” in The Oxford Handbook of Social Networks, edited by Ryan Light and James Moody (2021; online edn, Oxford Academic, 15 December 2020).

12.

McCormick

Tyler H.

Zheng

Tian

. 2007. “Adjusting for Recall Bias in “How Many X’s Do You Know?” Surveys.” Proceedings of the Joint Statistical Meetings.

13.

McPherson

Miller

Smith-Lovin

Lynn

Cook

James M.

. 2011. “Birds of a Feather: Homophily in Social Networks.” Annual Review of Sociology 27: 415–444.

14.

Rwanda Biomedical Center/Institute of HIV/AIDS, Disease Prevention and Control Department (RBC/IHDPC), School of Public Health (SPH) University of Rwanda, UNAIDS, and ICF International. 2012. “Estimating the size of populations through a household survey.”

15.

Salganik

Matthew J.

Mello

Maeve B.

Abdo

Alexandre H.

Bertoni

Neilane

Fazito

Dimitri

Bastos

Francisco I.

. 2011. “The Game of Contacts: Estimating the Social Visibility of Groups.” Social Networks 33: 70–78.

16.

Shelley

Gene A.

Bernard

H Russell

Killworth

Peter

Johnsen

Eugene

McCarty

Christopher

. 1995. “Who Knows Your HIV Status? What HIV+ Patients and Their Network Members Know About Each Other.” Social Networks 17: 189–217.

17.

Teo

Alvin Kuo Jing

Prem

Kiesha

IC Chen

Mark

Roellin

Adrian

Wong

Mee Lian

Hanh Hao

Cook

AlexR.

. 2019. “Estimating the Size of Key Populations for HIV in Singapore Using the Network Scale-up Method.” Sexually Transmitted Infections 95: 602–607.

18.

UNAIDS. 2021. “End Inequalities.” End AIDS. Global AIDS Strategy 2021-2026.

19.

Zheng

Tian

Salganik

Matthew J.

Gelman

Andrew

. 2006. “How Many People Do You Know in Prison? Using Overdispersion in Count Data to Estimate Social Structure in Networks.” Journal of the American Statistical Association 101: 409–423.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.24 MB