Free-text comments as a tool for developing the self-report method: Parents’ responses to a survey on violence against children

Self-report measures as a tool in quantitative research of delinquency and victimisation

Self-report measures are usually used in a survey setting. Surveys have several benefits for studying crime. Lower costs and time efficiency are often the referred to benefits, but also systemisation, precision and objectiveness are associated with quantitative criminology, whereas greater risks are linked to certain qualitative methods (i.e. ethnography to study criminals) (Jacques, 2014). However, it is also acknowledged that survey research has its limitations, and surveys can only achieve the reality of the phenomenon under observation to a certain extent.

In addition to these benefits and limitations concerning all kinds of quantitative research, specific benefits and limitations have been connected to self-report measures on victimisation and delinquency. The most significant benefit of a self-report delinquency or victimisation survey is that these surveys reveal the extent of hidden crime previously not tapped by official sources (Kivivuori, 2011). Police, child welfare and medical reports, for example, demonstrate only a fraction of violence experienced by children (e.g. Ellonen & Pösö, 2014). On the contrary, particular problems associated with self-report measures are related to over- and underreporting, as well as shortages in covering the whole range of the phenomenon under observation, limited response sets and a tendency to elicit reports of trivial acts (Ellis et al., 2010, p. 281). To cover also infrequent acts, very large sample sizes are often needed, which increases the cost of the study. Further, regardless of the sample size, surveys may still exclude certain groups from the sample, such as people with reading difficulties or people without an address.

Self-report measures, as a measurement tool, are also particularly sensitive. When studying crime and victimisation, people do not always agree on what constitutes violence and what does not. Identifying particular incidents from an individual’s own life may also be challenging, if the incident was, for example, meaningless from the perspective of the individual. On the contrary, reporting some painful incident in one question may cause such discomfort that an individual hesitates answering the rest of the questions.

Self-report measures in studying violence against children

Work by Murray Straus and his research team (Straus et al., 1998) as well as David Finkelhor and his research team (Finkelhor et al., 2005) can be seen as forerunners in asking children about their own violent experiences and asking parents about their violent behaviour towards their children. Since then, empirical studies measuring children’s victimisation have been conducted in many countries and in different fields of study (Clément & Chamberland, 2014; Ellonen et al., 2011, 2016). These studies have provided knowledge of the prevalence of the different types of violence against children, which cannot be detected in any other sources. Most of these studies have used modifications of measures originally designed by Murray Straus or David Finkelhor, such as different versions of the Conflict Tactics Scale (Straus et al., 1998) and the Juvenile Victimization Questionnaire (Finkelhor et al., 2005), which also make some international comparisons possible. For example, it seems that rates of violence against children at home are much lower in Nordic countries (Ellonen et al., 2011; Janson, Jernbro, & Långberg, 2011), than for example in Canada (Clément & Bouchard, 2005) and England (Radford et al., 2011). Based on self-report surveys several explanatory analysis of violence against children have also been conducted showing the significant risk factors of children’s victimisation (see literature review in Peltonen, Ellonen, Pösö and Lucas, 2014).

From the methodological point of view studying children with self-report measures differs from studying adults or even adolescents. The use of self-report measures always requires thorough ethical consideration. When studying children, this is especially true due to the sensitive nature of this particular research topic. Children are more vulnerable than adults and children may not be as able as adults to handle difficult issues or feelings regarding (the questions of) violence. In addition, small children may lack reading and writing skills, and thus the questions about victimisation of children must be addressed to parents (e.g. Janson et al., 2011). These ethical questions have thus been a particular interest in several studies (Ellonen & Fagerlund, 2015; Ellonen & Pösö, 2010; Helweg-Larsen & Bøving-Larsen, 2003).

Less attention has, however, been paid to other methodological questions, especially to developing the self-report measure in studying violence against/experienced by children. Existing studies and questionnaires with repeated data collections have been developed between sweeps by learning from the previous collection (Clément & Chamberland, 2014; Fagerlund et al., 2014). In addition, pilot studies have been used in testing questionnaires (Fagerlund et al., 2014). However, other developments are scarce. Taking into account the nature of these studies, knowledge on executed surveys should be used more broadly to develop self-report measures in studying violence against children.

A specific challenge for self-report violence studies is, for example, to methodologically examine and evaluate how people understand violence and how that understanding guides their answers. People do not always agree on what constitutes as violence, and they may not always be able to identify particular incidents from their lives as violence. For example, pushing the child in the sense of discipline is usually identified as violent behaviour whereas pushing the child to prevent him/her to hurt him/herself is necessarily not. These constructions of violence vary also between cultures. In some countries all violence against children is banned (e.g. Finland) and in others not, which naturally influences on self-reports.

Open-ended questions and free-text comments as a part of questionnaires

Surveys in general are typically seen as tools for gathering quantitative data. Sometimes surveys are even referred to as being unfit for the collection of qualitative data (Beckett & Clegg, 2007). At the same time survey respondents are often provided with the opportunity to write free-text comments as the final part of the questionnaire (Ong, Dunn, & Croft, 2006). In addition, open-ended questions are often used as a part of a structured survey. These two question types are sometimes referred to as synonyms (Ong et al., 2006) although a clear separation can also be made: Open-ended questions are a part of a structured question, whereas free-text comments refer to a space for free responses.

It has been argued that open-ended questions can be used to ‘come as close as possible to a natural conversation and to avoid closed alternatives hindering a verbatim description full of detail and clarification’ (Friborg & Rosenvinge, 2013, p. 1398). The same can be argued in the case of free-text comments. In addition, free-text comments may give important further information and offer a context for other responses, as well as bring out issues that quantitative survey data alone cannot reveal (Rich, Chojenta, & Loxton, 2013; Riiskjær, Ammentorp, & Kofoed, 2012). Besides additional and in-depth information, open-ended questions and free-text comments may also lead to results differentiating from those of closed measures (Friborg & Rosenvinge, 2013; Rich et al., 2013). It has also been suggested that open-ended questions are more suitable for asking about sensitive or potentially stigmatising topics (Friborg & Rosenvinge, 2013; Greenfield, Nayak, Bond, Ye, & Midanik, 2006).

The rich content and insightful information offered by free-text comments of surveys are thus recognised, however, the use of these is limited (Rich et al., 2013). A search for scientific articles on free-text comments and open-ended questions of surveys finds mostly medical studies and surveys related to healthcare (Friborg & Rosenvinge, 2013; Garcia, Evans, & Reshaw, 2004; Ong et al., 2006; Rich et al., 2013; Richards et al., 2009; Riiskjær et al., 2012) and some considering marketing research and consumer behaviour (e.g. Ares, Giménez, Barreiro, & Gámbaro, 2010; ten Kleij & Musters, 2003) but no articles from social science or criminological research. This may be because the researchers easily identify themselves as either quantitative or qualitative researchers and open-ended questions and free-text comments are seen as something in the middle and are thus excluded from any detailed analysis. The methodological literature also lacks guidance on what to do with these responses as a part of quantitative study (Garcia et al., 2004; Riiskjær et al., 2012). Instructions can be found for content coding of open-ended questions in personality psychology (Woike, 2009) as well as a comparison of the usage of open-ended questions asked in face-to-face interviewing compared to computer-assisted self-interviewing (Fairweather, Rinne, & Steel, 2012). Garcia et al. (2004) also discussed the potential of free-text comments for enhancing survey analysis in their study involving a sample survey on maternity care and a longitudinal cohort study of UK medical graduates. Yet to our knowledge, there are no methodological studies aiming to develop self-report surveys in the field of criminology or social sciences by utilising the information gathered from free-text comments.

Also from the research ethical point of view utilisation of free-text comments is not clear. Ethical responsibilities of a researcher to research subjects as well as fellow scientists are clear and widely discussed (Christians, 2000; Ellis et al., 2010) but responsibility concerning the utilisation or non-utilisation of data has seldom been discussed. For example, is it ethically justified to ask for free-text comments if there is no intention to analyse those? A few indications of such a consideration can be found, specifically concerning open-ended questions (Boynton & Greenhalgh, 2004; O’Cathain & Thomas, 2004). According to those considerations, asking for information without a plan to analyse it may be considered as a waste of both participants’ and researchers’ time and therefore this practise is ethically questionable (Boynton & Greenhalgh, 2004). A thorough discussion of this is, however, not available.

Taking into account the arguments of the benefits related to open-ended questions and free-text comments and the sensitive nature of the self-report victimisation or delinquency surveys, free-text comments could be seen as a valuable tool for developing self-report measures. Careful phrasing of the questions and structuring of the questionnaire makes self-report a sensitive tool. Respondents’ knowledge could therefore be useful in improving the development of a self-report measure.

Aim of the study

In this article, we will analyse parents’ free-text comments in response to a self-report survey of their own violent behaviour towards their children to answer two research questions. First, we will analyse the themes presented by parents concerning violence against children as a phenomenon, and second, we examine the methodological input of these free-text comments in self-report survey.

Data and methods

This study is based on survey data collected in mainland Finland in 2011. ¹ The survey was conducted among parents of children between 0 and 12 years of age. A randomised sample, taking into account a regional distribution, was collected from the Finnish Population Register, which includes all Finnish citizens. Although the sample was constructed based on the presence of 0- to 12-year-old children in the household, the questionnaire was addressed to the parents of those children. If more than one parent resided in the household, only one was asked to answer to the questionnaire, and they were free to decide which of them would do so. The total number of parents/guardians responding to the questionnaire was 3170. The response rate was 53%.

The questionnaire contained questions about parents’ violent behaviour towards their child (selected into the sample). These questions were based on questions originally developed through Swedish studies conducted in 2000, 2006–2007 and 2011 (Janson et al., 2011). For Finnish usage, the questionnaire was back-translated into Finnish and Finnish–Swedish (another official language in Finland). Parents were also asked to report other violent incidents experiences by the child outside of home to the best of their knowledge. These incidents were asked by using questions which were originally used in the Finnish Child Victim Survey conducted among 12–13 and 15–16-year-old children in 2008 (Ellonen et al., 2008) and 2013 (Fagerlund et al., 2014).

The final question of the survey was: How did it feel to answer the questions of this survey? You can also write other thoughts raised by the survey. This aim of this question was to invite free text. Of all respondents 2047 (65%) reported their thoughts in response to this question. Answers varied from one word to several paragraphs of text. Feelings about answering are analysed elsewhere (Ellonen & Fagerlund, 2015) and this analysis focuses on the comments on the studied subject and the survey in particular.

Making sense of the responses

This first free-text category includes thematic descriptions which aim to explain, or to make sense of, the responses given to individual questions. In particular, the connections between different questions are addressed as important issues which need to be explained in more detail. The descriptions focus on two themes, first, on providing more information about the family or the violent incident, and, second, the reasons for violent behaviour.

When describing the family and the violent incidents, the free-text descriptions provide more information and clarify what really happened in addition to just saying ‘yes’ to the structured question. The tone of these comments aims mostly to downgrade the information given before: parents emphasise in their writing that the incident, reported in the structured questions, is not in fact very serious. Most of these comments are about the violent incidents between siblings. Apparently answering ‘yes’ to the question ‘has anybody hit your child during past 12 months’ is seen as being more serious than the actual act as demonstrated by the following comments written by two parents.

I reported that my other child, who is 5 years old, has hit my other child a few times. But I think it hasn’t been serious, more like normal fighting between siblings.

In our case the 2-year-old little sister hits her brother when she is angry. She has to apologise to him every time. This hitting doesn’t cause any pain to my son.

The questions in self-report surveys are always about a concrete act such as hitting, since hitting is clearer as a definition for a majority of people, whereas the question about violence could mean different things to different people. These parents’ comments, however, demonstrate that although people may share an understanding of what constitutes hitting, they may have specific criteria to that act. For example, hitting is only considered as such when it causes pain. Or that hitting between siblings is a different kind of hitting than a hit from a stranger.

The free-text comments also included remarks about the family and its structures. In particular, the child care arrangements between separated parents are described in more detail to say what ‘separated family’ meant for this family, and children’s lives are described in a more detailed manner. For example, children’s hobbies are listed and school achievements are described to show that the children are coping well regardless of the negative experiences they may have. This suggests that the questions and the existing categories about family structures are not seen as informative enough or that the parents want to avoid giving any stereotyped picture of their situation, enhanced by the categories given in the questionnaire (e.g. separated family). One parent described their family situation in the following way.

My relationship with my children is good. I have gone through bitter divorce and now I am a single mother. Meetings between children and their father didn’t go well and children thought their father has abandoned them. But now things are better and my new husband is a great father figure for them.

Parenting challenges were described as difficulties in balancing between paid work, care of children and stress in parenting. These comments are also given by parents who had not personally acted violently. The parent below describes stress from a parenting perspective but also her/his skill to avoid violent acts.

It is reliving to notice that despite occasional tiredness I am able to recognise the boundary between acceptable and unacceptable behaviour. Sometimes it is very hard to combine work, taking the child to day care, going to the grocery store, housework and taking care of the child….

Some parents explain their violent behaviour by the characteristics of the child. In some comments the child is seen to be so ‘difficult’ that violence is the only option in some situations.

I think that a parent doesn’t always behave how he/she would like to behave. I don’t accept any kind of physical violence but sometimes the situations with children are so stressful that I grab by child by his/her hair, although I wouldn’t like to do that. I also yell at him/her although I know it doesn’t help and it just make us all feel bad  …

Violent behaviour is occasionally described as a necessary and reasoned act in certain situations. The first excerpt below describes the parent’s rationale in her/his use of violence, suggesting that it is a contract between her/him and the children, and the second excerpt explains the parents’ use of violence by the children’s behaviour due to their special needs.

Children have been told that I don’t mean to hurt them but if they won’t take my word on something, nothing else helps besides pulling hair

We have four children and two of them are slightly disabled. The older one of these children is 10 years old and the younger is 7 years old. Both of these children have issues with anger and fear. We as a parents experience physical violence from these two every day and we have to protect our other two children from hits and kicks. We try to avoid physical ways but it is not always possible.

Another perspective on the same issue describes the children’s characteristics as such that they provoke problem-free parenting. ‘It was easy to answer because Jessica is so kind and easy that we don’t have any problems’.

The third reason for using physical violence is that parents themselves have experienced physical punishment in their own childhood. Most of these comments, however, are presented to demonstrate the parent’s avoidance to behave in a similar way than his/her own parents, as highlighted by the first excerpt. If the parent had failed to avoid physical punishment, this failure was explained by their childhood experiences (the second extract).

Usually the use of violence is transmitted from one generation to another. I am proud that in my family we have been able to break this cycle. It hasn’t been easy but we have done a lot of work for it.

I have been pulled by the hair, slapped and hit with a belt in my childhood. I remember those times and I have decided I will never hit my child. Despite that I have lost my temper sometimes and I pulled my child’s hair and slapped her. I felt extremely guilty. After that I have learned my lesson and now I feel that I am a good mother.

Defining violence against children

The second category of free-text comments by parents can be read as aiming to define what violence against children is as a phenomenon. The definitional comments deal with a line between acceptable and unacceptable parental – their own or other parents’ – behaviour, and with the different arenas for violence. The latter descriptions aim to cover such arenas which were excluded from the questionnaire but which are seen as important by parents.

When writing about the line between acceptable and unacceptable parental behaviour, the comments are very much about what constitutes parental violence. In those textual comments the motive of the behaviour, and whether the act causes injury or not, are given a significant role. Similar themes are written about in the previous category that aims to make sense of violence. In this category, the themes are not used for the sake of explanation but for presenting one’s thoughts about the phenomenon and debating the issue.

All kinds of violence against children (or any other person) is wrong. But the purpose of, for example, hair pulling or flicking is just to “wake the child up”, not to hurt the child. You can’t compare hair pulling or flicking to, for example, hitting. These acts are in a different category.

When the purpose is to emphasise what the parent is saying and make the child listen, for example, flicking is not wrong.

According to the comment above, parental violence is defined either by its purpose or the harm caused by it. If the purpose is not to hurt the child, but to educate the child, it is not necessarily violence. Or if the act does not cause any pain to the child, it is not necessarily violence. Also the level of severity is seen to be meaningful: hitting could be violence but that is not the case of hair pulling, as highlighted by the first excerpt above.

Also the line between the siblings’ playful fighting and acts of violence was a topic of many comments. In the questionnaire physical violence between siblings was asked in questions including acts like slapping, hitting and bullying. The respondents were advised in the form to exclude playful fights from serious incidents. Although the questions were based on concrete acts, respondents connect those acts to the word ‘violence’ and argue that the word violence is too strong to describe ‘ordinary’ physical fighting between siblings. Some respondents, however, acknowledge that sometimes fighting can go too far and it becomes violence. The line between ‘playful wrestling’ and serious violence is, however, hard to define. The difficulty to draw the line is described in the following comments:

Fighting between siblings is hard to see as violence, which should be reported to authorities. Also in school, fights between boys are normal and hard to avoid even though everybody knows the rules.

Fights between siblings are normal and if a sister hits a brother, the experience isn’t as serious to the child as if some other peer hits her.

Regarding violence in hobbies, the comments acknowledge especially sport-related violence. It is described as severe. Below there is an excerpt in which the parent describes in many words what his/her son has experienced on his football team.

I want to tell about the kind of treatment children have to put up with during hobbies. Being bullied by a sister is nothing compared to it. My son plays football and the assistant coach singles him out and picks on him. My son has had to listen to the coaches criticising him about his football skills and during tournaments, they have even yelled profanities at him from the sidelines. Despite trying to create a supportive environment at home, where our son is very nice and well-behaved, we cannot protect him from this kind of outside violence.

This parent contrasts sport-related violence with that of the siblings and argues it to be more severe. Most importantly, when violence between siblings or peers is seen as hard to define by parents, violence in hobbies and in school is seen as very serious and the descriptions are given in a very straightforward manner.

Children and violence in society

In this third category, the free-text comments address society, its legislation and service system in particular. These comments are made mainly from a normative perspective such as opinions on legislation and other societal norms regarding the parental use of violence or about the policies and practices which society has in place to tackle violence.

Although all forms of violence against children are criminalised in Finland, including disciplinary violence, all parents do not agree with the legislation and they use the opportunity to express their opinions in the free text. The existing norms are criticised to restrict parental upbringing as demonstrated by the extract below.

I worry that the Child Welfare Act has become tightened. I support old-fashioned upbringing methods. There has to be discipline and to some extent a child can be physically disciplined. Even beating is allowed, if the child crosses the line and nothing is enough. The current gentle upbringing is not good for them in the long run and children become arrogant and slack young adults. But I do not accept violence or brooding.

It is argued that parents should have the right to physically punish their child. Similar statements are made in support of teachers’ rights. According to Finnish legislation teachers are not allowed to physically punish pupils. The right of teachers to use violence is seen as a preventative measure in some free-text comments: it could stop school-based violence as argued by the text below.

In school and on the way to school the amount of violence is cruel, and also some parents are quite aggressive toward stopping this but the violence cannot be solved while all tools for intervention have been taken away from teachers.

In addition to the parents’ opinions about legislation, the free-text comments include views about public services in conjunction with parenting and the use of violence. Some of the comments express opinions on a very general level but they all share the message that the services do not meet the parents’ and families’ needs.

I have been thinking about the tired, worried and poor mothers and fathers, of which we have a lot here in Finland. The government only takes from us and does not give back a lot. With regard to home help, for example, in the past you could get home help for a day for a reasonable price, if you wanted to go, for example, to the child health clinic with your other child, or you wanted to go to a store alone or anywhere else. Nowadays you can’t get any help unless you are a customer of child protective services, you are a drunk or mentally ill. There isn’t any preventative care. You do not want to be labelled as a mad person or a drunk because you use the social services of the social welfare agency.

Answering felt frustrating because for several years I have tried to get help for my boy but getting it is very hard and takes too much time.

Experiencing exclusion

The fourth category of free text included descriptions of the respondents’ experiences of exclusion from the questionnaire as they state that the questions were not suitable for them. One reason is the topic itself – violence only happens to other people – and the other reason is the formulation of some questions.

When the topic of the questionnaire is described as distant or unsuitable, the parents report their lack of experience in this matter, and therefore state the difficulty in answering the questionnaire. These respondents also express their worries that some children ‘out there’ experience these things; yet they themselves are excluded from the group of people knowing and experiencing parental use of violence. Another reason for some respondents seeing this topic as being unsuitable is the young age of their child. Some of the parents with an infant struggle with the questions because they do not think these things could happen to such small children, like seen in the second extract.

Survey made me realise how well things are in our family but not in every family.

My child is only 17 months old, so I wasn’t the right person to answer to this survey. We haven’t had any conflicts yet.

On the other hand, some remarks of being excluded from the study include comments about the questions. The comments address questions which are difficult to understand or which do not accurately reflect their personal life situation. Some comments are about the use of words in the questionnaire and how all questions were not applicable, for example, if you lived in a same-sex relationship.