Modifying researchers’ data management practices

International

The international research environment promotes the sharing of information awarded by publicly funded research and encourages the reuse of data sets for the betterment of the community by ensuring that practice reflects policy and guidelines. For example, in the Research Councils UK’s (RCUK) (2014a) Research Outcomes Overview, it is stipulated that researchers must ‘demonstrate the value and impact of research supported through public funding’ and ‘Information on research outcomes attributed to all RCUK funded awards are now collected online’. In addition, the RCUK policy (Research Council UK, 2015) mandates that research data be open and discoverable and that policies and procedures should be in place to ensure compliance with these practices. RCUK actively works with researchfish (Research Councils UK, 2014b), which is a portal where researchers can log the outputs, outcomes and impacts of their work.

Likewise, the United States’ National Science Foundation (NSF) is ‘dedicated to the support of fundamental research and education in all scientific and engineering disciplines’(National Science Foundation, 2016a). The NSF’s webpage on the dissemination and sharing of data states that all applicants must submit a data management plan and describe how it meets the policy on the dissemination and sharing of research results (National Science Foundation, 2016b).

Similarly the European Commission (EC) has instituted an Open Research Data Pilot as part of Horizon 2020 (European Commission, 2016). All research proposals must include a section on research data management which clearly outlines how the data will be disseminated, shared and retained. This informs the Digital Single Market, which aims to open up digital opportunities for citizens’ access to information and promote modern open government.

Australia

Similar to the United Kingdom and the United States, the Australian Research Council (ARC) (2014) considers data management planning an important part of the responsible conduct of research. The ARC strongly encourages the depositing of data arising from a research project in an appropriate, publicly accessible subject and/or institutional repository. Additionally, the National Health and Medical Research Council‘s (2014) Policy on the Dissemination of Research Findings states that the metadata from all NHMRC projects should also be made available in an open source journal within 12 months of publication.

The Australian Code for the Responsible Conduct of Research (National Health and Medical Research Council, 2007) promotes integrity in the proper management of research data, publishing, dissemination and attribution of authorship by use of a best practice framework. Researchers and institutions each have a responsibility to comply with legislation, guidelines, policy and codes of practice as part of responsible research.

Griffith University

Griffith University is a comprehensive, research-intensive university, ranking 37th in the 2015/16 QS University Rankings Top 50 Under 50 (Quacquarelli Symonds, 2015). Located in the rapidly growing corridor between Brisbane and the Gold Coast in Southeast Queensland, the University offers more than 200 degrees across five campuses to more than 43,000 students from 130 countries studying at undergraduate through to doctoral level in one of four broad academic groups: arts, education and law; business; science; and health. Griffith’s strategic research investment strategy has positioned it to be a world leader in the fields of Asian politics, trade and development; climate change adaptation; criminology; drug discovery and infectious disease; health; sustainable tourism; water science; music and the creative arts.

The Division of Information Services (INS), with which the authors are affiliated, has a long and proud tradition of providing quality service to Griffith students and staff. It also has an international reputation for being innovative and cutting-edge in the deployment of emerging technologies (Brown et al., 2015; Searle et al., 2015; Stanford Libraries, 2013).

In 2014 Griffith University introduced its Best Practice Guidelines for Researchers: Managing Research Data and Primary Materials, in response to the Australian Code for the Responsible Conduct of Research (National Health and Medical Research Council, 2007). The subsequent Griffith University Code for the Responsible Conduct of Research was endorsed by the University’s Academic Committee in July 2012 and subsequently updated in November 2015. According to the principles of the Code, the University is required to provide research infrastructure, training and professional development opportunities as well as elicit an understanding from researchers that they are to properly manage their data, while adhering to all applicable policies, legislation and standards with honesty and integrity.

Griffith University has invested significantly in the provision of solutions and services for capturing, storing, analysing, managing, sharing and publishing data. Examples of institutional storage solutions include (a) Research Storage Service (https://research-storage.griffith.edu.au/), a self-serve private cloud solution based on the open source application, ownCloud, and (b) a research data repository. Since implementing the ownCloud solution in 2015, as of May 2016 there were 580 Griffith users. Although a pleasing start, this constitutes only approximately 15% of potential users at the institution. Given the status of Griffith as a research intensive university, a minimum of 30% take-up by May 2016 seemed a reasonable expectation. However, it became apparent that there is a lack of incentives for researchers to use enterprise services. The research project described in this paper is but one response to bring these services to the attention of the academic community.

Griffith University is currently developing an institutional approach to research data support by utilising the Australian National Data Service (ANDS) Data Management Framework (Australian National Data Service, 2016b) and the Capability Maturity Model Integration (CMMI) (Australian National Data Service, 2016a). Key stakeholders of the University, i.e. Griffith Graduate Research School, Office for Research, and Information Services, are seeking a common understanding for the strategic direction of data management, education, training, services and solutions in order to develop a data support model and an institutional data service.

The research project described in this study, along with others currently being undertaken, will inform the process of developing a suitable framework.

A-COM-B Framework

In investigating interventions which could be used to improve researchers’ data management practices, Wolski and Richardson (2015) examined a number of behaviour and behaviour change theories and models. Ultimately they chose the COM-B system developed by Michie et al. (2011) as the simplest, yet most comprehensive framework on which to base their approach. In the COM-B system, C = Capability; O = Opportunity; and M = Motivation, all of which interact to generate behaviour (B).

However, influenced by the work of Piderit (2000), Wolski and Richardson modified the COM-B system by incorporating attitude as a key component in understanding researcher behaviour. Figure 1 offers a diagrammatic representation of this concept, now presented as A-COM-B. The single-headed and double-headed arrows represent the potential for influence between the various elements.

OPEN IN VIEWER

Figure 1.

A-COM-B framework for understanding behaviour.

Attitude is defined as ‘an individual’s evaluation or belief about something’ (Wolski and Richardson, 2015: 6); capability is the ‘the psychological or physical ability to enact the behaviour’ (Michie et al., 2011: 4). Motivation is defined as ‘all those brain processes that energize and direct behaviour, not just goals and conscious decision-making. It includes habitual processes, emotional responding, as well as analytical decision-making’ (Michie et al., 2011: 4). Opportunity is defined as ‘all the factors that lie outside the individual that make the behaviour possible or prompt it’ (Michie et al., 2011: 4). Behaviour is the result of the interaction between these four key elements.

In applying the framework to a situation in which one wishes to modify a current practice, the starting point is to (1) identify the underlying elemental behaviours that make up the practice and (2) identify which of these needs to be changed. Importantly the next step is to:

…identify current attitudes to the desired change in behaviour. However a challenge is that unlike behaviour, attitudes are more difficult to observe, measure and quantify. Therefore attention may need to be paid to employing techniques such as qualitative interviewing coupled with good listening skills. This is an important step as understanding the nature of attitudes will normally provide insights into the other elements of the framework, i.e. capability, motivation and opportunity. An understanding of all these elements creates a foundation for developing an intervention plan. (Wolski and Richardson, 2015: 8)

Applying the framework

The research team, comprising of three of the authors, selected a small but high-profile social sciences research centre to test the theories in Wolski and Richardson’s A-COM-B model. This centre was chosen because of the existing relationship which the authors had already built with its researchers and because this choice would align with Wolski and Richardson’s (2015: 8) theory that ‘to understand the current attitudes and to plan an ‘intervention’ plan, local service delivery teams may need to understand their local cohort to develop an effective response’.

To identify underlying attitudes and behaviour, the research project team designed a series of interview questions, categorised according to five broad headings:

How and where researchers store their data;

There are 23 questions in the survey, comprised of a mixture of multiple-choice and free-text formats.

The interviews were conversationally led by one research project team member, while a second team member noted responses to the questions in a Google form. Each interview was allocated 30 minutes. The interviews were also recorded and later transcribed.

It is estimated that the research project will take 12 months to complete. The initial phase of the project as described in this paper was carried out over a six-month period. It is expected that the next phase of the project will include devising and implementing intervention strategies, follow-up surveys and analysis, and the creation of a toolkit for service providers. It is anticipated that these activities will take a further six months.

Demographics of the research centre

The centre chosen for the study is a high-profile, interdisciplinary social science research group. It comprises researchers from several departments across the University, whose research focuses on human resource management, industrial relations and organisational behaviour. A recent benchmarking exercise, conducted by librarians in Library and Learning Services, placed the research centre among the top national and international research centres with a similar research focus.

For the purpose of the study, adjunct members and those on extended leave were excluded. The number of researchers invited to participate was 24.

As shown in Table 1, there is a fairly even distribution among early/mid-career and late career researchers.

OPEN IN VIEWER

Table 1.

Distribution of invited participants by career stage.

Early – mid career (Within 15 years of being awarded PhD)	Late career (15+ years since awarded PhD)	Total
10	14	24

Table 2 indicates that 20 (83%) of the participants have an academic rank of senior lecturer or above.

OPEN IN VIEWER

Table 2.

Distribution of invited participants by academic rank.

Research Assistant	Research Fellow	Lecturer	Senior Lecturer	Associate Professor	Professor	Total
1	1	2	6	6	8	24

At the time of writing this paper, 12 (50%) of the invited participants had been interviewed.

Types of data

Researchers were asked to describe their research areas and the types of data they collect. Most researchers reported collecting both qualitative and quantitative data, with only two respondents (16.7%) reporting that they collected qualitative data only. The most common method of data collection was audio recording interviews and transcribing to text-based format. Only two respondents (16.7%) used online surveys to collect data.

Managing data

Dropbox and hard drives were the most common ways of managing research data. Of the nine researchers (75% of total participants) who responded that they used Dropbox, five used the free Dropbox product and four used the paid Dropbox product.

All 12 of the researchers reported using more than one solution for managing their data, with most using Dropbox as well as their hard drive, Google Drive and a USB device. Only one participant (8.3%) used one of the University’s data storage services, i.e. Griffith Research Space (Figure 2). However, most researchers reported, in a separate section of the survey, that the size of their research data was less than 5GB, with only two (16.7%) reporting that it was more than 5GB.

OPEN IN VIEWER

Figure 2.

Solutions for managing research data.

Participants were asked to indicate the criteria which influenced them to choose the solutions identified in Figure 2 for managing their data. See Figure 3.

OPEN IN VIEWER

Figure 3.

Criteria for selecting data management solutions.

The most common reason given for their choice in data management options was ease of use. One researcher described Dropbox as ‘ridiculously easy’, and another cited difficulties using Google Drive that do not arise when using Dropbox.

Sharing data with collaborators was the most commonly reported problem with the way researchers manage their data (Figure 4). Some of the reported issues included complications arising from collaborators working between the free and the paid versions of Dropbox, and working with collaborators who used different systems. Size of data being shared was an issue for only one researcher (8.3%).

OPEN IN VIEWER

Figure 4.

Issues encountered with data management.

Participants were asked to identify all methods used for backing up their data (if applicable) (Figure 5). Using an external hard drive was the most popular way to back up research data, with 5 participants (41.6%) selecting this solution. Two researchers who considered Dropbox to be their data backup solution explained that they relied on Dropbox to do the backup automatically, and one researcher who used Google Drive ‘assumed that going to the inconvenience of using Google Drive [sic], it will be backed up’.

OPEN IN VIEWER

Figure 5.

Solutions for backing up data.

The one researcher (8.3%) who indicated that they did not back up their data cited ‘time constraints, laziness and naive trust in the systems along with optimism that all is good until the first time something goes wrong’ as the rationale.

Figure 6 shows that one-third of the participants (5) back up their data as frequently as on a daily basis. However, this result should be understood in the context that most of the researchers who reported backing up their data daily, were relying on it being backed up systematically by the storage products they used.

OPEN IN VIEWER

Figure 6.

Frequency of data backups.

Collaboration

Most researchers reported using Dropbox to share data with collaborators (see Figure 7). One researcher explained that it was the chief investigator’s responsibility to ensure that the data is stored and shared appropriately. Another researcher who spoke about sharing passwords and login information with collaborators to access confidential material said: ‘it’s get this done now, there isn’t time to work through the better way to do things’.

OPEN IN VIEWER

Figure 7.

Methods for sharing data with collaborators.

In response to a question as to who owns research data in collaborative research projects, the responses from researchers suggested that although they had not previously thought about data ownership, they had clear perceptions on where ownership should lie. Some of the responses included that it was ‘jointly owned’ for collaboratively collected data and ‘assume that it’s mine’. Others believed it was ‘wherever the grant sits’ or that the University owned it.

Data management plans

When asked if they had a data management plan, all of the researchers interviewed responded that they did not. One researcher referred to ‘an informal, unwritten one’ and another explained, ‘people just decide what to do as they go along’. Another researcher said it was ‘instinctive’.

Data sharing

When asked whether they share or intend to share data publicly, there was an overwhelmingly negative response from the researchers. Two researchers recounted hearing stories of data being put online, and others subsequently re-using the data to write papers. This practice was perceived as contentious. One researcher described the UK requirement to publish data from publicly funded research projects as ‘controversial’ and another researcher responded, ‘I would be horrified being forced to do that’. Others suggested that they could not see that their data was useful to anybody else. Only one researcher reported that they had thought about making their data open and that they did not perceive any problems in doing that.

An interesting finding, which corroborates surveys reported in the literature (Federer et al., 2015; Tenopir et al., 2011), was that a number of these same researchers reported re-using data from research projects or requesting data from researchers with whom they had worked previously.

Griffith research storage services

While most researchers answered that they were aware of Griffith’s research storage solution, only one researcher said they used it. The most common reason for not using the service was that they did not know how. When researchers were asked what would motivate them to use the storage solution, several referred to ease of use and accessibility as motivators. One researcher stated that they might start using the service ‘if it’s the right thing to do’ and added ‘it’s just another thing, it may be well and good but you have to learn it and it contributes to a sense of overload’. Another researcher commented: ‘if I had time to do it. I feel I am competent to do it, I just don’t have the time available’.

Managing data

From the interviews to date, it is apparent that researchers’ attitude is to choose the ‘easiest-to-use’ products for storage and moving data. Some researchers reported frustrations with using varied and differing systems and having to make choices regarding the use of them. As one respondent said: ‘I might use Google Drive because I know how to use it…and to go and use R drive and Research Storage Service and the Vault, I have to go and find out about it’. Another stated: ‘[there] isn’t time to work through the better way to do things’. This has translated into a behaviour of using their preferred solution, mainly Dropbox and a hard drive.

By using the A-COM-B model to design intervention strategies, the research project team could create intervention plans that will target researchers’ attitude towards managing data and address capability, motivation and opportunity factors.

If it can be shown that Griffith’s preferred data management solution is as easy to use as Dropbox, the research project team could target motivators and opportunities to trigger a change in behaviour. However, if Griffith’s preferred solution is not as easy to use as Dropbox, then rather than focus on motivation or capability, intervention strategies could target instead the researchers’ attitude that ‘easy-to-use’ is the best reason to choose a product.

Data backup

While only one researcher reported not backing up their data, most used an external hard drive and relied on automated backup utilities. While backing up data is good data management practice, using an external hard drive may expose the data to security risk through theft or damage.

The researchers’ responses demonstrate a casual and indifferent attitude towards data backup. While a few consciously and regularly backed up their data, many left it to the ‘system’ to do. As nothing had gone wrong in the past, they did not perceive any threat and therefore were not motivated to change their behaviour. One researcher, who regularly and deliberately backed up their data, explained that, although they had not suffered any loss of data, it was witnessing the consequences when close colleagues lost data that had made them act carefully with data backup. This is an example of a motivator that has triggered a behavioural change.

By applying the A-COM-B model to intervention strategies, the research project team could target the underlying attitudes surrounding data backup to reframe the task as being important enough to take considered, planned and careful steps to ensure data is backed up securely. Citing the example above might even be a useful intervention strategy, i.e. communicating the impact of actual cases of lost data. As the well-known axiom says, ‘Facts tell, stories sell’.

Collaboration

When asked what led them to choose their data management options, 25% of the researchers said that it was because a collaborator was using that option. Many explained that either whoever was in charge or the chief investigator on a research project normally decided on where and how data was stored and managed. The researchers’ attitude towards collaborative data management was that assumptions and unspoken understandings were sufficient and not necessarily in need of more accurate and specific definition.

Although many researchers believed that the chief investigator took the lead in data management, one senior researcher explained that they left it up to their research assistant to establish the data management systems of choice. They explained further that the reason Dropbox was used was because the research assistant had made that decision.

In targeting the attitude of researchers regarding collaborative data management, the research project team could design interventions to reframe researchers’ attitudes towards becoming more careful and considered when planning collaborative projects. Interventions targeted at senior levels might focus on establishing clear and uniform collaborative data management guidelines, whereas interventions targeted at more junior levels might focus on the selection of data management solutions.

Data management plans

Another finding was that many of the researchers considered that there was a direct correlation between the size of a dataset and the importance of managing that dataset. Their attitude was that their datasets were not large enough or important enough to warrant more conscientious planning. One researcher, who said they did not see themselves as a data manager, suggested that anyone who made the effort to manage their data was a ‘scientist with oodles of scientific data’. Another agreed that ‘we need to get it right’, but ‘it may not be seen as important’. With only relatively small datasets, the centre’s researchers may not have felt motivated to change their behaviour and conduct more robust data management planning.

When planning interventions, the research project team could target the researchers’ attitude that their data is unimportant or too small to manage effectively. One motivator could be ensuring data planning requirements from grant funders are addressed; providing data management plan templates and support could help to address issues around capability and opportunity.

Data sharing

The attitude of most of the interviewed researchers towards data sharing was that it was contentious, that their data would not be of interest to other researchers, and that sharing data might raise some methodological issues. As one researcher stated:

The data you collect is so project specific and so raises some general methodological questions you have to ask yourself about the merits or otherwise of using data collected for one purpose, or for one question, or one set of hypothesis, to address another one.

Although most of the researchers could not perceive the benefits of sharing their data with others, many had themselves benefited from re-using data from historical research projects.

An intervention strategy might leverage the attitude that re-using data from historical research projects or data previously collected by collaborators was acceptable practice. This could be reframed as data sharing can be beneficial. Motivators might include the citation advantage of data publication and the publication and re-use of data as a measure of research impact. Capability and opportunity might be enabled through support and education around platforms and systems used to publish and share data.

Griffith University’s research storage services

Although most of the researchers were aware of Griffith’s research storage solutions, only one said they had used it. The researcher who reported using it expressed frustration with the service: ‘I think I just found it difficult to use and I tried to get some assistance and I gave up…’

The attitude of the researchers’ towards Griffith’s research storage services reflects their attitude toward choosing data management solutions. Their current practices work effectively, they are easy to use, and as a result the researchers feel unmotivated to change.

Many researchers cited time constraints as a reason for not changing their data management behaviour. With increasing pressure on researchers to increase their research outputs, demonstrate impact and adhere to local and national policies and mandates, data management planning can be seen as an additional burden.

Interestingly, although Wolski and Richardson (2015) suggest that the mandated policies and guidelines will not guarantee improvement to data management practices, several researchers said that they would be motivated by ‘mandates’. One said ‘being told to do it’ would motivate them to change their data management behaviour and another said: ‘if it was the right thing to do’. One researcher, who reluctantly used the University’s Google Drive for research data storage, described attending a seminar where they were told they must use Google Drive and said that ‘put the fear of god into me about using Dropbox’. They continued, ‘Colleagues didn’t attend that seminar and they all use Dropbox because it’s easier and I agree with them, it’s much easier’.