Telling a story with metadata or Always drink upstream from the herd: What if your metadata isn’t properly represented in the stream?

1. Introduction

Discussions on open standards for scholarly communication primarily focus on text-based ‘traditional’ forms of research output, but what is not discussed are the implications for all the forms of research where outputs are multi-format, more focussed on form, or where the knowledge created is embodied in the activity of research than in a single retrospectively captured text. Through 2022, the Arts and Humanities Research Council (AHRC) funded Practice Research Voices (PR Voices) project sought to define how tools and standards could best be deployed to support this practice-based research, resulting in an infrastructure that makes this research accessible, and recognises its value and impact. We presented some of the project findings at the NISO Plus conference in February 2023.

The session: ‘Telling a story with metadata or Always drink upstream from the herd’ focussed on exploiting metadata to add context to how all these players work together, how could we leverage existing standards and initiatives to do so? How can we connect research outputs downstream? If we think about metadata early and often, we can tie research that follows to the original work and attribute appropriate credit to the author.

The first speaker in this session, Julie Zhu (IEEE), illustrated the complexities of the research/resource discovery landscape and the associated metadata ecosystem for text-based research outputs and products, using the helpful analogy of plumbers, pipes, leaks, and links to illustrate this [1]. Julie’s presentation highlighted several key points about how research is discovered in the existing academic publishing landscape in which the foundations are built on outputs and products. Explaining that discoverability landscape in terms of best practice tips for authors to improve discoverability; the wide-ranging number of standards and mechanisms for making this text-based research discoverable, the different tags needed for different discovery mechanisms and their use in linking; and the role of publishers and libraries in enabling discoverability via the metadata ecosystem.

In contrast, the Practice Research Voices (PR Voices) project work focussed on ‘non-traditional’ or ‘non-text’ outputs, primarily (but not only) created in arts and humanities disciplines, where the only formal option for any sort of ‘publication’ is via the landscape of institutional repositories. While institutional repositories enable access to text publications via open access, ‘non-traditional’ or ‘non-text’ research outputs remain under-recognized and disconnected from mainstream infrastructure. We have combined the work we shared in this NISO session with that of a follow-up session that delved deeper into the details we uncovered during our scoping work and related research on this topic carried out under the SPARKLE project [2], directed by researchers at the University of Leeds, also funded by the Arts and Humanities Research Council (AHRC).

2. Non-traditional outputs: Features and definitions

When thinking about a text-based ‘traditional’ research output – a journal article, conference proceeding, book chapter or book are likely to come to mind. These items tend to be made up of a single file, often a PDF, and are static (unless, for example, a correction is published). They reflect the end product of research that is retrospectively captured, and subsequently made available via repository, journal, publisher or pre-print service. More recently research ‘data’ has become a primary focus [3] and software is also gaining traction as a core output to underwrite the transparency and trust in research [4]. These concepts are grounded in a primarily science-focussed research recognition and rewards ecosystem that values the production of written outputs as the primary goal of research.

What then of those outputs that do not fall within this text-based ‘traditional’ outputs landscape? Depending on your geographical location or the purpose of the work, you may see them referred to as multi-component [5] or non-traditional research outputs [6]. These outputs are often perceived as being produced primarily by arts and humanities researchers, although they are produced across many disciplines [7].

The concept of contributorship for these outputs is much broader than authorship – with the primary role of creator joined by that of collaborator (e.g., curator, videographer, choreographer) and participant (either as an individual or group) being significant. The works are not static, they can be added to and change over time, and do not necessarily exist in isolation. There is also often the need to not only collect items together, but also to evidence how they relate to each other, as the way in which the work evolves is fundamental to communication about how the research progressed. They are often an attempt to represent performance or other ‘non-tangible’ events – so what is being captured is a remediation of an activity or output rather than the act itself. The research element can be embodied in the work or in the representation of the work, or it may need articulating using a research narrative.

Viewing this work from the lens of practice research moves the discussion on from simply focussing on the idea of non-text ‘non-traditional’ outputs as an add on to the existing text-based ‘traditional’ scholarly communications landscape. The ‘PRAG-UK’ Reports published in the UK in 2021 [7] built a foundation, identifying two clear definitions that can help inform this work:

(a) Practice research: “An umbrella term that describes all manners of research where practice is the significant method of research conveyed in a research output. This includes numerous discipline-specific formulations of practice research, which have distinct and unique balances of practice, research narrative and complementary methods within their projects.”

(b) Research narrative: “In a practice research output, a research narrative may be conjoined with, or embodied in, practice. A research narrative articulates the research enquiry that emerges in practice.”

3. Practice research within current infrastructure

The repositories landscape has reflected the traditional approach to sharing retrospectively captured research outputs, starting with the aim of making research Findable, Accessible, Interoperable and Re-usable (FAIR) [8], by broadening access to traditional text-based outputs, primarily versions of journal articles, and more recently datasets. The use of and respect for context hasn’t been well considered as this just hasn’t been a priority. The metadata templates in repositories that researchers need to engage with often reflect this – requesting ‘author’, ‘abstract’, ‘publisher’ and a single date. The advent of data repositories has seen the concept of ‘collection’ become common, but their definitions are based on an assumption that this is simply about grouping together individual files – and that the only changes to this collection will be the addition of files. This can be illustrated with prominent examples from both persistent identifier and metadata schema, including Dublin Core: “An aggregation of resources. A collection is described as a group; its parts may also be separately described.” [9] and DataCite: “An aggregation of resources, which may encompass collections of one resourceType as well as those of mixed types. A collection is described as a group; its parts may also be separately described.” [10].

It is this landscape, in which existing open standards treat practice research, in the same way that the scholarly communications landscape does – as ‘OTHER’ in which practice researchers have to fit. But every individual repository’s approach has been customised and retrofitted into a landscape that doesn’t quite work for practice research. So, in summary, we see practice research outputs that do not fit neatly into the mainstream scholarly landscape optimized for text publications. Institutional repositories often struggle to accommodate the complexity of multi-part, evolving practice research.

4. Practice Research Voices (PR Voices)

PR Voices aimed to scope a national practice research repository, using as a foundation the Cayuse (formerly Haplo) repository that had been developed in collaboration with Westminster to embed practice research from the beginning [11]. This repository would underpin recommendations on how to embed the requirements of practice research disciplines across the repository, discoverability, and interoperability landscape, rather than being an afterthought that is merely referred to as “OTHER”. We would bring together voices from many interested communities to form the PR Voices Choir and work with them to identify what FAIR and transparent research looks like for these disciplines, in a way that respects all contributors and their intellectual property. We used surveys, semi-structured interviews, a focus group, and a workshop to gather data.

The project recognised and benefitted from the existing partnership between Westminster, Jisc, the British Library, and Cayuse [12]. The creative and performing arts focus was expanded by bringing on board Kings College London, and inclusion of the Victoria and Albert Museum recognised that practice research is also carried out in galleries, libraries, archives, and museums (GLAM). The team that we put together recognised the wide range of stakeholders to this work, led by an experienced innovator (also a librarian and research manager), experienced practice researchers, persistent identifiers and metadata expertise, repository and research data management, repository software expertise, and project administration support.

The project focussed on three workstreams. The repositories workstream aimed to review the University of Westminster work, articulate the underpinning schema and map this to the British Library’s Shared Repository for cultural and heritage organisations [13] as while the primary use case was a national practice research repository – we strongly believed that all repositories should benefit from this work. The metadata and persistent identifiers workstream was key to improving existing open standards rather than creating new ones and was underpinned by the Cayuse schema. We prioritised three standards where we could see the most potential, with communities willing to engage with this work: DataCite, the Research Activity Identifier (RAiD) and the Contributor Roles Taxonomy (CRediT). The practice research community of practice workstream underpinned the project, to recognise what a community effort it is to capture, share and preserve this research. It brought together many different existing communities – researchers and practitioners, archivists, curators, funders, librarians, records managers, repository, research and research data managers and software developers. Figure 1 maps out the community of stakeholders to the PR Voices project.

OPEN IN VIEWER

Fig. 1.

The Practice Research Community of Practice and broader stakeholders [14].

The PR Voices final report [14] highlighted a range of findings and recommendations which need further validation in consultation with stakeholders from across the practice research community of practice.

Key challenges identified included:

Complexity of capturing iterative, multi-part outputs

This work in turn needs a national practice research infrastructure to be in place as a way to host developments, gather feedback and iterate changes, in co-design with community and which forms the basis for the open standards developments.

There is a need to agree on and articulate a standard approach for sharing practice research objects that better represents their complex nature, which will enable recognition by the wider scholarly landscape and their value in research evaluation mechanisms. This will also increase the level of discoverability and interoperability benefits to practice research, that those producing text-based outputs already have. Finally, improving the capture, preservation and discovery of practice outputs will underwrite an improved sharing culture.

There are several key sustainability challenges to this work, which include the complexity of capture, lack of capacity in infrastructure, and a need to consolidate existing good practice and develop a practice research specific programme which develops a network of experts, skills, training, expertise, and support material. This must include the embedding of digital preservation.

5.1. Repository

Mainstream repositories often struggle to move beyond a retrospective archive model oriented around individual text publications. In contrast, practice research requires an interactive, evolving platform reflecting process and context. That is, the practice research platform needs to become an interactive tool (scribe on the side) that reflects form AND function (beauty), while recognising contributors as well as the processes and products. This needs to sit neatly alongside the traditional aims of enabling discoverability, citation, and preservation of practice research.

Our report documents a first attempt at requirements for a practice research platform, including personas reflecting the wide range of specialists involved in sharing practice research and an associated user story gives a brief overview of what each persona might want from the platform. A mock-up of recommended changes to the WestminsterResearch implementation of the Cayuse repository is informed by the project findings and provides a foundation for moving forwards.

We investigated the suggestion recommended by the PRAG-UK 2 Report: “How can practice research be shared of a project item type – also known as a collection or portfolio.” [15].

We have identified the need to capture individual objects on an ongoing basis, and bring those together in a collection. We propose the ‘portfolio’ concept goes beyond the existing use of ‘collection’ in the landscape and which aggregates objects and overlays narrative and context. Figure 2 represents a first attempt at articulating the core schema attributes for practice research outputs in the exemplar repository.

OPEN IN VIEWER

Fig. 2.

Examining core schema attributes for practice research outputs in the exemplar repository [14].

We acknowledge that further work is needed to articulate the schema, identifying what is currently missing, and what sits in an enhanced template text for display rather than the structured schema. More importantly, the needs of the community should be represented in a schema that delivers a solid, agreed fundamental core set of attributes for outputs that can be widely implemented and deployed in infrastructure to ensure that the capture and dissemination of FAIR research becomes the norm across all technology mediation serving all disciplines.

“I think something about that, that iterative process and how the portfolio kind of almost sits alongside you as you’re undertaking your research, perhaps it’s not, necessarily a retrospective activity and there’s something perhaps about the way in which you develop it, that’s helpful in terms of understanding the insights that you’ve gained.” [16]

A validation exercise carried out against the British Library’s Shared Research Repository for cultural and heritage organisations confirmed that their repository uses ‘collection’ as a record type, but that this is different to the concept of a portfolio. This has been identified as a unique strength of the Cayuse repository, also evidenced by discussions in the focus group activity, as a distinction which merited wider adoption. The mismatches between the two were a mixture of platform and culture. It was agreed that the Cayuse schema work was both appropriate and met existing needs.

5.3. Community of practice

During the project’s work with the community, it became very clear that ongoing engagement and an agile approach to co-design in partnership with the practice research community must underpin any future work. They face a number of cultural, technological, and capacity barriers in presenting their practice research, with the PR Voices researcher and practitioner survey highlighting issues such as “Using institutional repositories that are designed for text-only outputs [20]” and “Institutional repository has too narrow a scope in terms of file type uploading and hosting. Particularly with interactive digital media.” [20]

The survey also showed thirteen of sixty respondents were unaware of what an ORCID is, with a further seven that are unsure how to use it; and twenty-five of fifty-five respondents not sure whether their materials have a persistent identifier such as a DOI associated. [20] This demonstrates a lack of awareness of open standards for research communication, which itself contributes to the lack of engagement with and interest in those infrastructures. Success of any future provision is rooted in maintaining a deep level of buy-in and commitment from practice researchers.

This aligns with the approach Westminster took to building the Cayuse repository (Fig. 3) and includes the need for a permissive space to enable this.

OPEN IN VIEWER

Fig. 3.

An agile approach to co-design of the Cayuse (formerly Haplo) repository at the University of Westminster [19].

The membership of this community of practice includes all of those with a stake in capturing, making discoverable and, preserving practice research – meeting the expectation of recognising all contributors to this work. The practice research community of practice is made up of diverse disciplines, specialists, and existing communities of practice, all with their own cultures and a range of motivations.

We have spotlighted practice research as a scholarly approach with lessons for all of academia. Practice research elucidates key weaknesses in the mainstream scholarly communication ecosystem related to representing diverse outputs, contributors, and forms of impact. Findings from PR Voices demonstrate that enhancing discovery and reuse of practice research will require shifts in culture, technology, and community participation.

Equitable integration requires embracing novel mindsets and structures. Challenges remain, but a route ahead exists through open standards, metadata, and co-design with communities. Although grounded in practice research disciplines, benefits will extend across research domains through flexible, connected infrastructure.

Crucially, we must recognize practice research’s distinct methods, contributors, and outputs at an international level. Transparent access (espoused by the FAIR Principles) underpinning data sharing meanwhile respects all contexts [21], is often more appropriate for this research and we must move beyond imposing text-based open access models on these communities. Open standards must actively support varied dissemination pathways, not just tolerate difference.

Our vision is an inclusive ecosystem truly valuing all research and contributors:

Continued co-design with community is key to developing systems that also optimize impact. Implementation must centre inclusive co-design and recognizing multifaceted value beyond text publications. We sound a clarion call to creators, repositories, publishers, funders, and identifiers to collaborate on technical and social infrastructure comprehensively representing practice research, enabling discovery, access, and re-use according to needs. The path presents challenges but is navigable. Our diverse ecosystem can become more welcoming of non-traditional outputs, transparent in controlled sharing where FAIR ≠ Open, and celebratory of varied contributions. Let us boldly take these next steps together.