Biological variation data: An important old topic with new standards and new look resources

Why should you read this editorial?

The new revision of the ISO 15189: 2022 standard clearly identifies that it is a patient-centred standard. ¹ Compliance with the new standard should ensure that a risk-based approach is adopted by service providers to address clinical risk and the impact of services upon patients. Consideration of patient care and safety are to be at the core of laboratory processes and fundamentals to be addressed in the design and delivery of a laboratories quality management systems. To meet the standard, it follows that analytical methods adopted and deployed by services must clearly be assessed and documented within a process that is focussed on risks to patients and clinical decision making. This demands an understanding of the analytical performance specifications (APS) for measurands required to enable clinical decisions that lead to desired outcomes, and validation of methods to ensure those deployed meet such specifications. Additionally, results/ reports of requested examinations need to be fit for purpose and interpretable for the population served. Consequently, risk assessment and management includes and requires:

• Identification and adoption of relevant APS.

Biological variation data (BVD), describing within-subject (CV_I) and between-subject (CV_G) variability, have established and important roles in these contexts. ² The Milan Hierarchy for Analytical Performance Specifications established during the 1st Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine, (EFLM) held in Milan in 2014, in particular highlighted the importance of these data sets to laboratories, and raised issues regarding the quality of BVD data available for this purpose at that time.^3,4 The three models for the identification of APS identified in the hierarchy those are based on:

1. The effect of test performance on clinical outcomes (preferred option, but few studies are available).

In the absence of clinical outcomes data, either direct or indirect, biological variation assumes a primary role in defining and documenting laboratory service quality for some, but not all measurands after considering all the information. ⁵

The BVD used for definition of APS and in other applications that impact upon patient care represent a class of reference data.^6,7 This delivers a driver for those identifying, reporting and publishing BVD to do so in a way that enables users to understand the provenance and quality of the data they are about to apply clinically. From the BVD user perspective, the need to enable safe, accurate and effective clinical applications of BVD delivers a driver for them to understand the concepts of biological variation, and to also understand the value and limitations of applications of BVD. This knowledge, accompanied by access to well-characterised quality-assessed BVD, can therefore be viewed as essential prerequisites for laboratory medicine specialist having to demonstrate compliance with ISO 15189:2022.

So, where we are today, is that a need for standards has emerged to assure and ensure the veracity of published BVD to be used as reference data in applications that have a role in defining and monitoring the quality of services. Absence of standards and guidance for the production and reporting of BVD will affect clinical decision making (a priori) and deliver a risk to patient care that needs to be managed. Logically, application of poor quality BVD, or erroneous application of good quality data in the wrong setting, will compromise the validity of decisions around method choice, performance and results interpretation and thereby impact upon the clinical effectiveness of an examination. Transportability of reference BVD for applications across a diversity of populations to enable those decisions requires that they are accessible and reported as reference data. There are parallels here with the ideas, concepts and practicalities of delivering reverence values as described by the IFCC. ⁸

Delivering standards and addressing quality

Currently, there are no internationally recognised standards for delivery and reporting of BVD. This is anomalous given their importance and their use globally. However, in the absence of a formally agreed standard for BVD delivery, the prospective experimental approach described by Fraser and Harris has emerged as a de facto standard. ⁹ This has been applied with varying degrees of rigour to the design and delivery of most BVD studies published to date. We therefore have 35 years of historical data delivered using this approach. The Task Group for the Biological Variation Database (TGBVD) and Biological Variation Woking Group (BVWG) of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) have studied this pool of publications and considered the many issues that impact upon the quality of the reports and data outputs of these studies. This has enabled identification of a standard for reporting that can be used to critically appraise historical studies and drive up the quality of reports of future studies. ¹⁰ The proposed Standard for Reporting Biological Variation Studies (STARBIV) can be found in the form of an interactive mind map hosted on the EFLM Biological Variation Database (E-BVD) website. ¹¹

STARBIV is one of many outputs from the EFLM developed to enable delivery of resources to support use and application biological variation data. It has emerged from the work leading up to, and including, the delivery of the E-BVD, (https://www.biologicalvariation.eu). Much of this work was undertaken in response to issues around the quality of BVD available for setting of APS raised at the EFLM strategic conference in 2014.^6,12 At that time, the ‘go to’ source for BVD, and BVD derived APS, was the database currently found on the ‘Westgard’ website. ¹³ The BVD content of that site was collated by the Spanish Society of Clinical Chemistry and Molecular Pathology (SEQC) and was last updated in 2014 using the criteria described by Perich and Ricos et.al. ¹⁴ Many of the SEQC team involved in the delivery of that data base subsequently became members of the EFLM Task Group set up in response to the concerns raised about data quality. The E-BVD has developed to the extent that it should now be seen as the ‘go to’ site for BVD of today. ¹¹ It is populated with BVD from many of the prospective experimental studies listed in the SEQC 2014 database, but this follows critical reappraisal of the studies producing those data by the EFLM groups using criteria and knowledge now embedded within the STARBIV mind map. Early in the E-BVD development pathway a case was made by the BVWG that BVD are reference data, and that they should be published in the database accompanied by a defined minimum data set (MDS) extracted from study reports to enable their transportability. ⁶ It is an approach now adopted to enable valid transport of BVD as reference data across time, geography and clinical practices. ¹² This means that those producing, collating and reporting BVD need to be aware of what constitutes a well-executed study and ensure that old and new reports of those studies have consistently delivered BVD accompanied by the MDS to support valid clinical applications.

The STARBIV opinion paper delivers a more in depth discussion of the rationale behind the standard and identifies the STARBIV mind map as a tool that is multifunctional. ¹⁰ While primarily developed to enables users to assess compliance of BVD study reports to the standard, it additionally provides a structured aide memoire for those designing new studies and serves as an educational resource to those with a general interest in the topic. The latter goes someway to addressing the knowledge requirement alluded to earlier. This is enabled via access to embedded presentations, notes and links to key full text references.

STARBIV compliant study reports will be of a quality that enables users to understand the all-important provenance of the data set and enable BVD transportability. Compliance involves an objective assessment of reports aided by the embedding of the Biological Variation Data Critical Appraisal Checklist (BIVAC) into STARBIV. ¹⁵ This delivers a comparative indicator of BVD quality. BIVAC requires assessment of 14 quality items applied to the study reports generating BVD; an overall grade for each study is generated on a scale of ‘A’ to ‘D’; where ‘A’ indicates full BIVAC compliance and a ‘D’ grade non-compliance to the extent that the studies are considered egregiously flawed and reporting BVD that may be unsafe for use in clinical applications. The headline grade reported is that of the lowest appearing in the profile. For example, a study with thirteen of 14 QIs graded ‘A’, and 1 graded ‘C’ will have an overall grade of ‘C’. The QI profile is recorded and viewable for each measurand listed in the E-BVD, as is the required MDS gleaned from reports. Together these are used to further identify studies suitable for inclusion in a metanalysis of BVD.^7,15 This process enables reporting in the database of point estimates of the biological variation of a measurand with improved confidence intervals. Online tools are under development to enable users to assess and record compliance with a STARBIV high level checklist and generate BIVAC grades. ¹⁶

Compliance with STARBIV will therefore enable convergence of new studies reporting BVD upon a standard to address quality and confidence issues around the data and enable their use and application as reference data. If widely adopted by the laboratory community and editorial boards of scientific journals the impact will be to drive up the quality of a BVD reference data needed by laboratories to address the issues bullet pointed at the beginning of this article. This is not unlike the process associated the Standards for Reporting of Diagnostic Accuracy (STARD) that has been shown to improve the quality of reporting in that context.^17,18

What else is new?

STARBIV is one of several initiatives that have been delivered by the EFLM to address current issues with delivery and application BVD. ¹⁹ The E-BVD now lists over 3000 biological variation estimates extracted from close to 600 publications. All entries are accompanied by a relevant MDS, BIVAC score breakdown. ¹¹ Additionally the database website further enables calculation and delivery of APS for assay imprecision, bias, maximum allowable uncertainty and total allowable error. The approaches adopted for defining these APS have been recently reviewed by Sandberg et al. ²⁰ The important issues around the validity of the widely used total error formula, as raised by Oosterhuis, are discussed in that article. ²¹

To date the various EFLM groups working in this area have collectively delivered over 40 published peer reviewed papers, that include reports of new studies of BVD that they have undertaken, critical reviews of biological variation of groups of measurands and general topics relating to biological variation. A bibliography and links to many of the full text versions are available on the EFLM website and video lectures are available via the EFLM Academy Syllabus Course.^22,23

The exemplar BIVAC grade ‘A’ European Biological Variation Study (EuBIVAS) is one of the other major outputs from the EFLM group. This study of biological variation over a period of 10 weeks was of 91 healthy volunteers from six European countries and has added a significant amount of new data to the biological variation database. ²⁴

A recent review by Sandberg et al. describes several of the current developments and identifies future challenges around biological variation. New and emerging developments include the use of data mining and big data approaches to identification of BVD.^19,25 The adoption of such approaches will see updates of the STARBIV standards and mind map in due course.

BVD: Reference intervals and significance of change

The use of BVD for the assessment of the validity of population-based reference intervals and reporting significance of change in serial results of examinations are important applications. These can be aided using a measurands index of individuality (II; the ratio of CV_I to CV_G) and reference change values (RCV), respectively. Both are important concepts in laboratory medicine. ² If the II is less than 0.6 it indicates that use of population-based reference intervals for diagnosis and monitoring of disease may be of little value for the individual subject. In such cases, a subject could have a test result that is highly unusual for them as an individual classified erroneously as ‘normal’ as it falls inside the wider population reference interval described by the reference limits. This is unlikely to happen if the ratio is greater than 1.4 and remains a possibility if the ratio is between 0.6 and 1.4. Where II is low then RCV is helpful in identifying whether the change is unusual; in that if the RCV is exceeded then it represents a change that is greater than to be expected based on the combined biological and analytical variation at a chosen level of probability. An RCV calculator has been included in the E-BVD. It enables users to calculate RCV at a chosen level of probability following input of BVD provided on the website and users local estimates of analytical imprecision. The calculator employs the asymmetric method described by Fokkema et al. ²⁶ This provides different values for upward and downward change for the individual. RCV is a tool that enables identification of a need to further investigate an observation that may be of clinical importance for the individual that may otherwise be classified as “normal”. There is also some degree of complexity with the use of RCV. Conceptually, it is a comparatively simple tools to understand and apply. The devil is however in the detail. A recent paper by Jones et. al. identifies issues around the potential effect of regression of values towards the mean upon RCV which may impact on the interpretation and application. ²⁷ Given that the higher proportion of workload in clinical labs is related to monitoring of the disease of a subject (you only diagnose a disease once), and that the majority of measurands have an II of less than 1.4, it is significance of change reporting that is surely of potentially more value to users in following the course of a patient’s disease?

An alternative to the use of RCV is the use of personalised reference intervals. This is a developing area of study. Like RCV, this approach has a dependency upon availability of well-characterised, high quality BVD.^28,29

Conclusion

In conclusion, it is the view of the author that BVD are not perceived by many laboratory professionals as being reference data requiring critical appraisal prior to application. As indicated earlier in this article, BVD applications are of value in service definition (APS), results interpretation (reference intervals and RCV) with many other applications. These are important areas of focus for laboratory services providers given the need address issues of risk management and patient centredness that is required for compliance with ISO15189:2022. The EFLM biological variation groups have responded to the extant and emerging needs of the international laboratory community to deliver and support the use of quality-assessed BVD in a format that enables transportability and application as reference data. They have developed tools, resources (BIVAC and E-BVD) and standards (STARBIV) to enable the study and reporting of BVD as reference data that should be a necessary quality-related focus of laboratory medicine specialists moving forward.