Interscenario Reliability of the Breast Inflammatory Symptoms Severity Index in Mothers with Inflammatory Conditions of the Lactating Breast

Abstract

Introduction:

A reliable method of measuring the severity of inflammatory conditions of the lactating breast (ICLB) enables health care professionals to provide individual care for mothers. There is a need to triage and prioritize mothers with severe ICLB symptoms reliably. The aim of this study was to determine the inter-scenario reliability of the Breast Inflammatory Symptoms Severity Index when completed electronically by the mother, verbally administered over the phone by administration staff and when completed face-to-face by a health care professional before treatment.

Methods:

An inter-scenario, repeated measures reliability study of the Index, which is an 80-point clinician-administered patient-reported outcome measure used to assess and monitor mothers with symptoms of inflammatory conditions of the lactating breast. The study was conducted with mothers across 3 scenarios with symptoms of inflammatory conditions of the lactating breast in Australia. Standard Error of Measurement (SEM), Measurement Error (ME) and Intraclass Correlation Coefficients (ICCs) were calculated to determine interscenario reliability of the Index.

Results:

Data were collected for 20 mothers. Statistical analysis identified that the Index had good reliability among the three scenarios, evidenced by low measurement errors and high correlation. When comparing administration mode, phone method was more reliable than electronic administration.

Conclusions:

Interscenario reliability was good, aiding the use of the Index in the community, research and clinical practice. This tool provides lactating mothers who experience lactating breast conditions and their clinicians with surety that they are using a reliable outcome measure for assessment when taken electronically or via phone.

Keywords

inflammatory conditions of the lactating breast (ICLB)breastfeeding postnatal care psychometrics assessment breast inflammatory symptoms severity index (BISSI)

Introduction

Inflammatory conditions of the lactating breast (ICLB) is an umbrella term that encompasses a spectrum of conditions including breast engorgement, blocked ducts, mastitis, and breast abscess.^1–3 Mothers with ICLB generally present with a range of local symptoms including breast pain, swelling, palpable lumps, erythema, and systemic symptoms such as fevers and malaise.^4–7 In Australia, 20–25% of lactating mothers may experience symptoms of ICLB, most commonly within the first 6 months postpartum.^5,7 If ICLB are inappropriately managed, this may result in premature cessation of breastfeeding,^4,7 which can have negative implications on the health outcomes of both mother and child.^8,9 The literature reports numerous short and long-term benefits of breastfeeding, some of which are reduced maternal risk of ovarian and breast cancer,^8–10 reduced infant risk of developing asthma in childhood,¹¹ and obesity¹² and diabetes^9,13 in adulthood. Timely assessment and treatment of mothers with ICLB is therefore crucial to ensure that breastfeeding successfully continues.

Patient-reported outcome measures (PROM) are commonly used in health care and aim to characterize and monitor a patient’s symptoms, response to treatment, and ultimately guide health care professionals’ decision-making.^1,14 There is no reliable outcome measure available for use in the assessment of ICLB. A reliable way of assessing and measuring the severity of signs and symptoms is important to enable health care professionals to provide evidence-based support and appropriate treatment plans for mothers with ICLB.

The Breast Inflammatory Symptoms Severity Index (BISSI) is a PROM typically administered face-to-face by a health care professional to assess and monitor mothers with symptoms of ICLB.^1–3 The 11-point numerical rating scale (0–10) was first proposed by Cooper et al.¹ and assesses eight domains: pain awareness, pain touch, sickness, hardness/tightness, temperature, redness, size, and impact of affected area. This gives a total possible score out of 80, with a higher BISSI score reflecting higher symptom severity.³ A clinician script supports the clinical use of the BISSI, ensuring efficiency and consistency.¹ At present, the BISSI is the only ICLB PROM to have undergone psychometric testing. Content and face validity of the BISSI were successfully achieved using a three-round Delphi technique,¹ while construct validity and internal consistency were determined in a retrospective audit on 160 patient records of mothers treated for ICLB.³

Determining if the BISSI can be reliably administered in different scenarios could have numerous benefits. In clinical practice, scheduling and appointment prioritization are often conducted by administration staff (receptionists, managers, or administrators), who may be the first point of contact for mothers. Automated messaging systems, becoming more prevalent globally, are also used as an alternate form of communication between patients, administrators and health care professionals.^15,16 In cases of ICLB, the utility of the BISSI across different scenarios may expand its use beyond the health care team, potentially enabling use by administrators to communicate with health professionals via established practice systems. An increased ability to reliably assess ICLB symptom severity may enable improvements in patient triage, symptom monitoring and monitoring patient responses to treatments.

Broadening the scenarios in which the BISSI is completed will help to enhance its robustness, thereby providing a means for health care professionals to assess, advise, and monitor mothers with ICLB who reside in regional and remote areas, or mothers who are unable to access clinical services in-person. Successful application of the BISSI in multiple scenarios may also positively impact future research of ICLB, expanding possibilities of how data is collected, when it is collected and where it can be collected from. This study, therefore, represents a step forward in the process of the BISSI becoming a reliable outcome measure to improve the assessment and care of mothers with ICLB.

Study aims

The aim of this study was to determine the interscenario reliability of the BISSI when administered electronically by the mother, verbally over the phone by administration staff and when administered face-to-face by a health care professional, prior to treatment.

Materials and Methods

Study design

An interscenario, repeated measures reliability study was conducted at a private physiotherapy clinic in Melbourne, Australia. The Australian healthcare system allows for mothers with ICLB to be treated privately for cost or publicly, usually at the choice of the mother. Women who have ICLB who have chosen to be seen privately can then choose which health care professional (including physiotherapists) they see. Prior to study commencement, three lactating mothers were consulted to seek feedback on the feasibility of the study and pilot the electronic BISSI. The Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network Guidelines for Reporting Reliability and Agreement Studies¹⁷ have been used to structure this article.

Participants and recruitment

A total of 20 mothers were deemed eligible by administration staff at the physiotherapy clinic, using predetermined criteria with a sequential method of sampling utilized. A recruitment script was used for consistency and to determine participant eligibility. Mothers were eligible if they were seeking or receiving physiotherapy treatment at the clinic for an ICLB, agreed to complete the BISSI in three different scenarios and were aged ≥18 years. Mothers were excluded if they were unable to understand English. If interest was shown at the time of booking, administration staff emailed the participant information sheet and a link to the consent form. Mothers who agreed to participate provided electronic written informed consent via Research Electronic Data Capture (REDCap) software.¹⁸ Mothers were free to withdraw their consent at any time without justification prior to completion of the data collection period. The mothers gave informed consent freely and were not directly approached by the research team to give consent on-the-spot.

The sole treating health care professional (N.K.) was an Australian Health Practitioner Regulation Agency registered physiotherapist who has completed the Australian Lactation for Health Professionals Course¹⁹ and has 3 years’ experience in treating mothers with ICLB. This health care professional administered the face-to-face BISSI.

Data collection

After consenting to participate, mothers were randomly allocated into one of two scenario orders: completion of electronic or phone BISSI first. The face-to-face BISSI occurred last in the scenario order ensuring all BISSIs were completed immediately prior to physiotherapy treatment. This is representative of how the BISSI would be used in clinical practice and ensured that treatment did not occur between the measurements, potentially affecting the BISSI scenario symptom scores.

The mother received an automated text message 2-hours prior to the appointment via the practice management system, Cliniko.²⁰ It was anticipated that medication and milk removal may impact directly on the BISSI scenario symptom scores. To account and control for this confounding factor, mothers who opted to medicate were asked to do so 2 hours before their appointment to provide a period of consistency during the time all three BISSIs were repeated. This allows for the medication absorption time of up to 1 hour prior to the completion of the first BISSI and a stable period of 2 hours for the remaining BISSI scores to be completed.^21–25 Mothers were also asked to breastfeed or express, as they normally would, from their affected breast 90-minutes prior to their appointment to reduce the need to feed the baby between scenarios. If baby required a feed between scenarios, mothers noted their time of feed.

Mothers were also encouraged to refrain from partaking in any other treatment(s) until all BISSI scores were submitted to reduce the impact of factors that would potentially change the BISSI score. One hour prior to the appointment, the mother completed either an electronic BISSI or received a phone call from the administration staff to complete the BISSI over the phone. The aim was to collect all three BISSI scores within the 90-minutes prior to physiotherapy treatment. Raters were blinded to prior scores.

Three administration staff undertook a BISSI training and familiarization session with the treating physiotherapist prior to commencing data collection. Administration staff read the clinician script to ensure consistent and accurate administration and recorded the mother’s answers on a hard copy version of the BISSI. These scores were passed to the treating physiotherapist to record in REDCap following the mother’s appointment.

On arrival to the clinic carpark the mother completed her second BISSI (electronic or phone). To decrease the potential of recalling prior scores, mothers completed a distracting task between each BISSI,^26,27 namely the transfer of self from home to carpark, and then from carpark to clinic.

At the commencement of the physiotherapy appointment, mothers were asked the method and time of last milk removal, and any medication including type and time of ingestion. Participant demographic characteristics including maternal and infant age, number of live births, number of prior ICLB episodes and affected breast were recorded and entered into REDCap. Residential postcodes were collected to determine the mother’s socioeconomic status according to the Australian Bureau of Statistics Socio-Economic Indexes.²⁸ Standard physiotherapy assessment and treatment ensued.

Data analysis

Data analysis was performed using SPSS, Version 29.²⁹ A p-value of 0.05 was used to denote statistical significance. Data were examined for normality by visual analysis of histograms, Q/Q plots and examining kurtosis and skewness z-scores to determine if they were within + or −1.96.³⁰ Where continuous data were not normally distributed, Median (quartile one, quartile three) were used to indicate a measure of central tendency and variance. To determine interscenario reliability the Standard Error of Measurement (SEM), Measurement Error (ME) (1.96 x √2 x SEM) and two-way mixed model intra-class correlations (ICCs) with confidence intervals were calculated.³¹ The SEM was calculated as the √MSE where MSE is the Mean Square Error from the between-subjects effects drawn from a repeated measures analysis of variance (RANOVA).³¹ Differences in BISSI scores between scenarios were examined using a RANOVA with means comparisons. These calculations were completed as a total comparison and individually comparing electronic and phone to face-to-face BISSI.

The confounding factors of time since medication and time since milk removal were examined using a Spearman’s correlation and scatterplot analysis to assess for potential impacts on the face-to-face BISSI scores. A Spearman’s test was used, as the time since medication was not normally distributed. An order effect was examined using an independent samples t-test with means comparisons in the electronic first group compared with the phone first group in the phone BISSI. For the electronic BISSI, as the data were not normally distributed, an independent samples Mann–Whitney U test was used to examine for an order effect.

An a-priori power analysis based on calculations by Stratford and Goldsmith³² using a maximum clinically acceptable SEM of 10 points, demonstrated that 20 mothers provided 80% power to detect up to an eight-point SEM.³³

Results

A total of 24 mothers were eligible, and 2 mothers withdrew prior to data collection as their symptoms resolved. Another two mothers were not contactable, resulting in 20 mothers entering the study (Fig. 1). The demographic characteristics of the 20 participating mothers are presented in Table 1. Mothers were aged between 29 and 40 years and their infants between 3 and 332 days old, with ICLB symptom onset a median (quartile one, quartile three), 2 (1, 3.75) days prior to enrolment into the study. Most mothers were of a high socioeconomic status, residing in more advantaged areas, as the most common (minimum, maximum) postcode decile was 10 (4, 10).

FIG. 1.

Recruitment flow chart.

Table 1.

Participant Demographic Characteristics of 20 Mothers with ICLB

Mother’s demographic characteristics	Value
Maternal age, mean (SD), years	33.5 (2.8)
Number of live births n (%)
1	8 (40)
2	9 (45)
3	3 (15)
Baby’s age, median (quartile one, quartile three), days	80 (18, 164)
No. of prior ICLB episodes this lactation, n (%)
0	10 (50)
1	6 (30)
≥2	4 (20)
Method of milk removal, n (%)
Exclusive breastfeeding	7 (35)
Exclusive pumping	2 (10)
Mixed—expressing and breastfeeding	11 (55)
Affected breast, n (%)
Right	11 (55)
Left	6 (30)
Bilateral	3 (15)
Taking antibiotics, n (%)
Yes	6 (30)
No	14 (70)
Taking pain or NSAID medications, n (%)
No	8 (40)
NSAID only	6 (30)
NSAID and paracetamol	6 (30)

ICLB, inflammatory conditions of the lactating breast; NSAID, nonsteroidal anti-inflammatory drug.

There was a reasonable spread of BISSI scores with the minimum and maximum scores for phone, electronic, and face-to-face equal to 15–54, 16–47, and 12–48 points, respectively. There was a normal distribution of phone, electronic, and face-to-face BISSI scores.

Reliability analysis

The mean total BISSI score for each scenario, SEM, ME, ICC, and RANOVA p-value are presented in Table 2. Overall, all SEM and ME scores were of low value considering the total BISSI score is out of 80, and ICC scores approached 1, indicative of good to excellent reliability.

Table 2.

Comparison of 3 Different Scenarios for BISSI Reliability in 20 Mothers with ICLB

Electronic BISSI mean (SD) points	Phone BISSI mean (SD) points	F2F BISSI mean (SD) points		SEM points	ME points	ICC (95% CI)	Repeated measures ANOVA (p value)
33.5 (8.8)	33.6 (10.2)	32.2 (9.9)	Total	2.5	6.9	0.932 (0.863–0.970)	0.137
			BISSI Electronic vs. F2F	2.7	7.5	0.911 (0.789–0.964)	0.129
			BISSI Phone vs F2F	2.0	5.5	0.952 (0.868–0.981)	0.042

ANOVA, analysis of variance; BISSI, breast inflammatory symptom severity index (scored /80); F2F, face-to-face; ICC, interclass correlation; ME, measurement error; SEM, standard error of measurement.

Confounding factors

The time in minutes from recording a mother’s first BISSI score to completing the face-to-face BISSI ranged from 12 to 127 minutes. Time between when nonsteroidal anti-inflammatory drugs were taken and the face-to-face BISSI score was a minimum of 2.3 hours. Time between when paracetamol was taken and the last BISSI score was a minimum of 2.1 hours. Mothers’ last feed time prior to face-to-face BISSI ranged between 21 and 295 minutes. Time since medication was taken or time since milk removal was not correlated with the face-to-face BISSI scores. Examination of scatterplots between the time since medication was taken or time since milk removal with the face-to-face BISSI scores revealed no non-parametric or parametric relationship. There was no order effect found in either the phone BISSI or the electronic BISSI, according to whether the phone score or the electronic score was taken first or second.

Discussion

Our findings suggest that the BISSI can be considered a reliable PROM to use for mothers with ICLB symptoms with high utility to assess symptom severity over the phone or electronically, compared with when clinically administered face-to-face. These findings were obtained despite the normal biological variability in lactation and the dynamic nature of ICLB.⁷ Thus, the results support the BISSI having good to excellent reliability and imply that the BISSI is appropriate to use in the multiple scenarios tested.

Considering the BISSI is comprised of eight domains, each scoring between 0 and 10, with a maximum possible score of 80, an SEM of 2.5 points means that we could expect a mother’s reported BISSI score to lie within 2.5 points in either direction when taken in different scenarios. The measurement error implies that if there was a 6.9 point (or more) difference between a mother’s BISSI scores between scenarios, we can be 95% sure this reflects a real difference.³¹ With a difference that lies between the SEM and ME (2.5 and 6.9 points), we can be less sure (between 68% and 95%) that we have measured a real difference.³¹ Further research is still needed to determine what change in BISSI score would demonstrate a clinically meaningful change in ICLB symptoms.

A strength of this study was that mothers were randomly allocated to one of two scenario orders (electronic or phone BISSI first) to reduce systematic bias. The face-to-face BISSI had to occur last in both scenarios as this coincided with physiotherapy treatment, which had the potential to affect BISSI scores. This is also a true representation of how the interscenario BISSI would be used in clinical practice, that is, prior to and directly after treatment. Also of note, the BISSI was able to be used by multiple (three) administration staff at the practice, due to the accompanying clinician script, which was designed to ensure novice users could still accurately utilize the outcome measure. This supports the high utility of the BISSI, demonstrating it can be administered reliably by non-health care professionals.

In establishing interscenario reliability of the BISSI in this study, along with the prior establishment of content and face validity,¹ construct validity and internal consistency,³ further contribute to the robustness of the BISSI as an ICLB-specific outcome measure. The BISSI across the multiple scenarios was quick to complete, and there were no technical malfunctions with the electronic version. Timely administration of the BISSI by both health care professionals and administration staff could enable improved management of ICLB. Administration staff can assist with appointment prioritization, thereby allowing increased time for the health care professionals to provide treatment. The BISSI could also be employed in the wider community, for mothers who reside in regional and remote areas, or who are unable to access clinical services in-person. In terms of future research, the BISSI could be reliably used as an outcome measure to compare treatment effectiveness in interventional studies on ICLB. Further establishment of the tool’s reliability would be to explore the intra and inter-rater reliability.

Prior to commencing the study, we consulted lactating mothers to seek feedback on the feasibility of the study’s procedure and pilot the electronic BISSI. One concern raised was that mothers early in their postpartum journey may have more difficulty completing the BISSI scores given their baby may need more frequent feeds and attention. All mothers completed all three BISSI scores, even with babies as young as three weeks, which suggests this tool is quick and convenient even for unwell mothers with young babies. It was noted that at least five mothers reported breastfeeding between completing all three BISSI scores. The need for these mothers to feed between scenarios mimics real-world clinical practice, and the BISSI tool remained reliable, despite the removal of milk from the breast.

The study design intended to collect all three BISSI scores within 90 minutes to account for confounding factors. However, it was difficult for the mothers to adhere to this timeframe, resulting in a 12- to 127-minute period in which all scores were obtained. It is known that reliabilities tend to drop as the time between tests increases.³⁴ Prolonged time between collection of scores could result in symptom deterioration or improvement occurring from the pathological process itself. In addition, the lactating breast is constantly producing milk,³⁵ which may impact the BISSI domain of “hardness/tightness.” Despite this, the BISSI tool demonstrated good reliability. Conversely, the shorter timeframes over which the scores were taken may have increased the potential for participant response bias, via recent recall of scores. To mitigate the potential impact of the response bias, the treating physiotherapist provided the mother with a distracting task, namely, additional clinical questioning before scoring the final face-to-face BISSI.

One limitation was that mothers were recruited through one private physiotherapy clinic and represented a higher socioeconomic status, which reduces the external applicability of these results to differing populations. Additionally, mothers were only eligible to participate if they spoke English and had internet access. Therefore, our sample may not be a true representation of all mothers with ICLB.

Conclusions

This study provides evidence supporting interscenario reliability of the BISSI and is another step towards psychometric validation of this tool. This aids the use of the tool in both clinical practice and research. Successful application of the BISSI in multiple scenarios is likely to positively impact the care of lactating mothers, potentially improving long-term health outcomes for both mother and baby. The BISSI may enable reliable distance monitoring of breast inflammatory symptoms, allowing clinicians to potentially improve triage and prioritize mothers prior to in-clinic consultations.

Authors’ Contributions

L.J.M.: This author provided conceptualization, validation, formal analysis, writing—review and editing, data curation, project administration, and supervision. N.K.: This author provided methodology, validation, formal analysis, investigation, data curation, writing—original draft, visualization. E.C.W.: This author provided methodology, validation, formal analysis, data curation, writing—original draft, and visualization. H.J.S.: This author provided methodology, validation, formal analysis, data curation, writing—original draft, and visualization. B.K.E.: This author provided methodology, validation, formal analysis, data curation, writing—original draft, and visualization. M.M.C.: This author provided methodology, writing—review and editing, and visualization. E.L.H.: This author provided methodology, formal, writing—review and editing, and visualization. A.M.M.: This author provided conceptualization, methodology, writing—review and editing, and supervision. All authors read and approved the final article.

Footnotes

Acknowledgments

The authors wish to thank all the clinical trial participants who participated in this study. The authors wish to acknowledge the private physiotherapy clinic and staff involved in the data collection and thank the clinic’s director, Kym Veale, for enabling this research.

Ethics Approval and Consent to Participate

Ethical approval for this study was granted by Curtin Human Research Ethics Committee (HRE2023-0428) on August 7, 2023, and reciprocal approval was received from Monash University (39989). All participants gave written informed consent.

The research was completed in accordance with the Declaration of Helsinki as revised in 2013.

Availability of Data and Materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Disclosure Statement

A potential conflict of interest is that the treating physiotherapist, N.K., is employed at the clinic where data collection was undertaken. The remaining authors declare no conflict of interest.

Funding Information

There has been no financial assistance for this project.

References

1. Cooper

, Lowe

, McArdle

. Development of a novel patient focussed symptom severity index for use in assessing and treating inflammatory conditions of the lactating breast: A delphi study. Int J Evid Based Healthc 2020;18(2):231–240; doi: 10.1097/XEB.0000000000000225

2. Heron

, Maselli

, McArdle

, et al. Exploring physiotherapists’ clinical definition and diagnosis of inflammatory conditions of the lactating breast in Australia: A mixed methods study. Int Breastfeed J 2020;15(1):48–58; doi: 10.1186/s13006-020-00294-9

3. Heron

, McArdle

, Karim

, et al. Construct validity and internal consistency of the breast inflammatory symptom severity index in lactating mothers with inflammatory breast conditions. PeerJ 2021;9:e12439; doi: 10.7717/peerj.12439

4. Deng

, Huang

, Ning

, et al. Maternal risk factors for lactation mastitis: A meta-analysis. West J Nurs Res 2021;43(7):698–708; doi: 10.1177/0193945920967674

5. Kinlay

, O’Connell

, Kinlay

. Incidence of mastitis in breastfeeding women during the six months after delivery: A prospective cohort study. Med J Aust 1998;169(6):310–312; doi: 10.5694/j.1326-5377.1998.tb140282.x

6. Mitchell

, Johnson

, Rodríguez

, et al. Academy of breastfeeding medicine clinical protocol #36: The mastitis spectrum, revised 2022. Breastfeed Med 2022;17(5):360–376; doi: 10.1089/bfm.2022.29207.kbm

7. Wilson

, Wood

, Benova

. Incidence of and risk factors for lactational mastitis: A systematic review. J Hum Lact 2020;36(4):673–686; doi: 10.1177/0890334420907898

8. Stuebe

, Schwarz

. The risks and benefits of infant feeding practices for women and their children. J Perinatol 2010;30(3):155–162; doi: 10.1038/jp.2009.107

9. Victora

CGP

, Bahl

RMD

, Barros

AJDP

, et al. Breastfeeding in the 21st century: Epidemiology, mechanisms, and lifelong effect. Lancet 2016;387(10017):475–490; doi: 10.1016/S0140-6736(15)01024-7

10.

10. Babic

, Sasamoto

, Rosner

, et al. Association between breastfeeding and ovarian cancer risk. JAMA Oncol 2020;6(6):e200421; doi: 10.1001/jamaoncol.2020.0421

11.

11. Green

. Infant feeding and asthma: Is breast milk best? Rev Econ Household 2011;9(4):487–504; doi: 10.1007/s11150-010-9108-z

12.

12. Horta

, de Lima

. Breastfeeding and type 2 diabetes: Systematic review and meta-analysis. Curr Diab Rep 2019;19(1):1–6; doi: 10.1007/s11892-019-1121-x

13.

13. Belfield

, Kelly

. The benefits of breastfeeding across the early years of childhood. J Hum Cap 2012;6(3):251–277; doi: 10.1086/667415

14.

14. Kyte

, Calvert

, van der Wees

, et al. An introduction to patient-reported outcome measures (PROMs) in physiotherapy. Physiotherapy 2015;101(2):119–125; doi: 10.1016/j.physio.2014.11.003

15.

15. Paré

, Trudel

M-C

, Forget

. Adoption, use, and impact of e-booking in private medical practices: Mixed-methods evaluation of a two-year showcase project in Canada. JMIR Med Inform 2014;2(2):e24–e35; doi: 10.2196/medinform.3669

16.

16. Tan

, Shah

, De Souza

, et al. Improving the patient booking service to reduce the number of missed appointments at East London NHS foundation trust community musculoskeletal physiotherapy service. BMJ Open Qual 2017;6(2):e000093; doi: 10.1136/bmjoq-2017-000093

17.

17. Kottner

, Audige

, Brorson

, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. Int J Nurs Stud 2011;48(6):661–671; doi: 10.1016/j.ijnurstu.2011.01.016

18.

18. Harris

, Taylor

, Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019;95:103208–103231; doi: 10.1016/j.jbi.2019.103208

19.

19.Inform Physiotherapy. Lactation for health professionals course. 2023. Available from: https://www.informphysio.com/education

20.

20. Friedlaender

, Dafner-Beach

. Cliniko: Allied health practice management system. 2023. Available from: https://www.cliniko.com

21.

21. Bushra

, Aslam

. An overview of clinical pharmacology of Ibuprofen. Oman Med J 2010;25(3):155–1661; doi: 10.5001/omj.2010.49

22.

22. Freo

, Ruocco

, Valerio

, et al. Paracetamol: A review of guideline recommendations. J Clin Med 2021;10(15):3420–3442; doi: 10.3390/jcm10153420

23.

23.MIMS Australia. Panadol. MIMS Australia.

24.

24.MIMS Australia. Dicloxacillin. MIMS Australia.

25.

25.MIMS Australia. Flucloxacillin. MIMS Australia.

26.

26. Glanzer

, Cunitz

. Two storage mechanisms in free recall. J Verbal Learning Verbal Behav 1966;5(4):351–360; doi: 10.1016/S0022-5371(66)80044-0

27.

27. Postman

, Phillips

. Short-term temporal changes in free recall. Q J Exp Psychol 1965;17(2):132–138; doi: 10.1080/17470216508416422

28.

28.Australian Bureau of Statistics. Socio-Economic Indexes for Areas (SEIFA). 2023. Available from: https://www.abs.gov.au/statistics/people/people-and-communities/socio-economic-indexes-areas-seifa-australia/2021

29.

29.IBM Corp. IBM SPSS Statistics. 2023. Available from: https://www.ibm.com/products/spss-statistics

30.

30. Kim

H-Y

. Statistical notes for clinical researchers: Assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod 2013;38(1):52–54; doi: 10.5395/rde.2013.38.1.52

31.

31. Weir

. Quantifying test-retest reliability using the intraclass correlation coefficient and the sem. J Strength Cond Res 2005;19(1):231–240; doi: 10.1519/15184.1

32.

32. Stratford

, Goldsmith

. Use of the standard error as a reliability index of interest: An applied example using elbow flexor strength data. Phys Ther 1997;77(7):745–750; doi: 10.1093/ptj/77.7.745

33.

33. Bujang

, Baharum

. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: A review. Archives of Orofacial Sciences 2017;12:1–11.

34.

34. Bhattacherjee

. Chapter 7: Scale Reliability and Validity. In: Social Science Research: Principles, Methods, and Practices. 2nd ed. Univeristy of South Florida; 2012.

35.

35. Kent

. How breastfeeding works. J Midwifery Womens Health 2007;52(6):564–570; doi: 10.1016/j.jmwh.2007.04.007