Abstract
Data quality is defined as a measure of data status that fulfills the following elements: accuracy, completeness, consistency, reliability, and if the data is current. The World Health Organization (WHO) reported that only 40% of all countries have an adequate system to collect information on birth and deaths. Even though the system is there, vital registration systems are inaccurate and incomplete in developing countries. In Rwanda, maternal health related data was over-reported more than other indicators. These are the main reasons for conducting the study to investigate the data quality of four maternal and newborn health indicators reported by Rwandan Western Province health centers. This concurrent-mixed method study included 61 data managers and 12 key informants. Routine data quality assessment tool and structured interview guide were used to collect data. Descriptive statistics were used to get proportion of respondents’ socio-demographic characteristics. The analysis was done for assessing median of data quality index. The results show that 55.7% of data managers were male while 58.3% of responsible of maternity were female. Majority (58.9%) of participants was in age’s category from 33-42, 61.6% have A1 education level and 53.4% have experience less than five years. Data quality index of one out of four (25%) MNH indicators was found below 95% accepted by WHO. The main reasons for insufiscient quality of data are lack of data validation meetings (57.5%) and incompleteness of reporting tools (36.4%). Monthly data validation meetings chaired by HC leaders are important to contribute to high-quality data in healthcare settings. Supportive supervisions done in data quality and management have to be organized in a supportive, and educative way.
Author Contributions
Academic Editor: Raul Isea, Fundación Instituto de Estudios Avanzados -IDEA
Checked for plagiarism: Yes
Review by: Single-blind
Copyright © 2022 Mathieu Niyonkuru, et al.
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Competing interests
The authors have declared that no competing interests exist.
Citation:
Introduction
Data of quality which are timely reported in HMIS by health facilities are a key foundation of health systems at all levels 1. These data are used for the purpose of planning diseases prevention, care, and treatment interventions there is a need that data is accurate, and well-analyzed before its' submission. The World Health Organization (WHO) reported that 40% of all countries have an adequate system to collect information on births and deaths. To ensure data quality of reports from health facilities, different countries started to use health management information systems. This system has been an important tool used for health sector reforms worldwide. The WHO created this system to ensure adequate data collection, treatment, management, and usage within the healthcare system, which defines the system's effectiveness 2. Even though different Governments have invested in this system, vital registration systems are inaccurate and incomplete in developing countries 3. These regions with poor data quality in system registration are the ones that have neonatal mortality rate which is very high 4.
The WHO has stated that data produced through regular health recording and reporting methods is of poor quality, fragmentary, and late 5. Several studies in Africa and Asia that evaluated the quality of routine health data found significant periods of incompleteness, delay in reporting, incomplete indicator level data, and erroneous facility reporting. Findings from a study conducted in Ethiopia, Nigeria have shown incompleteness, underreporting in facility-level reports 6.
Research conducted in different organizations to monitor the quality of maternal and newborn healthcare on 1 791 indicators reported that 19.3% of them were found duplicated. In the same study, only 6.7% indicators were found meeting all requirements for scientific soundness 7. Systematic review of maternal and newborn data from 34 studies highlighted heterogeneity 8.
Findings from a study conducted to evaluate quality of reports in low-middle income country (Botswana) revealed that 56 percent of maternal and newborn indicators were below the acceptable range for data quality, and 87 percent of discrepancy values were outside the acceptable range. This investigation revealed that MNH indicators had lower data quality than child health indicators 9.
The study of HMIS data quality in Ethiopia revealed that completeness, reporting timeliness, and correctness were all inadequate 10. A lack of data quality has been proven a flaw. The degree of data accuracy in health centers was significantly lower compared to the national target. Lack of training, failure to use supervisor feedback, less frequent DQA, and health care providers' poor skills are all shown to be linked issues 11. At the facility level, studies have noted remarkable poor data quality challenges related to inaccuracy data captured at monthly basis 12.
Health data users need high-quality data to be used confidently. Without quality data, there is low demand, decisions are made on real data, and health programs implementation and effectiveness will suffer 13. A good HIS guarantees that good quality information is available and that it is used to assist informed decision-making 14. Findings from several surveys reveal a high cost of damage in many countries linked to the problems associated with data quality 15.
Inaccurate data harms the economy as shown by studies, duplicated medical records can cause repetition of medical care and result to the cost of $1 950 an average per outpatient and over $800 per emergency visit 16. Once data are reported with poor quality, they contribute to inadequate decisions taken at different levels, causes insufiscient confidence in health systems, and result in non-validity of impact evaluation studies. Findings from 34 studies in published literature evaluated the quality of newborn data collected at health facility level in LMIC, revealed that there is no study reporting on the quality of newborn indicators and the systematic review done highlighted heterogeneity 17. In Rwanda, maternal health related data was over-reported more than other indicators 18. Data quality assessment sessions conducted in maternal, newborn and child health services for indicators reported in HMIS showed that there are issues that resulted in inaccurate data, due to insufficient knowledge of indicators 19. Regular quality of data assessments and use might contribute to higher data quality specific activities that identify poor data quality root causes. Human resource constraints make it less probable for facilities to completely adopt actions that increase data quality and usage 20.
In different coordination meetings attended in Western Province noted insufiscient quality of data reported in HMIS by districts health centers. In data presentations, data quality issues were identified such as outliers, typing error in general with particularity in maternal and newborn health indicators. Some of indicators with data quality issues were number of women giving birth who received uterotonics, number of newborns not breathing at birth who were resuscitated. These are the main reason which justified this study to be conducted in this region of Western Province to determine quality of maternal and newborn indicators reported by selected HCs of Western Province by assessing quarterly median data quality score of four MNH indicators, determine the common reasons for insufiscient data quality index.
Methods
Research Design
The present study was descriptive, cross-sectional study design. This study was conducted in 61 health centers of Ngororero, Nyabihu, Rubavu and Rutsiro. These four districts have 5 hospitals, 62 health centers, 80 health posts and 7 474 community health workers. This study involved 61 data managers and 12 responsible of maternity services from 16 HCs of Nyabihu, 13 from Rubavu, 15 Ngororero, and 17 from Rutsiro. Twelve key informants (responsible of maternity) are the ones who normally provide maternity and newborn health service. A purposive sample was carried out and one HC with the highest and two HCs per district with low data quality were selected.
Sample Size
To conduct this study, all 61 (100%) were health centers data managers and 12 responsible of maternity services were sampled because they are involved in the data management processes.
Sampling Technique
Quantitatively, a convened sampling technique has been used to select data managers. The desired sample size has been obtained from the total number of data managers who are affected in health centers of Nyabihu, Rubavu, Ngororero and Rutsiro Districts health centers. Every data manager has had a chance of being selected in this study once they consented to be part of it. Then a total of 61 data managers have had equal opportunity to be selected in this study.
Qualitatively, the study has used purposive sampling. This technique allowed the selection of responsible of maternity services (key informants) to provide delivery and post-natal services in four selected districts health centers. Two HCs per district with lowest score of data quality and one HC per district with highest score of data quality. Then a total of 12 responsible of maternity services have been selected in this study.
Data Collection Methods
The researcher used a routine data quality assessment tool for health facilities (Quantitative Aspect). This tool has been designed in an excel sheet and validated by Rwandan MoH to be used by the M&E team from hospitals to health centers in their DQA routine activities. In this research, the tool has been used to determine the quarterly median data quality score of four MNH indicators reported by selected HCs and common reasons for insufficient data quality score. Questionnaire has been developed, digitalized and pre-tested to facilitate the identification of the challenges, weaknesses, estimate time and effort required to respond to the questions for an adjustment before applying them to the whole study population.
Qualitative information has been collected to 12 keys informants from HCs with highest and lowest score of data quality. The semi-structured questionnaire has been developed and tested for collecting qualitative information in focus group discussion, to record strengths, challenges, suggestions to have a good quality of maternal and newborn indicators reported in HMIS.
Procedures of Data Collection
Data collection was done between April and June 2022. Quantitative data collection process was done by trained district health data analysts. These data collectors used smart phones in which digitalized questionnaire using KoBoCollect V 2021.2.4. was deployed. Researcher managed database filed and protected by a password on a computer.
For qualitative data collection, the researcher personally guided a focus group of 12 key informants (responsible of maternity) from eight HCs with the lowest score of data quality and four HCs with the highest quality of data quality. All respondents were informed of the objective and the study procedure before participating and they voluntarily signed a consent form before processing the responses. At the end of this focus group, one open-ended question was asked to respondents to enumerate factors that can affect quality of data reported in HMIS by health centers. A researcher transcribed responses and each participant got a code.
Data Analysis
Quantitative data has been recorded on DQA tool on a computer. The median data quality score per HC has been determined after the data collection. Descriptive statistics related to frequency distributions and percentages have supported a description of various variables under study using Stata/SE 17.0.
Qualitative data has manually and thematically been analyzed. The study findings have been presented in tables and graphs for the quantitative section whereas the narrative approach has been used to deliver qualitative results from 12 key informants selected in eight HCs with the lowest score of data quality and four from HCs with highest data quality score. The data collection process was done between April and June 2022 using KoBoCollect V 2021.2.4, and Stata/SE 17.0 was used for data analysis.
Qualitative data was thematically analyzed. Descriptive statistics were used to get proportion of respondents’ socio-demographic characteristics.
Ethical Consideration
Mount Kenya University authorities provided to researcher required academic letter for introduction to the sampled districts mayors to allow data collection in health centers. Informed consent has been developed and presented to the study participants before any data/information collection. Participants have been informed about the research purpose, procedures, benefits to the participants and society, and dangers via this informed consent form.
Participants have been notified that participation was completely optional, that they can withdraw at any moment without penalty, that the interview took approximately one hour, that the data and information provided are kept confidential. Then, at the bottom of the informed consent form, a statement acknowledging that the participant has read and understood the consent document, that he or she has had the opportunity to ask the researcher (or appropriate individual) questions, and that the participant consents to participate in the study, followed by the participant's signature.
Results
Demographic Characteristics of Respondents
The distribution of social-demographic characteristics of respondents who participated in this study are presented in table 1 and table 2
Data presented in table 1 shows that 61 data managers working in western province districts health centers were voluntarily recruited to participate in this study. Majority (55.7%) of data managers are male, 55.7% were found in ages category from 33-42 years, 55.8% have A1 education level, 59.0% did health sciences and 49.2% have experience which is less than five years.
Data presented in table 2 shows that 12 responsible of maternity services were voluntarily recruited to participate in this study to collect study qualitative data. Majority (58.3%) of maternity responsible were found female, 75.0% were found in ages category from 33-42 years, 91.7% were found with A1 education level, all of them (100%) did health sciences and 75% have experience which is less than five years.
Table 1. Characteristics of Sample (data managers)| Sample description | |||
|---|---|---|---|
| Frequency | Percent | ||
| District | Ngororero | 15 | 24.6 |
| Nyabihu | 16 | 26.2 | |
| Rubavu | 13 | 21.3 | |
| Rutsiro | 17 | 27.9 | |
| Gender | |||
| Female | 27 | 44.3 | |
| Male | 34 | 55.7 | |
| Age of respondents | |||
| 23-32 | 14 | 23.0 | |
| 33-42 | 34 | 55.7 | |
| 43-52 | 12 | 19.7 | |
| 53-62 | 1 | 1.6 | |
| Education level | |||
| A2 level | 3 | 4.9 | |
| A1 level | 34 | 55.8 | |
| Bachelor's and + | 24 | 39.3 | |
| Field of education | |||
| Health sciences | 36 | 59.0 | |
| Other fields | 25 | 41.0 | |
| Experience’s years | |||
| Less than 5 years | 30 | 49.2 | |
| 5-10 years | 9 | 14.8 | |
| 10 years and + | 22 | 36.1 | |
| Sample description | |||
|---|---|---|---|
| Frequency | Percent | ||
| District | Ngororero | 3 | 25.0 |
| Nyabihu | 3 | 25.0 | |
| Rubavu | 3 | 25.0 | |
| Rutsiro | 3 | 25.0 | |
| Gender | |||
| Female | 7 | 58.3 | |
| Male | 5 | 41.7 | |
| Age of respondents | |||
| 23-32 | 1 | 8.3 | |
| 33-42 | 9 | 75.0 | |
| 43-52 | 2 | 16.7 | |
| Education level | |||
| A1 level | 11 | 91.7 | |
| Bachelor's and + | 1 | 8.3 | |
| Field of education | |||
| Health sciences | 12 | 100.0 | |
| Experience’s years | |||
| Less than 5 years | 9 | 75.0 | |
| 5-10 years | 3 | 25.0 | |
Quarterly Median Data Quality Score of Four MNH Indicators Reported Western Province HCs
Quarterly median of four MNH indicators reported by western province districts health centers is presented in table 3. The researcher used the RDQA tool to calculate the score for each indicator. To assess quarterly data quality index, researcher used RDQA tool to get the score of each indicator. Based on score of each indicator, researcher calculated the mean of score attributed to these four MNH indicators and this was called data quality index.
Table 3. Median of 4 MNH data quality index| List of MNH indicators | Median DQI | IQR |
|---|---|---|
| 1. Number of women giving birth who received uterotonics in the third stage of labor | 96.3 | (81.8 - 100.0) |
| 2. Number of live births | 99.2 | (96.8 - 100.0) |
| 3. Number of newborns not breathing at birth who were resuscitated | 66.7 | (66.7 - 100.0) |
| 4. Number of newborns who receive PNC within 2 days of birth | 96.6 | (85.1 - 98.3) |
The results presented in table 3 reveal that the median data quality index for three out of four MNH indicators (75.0%) is over 95 accepted by the MoH, while one indicator (25%) is below 95 (66.7). Indicators number 3 (Number of newborns not breathing at birth who were resuscitated) is the one which was founded with lowest data quality index.
Reasons For Insufficient Data Quality Index of Four MNH Indicators Reported Western Province HCs
Common reasons for insufiscient data quality index of four MNH indicators were collected in selected health centers of western province. These reasons are presented in figure 1.
Figure 1.Common reasons for insufficient data quality index
Study findings presented in figure 1 shows that 36.4% of HCs has insufficient DQI due to incompleteness of data collection tools, 33.3% due to lack of monthly data analysis report, 24.2% due to lack of validation report prior data entry in HMIS, and 6.1% due to data discrepancies for few randomly selected indicators.
Discussion
The main objective of this research was to assess the data quality of MNH indicators reported in HMIS by western province health centers. Findings from this research reveals that data quality index of one out of four (25%) MNH indicators was found below 95%. Indicators number 3 (Number of newborns not breathing at birth who were resuscitated) is the one which was founded with low data quality index. There are no equivalent findings from studies conducted in similar circumstances found to compare the similarity. In contrast, our results differ from ones of study conducted in Ethiopia, which found data quality index ranged from 32 percent to 75 percent 21. Our results might be different from the one of Ethiopia as referred study examined eight indicators while we examined four MNH indicators. This indicator was found with insufiscient data quality index because it might be not well calculated as health care providers affected in maternity might subjectively assess APGAR
Normally the hospital evaluation team conduct evaluation sessions each quarter at the HC level. It seems that this evaluation is not contributing to the increase of quality of data as it is mandate. This is very crucial because normally this supportive supervision might be well prepared and conducted with purpose of educating HC staff as they are conducted by advanced skilled staff. Once conducted with routine, they are not contributing to the strengthening of health system or to solve identified gaps. Another way of solving data quality issues in health centers is data validation meetings which are mandate to be chaired by HCs heads and responsible of services each month. The role of data validation meetings to solve data quality issues is clearly defined in HMIS SOP document. The problem is that health centers data validation meetings are not chaired at regular basis. In the HCs which conducted these meetings, the minute shown gaps and the agenda seems to be not really the discussions and validation of data but what to show to evaluators in qualitative evaluation. The role of leadership of HC and hospital is very important to discourage this attitude of not focusing on data.
The main reasons of insufficient data quality, lack of monthly data analysis, validation meeting prior data entry in HMIS (56.5%) and incompleteness of data collection tools (36.4%), were found as main reasons of insufficient data quality. These results are completed and confirmed by key informants from HCs with insufficient DQI who highlighted that insufficient knowledge of MNH indicators, incompleteness of maternity registers, and shortage of staff, supervision which doesn’t go in deep for verification of data validity, HCs staff and leadership who do not take data validation meetings as significant, as main challenges. Our findings are similar with the study conducted in Benin which revealed that insufficient quality of data was associated to the incompleteness of reporting forms 22, misunderstanding of HMIS tools/indicators and insufficient skills, workload, and incentives 1. Our findings were found differ from one of study conducted in South Africa which revealed that the poor quality of the data was attributed to the insufficient competencies of health information personnel 23 and the one conducted in Iran on data quality and utilization found that poor quality was linked to the volume of data gathered, the frequency of HIS training, supervision and feedback, and an incentive mechanism 24. This is linked to the skills of nurses and midwives affected in maternity to provide maternal and newborn services who founded in report with insufficient knowledge to calculate APGAR score 19. Data from health centers with insufficient data quality index shown that lack of training, poor and delay of documenting primary data sources, shortage of staff, health care providers and leadership who don’t give values reported data and validation meetings are the main challenges that these health centers met to have poor data quality report. We can’t ignore the role of training for health care providers who don’t have knowledge on indicators to report.
As health care providers misunderstood MNH indicators, they can make errors in counting data. Once these providers don’t have knowledge of indicators, don’t know how numerator/ denominator, this result to the leaving empty reporting cages and contribute to the insufficient data quality. Ignoring filling reporting tools while these are quarterly checked in qualitative and quantitative evaluation is a problem.
The overloaded health care providers, cannot properly complete reporting tools, can commit errors in data counting, transcript. The problem that these health centers with insufficient data quality index is that they experienced the problem of fillings reporting tools, and data analysis and validation meeting prior data submission in HMIS. These problems which affect quality of data normally might be solved by validation meetings, but these are not taken by leadership as serious meeting. Again, supervisions from hospital level which were supposed to educate health care providers and data managers to play their role and responsibility don’t go in deep for verification of data validation. Hospital M&E teams or qualitative evaluators are mandate to pass at the HC level at quarterly basis for their routine activities of supporting health care providers. The problem is that these supervisions are not conducted just to report that they have been done for PBF not in educative way as highlighted by health care providers.
Conclusion
To ensure high data quality in rural health centers need a collaboration and well organization of teams from supervisions health institutions. These supervision teams have to conduct supervision in educative way to respond to the identified gaps to increase data quality of indicators reported by HCs. Heads of HCs might continue to create a positive working environment, and chair monthly data validation meeting mandated prior data submission. MoH and its’ stakeholders are encouraged to continue to provide more routine data management training to health care providers involved in data management focusing on how to fill reporting tools, particularly those in rural health settings. As this study was limited on its’ methodology, other researchers are encouraged to conduct research on how health centers comply with Standard Operating Procedures for Management of Routine Health Information.
References
- 1.Susan F Rumisha, L E. (2020) Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania.BMC Medical Informatics and Decision Making.
- 2.WHO. (2010) Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies.WHO Library Cataloguing-in-Publication Data.
- 3.Goudar S. Part of Springer Nature (2015) Data quality monitoring and performance metrics of a prospective, population based observational study of maternal and newborn health in low resource settings.BMC.
- 4.Margaret. (2017) Data accuracy in the Victorian Perinatal Data Collection: Results of a validation study of 2011 data.Health. , Information Management Journal
- 5.Karengera. (2016) Quality and use of routine health care data in selected district of eastern Province of Rwanda. , International Journal of Public Health Research., Karengera
- 6.WHO. (2017) Data verification and system assessment. WHO Document Production Services , Geneva, Switzerland .
- 7.Pedro. (2019) Indicators for monitoring maternal and neonatal quality care: a systematic review. BMC Preganncy and Childbirth.
- 8.Lundin Rebecca. (2022) Quality of routine health facility data used for newborn indicators in low- and middle-income countries: A systematic review.
- 9.Lebapotswe. (2019) Data quality self-assessment of child health and sexual reproductive health indicators in. , Botswana
- 10.Adane. (2021) Routine health management information system data in Ethiopia: consistency, trends, and challenges.Glob Health Action.
- 11.Salomon M. (2021) Data quality assessement and associated factors in the health management information system among health centers of southern Ethiopia.
- 12.Hotchkiss. (2020) Evaluation of the performance of routine information system management (PRISM) framework: evidence from Uganda.
- 13.Nutley T. (2013) Improving the use of health data for health system strengthening.Global Health Action.
- 14.Nguefack-Tsague G. (2020) Factors associated with the performance of routine health information system in Yaoundé-Cameroon: a cross-sectional survey.
- 16.Kent J. (2018) Patient Matching Issues Increase Costs, Threaten Patient Safety. Healthcare Analytics.Population Health Management, Healthcare Big Data.
- 17.Lundin R.(2022).Quality of routine health facility data used for newborn indicators in low- and middle-income countries: A systematic review.
- 18.Nshimyiryo. (2020) Health management information system (HMIS) data verification: a case study in four districts in Rwanda.
- 21.Ouedraogo Mariame. (2019) A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone. , Ethiopia
- 22.Yolaine Glele Ahanhanzo. (2015) Data quality assessment in the routine health information system: an application of the Lot Quality Assurance Sampling in Benin.Health Policy and Planning.