Introduction

During the last years, the diagnosis of mycosis has become a challenge for mycology laboratories. Causes such as the increase in fungal infections and the emergence of new fungal pathogens and the development of resistance to antifungal drugs generated the need to incorporate more complex and specific diagnostic procedures.

Moreover, there is a trend toward the implementation of quality standards such as the ISO 15189 International Standard to ensure accurate, reliable and timely labora- tory results and the continuous improvement of laboratory performance⁷. Participation in external quality assessment programs is one of the requirements of this standard, being essential to verify laboratory performance and, eventually, to improve it.

Argentina is a developing country with a surface area of 3 761 274km² and an average population density of 10.7 inhabitants per km² (https://www.ign.gob.ar). The country has large geographical areas with low population densities, where health facilities are generally scarce or limited; in contrast, city centers have higher population density and more health facilities (https://www.ign.gob.ar). To over- come this limitation, a laboratory network was created in 1996 in order to organize the referral of complex samples from low complexity laboratories to high complexity labo ratories. In the same year, the Mycology Department (MD) of the National Institute of Infectious Diseases, introduced a National External Program of Quality Control in Mycology (PNCCM, for its acronym in Spanish), which offered qual ity assessments to the members of the network and other mycology laboratories. It aims to improve diagnostic quality, to help establish and to set up standardized procedures and training of laboratory staff. Participation in the program is not compulsory, and participating laboratories are grouped into three categories depending on their complexity.

The aim of this study was to evaluate the effectiveness of the PNCCM and the performance of the laboratories par- ticipating in the PNCCM in the period 1996-2018.

Methods

PNCCM organization: The program was organized into three components. The first component was the coordinating cen- ter (CC), located at the MD. The CC was responsible for the program direction, specimen preparation and, performance evaluation of the participating laboratories. The regional coordinators (RCs) constituted the second component (one per jurisdiction), and were responsible for specimen distri- bution. Jurisdiction refers to the City of Buenos Aires and the 23 provinces of Argentina. The third component encom- passed all participating laboratories.

Figure 1: Number of participants per year. Reported: Laboratories that answered the testing event; Not reported: Laboratoriesthat did not answer the testing event. 

The object of study of the program was the assessment of hospital laboratory performance. Hospital laboratories were categorized by complexity (low, intermediate, or high) according to the medical practices offered and the associ- ated fungal infections.

From its inception in 1996 until 1998, there was one test- ing event per year, comprising three specimens. Since 1998, there were two testing events per year. Then, since 2006 there have been two testing events per year, consisting of four specimens.

Specimen preparation: Selected specimens were clinical samples or fungal collection strains with different lev- els of difficulty for their identification. Clinical samples included hyperimmune sera, homogenized clinical samples, and simulated clinical samples inactivated with formalin. Fungal strains included both common and uncommon or emerging pathogens. Samples included an information sheet with epidemiological, geographical, and clinical data.

A batch of each specimen was produced and controlled. Evaluation of homogeneity of clinical samples and fungal strain batches involved a random sample of 10%. Viabil- ity, purity, and main characteristics were assessed for each specimen. Once homogeneity of the batch was confirmed, a new random sample of 10% was stored in order to assay its stability in case of unexpected troubles in the testing event.

The specimens were shipped into a triple packaging sys- tem according to the Guidelines for the Safe Transport of Infectious Substances and Diagnostic Specimens¹⁶.

Difficulty level of the samples: Specimens were divided into three categories according to the difficulty to be solved: Low difficulty level (L1) included fungal strains belonging to common dermatophyte species and Candida albicans, and clinical samples of skin, hair, and nails. Intermediate difficulty level (L2) included fungal strains belonging to Aspergillus fumigatus, A. niger and frequently isolated yeast species; as well as clinical samples for the determination of opportunist mycoses by direct examination (BAL, CSF).

High difficulty level (L3) included fungal strains belong- ing to Mucorales, dematiaceous, other hyphomycetes, and less frequent yeast species. Clinical samples included the diagnosis of endemic mycoses (coccidioidomycosis, paracoccidioidomycosis, and histoplasmosis) and chronic aspergillosis.

Figure 2: Percentage of correct answers per testing event. The two periods considered include 1---13 and 27---43 testing events,respectively. 

Sample analysis and response evaluation: To process and analyze the specimens, the laboratories were required to perform the same procedures they use for routine work. Laboratories submitted their results by filling out online standardized forms within two months of the shipment.

Fungal strains were evaluated based on the iden- tification at genus or species level. Furthermore, the macro-and micromorphological, biochemical, physiological, and/or nutrition characteristics were submitted. Clinical samples and simulated clinical samples were evaluated by microscopic observation. The criterion for evaluation of the hyperimmune sera was the diagnostic result.

The data submitted by laboratories were stored and analyzed using a database management system designed in-house.

Reference responses: Each year, the six best-performing laboratories were selected. Their results were an internal check of the responses of each specimen. If at least three out of the six laboratories failed to identify a specimen, it was withdrawn from the testing event and not considered in the participants’ overall performance.

Reporting: Each participant received an individual report and a global report that summarized the results of all par- ticipating laboratories by complexity level.

When similar methodological errors arose in a signifi- cant number of laboratories, the evaluation report included additional information. RCs received a jurisdictional report summarizing the overall results. Specific information regard- ing individual laboratory identity and performance remained confidential.

Effectiveness of the PNCCM: Five indicators assessed the performance of the PNCCM. 1 - Number of labora tories and jurisdictions controlled. 2 - User adherence: percentage of laboratories participating in 2018 over the total number of laboratories enrolled since 1996. 3 - User satisfaction: five satisfaction surveys were carried out in the period 2006-2018. 4 - Improvement of laboratory results by sample complexity level: possible percentage increase in the number of correct results comparing two periods: the first from 1996 to 2003 and the other from 2012 to 2018. 5 - Labo- ratory performance: the average value of correct responses was calculated for each sample complexity level, and was used as a cut-off value to define whether each laboratory reached its level of complexity. A level 3 laboratory must reach three cut-offs (L3, L2, and L1). A level 2 laboratory must reach L2 and L1 cut-offs, and a level 1 laboratory only has to reach L1 cut-off.

This analysis included laboratories that responded at least six testing events from 1996 to 2018. In addition, the percentage of correct answers by specimen and sample com plexity level was determined.

Results

Effectiveness of the PNCCM

- Number of laboratories and jurisdictions controlled: In 1996, 59 laboratories enrolled in the PNCCM, but only 29 answered the first testing event. Participants belonged to 14 out of the 24 jurisdictions. Since 2009,148 laboratories have enrolled in the PNCCM, and 139 answered the testing events. From that moment on, the program was able to control all jurisdictions.

Figure 1 shows the increase in the number of partici- pating laboratories by year. The percentage of participants who answered each testing event increased from 49% to 91% between 1996 and 2000 and thereafter fluctuated between 80% and 97% (Fig. 1).

- User adherence: Overall adherence of the partici pants during the evaluated period was 72% (166/229). In the public sector, adherence was 78% (140/179), while in the private sector was 52% (26/50).

- User satisfaction: The satisfaction surveys were answered by 26% of the participating laboratories, consider- ing the PNCCM to be very good (84%) or satisfactory (16%).

- Improvement of laboratory results: The percentage of correct answers was 76% and 82% in the periods 1996-2003 and 2012-2018, respectively. Figure 2 shows the upward trend in the average correct answers per testing event. The overall average of correct answers for the testing events from 1996 to 2018 was 79%.

The overall improvement in the quality of the results for the evaluated period was 15% for low complexity sam ples, 7% for intermediate complexity samples, and 14% for improvement in the identification of level 3 strains. Sero- logical techniques results showed no improvement in the evaluated period.

The percentage of correct answers for the identifica- tion of yeasts frequently associated with fungal infections in Argentina^4,9,10,14 reached 80% (yeast of L1 and L2 sam ples). In the case of C. albicans, the percentage increased to 95%, while it markedly decreased when the species were emerging pathogens such as Candida haemulonii (13%), Cryptococcus gattii (28%), and Wickerhamomyces anomalus (33%).

Aspergillus spp. were correctly identified at the species level in 84% of the cases. Even emerging pathogens (level 3), such as A. terreus, were correctly identified at the species level by 48% of the laboratories in 2007 and by 68% of the laboratories in 2015.

Filamentous fungi uncommonly associated with human infections were correctly identified, at least at the genus level, 77% of the time. Moreover, less than 50% of the lab- oratories were able to identify other emerging pathogens, such as Scedosporium boydii and Purpureocillium lilacinum, which were only correctly identified by 44 and 29% of the participants, respectively.

Table S1 shows the percentage of correct answers by type of specimen and difficulty level (L1: low; L2: intermediate; L3: high, supplementary material).

- Laboratory performance: About 134 laboratories were included in the calculation of the cut-off value, which was 83% for level 1, 78% for level 2 and level 3, 74% for species identification and 88% for serological diagnosis. Figure 3 shows the distribution of the percentage of correct answers (score) according to the sample complexity level. Of 134 laboratories, 87 achieved the L1 sample cut-off value, 62 achieved the L2 sample cut-off value, and 54 achieved the L3 sample cut-off value for strains. Of 44 laboratories that performed serological diagnosis, 13 achieved the L3 sample cut-off value for antisera.

Over 82% of the laboratories answered correctly more than 70% of the samples sent, as shown by the black bars in Figure 3.

Figure 3: Distribution of the scores obtained by the laboratories according to the level of complexity of the samples (L1: low; L2:intermediate; L3 strain: high, only strains; L3 antisera: high, only antisera; Average: global average of correct answer). 

Discussion

Argentina does not have a mandatory external quality con trol program in mycology; however, the Ministry of Health and Social Development strongly recommends National Refe- rence Laboratories to offer quality assessment programs for

national laboratories networks and to evaluate their perfor mance. In addition to the PNCCM, the National Reference Laboratory in Clinical Mycology offers continuous training, produces, and provides diagnostic reagents and reference strains for technology transfer. Through these activities, this institution aims to increase the diagnostic capacity of fungal infections in Argentina, and particularly the National Mycol ogy Laboratory Network. Professionals widely accepted the importance of quality assurance, as shown by a growing interest in PNCCM since its inception until today.

In this study, we evaluate the usefulness of the PNCCM by analyzing its effectiveness through indicators and deter- mining the performance of the laboratories. This analysis showed that, in the studied period, the number of con- trolled laboratories increased more than 4 times and the number of jurisdictions increased from 18 to 24. As a result, all jurisdictions have at least one laboratory to perform the diagnosis of fungal diseases in their area, increasing the coverage to 100% of the jurisdictions in Argentina. Qual ity surveys showed a high level of satisfaction. However, a low number of laboratories answered them. The degree of adherence was high, particularly in public laboratories that have shown an adherence of 78%. The majority of the labo- ratories have participated continuously since they enrolled in the program. On the other hand, some laboratories have left the PNCCM for reasons not related to the program, including laboratories that stopped providing services. We observed an improvement in the quality of laboratory results through time, which may be a consequence of the continu- ous training offered by the PNCCM. However, we detected a low percentage of correct answers in the identification of infrequent pathogens. It is necessary to keep training the laboratories in the identification of emerging pathogens, as was evidenced by the low percentages of correct responses (<45%) of C. gattii, W. anomalus, C. haemulloni, P lilac- inum, and S. boydii.

More than 75% of laboratories correctly identified A. fumigatus and A. niger. On the other hand, the improvement in the identification of A. terreus, an emerging pathogen of high complexity, suggests that laboratories have a good training in the resolution of this genus. Finally, the per formance of the laboratories over the period 1996-2018 showed that 82% of them correctly identified over 70% of the specimens. This analysis evidenced which laboratories should be trained. Unfortunately, few reports about this sub- ject are available and it was not possible to compare the effectiveness of the PNCCM with other proficiency testing programs.

This study showed that the PNCCM was effective in detecting methodological or interpretation errors, identi- fying the needs for training and evaluating the performance of the laboratories. The PNCCM created uniform quality standards for all laboratory tests to ensure the accuracy, reliability, and timeliness of patient test results, regardless of the laboratory involved. These requirements depend on the assay and the complexity of the laboratory.

Finally, in agreement with Stan et al.¹¹, we believe that a proficiency testing program is a valuable tool for assessing laboratory performance as well as for improving the quality of the results.

Conflict of interest

The authors declare that they have no conflicts of interest.

Acknowledgements

The present study has not received specific aid from agencies from the public sector, commercial sector or non-profit entities.

We want to thank all the participants that voluntarily joined the PNCCM.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at