A Toolkit for measuring the Quality of Leprosy Services.

 

1. INTRODUCTION

Quality of service provision may be defined in different ways by different stakeholders.  In the provision of healthcare there are two main groups to be considered, the providers and the service users.

Providers

Providers have a mandate to make healthcare services available to their target population, with the long-term goal of improving health outcomes.  Health services must be accessible to those who need them; access involves adequate coverage of a certain geographical area, without leaving underserved areas; it also involves equal and timely access to all sub-groups of the population, including women and children, ethnic, religious and lifestyle minorities, the poor and those with stigmatizing conditions, such as leprosy and HIV/AIDS.  In addition, access involves entry to basic health services (primary health care) as well as referral for specialist care.  Almost by definition, specialist care is not available everywhere, but is concentrated in centres of excellence; in a high-quality health service, provision will be made to facilitate access to such centres, as needed, by providing transport, for example.

In addition to accessibility, quality of service provision implies a level of care which exceeds certain minimum standards.  Care of the patient includes administrative matters, such as patient records, appointments, waiting times, etc.; issues of courtesy and privacy in all forms of communication; and technical matters, including the diagnosis and appropriate treatment of medical conditions.

Quality services depend on various factors.  As a foundation, the minimum acceptable standards must be defined in each area and staff must be trained to work accordingly.  Regular technical supervision should identify local problems and remedy them quickly through on-the-job training.  Medium to long-term surveillance of quality involves monitoring key indicators, including indicators of health outcomes, so that more systemic failures can be identified and corrected.  Consistently high quality care is dependent on the motivation of health staff.

Service users

When quality is viewed from the perspective of the service users, issues of access, administrative efficiency and courtesy assume greater importance, as patients are generally not in a position to judge whether all the technical aspects of their care have exceeded the appropriate standards.  However, it is clear that in the 21st century, no health service provision could be regarded as of high quality without routine provision for examining and acting upon the views of service users.


2.     INDICATORS

Indicators have two main functions:

  • firstly, to indicate progress towards objectives, which is required for reporting purposes and for advocacy
  • secondly, to indicate obstacles which may prevent the achieving of objectives, which is necessary to achieve continuing improvement of the service.

There are three types of indicators which can be considered for use in measuring quality:

2.1 Proxy indicators – single-item measurements which give clues about overall quality of the program.  A proxy indicator on its own is not reliable as an indicator of quality, as there may be serious flaws in other parts of the program not reflected in this indicator.
Proxy indicators of quality in leprosy programs include the following:

  • The treatment completion rate: this measures whether patients have attended for treatment as required; it is simple to measure and is closely correlated with cure of the disease, an important health outcome.  If the treatment completion rate is high (above 85%), it suggests that patients are receiving reasonably acceptable care, otherwise they would not attend.  Aspects of leprosy work not measured by this indicator include case-finding activities and access to the services (for example, if women or members of a certain ethnic group have reduced access to the services, they will not appear in the figures and the treatment completion rate for those that are able to attend could be very high).  The prevention and treatment of nerve damage and other complications of leprosy is also not reflected in this indicator (patients may receive MDT as prescribed, but minimal care for other problems and thus complete treatment with an unacceptable level of impairment).
  • The proportion of Disability Grades 1 & 2 amongst new cases is a well-established indicator of case-finding efficiency.  If cases are diagnosed late, a greater proportion will have impairments at the time of diagnosis.  This indicator may be unreliable if staff are not well trained in assessing disability grades, numbers with grade 1 disability being often underestimated, for example.  This indicator does not say anything about the quality of care after diagnosis, but taken together, the first two proxy indicators give a fair estimation of the quality of the program.
  • The proportion of new cases verified as correctly diagnosed has been used in specific instances in which the reliability of the diagnosis is questioned.  It is cumbersome to implement, as it requires patients to be examined by a second person, and it says little about other aspects of the program, besides diagnostic skills.  It is no longer widely used.
  • The proportion of clinics found to be without an adequate stock of MDT drugs may be important if there are supply problems.
  • Delay in detection of new cases is of interest, but as it relies on patient recall, it may not be robust enough to use as a routine indicator of performance.
  • The proportion of contacts examined could assume greater importance if contact examination becomes more widely practiced, perhaps as part of a chemoprophylaxis program.
  • Gender: equality of access is not easy to measure, but the gender balance in new case detection can be monitored.

2.2 Process indicators – these measure activities that promote quality: e.g. indicators of training and supervision activities which are carried out within the program.  By definition, process indicators do not look at outcomes, so they do not show, for example, whether the training or supervision activities lead to better care.

Process indicators in leprosy programs have been used quite widely by the Damien Foundation with good results.  Examples include:

  • The proportion of training sessions that took place among those initially planned
  • The proportion of supervision visits that took place among those initially planned
  • The existence of a checklist for supervision

Other suggested process indicators include:

  • The percentage of health workers in health facilities with registered cases who have been trained in the past 3 years
  • The percentage of health workers who have received training in counseling

A problem with using training as an indicator is that the definition of training is very broad; it may occur in a wide variety of settings (including on-the-job training) and last for almost any period of time, from a few minutes to a few weeks.  It may be formal or informal, and may or may not be evaluated by testing the trainees at the end.  Indicators that measure such a poorly defined activity are not likely to be very useful.

2.3 Direct indicators of quality - for example, measuring a health outcome in the population being served. Direct indicators of quality could include:

  • The proportion of patients who develop new/additional disability during MDT.
  • A problem with disability as an indicator is that it may not be measured consistently by health staff, and this appears to affect the reliability and completeness of current data on impairment and disability.  The WHO Disability Grade, and the related EHF-score, are however, rather simple and can be applied by anyone with the appropriate orientation.  Although routinely reported for cases at diagnosis, not many programs are reporting the disability of all cases at the end of treatment, which is necessary in order to calculate this indicator.
  • The expressed opinions of service users give a direct indication of quality from their perspective.  These can be assessed by means of exit interviews or focus group discussions.  See the Annex for further information about these methods


3.    PROPOSED INDICATORS OF QUALITY

In general, proxy indicators are well understood and are used in almost all programs to a greater or lesser extent.  Process indicators and direct measures of quality are less well known and are rarely used at present.

The top priority is to promote the three most important indicators of quality which are included in the WHO Enhanced Global Strategy and Operational Guidelines (2011 – 2015); considerable effort will be required to get them reported reliably and comprehensively:

  • The treatment completion rate
  • The proportion of Disability Grade 2 amongst new cases
  • The proportion of patients who develop new disability during MDT


There are two more indicators which will give very useful information about quality without being too onerous to collect every year:

  • Assessing the views of a sample of patients
  • The existence of a checklist for supervision


4.    PROCEDURES FOR USING THE FIVE INDICATORS OF QUALITY

4.1 The treatment completion rate
The method for calculating the treatment completion rate is explained in the 2011-2015 Operational Guidelines (pages 51-52)

4.2 The proportion of Disability Grade 2 amongst new cases
This is also covered in the 2011-2015 Operational Guidelines (page 53; see also pages 22-25)

4.3 The proportion of patients who develop new disability during MDT
This is described in the 2011-2015 Operational Guidelines (pages 54-55)

4.4 Assessing patients’ views through regular exit interviews or focus group discussions:
see Annex for further information.
The format of this indicator could be as a process indicator (e.g. the number of patients involved in either type of assessment, per year), or as an outcome indicator (e.g. the percentage of patients who were satisfied with the services provided).

4.5 The existence of a checklist for supervision
Each supervisor should have available a copy of a checklist for use on visits to a leprosy clinic.


5.    CONCLUSIONS

In conclusion, the careful, regular measurement and reporting of five simple indicators would give a very good indication of the quality of a leprosy control program.  These would encompass several aspects of the provision of services and some indication of user satisfaction.

The first priority is to implement the three indicators which are included in the current Global Strategy (2011 – 2015).  Of the new indicators proposed, the most significant will be the involvement of patients through exit interviews or focus group discussions.  The involvement of even a small sample of patients through this process is likely to give very useful feedback and allow the quality of service provision to be significantly improved.


ANNEX: ASSESSING PATIENTS’ VIEWS

A. FOCUS GROUP RESEARCH METHOD

The Focus Group research method is considered relevant for surveying users’ views in primary health care. Research shows ‘that users do have clear expectations, make active judgements of providers and are willing to share these views1.’  Focus Groups are structured group interviews. A Focus Group consists of six to eight participants who all have a similar background, for example, mothers of young children, or adult men with leprosy, etc. Different groups are formed to gather the views of different segments of the community. A trained moderator works from a pre-determined set of discussion topics to guide the interview while the group discusses the topics raised. The discussion is recorded and comments by group members during the discussion are the essential data for analysis.

Ethics

The basic goal in using Focus Groups is to hear the views of the members. Groups that limit the opportunity of participants to present their own feelings, opinions, and experiences are counter to this goal. For example, groups that cross authority lines are certain to make lower-ranking participants feel uncomfortable. Therefore, when recruiting a Focus Group, it is important to consider the social, economic and ethnic background of the participants.

Privacy is the central ethical concern in Focus Group research. Focus Group projects should promise confidentiality, which can be achieved through the following steps:

  • Only the researcher will have access to any of the recruitment information, and these records will be destroyed at the conclusion of the project.
  • During the discussion, participants will be identified only by first names or pseudonyms.
  • Only the researcher will have access to the recordings that were made, and these will be destroyed at the conclusion of the project.
  • In any transcripts that are made from the recordings, all names and any other potentially identifying information (e.g. mention of specific individuals, events or places) will be either removed or modified.

Analysis

The recording or transcript is examined systematically in order to create a report which conveys both the consensual and minority views that were expressed, and gives realistic and relevant conclusions and recommendations.

Advantages

  • Bridges the social, professional or other distance between researchers and the target audience (health practitioners, community leaders or patients).
  • Allows the investigation of complex behaviour and motivation, while providing a better understanding of the diversity of experience and thought.
  • Friendly and respectful research method.
  • Focus Group participants have the benefit of distance, in terms of time and space, from the health care service.

Limitations

  • The method invites participants to provide dramatic accounts of what may be a banal everyday reality. (For instance, experience of dehumanization may be reflected in the Focus Groups in form of ‘atrocity’ stories1.)
  • The outcome of the Focus Group session depends on the quality of the group facilitator (For instance, there is ‘a danger that a poor facilitator will ask leading questions which suggest they want certain answers, or there may also be a tendency for the statements of the more vocal participants to take on the weight of consensus and for dissonant views to be censored’1)
  • Eloquence of certain participants may lead researchers to select parts of a transcript as summing up a community’s opinion and ignoring contrary views and opinions.
  • Focus Group data were most valuable not in comparing sites, but in identifying the aspects of health care which matter most to users of health services.

B. EXIT INTERVIEW METHODOLOGY

In an Exit Interview, a researcher discusses various topics of interest with a person who has just completed a visit to a health facility.  The interview may be either informal or semi-structured.  The aim is to measure two principal dimensions of quality of care for individual patients: access and patient satisfaction.  Immediate recall of health education messages can also be tested using this method.

Purposes

  • To develop a profile of patients and patterns service of utilization
  • Evaluate rational drug prescribing
  • Obtain patient views on the service and the care received.

Advantages

  • Exit Interviews provide immediate and spontaneous views, and may give specific details that would otherwise be forgotten.
  • Exit Interviews are more immediate and spontaneous than Focus Group discussions, and can be useful in examining researcher-defined issues of care, such as:
    • Specific details of accessibility and delay
    • Specific details of the procedures which were undertaken in the clinic, such as the clinical examination, tests performed and advice given, etc.
    • Immediate evaluation as to how helpful the health visit is perceived

Limitations

  • Exit Interviews are ‘face to face’ interviews and therefore provide only limited access to the patient’s opinion.  Many opinions and criticisms may remain unstated. The lack of time and distance may lead to biased judgments.
  • In a ‘one-to-one’ interview it may be more difficult for interviewees to disclose negative views, whereas in a meeting with other service users, it can be less threatening for participants if such views come from the group, rather than from one dissatisfied individual.
  • Exit Interviews show little variability and may bear little relationship to the findings of the Focus Groups; however, they can be useful in assessing researcher - defined interpersonal dimensions of care.

Ethics

  • the right not to be interviewed should be made clear
  • the interview should not be done by the health provider
  • the anonymity of the patient should be secured

Conclusions

  • The power relations operating in Exit Interviews and Focus Groups are very different.
  • The data obtained through the methods differ substantially.
  • In order to establish the views of a user population, a combination of both methods is required.

References

1  Schneider, H. and Palmer, N. 2002, ‘Getting to the truth? Researching user views of primary health care’, Health Policy and Planning, 17; 32-41.

Further reading

General framework literature for the Focus Group method:
The Focus Group Guidebook, David L. Morgan, Focus Group Kit 1,  Sage Publication, Thousand Oaks,London, New Delhi.1998.

General framework literature on Exit Interviews:
Qualitative Methods for Health Research, Judith Green & Nicki Thorogood, Los Angeles, London, New       Delhi, Singapore, Washington DC:  Sage, 2009.

S. M. Campbell , M. O. Roland and S. A. Buetow, ‘Defining quality of care’, Social Science & Medicine, Volume 51, Issue 11, December 2000, Pages 1611-1625.

Henbest RJ and Fehrsen GS.1992. ‘Patient -centredness: is it applicable outside the West? Its measurement and effect on outcome.’ Family Practice 9; 311-17.
ILEP TECHNICAL BULLETIN: Issue No. 17, August 2010

Marieke van Dijk,  Royal Tropical Institute, Clients’ Perspective – Guidelines for assessing the quality of leprosy services from the clients’ point of view. 2nd Edition January 2010


ILEP is a Federation of autonomous anti-leprosy associations. The text contained in this Bulletin is not binding on ILEP Members.

The text can be freely quoted subject to acknowledgement of its source.