Exploring information superiority: a methodology for measuring the quality of information and its impact on shared awareness / Walter Perry, David Signori.
Table of contents
- The significance of management information systems for enhancing strategic and tactical planning
- Services on Demand
- Evaluating the Quality of Medical Care
- Access Check
The significance of management information systems for enhancing strategic and tactical planning. Ahlia University, Manama, Kingdom of Bahrain. Management Information Systems MIS is the key factor to facilitate and attain efficient decision making in an organization.
This research explores the extent to which management information systems implemented to make successful decisions at two selected financial organizations. The research examined whether the selected financial institutions of Bahrain vary as to the use of Management Information Systems leadership of decision making for strategic and tactical planning purposes. The research adapted the quantitative research design to examine two research hypotheses.
A total of forms were equally distributed to those who are working at different management levels at the selected organizations. The results of the research showed that MIS was primarily used to enhance strategic planning in both financial institutions. The regression analysis revealed that Tactical planning is found to have no effect on Decision Making, while Strategic planning has a clear effect on the Decision Making Effectiveness in both organizations.
Currently, organizations are in the race for enhancing their capability in order to survive in the competitions of the new century global market. Therefore, organizations are attempting to advance their agility level by improving the decision making process to be more efficient and highly effective to meet the successive fluctuations of the market. In an effort to achieve this, many modern organizations, either mid or large sized, have concerned with a cycle of progressive investments in and adopted new management information systems components.
During last decade, a high percentage of financial organizations frequently used Management Information Systems to facilitate the provision of services; and that the speed of the adoption is expected to grow further as the technology expands. In a Bank's information system, there is always a potential crisis which makes the bank endure an insufficiency; thus, an advanced information system supported by a superior mechanism control is required to make certain that an information system has achieved the required processes.
If the relevant information required in a decision-making process or an organization planning is not available at the appropriate time, then there is a good change to be a poor organization planning, inappropriate decision-making, poor priority of needs, and defective programming or scheduling of activities Adebayo, Information is essential for the endurance of a financial organization in the global and competitive market.
The nature of globalization and competitiveness in the market stress on the importance of developing an organization capability through better enhancing MIS. Accordingly, the stored information must then be recalled and distributed for the use of an organization leadership and top management as well as mid-level managers to take effective long term strategic and short term Tactical decision-making.
MIS is deemed to be a system which provides organizations top management and, even lower level management, with appropriate information based on data from both internal and external sources, to allow them to make effective and timely decisions that best achieve their organization goals and satisfy stakeholder requirements Argyris, , p. The conception of information catches the attention of different professionals from different fields such as computer science, economics, business and management, political science, statistics, communication and information studies Newman However, the question is "what type of information"?
How Information management can play an essential role in the decision making process? This paper focuses on how information management is needed to generate proper planning and then decisions at both strategic and tactical levels in the two selected financial organizations. The process of dealing with the financial institutions was tainted by a lot of sensitivity, because of the refusal of those institutions to reveal their decision-making mechanism due to their Disclosure Rules.
So we decided to call the first selected institutions case one and case two referring to the second selected organizations. Few authors have explored that the critical information required by midlevel and strategic level management is efficiently provided by MIS. A small amount of research has deliberated that the limitations and deficiencies in the process of management information system performance are the main reason for diminishing the efficiency of decision-making process in the organization Fabunmi, ; Knight Moore, The questions related to what extent the managing of these information systems assists different decisions at different management levels and the type of responsibility of the financial institution's senior and tactical management in enhancing the management information has been raised with low empirically investigation and examination.
The purpose of this research is to explore the extent to which management information systems are used to make effective decisions of long and short term planning in two financial organizations at the Kingdom of Bahrain. The study will examine whether the government financial institution Case one and the Private financial institution Case two differ as to the use of management information systems for leadership decision makes in short and long term planning.
This paper aims to evaluate the impact of current MIS models being developed at the selected organizations, and how far they practice this concept in order to enhance their tactical and strategic planning. The remainder of this paper is organized as follows. Sections 2 and 3 discuss the literature review and research methods. In Section 4, we present results and analysis.
In section 5, a discussion will be presented. Finally, conclusion and recommendations are presented in sections 6 and 7, respectively. There is a lof of research on the approaches, techniques and technologies for the design and development of MIS. However, there are a few articles that cover the impact of Management Information Systems on planning strategies and decision making. While there are no universally accepted definitions of MIS and those that exist in literatures are just prejudices of the researchers Adeoti-Adekeye, Lee, defined MIS as " a system or process that provides information needed to manage organizations effectively".
Additionally, Baskerville and Myers broadly define MIS as " the development, use and application of information systems by individuals, organizations and society". In his study, Becta describes an information system as " a system consisting of the network of all communication channels used within an organization ". In their study, Laudon and Laudon have defined MIS as "the study of information systems focusing on their use in business and management". The abovementioned definitions showed that MIS has underlined the development, application and validation of relevant theories and models in attempts to encourage quality work in the area.
Referring to the literatures, the field of Management Information Systems MIS has had a variegated development in its relatively short life span. MIS has developed its own theme of research and studies Baskerville and Myers, Tracing previous literatures, we can report that during its first few decades, MIS concentrated on the information in the context of: Only during the last two decades, the MIS field has shifted to the primary, considered the second type of communication, namely, instruction-based.
This has become known as the domain of expert systems Sasan Rahmatian, In attempts to review published studies on MIS and articles, Alavi and Carlson have identified popular research topics, the dominant research perspective, and the relationship between MIS research and practice. In contrast, Baskerville and Myers have examined the MIS field and found a constant shift of MIS research from a technical focus to a technology-organizational and management-social focus.
Skyrius underlines the decision maker's attitudes towards different factors influencing the quality of business decisions; these factors include information sources, analytical tools, and the role of information technologies. Handzic also pays attention to the impact of information availability on people's ability to process and use information in short and long term planning and in decision making tasks. He revealed that the better the availability of information, the better the impact on both efficiency and accuracy of business decisions.
Liu and Young talk about key information models and their relationships in business decision support in three different scenarios. In order to improve the financial organizational capability and enhance its level of competition in the market, financial organizations should understand the dimensions of the Information Management, and clearly define and develop the resources in case of human, technological, and internal operations, among others,, and manage them well across the organizational boundaries.
However, establishing the link between Information System Management, planning and decision making is, at best, tricky. In an article by Shu and Strassmann , a survey was conducted at 12 banks in the US between and They noticed that even though Information Technology had been one of the most essentially dynamic factors relating all efforts, it could not improve banks' earnings. However, conversely, there are many literatures approving the positive impacts of Information Technology expenses on business value.
Kozak investigates the influence of the evolution in Information Technology on the profit and cost effectiveness of the banking zone during the period between and The study indicates an optimistic relationship among the executed Information Technology, productivity and cost savings. MIS enables the exchange of experiences, which transfers the required information to the management levels to sustain competitive advantage since it affects the decision making to improve the quality of services provided. Therefore, Barachini et al.
Management Information System will give the banking management a new dimension in managing its knowledge and help in carrying out and maximizing the management's initiatives in harmonizing the appropriate strategies in the short and long planning Edmondson, In his study, Obi suggested that MIS is indispensible in the area of decision-making as it can monitor by itself the instability in a system, verify a course of action and take action to keep the system in control.
Literatures also suggested that non-programmed decisions are relevant as they provide support by supplying information to the search, the analysis, the evaluation and the choice and implementation process of decision making.
The significance of management information systems for enhancing strategic and tactical planning
More recently, Adebayo explained that the existence of MIS is needed to improve and enhance decision making on the issues affecting human and material resources. From the literatures presented, we can easily perceive that the importance of the role of both middle and top management to maintain a consistent approach to develop, use, and evaluate MIS systems within the institution. To financial institutions, MIS is used at various levels by top-management, middle and even by the operational staff as a support for decision making that aims to meet strategic goals and strategic objectives.
The above literatures also explore the importance of MIS in providing decision makers with facts, which consequently support and enhance the entire decision-making process. Furthermore, at the most senior level, MIS and DSS supply the data and required information to assist the board of directors and management levels to make an accurate and on time strategic decisions. The current study attempts to explain the relationship between various factors.
Due to the nature of the current study and its hypothesis, the primary research purpose of the current study is, thus, explanatory. Explanatory or causal explains the complexity of the interrelated variables identified that were posited in the hypothesis and research. By developing several hypotheses, the study thus adapt the quantitative research design to better test those hypotheses. Quantitative research uses survey as the main instrument to collect data.
To achieve the purpose of the current study, the following research questions have been formulated: To what extent is MIS being utilized to support Strategic planning for decisions in Bahrain's financial organizations? To what extent is MIS being utilized to support Tactical planning for decisions in Bahrain's financial organizations?
To answer these questions, the current study carries out various hypotheses that developed from previous literatures and studies Ajayi et. Figure 1 presents the proposed model and factors affecting the process of decision making. Thus, we consider the hypothesis below: The Tactical Planning short term generated by MIS is positively affecting the decision making process. The Strategic Planning long term generated by MIS is positively affecting the decision making process.
The participants were asked to indicate their perception on a likert scales with response ranging from "strongly disagree" to "strongly agree". The collected data were analyzed based on correlation and regression analyses using the statistical package for social sciences SPSS. The questionnaire of this study is adapted from previous literature and studies e. The main reason why we need to refer to literatures when developing a questionnaire is to ensure the high reliability and validity of the survey.
The questionnaire we prepared for this paper was divided into 2 sections. In the second section we were interested in gathering information about the importance of MIS and its use in Case one and Case two of financial institutions in Bahrain. In the first two instances representative samples of potential providers or recipients are required, as well as representative samples of care provided or received.
In the third instance a representative sample of providers is needed, but not necessarily a representative sample of care. A more important aspect is to select, uniformly of course, significant dimensions of care. Perhaps performance should be studied in certain clinical situations that are particularly stressful and therefore more revealing of latent capacities or weaknesses in performance.
Hypothetical test situations may even be set up to assess the capacity to perform in selected dimensions of care. By these criteria, some studies belong in one category or another, but some seem to combine features of several in such a way that generalization becomes difficult. For example, in the first study of the quality of care received by Teamster families, the findings are meant to apply only to the management of specific categories of hospitalized illness in a specified population group.
The degree of homogeneity in the universe to be sampled is, of course, a matter of great importance in any scheme of sampling or selection. The question that must be asked is to what extent the care provided by a physician maintains a consistent level. Do specific diagnostic categories, levels of difficulty or dimensions of care exist in which a physician performs better than in others? One might, similarly, ask whether the care provided by all subdivisions of an institution are at about the same level in absolute terms or in relation to performance in comparable institutions.
Services on Demand
The medical care provided to one or another individual is valid evidence of quality and there should be little or no chance variation which is affected by adjusting the size of the sample. The empirical evidence concerning homogeneity is not extensive. Both the Peterson and Clute studies of general practice 18 , 19 showed a high degree of correlation between performance of physicians in different components or dimensions of care history, physical examination, treatment, etc.
Rosenfeld demonstrated that the differences in quality ratings among several diagnoses selected within each area of practice medicine, surgery and obstetrics-gynecology were not large. Although the differences among hospitals by area of practice appeared by inspection to be larger, they were not large enough to alter the rankings of the three hospitals studied. The two studies of care received by Teamster families 26 , 28 arrived at almost identical proportions of optimal and less than optimal care for the entire populations studied.
This must have been coincidental, since the percent of optimal care, in the second study, varied greatly by diagnostic category from 31 per cent for medicine to per cent for ophthalmology nine cases only. That the same factor may produce effects in two opposite directions is an indication of the complex interactions that the researcher must consider.
The earlier study consisted primarily of major abdominal surgery, whereas this randomly selected group contained few such cases and had more patients with minor conditions. Sampling and selection influence, and are influenced by, a number of considerations in addition to generalization and homogeneity. The specific dimensions of care that interest one preventive management or surgical technique, to mention two rather different examples may dictate the selection of medical care situations for evaluation.
The situations chosen are also related to the nature of the criteria and standards used and of the rating and scoring system adopted. Attempts to sample problem situations, rather than traditional diagnoses or operations, can be very difficult, because of the manner in which clinical records are filed and indexed. This is unfortunate, because a review of operations or established diagnoses gives an insight into the bases upon which the diagnosis was made or the operation performed.
It leaves unexplored a complementary segment of practice, namely the situations in which a similar diagnosis or treatment may have been indicated but not made or performed. Measurement depends on the development of standards. In the assessment of quality standards derive from two sources. Empirical standards are derived from actual practice and are generally used to compare medical care in one setting with that in another, or with statistical averages and ranges obtained from a larger number of similar settings. The Professional Activities Study is based, in part, on this approach.
Empirical standards rest on demonstrably attainable levels of care and, for that reason, enjoy a certain degree of credibility and acceptability. Moreover, without clear normative standards, empirical observations in selected settings must be made to serve the purpose. An interesting example is provided by Furstenberg et al. In using empirical standards one needs some assurance that the clinical material in the settings being compared is similar.
The Professional Activities Study makes some allowance for this by reporting patterns of care for hospitals grouped by size. The major shortcoming, however, is that care may appear to be adequate in comparison to that in other situations and yet fall short of what is attainable through the full application of current medical knowledge. Normative standards derive, in principle, from the sources that legitimately set the standards of knowledge and practice in the dominant medical care system.
In practice, they are set by standard textbooks or publications, 10 panels of physicians, 25 highly qualified practitioners who serve as judges 26 or a research staff in consultation with qualified practitioners. In any event, their distinctive characteristic is that they stem from a body of legitimate knowledge and values rather than from specific examples of actual practice.
As such, they depend for their validity on the extent of agreement concerning facts and values within the profession or, at least, among its leadership. Where equally legitimate sources differ in their views, judgments concerning quality become correspondingly ambiguous. The relevance of certain normative standards, developed by one group, to the field of practice of another group, has been questioned. For example, Peterson and Barsamian report that although spermatic fluid examination of the husband should precede surgery for the Stein-Leventhal syndrome, not one instance of such examination was noted, and that this requirement was dropped from the criteria for assessment.
The major studies of general practice have made allowances for this. Some researchers have used both types of standards, normative and empirical, in the assessment of care. Rosenfeld used normative standards but included in his design a comparison between university affiliated and community hospitals. No written standards, no matter how carefully drawn, would be adequate in five years.
This factor, expressed in terms of an acceptable percent of compliance with the standard, was designed to take account of contingencies not foreseen in the standards themselves. It does, however, have the effect of being more realistically permissive as well. This is because the correction factor is likely to be made up partly of acceptable departures from the norm and partly of deviations that might be unacceptable. Standards can also be differentiated by the extent of their specificity and directiveness. At one extreme the assessing physician may be very simply instructed as follows: Highly precise and directive standards are associated with the selection of specific diagnostic categories for assessment.
When a representative sample of all the care provided is to be assessed, little more than general guides can be given to the assessor. Lembcke, who has stressed the need for specific criteria, has had to develop a correspondingly detailed diagnostic classification of pelvic surgery, for example. Certain diagnoses, such as surgical operations, lend themselves more readily to this approach.
This is evident in Lembcke's attempt to extend his system of audits to nonsurgical diagnoses. The data abstracted under each diagnostic rubric are more like descriptions of patterns of management, with insufficient normative criteria for decisive evaluation. The alternative adopted is comparison with a criterion institution. Obviously, the more general and nondirective the standards are, the more one must depend on the interpretations and norms of the person entrusted with the actual assessment of care.
With greater specificity, the research team is able, collectively, to exercise much greater control over what dimensions of care require emphasis and what the acceptable standards are. When standards are not very specific and the assessor must exercise his own judgment in arriving at an evaluation, very expert and careful judges must be used. Lembcke claims that a much more precise and directive system such as his does not require expert judges. The same is true, and to about the same extent, of the medical audit using objective criteria; anyone who knows enough medical terminology to understand the definitions and criteria can prepare the case abstracts and tables for the medical audit.
However, the final acceptance, interpretation and application of the findings must be the responsibility of a physician or group of physicians. The dimensions of care and the values that one uses to judge them are, of course, embodied in the criteria and standards used to assess care. The dimensions selected and the value judgments attached to them constitute the operationalized definition of quality in each study.
The preselection of dimensions makes possible, as already pointed out, the development of precise procedures, standards and criteria. Lembcke 10 has put much stress on the need for selecting a few specific dimensions of care within specified diagnostic categories rather than attempting general evaluations of unspecified dimensions which, he feels, lack precision. He uses dimensions such as the following: Within each dimension, and for each diagnostic category, one or more previously defined activities are often used to characterize performance for that dimension as a whole.
Examples are the compatibility of the diagnosis of pancreatitis with serum amylase levels or of liver cirrhosis with biopsy findings, the performance of sensitivity tests prior to antibiotic therapy in acute bronchitis, and the control of blood sugar levels in diabetes. In addition to the extent to which preselection of dimensions takes place, assessments of quality differ with respect to the number of dimensions used and the exhaustiveness with which performance in each dimension is explored.
Evaluating the Quality of Medical Care
For example, Peterson et al. Peterson and Barsamian, 38 , 39 on the other hand, concentrate on two basic dimensions, justification of diagnosis and of therapy, but require complete proof of justification. A much more simplified approach is illustrated by Huntley et al. Judgments of quality are incomplete when only a few dimensions are used and decisions about each dimension are made on the basis of partial evidence.
Some dimensions, such as preventive care or the psychological and social management of health and illness, are often excluded from the definition of quality and the standards and criteria that make it operational. Examples are the intentional exclusion of psychiatric care from the Peterson study 18 and the planned exclusion of the patient-physician relationship and the attitudes of physicians in studies of the quality of care in the Health Insurance Plan of Greater New York.
In the absence of specific instructions to the judges, the study by Morehead et al. Another characteristic of measurement is the level at which the standard is set. The ability to discriminate different levels of performance depends on the scale of measurement used. Other studies assign scores to performance of specified components of care and cumulate these to obtain a numerical index usually ranging from 0— These practices raise questions relative to scales of measurement and legitimate operations on these scales. Some of these are described below.
Those who adhere to the first practice point out that any greater degree of precision is not possible with present methods. Some have even reduced the categories to only two: Clute 19 uses three, of which the middle one is acknowledged to be doubtful or indeterminate. Also, medical care has an all-or-none aspect that the usual numerical scores do not reflect. Care can be good in many of its parts and be disastrously inadequate in the aggregate due to a vital error in one component.
This is, of course, less often a problem if it is demonstrated that performance on different components of care is highly intercorrelated. Those who have used numerical scores have pointed out much loss of information in the use of overall judgments, 38 and that numerical scores, cumulated from specified subscores, give a picture not only of the whole but also of the evaluation of individual parts. Rosenfeld 22 has handled this problem by using a system of assigning qualitative scores to component parts of care and an overall qualitative score based on arbitrary rules of combination that allow for the all-or-none attribute of the quality of medical care.
As already pointed out, a high degree of agreement was found between intuitive and structured ratings in the Rosenfeld study 22 and between qualitative and quantitative ratings in the study by Peterson et al. A major problem, yet unsolved, in the construction of numerical scores, is the manner in which the different components are to be weighted in the process of arriving at the total. At present this is an arbitrary matter. Daily and Morehead 24 assign different weights as follows: Furthermore, therapy is in the process of constant change, while the form of history and physical examination has changed very little over the years.
The problem of seeking external confirmation remains. The problem of weights is related to the more general problem of value of items of information or of procedures in the medical care process. A problem in the interpretation of numerical scores is the meaning of the numerical interval between points on the scale. Numerical scores derived for the assessment of quality are not likely to have the property of equal intervals. They should not be used as if they had.
- Contemporary Theatres in Europe: A Critical Companion;
- Voyage au Pays des Chiffres (FICTION) (French Edition);
- Bobby and the Drug Store Robbery!
- Evaluating the Quality of Medical Care!
- Definition of Quality!
The reliability of assessments is a major consideration in studies of quality, where so much depends on judgment even when the directive types of standards are used. Several studies have given some attention to agreement between judges. The impression gained is that this is considered to be at an acceptable level. The differences in the scores of the family physicians based on these separate ratings did not exceed 7 per cent. These indicate a fair amount of agreement, but a precise evaluation is difficult since no other investigator is known to have used these same measures.
This was raised to 92 per cent following reevaluation of disagreements by the two judges. By contrast to between-judge reliability, very little has been reported about the reliability of repeated judgments of quality made by the same person. To test within-observer variation, Peterson et al.
The level of agreement was lower within observers than between observers, partly because revisits lasted a shorter period of time and related, therefore, to a smaller sample of practice. The major mechanism for achieving higher levels of reliability is the detailed specification of criteria, standards and procedures used for the assessment of care. Striving for reproducibility was, in fact, a major impetus in the development of the more rigorous rating systems by Lembcke, and by Peterson and Barsarmian.
Unfortunately, no comparative studies of reliability exist using highly directive versus nondirective methods of assessment. Unreported data by Morehead et al. The partial data that have been published suggest that the post-review reliability achieved by Morehead et al. It is their unanimous opinion that it is as important for the surveyors to have flexibility in the judgment of an individual case as it is for a competent physician when confronting a clinical problem in a given patient.
The reasons for disagreement between judges throw some light on the problems of evaluation and the prospects of achieving greater reliability. In another quarter differences developed around questions of fact, because one consultant missed a significant item of information in the record. It would therefore appear that with revised standards, and improved methods of orienting consultants, a substantially higher degree of agreement could be achieved. This is a function of ambiguity in the medical care system and sets an upper limit of reproducibility.
Of the small number of unresolved disagreements eight per cent of all admissions and 36 per cent of initial disagreements more than half were due to honest differences of opinion regarding the clinical handling of the problem. The remainder arose out of differences in interpreting inadequate records, or the technical problems of where to assess unsatisfactory care in a series of admissions. A final aspect of reliability is the occasional breakdown in the performance of an assessor, as so dramatically demonstrated in the Rosenfeld study. When several observers or judges describe and evaluate the process of medical care, one of them may consistently employ more rigid standards than another, or interpret predetermined standards more strictly.
Rosenfeld 22 showed that, of two assessors, one regularly awarded lower ratings to the same cases assessed by both. An examination of individual cases of disagreement in the study by Morehead et al. For surgical cases, one surveyor rated the care lower than the other in all eight instances of disagreement. The impression is gained from examining reasons for disagreement on medical cases that one of the judges had a special interest in cardiology and was more demanding of clarity and certainty in the management of cardiac cases.
The clear indication of these findings is that bias must be accepted as the rule rather than the exception, and that studies of quality must be designed with this in mind. In the Rosenfeld study, 22 for example, either of the two raters used for each area of practice would have ranked the several hospitals in the same order, even though one was consistently more generous than the other.
The Clute study of general practice in Canada, 19 on the other hand, has been criticized for comparing the quality of care in two geographic areas even though different observers examined the care in the two areas in question. Predetermined order or regularity in the process of study may be associated with bias. Therefore, some carefully planned procedures may have to be introduced into the research design for randomization. The study by Peterson et al. Another important source of bias is knowledge, by the assessor, of the identity of the physician who provided the care or of the hospital in which care was given.
The question of removing identifying features from charts under review has been raised, 3 but little is known about the feasibility of this procedure and its effects on the ratings assigned. To the extent that this is true, or suspected to be true, appropriate precautions need to be taken in the recruitment and allocation of judges.
The effectiveness of care as has been stated, in achieving or producing health and satisfaction, as defined for its individual members by a particular society or subculture, is the ultimate validator of the quality of care. The validity of all other phenomena as indicators of quality depends, ultimately, on the relationship between these phenomena and the achievement of health and satisfaction.
Nevertheless, conformity of practice to accepted standards has a kind of conditional or interim validity which may be more relevant to the purposes of assessment in specific instances. The validation of the details of medical practice by their effect on health is the particular concern of the clinical sciences. In the clinical literature one seeks data on whether penicillin promotes recovery in certain types of pneumonia, anticoagulants in coronary thrombosis, or corticosteroids in rheumatic carditis; what certain tests indicate about the function of the liver; and whether simple or radical mastectomy is the more life-prolonging procedure in given types of breast cancer.
From the general body of knowledge concerning such relationships arise the standards of practice, more or less fully validated, by which the medical care process is ordinarily judged. Intermediate, or procedural, end points often represent larger bundles of care.
Their relationship to outcome has attracted the attention of both the clinical investigator and the student of medical care organization. Some examples of the latter are studies of relationships between prenatal care and the health of mothers and infants 46 , 47 and the relationship between multiple screening examinations and subsequent health.
Many studies reviewed 18 , 19 , 23 , 26 , 28 attempt to study the relationship between structural properties and the assessment of the process of care. Several of these studies have shown, for example, a relationship between the training and qualifications of physicians and the quality of care they provide. The relationship is, however, a complex one, and is influenced by the type of training, its duration and the type of hospital within which it was obtained. The two studies of general practice 18 , 19 have shown additional positive relationships between quality and better office facilities for practice, the presence or availability of laboratory equipment, and the institution of an appointment system.
No relationship was shown between quality and membership of professional associations, the income of the physician or the presence of x-ray equipment in the office. The two studies do not agree fully on the nature of the relationship between quality of practice and whether the physician obtained his training in a teaching hospital or not, the number of hours worked or the nature of the physician's hospital affiliation. Hospital accreditation, presumably a mark of quality conferred mainly for compliance with a wide range of organizational standards, does not appear, in and of itself, to be related to the quality of care, at least in New York City.
Although structure and process are no doubt related, the few examples cited above indicate clearly the complexity and ambiguity of these relationships. This is the result partly of the many factors involved, and partly of the poorly understood interactions among these factors. For example, one could reasonably propose, based on several findings 26 , 38 that both hospital factors and physician factors influence the quality of care rendered in the hospital, but that differences between physicians are obliterated in the best and worst hospital and express themselves, in varying degrees, in hospitals of intermediate quality.
An approach particularly favored by students of medical care organization is to examine the relations between structure and outcome without reference to the complex processes that tie them together. Some examples of such studies have been cited already. This brief review indicates the kinds of evidence pertaining to the validity of the various approaches to the evaluation of quality of care. Clearly, the relationships between process and outcome, and between structure and both process and outcome, are not fully understood.
With regard to this, the requirements of validation are best expressed by the concept, already referred to, of a chain of events in which each event is an end to the one that comes before it and a necessary condition to the one that follows.
- JSTOR: Access Check;
- Panchito Puma (Spanish Edition).
- Access denied;
- Access denied | Department of Computer Science & Information Technology?
This indicates that the means-end relationship between each adjacent pair requires validation in any chain of hypothetical or real events. More commonly, as has been shown, the intervening links are ignored. The result is that causal inferences become attenuated in proportion to the distance separating the two events on the chain.
Unfortunately, very little information is available on actual assessments of quality using more than one method of evaluation concurrently. Makover has studied specifically the relationships between multifactorial assessments of structure and of process in the same medical groups. However, the exceptions were sufficiently marked, both in number and degree, to induce one to question the reliability 56 of one or the other rating method when applied to any one medical group. It would seem that further comparison of these two methods of rating is clearly indicated. Since a multidimensional assessment of medical care is a costly and laborious undertaking, the search continues for discrete, readily measurable data that can provide information about the quality of medical care.
The data used may be about aspects of structure, process or outcome. The chief requirement is that they be easily, sometimes routinely, measurable and be reasonably valid. Among the studies of quality using this approach are those of the Professional Activities Study, 36 Ciocco et al. Such indices have the advantage of convenience; but the inferences that are drawn from them may be of doubtful validity.
Myers has pointed out the many limitations of the traditional indices of the quality of hospital care, including rates of total and postoperative mortality, complications, postoperative infection, Caesarian section, consultation and removal of normal tissue at operation.
More important still, serious questions may be raised about what each index means since so many factors are involved in producing the phenomenon which it measures.
Access Check
Eislee has pointed out, on the other hand, that at least certain indices can be helpful, if used with care. The search for easy ways to measure a highly complex phenomenon such as medical care may be pursuing a will-o'-the-wisp. The use of simple indices in lieu of more complex measures may be justified by demonstrating high correlations among them. On the other hand, each index can be a measure of a dimension or ingredient of care.
Judiciously selected multiple indices may, therefore, constitute the equivalent of borings in a geological survey which yield sufficient information about the parts to permit reconstruction of the whole. The validity of inferences about the whole will depend, of course, on the extent of internal continuities in the individual or institutional practice of medicine. Some of the special difficulties in assessing the quality of ambulatory care have already been mentioned. These include the paucity of recorded information, and the prior knowledge, by the managing physician, of the patient's medical and social history.
The first of these problems has led to the use of trained observers and the second to the observation of cases for which prior knowledge is not a factor in current management. The degree of relevance to general practice of standards and strategies of care developed by hospital centered and academically oriented physicians has also been questioned. Another problem is the difficulty of defining the segment of care that may be properly the object of evaluation in ambulatory care.
For hospital care, a single admission is usually the appropriate unit. Usually the answer has been to choose an arbitrary time period to define the relevant episode of care. This review has attempted to give an impression of the various approaches and methods that have been used for evaluating the quality of medical care, and to point out certain issues and problems that these approaches and methods bring up for consideration. The methods used may easily be said to have been of doubtful value and more frequently lacking in rigor and precision. But how precise do estimates of quality have to be?
At least the better methods have been adequate for the administrative and social policy purposes that have brought them into being. The search for perfection should not blind one to the fact that present techniques of evaluating quality, crude as they are, have revealed a range of quality from outstanding to deplorable. Tools are now available for making broad judgments of this kind with considerable assurance. This degree of assurance is supported by findings, already referred to, that suggest acceptable levels of homogeneity in individual practice and of reproducibility of qualitative judgments based on a minimally structured approach to evaluation.
This is not to say that a great deal does not remain to be accomplished in developing the greater precision necessary for certain other purposes. One might begin a catalogue of needed refinements by considering the nature of the information which is the basis for judgments of quality.
More must be known about the effect of the observer on the practice being observed, as well as about the process of observation itself—its reliability and validity. Comparisons need to be made between direct observation and recorded information both with and without supplementation by interview with the managing physician. Recording agreement or disagreement is not sufficient. More detailed study is needed of the nature of, and reasons for, discrepancy in various settings.
Similarly, using abstracts of records needs to be tested against using the records themselves. The process of evaluation itself requires much further study. A great deal of effort goes into the development of criteria and standards which are presumed to lend stability and uniformity to judgments of quality; and yet this presumed effect has not been empirically demonstrated. How far explicit standardization must go before appreciable gains in reliability are realized is not known. One must also consider whether, with increasing standardization, so much loss of the ability to account for unforeseen elements in the clinical situation occurs that one obtains reliability at the cost of validity.
Assessments of the same set of records using progressively more structured standards and criteria should yield valuable information on these points. The contention that less well trained assessors using exhaustive criteria can come up with reliable and valid judgments can also be tested in this way. Attention has already been drawn, in the body of the review, to the little that is known about reliability and bias when two or more judges are compared, and about the reliability of repeated judgments of the same items of care by the same assessor.
Similarly, very little is known about the effects on reliability and validity, of certain characteristics of judges including experience, areas of special interest and personality factors. Much may be learned concerning these and related matters by making explicit the process of judging and subjecting it to careful study. This should reveal the dimensions and values used by the various judges and show how differences are resolved when two or more judges discuss their points of view. Some doubt now exists about the validity of group reconciliations in which one point of view may dominate, not necessarily because it is more valid.
What is proposed here is not only to demonstrate differences or similarities in overall judgments, but to attempt, by making explicit the thought processes of the judges, to determine how the differences and similarities arise, and how differences are resolved. In addition to defects in method, most studies of quality suffer from having adopted too narrow a definition of quality. In general, they concern themselves with the technical management of illness and pay little attention to prevention, rehabilitation, coordination and continuity of care, or handling the patient-physician relationship.
Presumably, the reason for this is that the technical requirements of management are more widely recognized and better standardized. Therefore, more complete conceptual and empirical exploration of the definition of quality is needed. Two types of efficiency might be distinguished: Logical efficiency concerns the use of information to arrive at decisions. Here the issue might be whether the information obtained by the physician is relevant or irrelevant to the clinical business to be transacted. If relevant, one might consider the degree of replication or duplication in information obtained and the extent to which it exceeds the requirements of decision making in a given situation.
If parsimony is a value in medical care, the identification of redundancy becomes an element in the evaluation of care. Economic efficiency deals with the relationships between inputs and outputs and asks whether a given output is produced at least cost. It is, of course, influenced by logical efficiency, since the accumulation of unnecessary or unused information is a costly procedure which yields no benefit. Typically it goes beyond the individual and is concerned with the social product of medical care effort. In another sense, it may have been brilliant strategy in terms of making available to the largest number of women the combined skills of a medical care team.
In addition to conceptual exploration of the meaning of quality, in terms of dimensions of care and the values attached to them, empirical studies are needed of what are the prevailing dimensions and values in relevant population groups. This is an area of research significant to medical education as well as quality. Empirical studies of the medical care process should also contribute greatly to the identification of dimensions and values to be incorporated into the definition of quality. A review of the studies of quality shows a certain discouraging repetitiousness in basic concepts, approaches and methods.
Further substantive progress, beyond refinements in methodology, is likely to come from a program of research in the medical care process itself rather than from frontal attacks on the problem of quality. This is believed to be so because, before one can make judgments about quality, one needs to understand how patients and physicians interact and how physicians function in the process of providing care.
Once the elements of process and their interrelationships are understood, one can attach value judgments to them in terms of their contributions to intermediate and ultimate goals. Assume, for example, that authoritarianism-permissiveness is one dimension of the patient-physician relationship. An empirical study may show that physicians are in fact differentiated by this attribute. One might then ask whether authoritarianism or permissiveness should be the criterion of quality. The answer could be derived from the general values of society that may endorse one or the other as the more desirable attribute in social interactions.
This is one form of quality judgment, and is perfectly valid, provided its rationale and bases are explicit. The study of the medical care process itself may however offer an alternative, and more pragmatic, approach. Assume, for the time being, that compliance with the recommendations of the physician is a goal and value in the medical care system. The value of authoritarianism or permissiveness can be determined, in part, by its contribution to compliance. Compliance is itself subject to validation by the higher order criterion of health outcomes.
The true state of affairs is likely to be more complex than the hypothetical example given. The criterion of quality may prove to be congruence with patient expectations, or a more complex adaptation to specific clinical and social situations, rather than authoritarianism or permissiveness as a predominant mode.
Also, certain goals in the medical care process may not be compatible with other goals, and one may not speak of quality in global terms but of quality in specified dimensions and for specified purposes. Assessments of quality will not, therefore, result in a summary judgment but in a complex profile, as Sheps has suggested. A large portion of research in the medical care process will, of course, deal with the manner in which physicians gather clinically relevant information, and arrive at diagnostic and therapeutic decisions.
This is not the place to present a conceptual framework for research in this portion of the medical care process. Certain specific studies may, however, be mentioned and some directions for further research indicated. Research on information gathering includes studies of the perception and interpretation of physical signs. Faulty diagnosis, as judged by comparison with a criterion, was the result of these two errors.
The work of Peterson and Barsamian 38 , 39 represents the nearest approach to a rigorous evaluation of diagnostic and therapeutic decision making. As such, it is possibly the most significant recent advance in the methods of quality assessment. But this method is based on record reviews and is almost exclusively preoccupied with the justification of diagnosis and therapy. As a result, many important dimensions of care are not included in the evaluation. Some of these are considerations of efficiency, and of styles and strategies in problem solving.
Styles and strategies in problem solving can be studied through actual observation of practice, as was done so effectively by Peterson et al. Although such test situations have certain limitations arising out of their artificiality, 64 the greater simplicity and control that they provide can be very helpful. At first sight, the student of medical care might expect to be helped by knowledge and skill developed in the general field of research in problem solving.
Unfortunately, no well developed theoretical base is available which can be exploited readily in studies of medical care. Some of the empirical studies in problem solving might however, suggest methods and ideas applicable to medical care situations. These and similar studies have identified behavioral characteristics that might be used to categorize styles in clinical management. Decision making theory may also offer conceptual tools of research in the medical care process.
Ledley and Lusted, 68 , 69 among others, have attempted to apply models based on conditional probabilities to the process of diagnosis and therapy. Peterson and Barsamian 38 , 39 decided against using probabilities in their logic systems for the very good reason that the necessary data the independent probabilities of diseases and of symptoms, and the probabilities of specified symptoms in specified diseases were not available. But Edwards et al. A basic question that has arisen frequently in this review is the degree to which performance in medical care is a homogeneous or heterogeneous phenomenon.
This was seen, for example, to be relevant to sampling, the use of indices in place of multidimensional measurements, and the construction of scales that purport to judge total performance. When this question is raised with respect to individual physicians, the object of study is the integration of various kinds of knowledge and of skills in the personality and behavior of the physician.
When it is raised with respect to institutions and social systems the factors are completely different. Here one is concerned with the formal and informal mechanisms for organizing, influencing and directing human effort in general, and the practice of medicine in particular. Research in all these areas is expected to contribute to greater sophistication in the measurement of quality. Some of the conventions accepted in this review are, in themselves, obstacles to more meaningful study of quality.
The separation of hospital and ambulatory care is also largely artificial. The units of care which are the proper objects of study include the contributions of many persons during a sequence which may include care in a variety of settings. The manner in which these sequences are defined and identified has implications for sampling, methods of obtaining information, and standards and criteria of evaluation.