Summary:
“Do not believe anything you read in the patient’s chart - it is full of lies”. This was the first lesson taught to new medical students by our hospital’s most esteemed diagnostician. I have found this assessment generally to be true - both for paper and electronic medical records.
Often, those times when one gets ‘burned’ in medicine, is when they have relied on something they read in a patient’s chart…which turned out to not be true.
However, in theory, we should be able to remedy this problem with electronic medical records.
This post is written across two parts,
Part 1: (i) how do we determine if data in the past medical history is true, (ii) how fake data enters the medical chart, and (iii) in practice, how do clinicians verify the truth of medical data?
Part 2 proposes ways to help verify medical diagnosis. Unfortunately, the solutions are very high level and abstract. For more prescriptive answers to this problem, I hope that others who have studied this write in their solutions: gschmidt@medmb.ca @_GregSchmidt
1. Is this true?
When trying to judge the truthfulness of a fact in the medical chart there are two questions:
Q1) What is the evidence that supports (or refutes) the fact?
Q2) What is the strength of that evidence?
In philosophy you may say… P1 + P2 + P3 = C1
Q1) What are the premises that support the conclusion (C)?
Q2) What is the strength of each premises (P)?
Let us look at two examples where the conclusion is an issue in the patient’s Past Medical History. These two examples were chosen because they are common and reasonably straight forward from a diagnostic perspective (meaning the diagnostic criteria rely on ‘objective bloodwork’ rather than clinical opinion or expertise).
1.1.eg. Past Medical History: “HIV”
Q1a. If someone write in the problem list, “HIV Positive”, what is the evidence for this? Was this fact linked to a walk-in point of care HIV test? Such a test may have a high false positive rate. Therefore the truth the patient is “HIV Positive” is actually not high. However, if this was linked to a HIV Viral load result that was elevated, that would certainly support this claim. Two elevated viral load results for the same patient would provide even higher level of support the patient in fact has HIV (if only one was recorded, perhaps it was a clerical mistake). Linking it to an undetectable HIV viral load result may not ‘negate’ this claim, as perhaps the result is undetectable because therapy is effective.
If the linked ‘support’ for the “HIV Positive” status is the medications the patient is on, that may provide a high level of support for the diagnosis. However, we must keep in mind this is a surrogate marker to infer a diagnosis, than a primary diagnostic marker of the condition.
Q2) But then, the second question is, what is the quality of evidence that supports this claim? Did that lab test come from patient recall? Was it manually entered into the medical record? Was it entered the day of the test? or 6 months later? Was the entered result double checked by another clinician? Was it ‘auto-captured’ using an OCR technology? Did the test come directly from the analyzer? All of this impacts the strength of the evidence itself that is being used to support the conclusion.
Precisely how the evidence supports the conclusion, and precisely how the strength of that evidence affects the outcomes is a matter of debate for another time. But as you can see, the process is actually a bit tricky.
eg1b) Alternatively, If the Past Medical History issue was instead written as “Positive rapid HIV test, November 4, 2017”, this claim is supported fully by evidence of a walk in point of care test. This example shows, one of the easiest ways to simplify determining if something is true, is to be more specific in what we are trying to quantify.
In this way, there are different levels of claims
- Positive rapid HIV Test, November 4, 2017
- Positive HIV western blot, November 4, 2017
- Positive HIV p24 Ag, Nov 4, 2017
- HIV Positive with elevated viral load, November 10, 2017
- HIV Positive, first diagnosed November 4, 2017
- HIV Positive, with supressed viral load, December 30, 2018
Each of these claims requires a different level of evidence. As they are more specific than “HIV Positive”, it makes it easy to know what level of evidence is required to support that claim. Similarly, to state “On HIV antiviral medications”, is a different claim and requires a different level of support than to state, “HIV Positive”. One may have a high certainty a patient has a “positive rapid HIV test”, but with that level of evidence, only moderate certainty they are “HIV Positive”.
1.2.eg. Past Medical History: “Diabetes: Type 2”
(Q1) The chart states the patient has diabetes. To determine the level of certainty of this claim, what evidence is being used? Is it a lab result of a patient’s hemoglobin A1C at 10%. Or did the clinician infer that because the patient takes the medication Metformin, they have diabetes?
(Q2) Where does the lab data of the A1C of 10% come from? Was that lab directly imported from the analyzer? entered by hand? scanned in? recalled by the patient? What about the medication “Metformin” - is that on a list of medications the patient’s spouse pulled out of their wallet dated from 7 years ago? is it a list imported from their physician’s EHR, a list from their pharmacy of dispensed medications? the pill bottles brought in by the patient? In each of these scenarios the evidence has different levels of certainty; and support different conclusions with different weight.
Even in a case as straightforward as this, there are still problems in trying to determine the certainty of these statements. How recent is that test? Was it a single measurement? What if the A1C was elevated because of drug interactions, certain anemias, or genetic conditions by the patient?
Perhaps the strength of evidence the patient is on Metformin is high. But that drug was being used for something other than diabetes, such as to assist with conception? This shows we ideally require both strong evidence attached to support a strong conclusion (the premises each must be strong, and they must link properly to the conclusion).
A comment: it is important to remember - displaying if something is internally true within the record (the patient is on metformin), remains a separate task from displaying if the record itself is actually a reflection of reality (the patient has diabetes).
As we can see, trying to display the level of certainty beside data in the medical record is complicated. But how does fake data enter into the Past Medical History today?
2. How fake data enters the past medical history
It is actually rather easy. And this happens both on paper and digital charts.
The typical sequence is that a clinician writes in their note under the list of conditions in ‘Past Medical History’ a word such as ‘COPD’ (chronic obstructive lung disease). Then, going forward in that hospital admission other clinicians copy the initial Past Medical History list written by the first clinician. Subsequent clinicians may add to it, but they may not go back and re-verify from primary material every item listed. Perhaps, only years later when the patient presents with a respiratory condition, does someone dig deeper into the Past Medical History and discover no evidence to support the claim of COPD. The patient never had it, only the chart did.
The same could happen if someone writes the patient has a history of depression, or hypertension, or a stroke in 2003.
Below is a list of the major sections in a medical chart. Any one of these can contain fake data:
- Past medical history & surgical history
- Allergies & medications (current and past)
- Consultant notes, patient symptom history / story
- Family medical history
- Social history, such as risk factors, exposures, lifestyle, and economic factors
- Results: labs, imaging, other tests and investigations
- Clinician impression, diagnosis, and plan
Each of these different sections may generate fake data by different means. In this article, let us focus primarily on the Past Medical History. Though these concepts can be generalized to the other sections as well.
3. How do we determine truth in medical records today?
Determining what is true (both in the conclusions, as well as assessing the quality of data) seems really difficult. How do clinicians do this?
3.1 Manual fact check
Each clinician has to fact check the chart themselves. Based on the time they have, and how critical that piece of information is to their work, an individual clinician must trace back each fact in the chart to original source documents. Source documents may include the patient themselves re-answering questions, finding original copies of labs tests, and contacting other offices to fax old consult letters and documentation. It is a time consuming process.
Some clinicians may fact check a majority of the chart (and often find large mistakes in the written record). Other clinicians rarely fact check the written record (and likely don’t even realize they’ve screwed up when treating the patient).
The criteria that physicians use when reviewing this data is unique to the individual, their training, their knowledge and expertise; as well as how busy they are, how interested in the case they are, how important the patient is, and how ‘crucial’ trying to get to the bottom of the issue is.
Although diagnostic guidelines and criteria exist, it seems more often precisely what criteria that physician is using, at that time, for that patient, may be quite variable.
Therefore to conclude: determining what is true in the medical record is a time consuming and complex task, that is prone to variation in the physician performing the task at that moment.
3.2 Fact Check by proxy
Clinicians have adopted ‘work arounds’ to get a general fact check of the chart, without having to do the above process themselves. The most common work around I have seen is based on looking at who authored a note. Was it the third year medical student? You likely should take everything written in there with a hefty cargo ship of salt. [aside: do automated algorithms know that medical student notes often miss the mark in accuracy?].
Conversely, some clinicians in a hospital are universally recognized for writing comprehensive and accurate notes. When a patient presents to the Emergency Room, if the physician on that evening sees a past note by the ‘brilliant clinician’, they likely will copy the Past Medical History from that clinician. Why? Because likely that person has a reputation of doing the manual fact checking process themselves.
Is this clinical freeloading? medical plagiarism? or just the standard of care? I’ll leave that to you to decide.
Although this is in practice how healthcare operates, you can likely compile many reasons why this short-cut for data verification is prone to error, and can perpetuate and copy forward incorrect data for decades to come. Which gets us back to the start, “Do not believe anything you read in the patient’s chart - it is full of lies”.
Continue to Part 2: Is this true? Confirming validity of Past Medical History data
You may also like