How Academic and Government "Anecdotes Are Not Data" Ideologues Kill People

I'm already receiving comments that, regarding Prof. Jon Patrick's detailed exposé of the dangers of ill-suited-for-purpose ED EHR's, Patrick's observations are:

... not really valid because they're not peer reviewed; they're just anecdotal.

Only an egghead could pen such words.

I always get hives immediately after eating strawberries. But without a scientifically controlled experiment with all the right peer review, it's not reliable data. So I continue to eat strawberries every day, since I can't tell if they cause hives.

I'd already written about anecdotalist refrains at my Mar. 7, 2011 post "Australian ED EHR Study: Putting the Lie to the Line "Your Evidence Is Anecdotal, Thus Worthless" Used by Eggheads, Fools and Gonifs." In that essay I cite Dr. Patrick himself on "anecdotal evidence", regarding which he hit the ball out of the Southern Hemisphere in an editorial in "Applied Clinical Informatics" entitled "The Validity of Personal Experiences in Evaluating HIT."

Aside from the fact that eggheads also don't seem to care about the issues of faulty peer review, especially in profitable biomedical sectors, such as at "The Lancet Emphasizes the Threats to the Academic Medical Mission" with its embedded links, and "Has Ghostwriting Infected The Experts With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?", there's this simple fact:

Public health catastrophe warnings from responsible sources don't need peer review, they need investigation.

Yes, there were those pesky, off-narrative journalistic reports that the Japanese nuclear reactors were not entirely safe, that Bernie Madoff was a fraud, that mortgages for everyone was not a good idea, that the O-rings couldn't stand sub-freezing temperatures, that the the foam that broke off the Columbia launch tank caused a danger, and that the Titanic didn't have enough lifeboats, but they weren't peer reviewed...so we ignored them. Saved us a lot of money, too.

-- SS

Addendum:

At my post "Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2", I opined:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

... researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

I stand by this assertion.

Finally, I ask: at what point does ignoring work such as Prof. Patrick's, if patient harm is caused by the system he reviewed, constitute reckless endangerment and perhaps criminal negligence by hospital and government officials?

-- SS

Addendum Mar. 23:

As if on cue, this story appeared in the WSJ:

March 23, 2011

Japan Ignored Warning of Nuclear Vulnerability

TOKYO—Japanese regulators discussed in recent months the use of new cooling technologies at nuclear plants that could have lessened or prevented the disaster that struck this month when a tsunami wiped out the electricity at the stricken Fukushima Daiichi power facility.

However, they chose to ignore the vulnerability at existing reactors and instead focused on fixing the issue in future ones, government and corporate documents show. There was no serious discussion of retrofitting older plants with the alternative technology

I guess the "vulnerability reports" just weren't peer reviewed, therefore meaningless - or not reviewed by the "right" peers.

This sounds like our own FDA, ONC office and Institute of Medicine (via the Committee on Patient Safety and Health Information Technology), "choosing to ignore" health IT "vulnerabilities" (such as the aforementioned) and focusing on future issues such as comparative effectiveness research, "the common good", etc. instead.

I call this attitude "reckless endangerment" and hope plaintiff attorneys are paying close attention.

-- SS

Saving My Mother Yet Again. EHR Legible Gibberish - Another Example, the ED EHR Allergy List - And Legal Threats for Exposing Problems

My brain-injured mother was admitted to a suburban hospital (recently acquired by the "big hospital" where her EHR-related injuries of 2010 had occurred) Saturday morning.

She was again in a confusional state (delirium) of unknown cause, probably recurrent infection.

Of note, almost every time I see hospital EHRs, I note a problem.

In my Jan. 2011 post on this issue at the same organization, "EHR Problems? No, They're Merely Anecodotal; the Truth Must Be That I Attract Bad Electrons and Stale Bits" I observed a nurse-stated "glitchy-ness" that day that manifested as unreliability in pulling up the patients' current med lists. I had to be the conduit of my mother's meds, despite having gone through them in detail for computer entry in the exact same ED just 24 hours prior after a fall:

... My mother was having a repeat of the ischemia to the brain or "TIA" (transient ischemic attack, i.e., threatening to have a stroke), only this time the ED EHR itself was also having a TIA.

This was not the "FirstNet" ED EHR by Cerner forensically analyzed by Dr. Jon Patrick (as I wrote about here), but another ED EHR, by a company whose ICU physiological monitoring system I once as CMIO struggled with due to repeated, unexplained crashing.

On this most recent ED visit/admission to the satellite just days ago, I noted another problem with the ED EHR system (the same one that started my mother's travails at the main facility in May 2010, and now in use at the satellite).

When the ED nurse brought up my mother's allergies, they were repeated over and over and over on the ED screen, in a long recurrent list dozens of lines long, as if they'd been cut-and-pasted multiple times at each visit. She apologized to me. See images of a printout that was provided to me by an ED attendant upon my request as my mother's POA, below (names of hospital, patient, EHR, and EHR screen layout digitally redacted):

Allergy list, page 1 (excerpted from an on-screen continuous list). Click to enlarge.

Allergy list, page 2 (excerpted from an on-screen continuous list). Click to enlarge.

Allergy list, page 3 (excerpted from an on-screen continuous list). Click to enlarge.

Note that on-screen these appeared as a long, confusing list.

The repetition made the list near useless to the ED personnel (for example, they don't have time to look for the one crucial item that ISN'T a duplicate in the mess).

Legible gibberish indeed.

The ED RN just asked me about my mother's allergies, saying she could not make sense of the computer list and wanted to make sure no mistakes occurred. This is an appropriate attitude - the only appropriate attitude - for a clinician. Fortunately, I'm a doctor and know the allergies well.

The hospitalist then called me that night suggesting she would give my mother Levaquin, an antibiotic. For the umpteenth time I had to tell a doctor at these facilities my mother was allergic to Levaquin. This was in fact one of my complaints on my April 2010 warning letter to the hospital's CEO and CMO on EHR deficiencies I'd noted in my mother's care. This was just one month prior to her catastrophe, when a critical heart medication "disappeared" in the ED EHR, causing a cascade of medication continuity failure.

Yesterday I insisted the duplicate entries be removed (more precisely, "made inactive" - they still appear, but in a different color than "active" entries).

It is, on first principles, inherently harmful to the public to have critical patient data stored in disarray in an Emergency Room electronic health record.

See the above images, and ask if this is what you'd want busy ED doctors to have to wade through to figure out if a drug they're about to administer might injure or kill you.

-- SS

Post legal-threat addendum:

I'd originally posted actual screen shots (PHI and hospital name redacted) of the allergy lists provided to me by an ED attendant upon my request as my mother's POA.

On April 8, 2011, however, I received a threatening letter from the attorney representing the hospital claiming these screens were viewed by the client as "copyrighted and proprietary information" that I had "misappropriated" (stolen).

(This raises the question as to who, exactly, the "client" is - the hospital, or the EHR vendor?)

In any case, I was asked to "retract from the blog the copyrighted and proprietary information" under threat of the hospital "pursuing all remedies under the state's trade secret laws and Federal Law." Further, I was accused of "inappropriate behavior" in trying to protect my mother from further EHR-related accidents.

The allegations (actually, fabrications) of "medical records misappropriation" and "inappropriate behavior" were especially outrageous and unprofessional, considering the hospital had already altered my mother's medical record by adding the medication they missed to the ICU H&P as at this post, was caught at it, and had admitted it to me.

I have now done as asked, only posting the allergy information and dates without the background EHR tabs of the screen header, the only component that could even remotely be viewed as protected IP (i.e., of the EHR vendor).

I had a followup discussion with a senior nurse involved in the EMR project about those screens and the legal threats, which I viewed as potential retaliation aimed at discriminatorily denying my mother and I use of public accommodations, i.e., the hospital, through intimidation. (If my seeking records legitimately was "inappropriate behavior", who knew what else I might be falsely accused of to "discourage" my return?)

I was informed with a straight face that the allergy repetition was a "feature", not a bug.

The problem was "the nurse in the ED", who did not understand they needed to "look at the dates" to understand the allergy list. ["Blame the user" is typical in this domain - ed.] This senior nurse clearly had an amateur's understanding of HCI and clarity of presentation of information - or was simply talking down to me.

I provided a reminder that with 20 years of Ivy academic, big hospital (much larger than hers) and Big Pharma experience in this domain, I found her arguments specious.

Amateurism on presentation of information is a factor that promotes EHR-related error. (It is my hope the original ED nurse is not punished for protecting patients instead of "protecting the computer" and its faults.)

-- SS

"Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2

At my Mar. 15, 2011 post "What does a "problem list" of typical health IT look like?" I displayed a chart by Jon Patrick at U. Sydney on the ill-suited nature of an American EHR system for use in Emergency Dept. settings. That system had been mandated for rollout in public ED's in the entire Australian state of New South Wales.

As it turns out, that chart was just preliminary.

A new chart is up entitled "Analysis of Problems Defined by ED Directors", divided into four sections:

1. Workarounds and Abandonments (27 elements)
2. Functions Lost from the Pre-FirstNet System or Desirable Functions (31 elements)
3. Processes with Added Risk to the Integrity of the EMR (11 elements)
4. General Problem List - What is the Potential for Resolution? (60 elements)

The entire chart can be viewed at this link (best with browsers other than MS Internet Explorer): http://sydney.edu.au/engineering/it/~hitru/index.php?option=com_content&task=view&id=120&Itemid=116 .

It is lengthy, detailed and - stunning.

Prof. Patrick concludes (emphases mine):

Reviewing this compendium of difficulties and obstacles created for staff makes it entirely unsurprising that the patient throughput of most EDs dropped by 50% on the day FirstNet was introduced and now some years later throughputs are only just beginning to recover as staff have been able to instigate work practices to minimise the worst aspects of the system.

[A spectacular waste of clinician energy - ed.]

The workarounds and abandonments give an expression of the frustration of staff and their strategy for retaining equilibrium in their work practices despite FirstNet's presence. In a number of cases we have seen the practice guidelines of NSWHealth surrendered by the imperatives of the technology with the imposition of the HSS. It is astounding that practices defined from years of clinical experience can be discarded so whimsically.

[As I have written, the IT industry has invaded the healthcare sector, and this is the absolutely non-whimsical result - ed.]

Fortunately, in the case of one pathology laboratory, patient safety was put ahead of the technological imperative lest it jeopardise the registration of the laboratory. The described function-losses with FirstNet compared to the pre-FirstNet systems, and the functionality needs expressed by the staff indicate that they are acutely aware of the value of good technology and have a strong desire to be equipped with something that works properly without creating unacceptable risks to patients and a draconian reduction in their efficiency. The risks posed by the system to maintaining the integrity of the medical record is something that staff are acutely concerned about as they feel it fails to fulfil their legal obligations.

Emergency Departments are too important to have to endure these stressful and unproductive conditions.

[This is a first principle, as as such is not open to debate - ed.]

It is time that the knowledge and experience of the Directors and their staff were listened to and taken seriously [actually, six decades or so into the "computer revolution", I'd say that time was long ago - ed.] for the sake of improving our hospitals's use of technology. After all we have to ask: What business would commit to an interloping "integrated" system whose services are being necessarily dismembered piecemeal as a matter of survival by the users? This is a system whose pieces are not used by the staff, but rather are shadow mirrored by them, not for redundancy but primacy.

Who would want a system that is progressively de-activated by the staff to overcome the hazards and operational inefficiencies it has introduced?

[My answer: those who profit handsomely, and at no liability to themselves, from this arrangement. I leave it to the reader to decide who might fit in that category - ed.]

As a physician and medical informatics specialist myself, I would not want my ED care or that of my family interfered with by such IT.

The interference in care of such systems already nearly killed my relative in 2010, and may yet succeed in doing so. She is hospitalized and in extremis once more as of this day.

Several questions:

• How did a government for an entire state of a major country come to allow themselves to believe an EHR system such as this would improve conditions in the most mission critical section of their hospitals, the ED's?

• What testing and validation of the software was done by officials and representatives of said government, and who were they, exactly?

• What experience and background did the validators possess?

• How were clinician complaints during implementation, which has apparently been underway for several years now, handled?

• What other countries are going down the same path?

• Why is not all health IT subject to the same type of government regulator-led validation as this system was put under by a private academic researcher? (Note that the U.S., pharma IT validation is led by the FDA, but that same agency has essentially shied away from healthcare IT validation.)
  

• Would a country buy software as ill suited to purpose for, say, mitigating disaster risk in their nuclear power facilities?
  

Finally, I ask:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

I opine that researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

-- SS

New Ancestral Diet Review Paper

Pedro Carrera-Bastos and his colleagues Maelan Fontes-Villalba, James H. O'Keefe, Staffan Lindeberg and Loren Cordain have published an excellent new review article titled "The Western Diet and Lifestyle and Diseases of Civilization" (1). The paper reviews the health consequences of transitioning from a traditional to a modern Western diet and lifestyle. Pedro is a knowledgeable and tireless advocate of ancestral, primarily paleolithic-style nutrition, and it has been my privilege to correspond with him regularly. His new paper is the best review of the underlying causes of the "diseases of civilization" that I've encountered. Here's the abstract:

It is increasingly recognized that certain fundamental changes in diet and lifestyle that occurred after the Neolithic Revolution, and especially after the Industrial Revolution and the Modern Age, are too recent, on an evolutionary time scale, for the human genome to have completely adapted. This mismatch between our ancient physiology and the western diet and lifestyle underlies many so-called diseases of civilization, including coronary heart disease, obesity, hypertension, type 2 diabetes, epithelial cell cancers, autoimmune disease, and osteoporosis, which are rare or virtually absent in hunter–gatherers and other non-westernized populations. It is therefore proposed that the adoption of diet and lifestyle that mimic the beneficial characteristics of the preagricultural environment is an effective strategy to reduce the risk of chronic degenerative diseases.

At 343 references, the paper is an excellent resource for anyone with an academic interest in ancestral health, and in that sense it reminds me of Staffan Lindeberg's book Food and Western Disease. One of the things I like most about the paper is that it acknowledges the significant genetic adaptation to agriculture and pastoralism that has occurred in populations that have been practicing it for thousands of years. It hypothesizes that the main detrimental change was not the adoption of agriculture, but the more recent industrialization of the food system. I agree.

I gave Pedro my comments on the manuscript as he was editing it, and he was kind enough to include me in the acknowledgments.

New Ancestral Diet Review Paper

Pedro Carrera-Bastos and his colleagues Maelan Fontes-Villalba, James H. O'Keefe, Staffan Lindeberg and Loren Cordain have published an excellent new review article titled "The Western Diet and Lifestyle and Diseases of Civilization" (1). The paper reviews the health consequences of transitioning from a traditional to a modern Western diet and lifestyle. Pedro is a knowledgeable and tireless advocate of ancestral, primarily paleolithic-style nutrition, and it has been my privilege to correspond with him regularly. His new paper is the best review of the underlying causes of the "diseases of civilization" that I've encountered. Here's the abstract:

It is increasingly recognized that certain fundamental changes in diet and lifestyle that occurred after the Neolithic Revolution, and especially after the Industrial Revolution and the Modern Age, are too recent, on an evolutionary time scale, for the human genome to have completely adapted. This mismatch between our ancient physiology and the western diet and lifestyle underlies many so-called diseases of civilization, including coronary heart disease, obesity, hypertension, type 2 diabetes, epithelial cell cancers, autoimmune disease, and osteoporosis, which are rare or virtually absent in hunter–gatherers and other non-westernized populations. It is therefore proposed that the adoption of diet and lifestyle that mimic the beneficial characteristics of the preagricultural environment is an effective strategy to reduce the risk of chronic degenerative diseases.

At 343 references, the paper is an excellent resource for anyone with an academic interest in ancestral health, and in that sense it reminds me of Staffan Lindeberg's book Food and Western Disease. One of the things I like most about the paper is that it acknowledges the significant genetic adaptation to agriculture and pastoralism that has occurred in populations that have been practicing it for thousands of years. It hypothesizes that the main detrimental change was not the adoption of agriculture, but the more recent industrialization of the food system. I agree.

I gave Pedro my comments on the manuscript as he was editing it, and he was kind enough to include me in the acknowledgments.