How not to randomize a study

If you are designing a randomized study, make sure you actualize randomize your patients. Schiffl messed this up in his study of daily versus three days a week dialysis, for acute renal failure. Schiffl achieved randomization by alternating eligible patients to three days a week versus daily dialysis. One key aspect about randomization, especially in non-blinded studies, is that the investigators cannot know what arm of the study the patient will be in prior to enrolling the patient. With alternating patients every investigator knows which arm the next patient will end-up in and they can make subtle decisions on the appropriateness of the patient or in how they present the consent form to influence the composition of the study arms.

This comes up, because my fellow sent me a paper on prophylactic dialysis prior to CABG (PDF). From the paper comes this gem:

Repeat after me, “If you know what arm the patient will be in prior to enrolling the patient, you are not running a randomized trial.”

Urinary anion gap

My fellow remembers the urinary anion gap by saying:

NeGUTive

So a negative urinary anion gap is due to gut losses as opposed to an RTA.

Cute.

iPhone, Blackberry: Fight!


I have a resident in my Farmington office this month. His primary phone is a Blackberry but he also carries an iPod Touch for medical applications. It’s getting to the point that the battle for the mainstream medical computer platform is the iPhone OS.

iPhone OS 3.0 is another major nail in the coffin of the Blackberry, Palm Pre and Windows. With every revision Apple fills in the holes. The critics are running out of ammunition.

Highest creatinine I have seen in acute kidney injury

We had a patient earlier this month who presented with a creatinine that was 20 mg/dL on admission and rose to 22 on the repeat. That is the highest creatinine I have ever seen in a patient with acute kidney injury. I have a seen two patients with advanced CKD with creatinines in the mid to high thirties. (34 and 37 mg/dL).

When my fellow described the patient I was sure this was going to be CKD until she mentioned, rather triumphantly, that when she examined the patient she palpated a large bladder. She had a Foley placed and the patient voided 1300 mL of urine in the next hour.
Obstructive uropathy in a woman is cervical cancer until proven otherwise. Sure enough, a subsequent CT scan of the pelvis revealed a pelvic mass which was diagnosed as cervical cancer.
The patient was discharged with a creatinine of 1.9 mg/dL.
A few aspects of the case were interesting and surprising:
  • Obstructive uropathy causes an electrogenic type 1 RTA (hyperkalemic type 1 RTA as opposed to the hypokalemic classic type 1 RTA). Because of the RTA, these patients often have hyperkalemia out of proportion to the degree of renal failure. She was not hyperkalemic and presented with a potassium of 4.6 mEq/L.
  • The patient had a pH of 7.2, bicarbonate of 4 and a pCO2 of 8, giving her a metabolic acidosis and a respiratory alkalosis (predicted pCO2 by Winter’s formula is 14±2). I had been taught that patients cannot blow off CO2 below 14 mmHG. I guess she had super lungs. As best we could tell, the respiratory alkalosis was due to anxiety and resolved the following day.
  • My fellow wanted to give bicarbonate for the metabolic acidosis, but I did not. The pH of 7.2 is fine and the patient was hemodynamically stable. Her total calcium was 4.6 and her phosphorous was 10. I was worried that giving bicarbonate would correct the acidosis which at the time was essential to prevent the hypocalcemia from causing tetany or worse. The acidosis shifts bound inactive calcium to the unbound and active ionized form.

Journal Club: Dry weight adjustment in dialysis

Agarwal continues his streak of important studies on blood pressure in dialysis patients. This study shows that reducing the dry weight results in reductions in ambulatory blood pressures done between dialysis sessions. Agarwal had previously demonstrated that in-center blood pressure readings poorly correlated with ambulatory blood pressure. One of the key findings was that the systolic fell twice as much as the dialtolic blood pressure. This means they did not only reduce the blood pressure but they also reduced the pulse pressure, something which we really are unable to do with antihypertensive drug therapy (which reduces both the systolic and diastolic blod pressure and have little affect on the pulse pressure).

Personal note: Agarwal was one of my attendings when I was a resident and wrote one of my letters of recommendation for fellowship.

GOLDMARK: the real story.

I co-wrote a fluid and electrolyte book while in residency. During the final push to finish the book we enlisted some friends to help with proof reading and editing in a week-long proof-reading orgy. Joel Smith, a wayward cellular molecular biologist who ended up a lawyer asked, “Is there something special about the mnemonic for anion-gap metabolic acidosis, MUDSLEEPS? Is the word important? Or just the letters?”

I explained that it was a standard mnemonic along with its cousin MULEPILES. He said that’s stupid and that we should make-up our own mnemonic. Five minutes later he came up with PLUMSEEDS, an exact anagram of mudsleeps, and we used that in the book. I thought it would be a marker of who used our book to learn acid-base, if they used plumseeds they were our’s otherwise, not so much.

9 years have passed and I have yet to hear anyone use PLUMSEEDS.

FAIL

This past September, my partner Susan Steigerwalt, put a letter on my desk she photocopied (she’s old school) from Lancet. The letter described a new mnemonic for the differential of anion-gap metabolic acidosis: GOLDMARK. This reworked mnemonic had more going for it than an ego test, it was a complete reworking of the old and busted mnemonic for new hotness.

I blogged about GOLDMARK a few months ago and received an e-mail from the lead author. I have since e-mailed all three authors. Here’s their story.

In May of 2008, Josh Emmett a second-year medical student at University of Texas Southwestern was having dinner with his dad Dr. Michael Emmett, Chief of Nephrology at Baylor University in Dallas. Dr. Emmett was telling Josh that he and a fellow were frustrated with MUDSLEEPS/MULEPILES/KUSMALE because of its obvious shortcomings: paraldehyde? No one uses that. DKA, Starvation and Ethanol, all of those cause ketoacidosis. Isoniazid/iron as causes of lactic acidosis? The next time I see that will be the first time I see that. Plus no D-lactic acid, no oxoproline, an issue that must have particularly rankled Dr. Emmett as he was an author on the definitive article on the subject.

Josh, looking for a distraction from his studies volunteered to help craft a new mnemonic. Dr. Emmett and a third-year IM resident, Ankit Mehta (who has subsequentky become a nephrology fellow with Dr. Emmett), came up with the letters they would use and the synonyms for different diseases:

Uremia could be U, R or K (Renal, Kidney)
Ethylene glycol could be A, E or G (Antifreeze, Glycol)
Oxoproline could be O or P (Pyroglutamic acid)
Aspirin could A or S (Salicylate)
Ketoacidosis could be K or D (Diabetes, though that is not nearly as good as ketoacidosis because there are other causes of ketosis besides DKA)

With that list in hand Josh hit the internets and plugged the letters into some mnemonic generating websites and came up with:

  • ELK DUMP
  • SUDOKKU
  • PULSE something
  • MOPED

After a few days of vetting the possibilities they settled on GOLDMARK.

GOLDMARK has become my standard AGMA mnemonic. Bye bye PLUMSEEDS.

  • Glycol: ethylene glycol
  • Oxoproline: Pyroglutamic acid
  • L-lactic acid
  • D-lactic acid
  • Methanol
  • Aspirin
  • Renal failure
  • Ketoacidosis
UPDATE: Dr. Ankit Mehta sent me some notes from when they were trying to find mnemonic:

Hi Dr.Topf,
I was cleaning my desk over the weekend and found some papers on which i was scratching some other mnemonics for agma:

  • MOLDS REEK
  • DUKES MOLE
  • LU SMOKED
  • DUSK MOLE
  • SMOK(ing) ALE
  • LAME SUDOK(u).

As you see none of them are as good as GOLD MARK. Also, some are a stretch of imagination!
hope this helps,
Ankit.

I weep that I won’t ever get to pimp medical students on the meaning of SMOKing ALE

eGFR: the problem of false positives

Last week I saw a 58 year old African American woman who was referred to me for an eGFR (i.e. MDRD equation) of 58. Her insurance company notified her primary care doctor about this decreased GFR. I saw the letter that Blue Cross sent and it did not give the physician any guidance on what to do with this information. The insurance company just wanted to make sure the physician was aware that the patient was flagged as having CKD. The primary care doctor sent her to me for further evaluation.

The patient, however, was completely freaked out. She went on the internet and started to learn about kidney disease and to her horror found (correctly) that her lisinopril and simvastatin could cause kidney disease. Since both of these medications had been started in the last few years she suspected (wrongly) that they were the cause of her kidney disease and stoppd both of them.

Her GFRs for three years before referral had been: 65, 63, 65 and 58. When I repeated her GFR it was 62.

This is a classic case of what Dr. Harold Feldman was writing about in the Feburary CJASN (PDF). Here is a patient who stopped the two most important drugs for her future health (statin, ACEi) because of a false positive eGFR.

This article uses a Markov chain Monte Carlo method to simulate use of serum Cr or serum Cr plus eGFR for CKD screening. The model they used is illustrated below:

In the model patients gets screened once a year (a cycle) from age 60 to 78. There are 6 states patients must be assigned to:

  1. No kidney disease (CKD stage 0)
  2. No kidney disease but false positive screening test (my patient)
  3. CKD, diagnosed
  4. CKD undiagnosed (false negative screening test)
  5. ESRD
  6. Death

In each cycle every patient must be assigned to a state. Dead patients must remain dead, ESRD patients can remain ESRD or die. Patients must develop CKD (state 3 or 4) at least one cycle prior to progressing to ESRD. Patients in any living state can die. Patients with CKD (state 3 or 4) can not tranition to no CKD (state 1 or 2). And according to the text but not the figure, patients without kidney disease but false positive screening (state 2) would go back to state 1 for the next cycle.

Some assumptions in the calculation:

  • Incidence of CKD 0.7% until age 65 then 2.3%
  • Mortality without CKD 0.97% from age 60 to 70, then 2.4% after age 70
  • Mortality with CKD 0.050% (why this would be half the rate of non-CKD makes no sense)
  • Mortality with ESRD age 62-67: 15%; 67-75: 19%; and 75+: 26%
  • Annual rate of progressing from CKD to ESRD 0.076%
  • Treatment of CKD had no effect on mortality
  • Treatment of CKD reduced the annual rate of progression to ESRD by 21% (from 0.076% to 0.055%)
The Baysean test characteristics for eGFR and sCr:
  • Sensitivity of eGFR: 0.924
  • Specificity of eGFR: 0.835
  • Sensitivity of serum Cr: 0.559
  • Specificity of serum Cr: 0.950

The model used the following evaluation of CKD

  • Two clinic visits with a nephrologist
  • Limited renal ultrasound
  • Renal function panel
  • U/A, urine protein, urine creatinine

The costs for the different states are outlined in the table below:


In the initial analysis eGFR was more accurate and more cost-effective than the serum Cr. Use of eGFR kept patients off dialysis (29 patients) and reduced deaths (13 patients) at the expence of an ocean of false positives:


But when you assigned a false positive CKD a slightly lower quality of life than a true negative, 0.98 versus 1.0, the serum creatinine came out more cost effective per quality adjusted life year (QALY).

Summary: the better test (as measured by the area under the curve of a receiver operator characteristics curve) loses to the worse test because of the decreased quality of life that results from a false positive reading. The false positives were so much more prevelant that they overwelmed the benefit from the decrease in death and ESRD found with the more accurate eGFR test.

This study has received tremendous publicity, likely because noone had looked at eGFR in this way before and it had a contrarian view. While the whole nephrology community has been pushing for routine eGFR reporting along with creatinine, Feldman comes and publishes a scathing indictment. Additionally, it makes good copy to say that nephrologists developed a new way to measure renal function that dramatically increases the demand for nephrology services.

My primary concearn with this study is two fold:

  1. The eGFR equation is used as a screening test in this study and in real- life. Screening tests need to be as sensitive as possible even at the expense of specificity. The thought is that the increased false positives will be picked up with secondary testing. But a screening test never wants to give patients a clean bill of health when in actuality they have smoldering unrecognized disease. In Feldman’s analysis the eGFR works perfectly as a screening test by picking up nearly all of the patients with CKD but the decreased specificity results in numerous false positives. It is interesting that it is not the cost of evaluating the false positives that results in the cost ineffectiveness of eGFR but it is the decreased quality of life that results from the anxiety associated with the initially positive diagnosis of CKD. In my mind this means we need to do a better job educating patients and providers to the nature of the eGFR test.
  2. The other problem with the study is it threatens to throw out the baby with the bath water. Even if the study does show that eGFR is not cost effective, part of the problem is that the only utility given to the eGFR is in the early diagnosis of CKD to prevent ESRD. However, I more often use the eGFR to dose adjust medications, to estimate the risk of contrast nephropathy, guide the use of loop versus thiazide diuretics. All of these uses of the eGFR cannot be replaced by a serum creatinine because the serum creatinine does not account for age, gender and race.

False positive diagnosis of CKD by the eGFR are real problems and Dr. Feldman has done the nephrology community a favor by bringing this issue to light. It would be interesting for Feldman to re-run his Monte Carlo simulation with various definitions of CKD, does an eGFR of 50 ml/min reduce the false positives enough to reduce the cost below the benefits? What about 45 ml/min (sometimes called CKD 3b)? It is important for a dialog to be initiated among primary care doctors, nephrologists and payers to come up with better definitions of CKD that don’t freak our patients unnecessarily while providing the best care we can.

Lead time bias

As I go through the literature on early nephrology referral, I am troubled by the possibility of lead time bias. This was a large issue in the debate surrounding the optimal time to initiate dialysis. The problem comes from measuring survival from the initiation of dialysis.

Patients with good nephrologic care regularly get started on dialysis earlier than their counterparts with poor or non-existent CKD care. This is evidenced by the lower creatinines at the time of initiation of dialysis in patients with early referral seen in multiple studies.

This is consistent with my practice where I tell my patients that we “…want to delay dialysis as long as possible, but not longer than possible…” because if they have profound malnutrition or advanced heart disease due to the delay of dialysis they will do poorly once they transition to dialysis.

Because of this skew in the initiation of dialysis it is important to account for that in any analysis of survival on dialysis. I hope this short slide show makes this clear.

Note: I do not know if lead time bias is responsible for the prolonged survival with early referral to nephrology I just know that it needs to be accounted for and most literature ignores this potential source of error.