Years and years and years ago

The Comedy of Consulting

A talk given to the San Diego chapter of the ASA

Best read by narrowing this window

Years and years and years ago,

When there were wolves in Wales,

I was employed full time at Upjohn as a member of the team that developed Motrin (also known as Ibuprofen or Advil). What a privilege! The experience was like…like being a utility infielder for the Yankees. One's role may be small, but sooner or later, one gets to wave a World Series ring in the air.

After leaving Upjohn, I became a consultant. I haven’t been part of a winning team since.

Mega corporations, medium-sized firms, start-ups. The experiences may be different, but one and all, the projects end in disaster.

My first consulting job was for Shiley where I worked for a woman I’d met while we both completed our post-docs. Characterize the failure rate of our heart valves, I was asked. “Your devices have an accelerating failure rate,” I soon warned, “Take precautions.” This was not a message Shiley wanted to hear, nor did they take precautions. A few years later, they were bankrupt, and Pfizer bought the shell. Typical, including the buyout. No, I did not get further consulting work from them.

With a large well-established firm, there is one and only one reason you are being hired. The project is a disaster and every knowledgeable employee has already bailed. I found myself working for the Bumbling Pharmaceutical and Device Company—that’s the pseudonym I use to protect the guilty in my book, Manager’s Guide to Design and Conduct of Clinical Trials. The output of the trials in question had lain undisturbed for almost three years. That’s because Bumbling discovered early on that the device was a failure. Of course, the FDA still wanted to see a write-up, so they had to hire someone—me. I also got some poor wretch, a newly hired employee to work with me. Guess what? Most of my job consisted of drawing up lists and lists of still missing data while the poor wretch’s job was to track the data down. The job paid the rent.

Right here in San Diego, I once consulted for a mid-size firm that had also let the forms pile up. This firm is responsible for my adopting an ironclad rule, one every consultant ought share with prostitutes: Get Paid Up Front. See, the guy who hired me knew going in he’d screwed up, but the blame was supposed to fall on me, not him. As it was, the job had two sets of problems—the ones he knew about and the ones he didn’t.

The one he knew about was that the microcomputer software he’d purchased was inadequate. This was a major problem back in the ‘80’s and ‘90’s. People would try to maintain databases on microcomputers with 64K of memory, most of which was taken up by the operating system. They try to do the same thing today on microcomputers with 256K, most of which is taken up by the operating system.

Anyhow, the first week or so of the job was cursed by constant system failures, which somehow were my fault. “I thought you knew how to use this software?!” In week 3, the FDA liaison needed a report. I did a double shift or three only to report the real problem: “You have 300 patients, but only 75 have follow ups.” Turns out that the President of the company, an MD, had been making sure that his friends, the other MD’s on the project, were all getting paid, whether or not their follow-ups had been submitted. Bad idea.

Enough stories. Time for some practical advice. I’ve said it once and now I’ll say it again: Do what the prosties do, always collect payment in advance. With too few exceptions, I've had to fight for my money in every consulting project I've taken on in the past twenty years.

In almost every case, I’ve been hired by someone who has lied about his own abilities to his superiors or by a team of individuals who’ve built their working lives around a series of no longer tenable lies.

The man (and he may well have been a woman, I use “he” only for simplicity) has persuaded his boss that somehow he has the necessary knowledge to conduct a statistical investigation. To be fair, it is often the pointy-haired boss who, believing anyone can do research, has said to the employee, you’re it. So I am hired to do programming or data management. Only as a seeming afterthought do they ask, “You can do a survival analysis, can’t you?

Xerox was developing a new operating system for its ill-fated line of computers. Their testing group was falling behind. Learning I had a statistical background as well as one in programming, I was asked to do two things. First, tell us why no matter how many testers we hire, we never seem to get ahead. Second, tell us when the testing will be complete.

The second question was the easiest. Each week, the testing group was discovering more and more errors at an accelerating rate. “Never, the testing will never be complete” was the answer.

The first question was also easy. They already knew the answer. They just wanted me, their consultant, that is, their patsy to be the one responsible for pointing it out. Seems the testing group had been given exactly four computers. Whenever a bug was discovered, the offending computer would sit idle until a member of the programming group could swing by and do a dump. So mostly, the testers played cards, gossiped or used the phones. When we got further and further behind, we were asked to come in Saturdays. The programming group never worked on Saturdays, so that really wasn’t much of a plan.

On my next consulting gig, my new supervisor not only lied about the job, he lied about me. I call this one, the case of the missing data, also known (to me)as the Christmas Eve massacre:

I met Mike, not his real name, at a swap meet. I was selling old software; he was selling old disk drives. Very affable, very outgoing, very high energy, Mike soon learned all about me. I learned that he was working for a startup company, was about to become their chief operating officer. “ And you’ll be our new chief statistician.” Of course, there was a catch. Two of them. I’d have to start as a consultant. And I’d have to wait to get paid till they got the next influx of venture capital. But in return for waiting, I’d get a premium over my normal fee.

Rule number two: if it sounds too good to be true, it probably is

It all sounded good and after a short delay it was all in writing. Mike’s offices were real and the sign on his door affirmed he was the COO. He drove a leased luxury car. I was given my own computer. In no time at all, I was happily designing trials and analyzing the data Mike gave me. One problem, well, two problems later on. Mike was giving me summary data and I wanted to get my hands on the original stuff. “I’m the statistician Mike, you shouldn’t be wasting your time.”

“I’m a statistician, too. You remember my dissertation.” I did remember. He was finishing up his doctorate in clinical psycology and he’d paid me—very little--to do the statistics.

The second problem arose the week before Christmas. I learned Mike’s company had got their venture capital, had had it for a couple of weeks, only I hadn’t been paid. I rushed to Mike’s office. Mike’s boss was there too along with several miscellaneous flunkies. I didn’t have a chance to discuss money. Mike’s boss wanted the analysis, wanted it now. Mike looked at me. “I need the raw data,” I said, “ I also need my mon…”

“Give him the data,” Mike’s boss interrupted. He didn’t use my name; I wondered if he knew it?

It took a weekend to wrest the data from Mike. Half a dozen trips to his house—I was given the wrong files twice—endless waits, plus arguments with Mike’s boss over the delays and over my insistence that I be paid just for waiting. O.K., Mike’s boss wanted results. Frankly, so did I.

It was late on a Saturday night when I was finally given the correct files. Sunday morning, I got to employ that most invaluable of SAS procedures, PROC MEANS, which among its other features displays data ranges. On Monday I was able to ask one of the technicians whether zero was a legitimate reading. It wasn’t, yet more than half the readings consisted of zeros. Moreover, at least a dozen of the records in the database were near duplicates, similar names, slightly different addresses and birth dates, but otherwise identical. Conclusion: The data had been faked.

I’d no choice but to bring the news to Mike’s boss. But first things first, “I need to get paid.” Mike’s boss looked at me the way you or I might consider some creature that has just crawled out from between the floorboards. “I can you pay now,” he said, “But if I have to, if you force me to, you’ll never work for me again.”

“I’ll take the money.” The check was for half the amount due and was written against some obscure one-branch bank an hour’s drive away. I drove to the bank, cashed the check and had the money in my hands within the hour. No “Chief Statistician” job for me.

Two months later, I heard from Milton, Mike’s boss (we were on a first name basis now). Mike’s behavior had grown increasingly bizarre and eventually he’d had to be hospitalized. (Methamphetamine does that to you). Milton had hired a second statistician, finally, and verified that the data was faked. “I just thought you were another of Mike’s crazy friends, I didn’t take you seriously.” He offered me another consulting job. I passed.

But maybe you still believe in the Easter Bunny, assume my problems (all of them) are unique to me and stem from working for under-capitalized startups like Xerox and Guidant. Not true. Think about why companies call on a consultant’s services in the first place. (I know they told you they had a hiring freeze, but they lied.)

1. The project is doomed. (In the corporate world a project is doomed once higher management abandons it. Abandonment may be for any of a myriad reasons, some related to the project and some not: Interim analysis reveals a new treatment to be ineffectual, but the FDA requires a final summary and analysis anyway. Or a new executive broom, busy downsizing, declares his predecessor's pet project redundant. Let a consultant do it.)

2. The project is flawed and someone must be blamed. (Don't expect them to thank you when you tell them, correctly, that they ought to have called in a statistician during the design phase.)

3. They’re hoping for a miracle. ("The last statistician didn't think anything could be done either, but since your rates were so much higher....")

It doesn’t take long to realize that your client’s problems stem from failures in data collection and data management, to say nothing of their frequent practice of making the decision first, then gathering the supporting data afterward. Such problems are readily avoidable, but, surprisingly, clients never express interest in how similar problems might be prevented, nor, the consultant soon learns, are you being paid to tell them.

Don’t expect praise for your efforts and, even with a written contract, expect to have to fight for your final week’s pay.

Here are some preventive measures for you: Get paid up front. Offer a discount for advance payment, if necessary. If the project is long-term (that is, longer than the client is willing to pay for up front) get a written contract. Maintain a regular line of communication with the person who authorizes payment (not infrequently, you’ll speak with this person only once and then be shunted off to subordinates.)

Cover your ass. Communicate. Tell the client what you feel should be done and how long you expect it to take. Bill for the initial consultation and begin work on the principal project only after you’ve received written authorization to start the work (an email will do). Break the work into parts, obtain written authorization prior to starting each part, and bill as you go.

Follow up on your collections. Money for external consultants often falls in some accounting never land; it’s not that they don’t intend to pay you, it’s just that with some critical person on vacation or maternity leave, they don’t know what account to pay you from.

As noted at the start of my talk, I was far more successful and happy as an Internal consultant. What did I do at Upjohn that helped me to suceed? I read the journals and internal memos in the fields I was supporting—gi, immunology, cns, cardiology. I joined their journal clubs. I even volunteered as a lab assistant when people were out sick or on maternity leave. I know how to kill mice in at least four differ ways.

So, Last, but by no means, least: Learn to speak the language your client's use. Don't ask what “significance level” or “power” they want to use. They won't know the meaning of the terms and are unlikely to have answers should you try to explain them. Need values? Learn your client's language, then glean the needed values from their literature.