Wednesday, 12 June 2013

Day 152: Opinions on opinions about opinions

Most pollsters hope to be considered correct 19 times out of 20. But as the experts hired by the tobacco industry would have it, the pollster who reviewed the data collected by Imperial Tobacco over decades was pretty much wrong in every conceivable way.

Mr. Christian Bourque first appeared at the Montreal Tobacco trials almost 5 months ago, bringing with him a report (Exhibit 1380) on secret surveys conducted by Imperial Tobacco over decades into smokers' knowledge and beliefs about the harms of smoking.

The report of this plaintiff's witness has already been the subject of stinging criticism from a survey methodologist now based at Oxford University. Mr. Raymond Duch, when he testified last month, even levied the charge that "Mr. Bourque is misleading the Court" with his conclusions.

A second round of criticism against this well-respected Montreal pollster was launched today with the appearance of Claire Durand, a professor of sociology at the University of Montreal. (Mr. Duch worked for JTI-Macdonald and Rothmans, Benson and Hedges. Ms. Durand was hired by Imperial Tobacco).

Her report (Exhibit 20066) concludes that Mr. Bourque's work was flawed from the get-go, as it suffered from "methodological problems" and deviated from "the rule of scientific neutrality."  She goes further, taking a fine-tooth comb to enumerate what she sees as many flaws in his work.

Claire Durand
Ms. Durand is one of the few women to have appeared at this trial - but it was in other regards that she was a most unusual witness. Despite having attended the court during the appearance of Mr. Bourque and Duch, she failed to have picked up any tips on the particular dance that takes place between lawyer and witness and judge.

The redoubtable Ms. Suzanne Coté had her job cut out for her with a witness prone to ramble, and obviously not skilled at taking guidance or responding to cues to be silent when enough has been said. (As someone similarly afflicted, I say this with some sympathy - but it certainly did not help her clients' cause!).

A speedy exposition kept on track

In the morning, with her trade-mark speed and organization, Ms. Coté took Ms. Durand through her principal criticisms of Mr. Bourque's synthesis of industry data.

The criticisms were plentiful, and seemed more forceful with the pace with which Ms. Coté moved her witness through the set.

Ms. Durand said that Mr. Bourque's report was incorrect in:
* assessing smokers' perceptions as these were not properly measurable,
* drawing inferences from quota-based samples that should only properly be made for randomly selected samples,
* failing to clarify that only smokers' views had been gathered,
* not properly citing sources,
* failing to provide a list of events to which he said the industry was reacting,
* attributing statistical significance to numbers that could not be properly compared,
* committing ecological errors and displaying the Robinson effect,
* improperly attributing results to a provincial or regional level when in fact they were based on surveys of urban dwellers only.

Oh, and yes, he was biased too.

Her presentation was on the court record well before lunch time.

A derailing cross-examination

Bruce Johnston asked for - and received - an extended lunch break to allow him to prepare for his cross-examination. When the session resumed at 3:00, he entered the court with papers in hand and a spring in his step.

Body language in this court is pretty subtle, but it is hard for even the most poker-faced teams to hide it when they are having fun or when things are going quite wrong. By the end of this afternoon both were evident - the enjoyment on the plaintiff's side mirrored by red-faces and slumping shoulders on Imperial Tobacco's bench.

Ms. Durand seemed to be the victim of her own testimony. She painted herself into a corner by stating views in such absolute terms that they either defied belief, or left her open to embarrassing exposure

Perhaps unaware that she was being goaded into saying increasingly ridiculous things, she engaged in lengthy and almost argumentative replies with Mr. Johnston. When Ms. Coté tried to intervene with objections, Ms. Durand ignored her lawyer and kept on talking - not just once, but a few times.

How many cigarettes are safe:  Is that Fact or Opinion?

Twice a year, at the end of its long survey of brand preferences, Imperial Tobacco asked smokers an open ended question: How many cigarettes of your own brand can you safely smoke without harming your health?

Mr. Bourque had observed that the results showed that "it was clear that not everyone was aware that smoking cigarettes - even a small number - could have a negative impact on their health."

Mr. Johnston asked Ms. Durand how she would answer the question, but she demurred and repeatedly refused to answer.  ("I am an expert in asking questions, not answering them," she said.)

The problem, she said, was this was NOT a factual question, and because it did not have a verifiable answer, it could not properly be answered. Her answers began to be a little tangled, as Mr. Johnston asked at length how these questions were different from those he was reading from the notes to the courses she taught.

The real kicker to her lengthy explanation that these questions were flawed in not being based in verifiable fact was revealed to the witness when Mr. Johnston showed her the testimony of Mr. Duch on the same set of questions.

Mr. Duch had complained that such questions were not a good indication of beliefs because they WERE factual questions.

Two industry experts -- two opposing views -- one entertaining court moment.

So do cigarettes cause cancer? And why can't you say so?

There is some poetry in asking polling questions of a polling specialist, and Mr. Johnston spent some time putting Ms. Durand through the experience of being asked some of the questions that had been tracked by Imperial Tobacco and others.

Having talked at length about respondents who "didn't know", Ms. Durand now exposed herself as a member of that tribe. She "didn't know" how many cigarettes one could smoke safely. She "didn't know" whether smokers had a shorter life.

When Mr. Johnston asked her whether cigarettes caused cancer - yes or no -  she hesitated for a long time. She tried to deflect from her difficulties by saying that she suffered from the professional disease of not wanting to answer questions "yes" or "no".  ("I noticed!" said Justice Riordan, but kindly).

 "I would refuse to answer [such a polling question]" she said.

The company is arguing that "everyone knows" that cigarettes caused cancer. But some are curiously reluctant to say so!

 Questions from the bench

Imperial Tobacco tracked smokers'
perceptions of health risks - 
Ms. Durand says the results were unreliable

It was Justice Riordan who - more than once today - raised the question that the critics of Mr. Bourque's report have steadfastly avoided: What difference did it make that the companies' studies were flawed if the believed they were good and kept them in place for decades?

"Nonetheless, the companies were paying for these studies," he pointed out. Even if they were poorly done, you could still say it was what the companies thought was going on." [very rough translation]

"I can't speculate on what the companies believed," Ms. Durand replied. "But if there was a good research service, it would have told the companies that they couldn't count on this information."

Later she said that if the companies had wanted to know what smokers' perceptions were, they did it the wrong way. "These results are not reliable."

Tomorrow both of this week's witnesses will complete their testimonies -- Mr. Choinière in the morning, and Ms. Durand in the afternoon.  Next week, former health minister, Marc Lalonde, will appear and David Flaherty will return.