Day 225: Hidden methods and reluctant answers

The cross-examination of James Heckman at the Montreal tobacco trial was likely more interesting than it felt.

Plaintiff lawyer André Lespérance and Bruce Johnston gave a sustained pounding to the report of this prominent American economist, raising a number of issues that by themselves could be the fodder for a lively debate.

But there was something that seemed rob the day of the dramatic tension it deserved. Was it the irritating way that Mr. Heckman quibbled over the simplest of questions, or refused to give clear answers? Is there a fin-de-siècle ennui as this trial sputters to an uncertain finish line? Was it Justice Riordan's apparent fatigue or disinterest with this witness?

For whatever set of reasons, this might be a rare day in the trial where the transcript is a more enjoyable read than sitting through the hearing. Too bad for the visitors from other plaintiff teams who were in court watching developments!

Filling in the blanks

James Heckman produced two econometric analyses for his clients, Imperial Tobacco and Rothmans, Benson and Hedges. One looked at "whether restrictions on advertising and warning label requirements on packages were followed by decreased youth smoking." (Expert report, part I, Exhibit 21320.1)

The other sought to evaluate whether "tobacco companies’ marketing of lights in Québec led consumers to erroneously regard lights as a less risky alternative to full-flavored cigarettes, which in turn allegedly resulted in a lower level of quitting among proposed class members than would otherwise have been the case absent this alleged misconduct." (Expert report, part 2, Exhibit 21320.2).

In both cases his answer "no" came in the form of a table that would make sense only if one had some familiarity with regression studies. Explanation of the results, and the methods used to get there, were sparsely provided -- and even more parsimoniously presented during the introduction of this witnesses' opinions yesterday by Imperial Tobacco lawyer, Deborah Glendinning. Perhaps she thought his pedigree meant he didn't have to actually explain his methods

How James Heckman presented his results:
Advertising bans do not affect youth smoking

Fortunately for us observers, there was a lot of economic experience in the room that could help Mr. Heckman fill in these important missing blanks. (Both Justice Riordan and plaintiff lawyer, André Lespérance, completed graduate studies in economics before turning to law as a career. )

And so Ms. Glendinning's somewhat perfunctory reveal of her witnesses' conclusions was complemented by today's more detailed review of what did -- and more importantly, what did not -- go into Mr. Heckman's calculations.

This is not a graduate seminar

Although Mr. Heckman has court experience, he seemed to have difficulty understanding that the choice of topics was not his to make, and that someone else had the right to frame the questions.

Mr. Heckman quibbled with the use of some words. He didn't want to answer hypothetical questions. He wandered into the weeds of unrelated details. On several occasions he was directed by the lawyers - and also by Justice Riordan - to answer the question. Not once did I get the impression that it dawned on this witness that this style was not helpful to his cause.

On a handful of occasions, he was told that the point he wanted to make could be picked up by his lawyer at the end of the day. But when that moment came, Deborah Glendinning too gave it a pass. Again, she acted in ways consistent with wanting to minimize his time on the stand.

Criticisms but no answers

André Lespérance began the day by pointing to the relatively small attention given to tobacco issues in Mr. Heckman's long research career.  He really only published one paper on the topic (Exhibit 21320.5).

Mr. Lespérance pointed out that this paper had criticized the studies of other economists, and had recommended an improved "framework" to address what lead young people to smoke. Yet in the following 8 years, Mr. Heckman has never actually applied that framework to produce his own estimate.

This echoed the testimony last month of Laurentius Marais, Kenneth Mundt, and Bertram Price, who were long on criticism of public health epidemiology, but very short on their own contributions to the research questions that they thought others had improperly answered.

Under the hood

In listening to Mr. Heckman yesterday, and reading his report, I had identified a few flaws in his approach. But my short list was a fraction of the criticisms leveled against the report by Mr. Lespérance.

Over the course of the morning, the lawyer showed the witness many factors that should have been included in Mr. Heckmans study, but were missing. These included:

* not accounting for the addictive nature of cigarettes.
Mr. Heckman acknowledged that cigarettes were addictive, and said that there were many ways to model for addictive behaviour, some of which were controversial. But the only adjustment he had made to his analysis was to "control in a simple way by conditioning on previous smoking". He eventually agreed with Mr. Lespérance that addiction "is a background variable and it may play a role."

* his model discussed the 1989 Tobacco Products Control Act (after which corporate sponsorship advertising was permitted), but did not identify the more comprehensive restrictions of the 1998 Tobacco Act 
Mr. Heckman said the reason he highlighted the earlier law was because he was responding to a paper by an author whose methods he criticized. (Saffer and Chaloupka, Exhibit 21320.13). After a while, he said that he had modeled the later law in his sensitivity analysis - a part of his report that has not been shared with the court. He was asked to provide the court with that sensitivity analysis.

* his model did not reflect the fact that tobacco companies substituted direct advertising with sponsorship advertising in the period he studied.
When asked why he did not consider sponsorship promotion in his model, Mr. Heckman gave the quintessential economist's reply:: if sponsorship advertising had been as effective as direct advertising, the companies would have used it before. Therefore it could not be seen as an effective alternative.

* his model did not address the availability of cheaper contraband products in the period 1989 to 1994, even though his client companies had pleaded guilty to contraband offences in that period.
Mr. Heckman explained that it had been methodologically difficult to assess a "cigarette price index" in the period. He acknowledged that the lowered prices in this period "could be a factor" in prevalence not falling.

* his model did not include actual advertising expenditures, but only a "dummy variable" for whether an ad ban was in effect.
Mr. Lespérance showed him ACNeilsen estimates of advertising expenditure that were available on the same web-site Mr. Heckman's researchers had used to obtain other data. Mr. Heckman said it was not specific levels of advertising expenditures that would make a difference, "but the accummulated stock" of many years exposure to advertising. If he realized that acknowledging such lag effects undermined his own study methods, he did not show it.

ACNielsen estimates of tobacco advertising
expenditures 1987 - 2000

* his model did not look at the impact of advertising on perceptions of risk, or on the reception to health warnings.

Mr. Heckman was asked to consider whether advertising which associated smoking with health, activity and risk taking could influence the environment in which government information about smoking was being presented. He admitted that this too had not been put into his model. 

So what would he tell a health minister?

Mid-morning, Mr. Lespérance presented Mr. Heckman with the scenario where he was asked to advise a Health Minister who was considering responding to his conclusions by allowing cigarette advertising to return to television - including during prime time youth programming.  Would Mr. Heckman tell the minister that this was a safe thing to do, given his view that advertising did not increase smoking?.

The witnesses' discomfort with the question was answer enough. He tried to point out that this was an extremely unlikely scenario, given the tenor of the times, and that even tobacco companies were no longer in the business of advertising in this way.

But under force to answer, he admitted that it would "not be a good public policy initiative ... I think I would not want to run the risk."  He then identified a reason other than the potential impact of advertising to justify his qualms: a policy reversal by government, he said, would be received as "new information" and the subsequent changed perception of the harmfulness of smoking might lead to increased use.

Switching it up

On the plaintiffs' team, it is André Lespérance and Bruce Johnston who often conduct cross-examination. Usually they follow the good cop/bad cop pattern of gentle questions by Mr. Lespérance followed by more forceful attempts at admissions by Mr. Johnston.

Today, they broke this pattern. In the morning, Mr. Lespérance had responded to Mr. Heckman's intransigent answers by speaking unusually sharply (for him) to the witness. Yet in the afternoon, the admissions that Mr. Heckman was volunteering seemed to encourage Mr. Johnston to keep things in a nodding-along tea-party mode.

The Compass Lexecon Team

Although Mr. Heckman has only ever published one paper on tobacco advertising (Exhibit 21320.5),  and  it greatly resembles the opinion he was paid to present to Justice Gladys Kessler,

His co-authors were Fredrick Flyer and Colleen Loughlin, who are senior executives of Compass Lexecon, a company which provided several consultant witnesses for the tobacco industry during the U.S. Department of Justice Trial, including Mr. Heckman. (Exhbit 1740R)

Mr. Johnston seemed to think that it was a little unlikely that Compass Lexecon would have provided the service of reworking their court opinions into a paper for publication without support from their client. Mr. Heckman repeated that he had not billed for his work on the paper - but he did admit that he "did not know for a fact" whether his co-authors had been paid.

Mr. Flyer and Ms. Loughlin were sitting in the court. Pointing to them, Mr. Heckman said that "they drafted the report" he was presenting to Justice Riordan, before correcting himself to say "they helped draft the report."

Rational economic agents who run tobacco companies

Mr. Johnston seemed to hit a sweet spot when he asked Mr. Heckman to comment on the behaviour of tobacco companies as "economic actors". Mr. Heckman uncrossed his arms and spoke openly about what companies could be expected to want to do to increase their profits.

He agreed that the companies would advertise to recruit new smokers, and would advertise to prevent smokers from quitting as long as the costs of doing so were not greater than the rewards. "If it were shown to be a profitable activity, I an sure they would pursue it."

He cast doubt, however, on whether such advertising activities would bear fruit. "If they did try, it has been a pretty miserable effort because the total marketing is shrinking."

Mr. Johnston elicited his agreement to many of the statements made by Rick Pollay regarding the execution of advertising activities -- that cigarettes were portrayed as part of the "good life", and that it "made economic good sense" to design adds to reassure and retain conflicted smokers.

Mr. Heckman was reminded that there was a time when some were encouraged to deny causality between smoking and disease. Mr. Johnston asked lightly whether he might not be in a similar role, denying causality between advertising and consumption. "I did not deny causality," said the economist. "I just said it hadn’t been proven."

Mr. Johnston pulled him into a personal endorsement of the lingering power of tobacco advertising. He showed an extract of Allan Brandt's book the Cigarette Century. (Exhibit 1544.2)  Mr. Heckman had not read the book, and was not familiar with the work of this Harvard historian. At first, he did not remember the Camel billboard in Times Square that Mr. Brandt discussed... and then the memory popped up.

Oh yes, he remembered -- "The one that blew smoke rings!"

Justice Riordan and light cigarettes

Earlier in the day, Mr. Heckman had been challenged about his choice of 1975 as the first period he used in studying whether light cigarettes had or had not affected quitting rates. Since this was the year in which cigarettes labelled "light" had first been used, his analysis did not allow a comparison with quit rates in pre-lights years.

Justice Riordan returned to this analysis with his short round of questions at the end of the day.

He pointed to the flat line of quit rates that Mr. Heckman had found for the years when the proportion of light cigarettes was growing. He wanted to know whether the quit rates should have been expected to increase in that time, given that there was growing concerns about smoking. "My intuitive reaction is to say that during that time, given those circumstances, the quit rates should have been rising. And yet you make a convincing portrayal that they were not."

Mr. Heckman's illustration of why light cigarettes
did not influence quit rates - Exhibit 21322

Mr. Heckman's answer was a gift to the plaintiffs. It was a logical possibility, he said, but he "didn't have access" to information on growing awareness.

The rules allow for questions which respond directly to the Judge's inquiry - and Bruce Johnston took advantage of the moment to show Mr. Heckman that his clients had indeed tabulated growing awareness of smoking as a health risk -- and that it had climbed steadily during this period. (Exhibit 62).

Deborah Glendinning had no questions for her witness today. Mr. Simon Potter, who represents his Philip Morris International-owned client, was not even in the room.

Well before 5, Mr. Heckman and his associates were trying to figure out whether they could catch a flight back to Chicago this evening. At $2,300 an hour, his two days before Justice Riordan would provided Mr. Heckman with more money than the median annual income of Canadians. It's a heterogenous world indeed!

Tomorrow the last of the industry expert witnesses will testify. Mr. David Soberman is an economist with the University of Toronto. Arguments will also be heard on a motion to block the return of JTI-Macdonald president, Michel Poirier

Day 224: James Heckman: the dismal scientist

By most standards, Mr. James Heckman would have to be considered a star witness for the two defendant tobacco companies who hired him to testify on their behalf at the Montreal tobacco trials.

As economists go, this man is no slouch.

Winner of the Nobel Memorial Prize in Economic Sciences. Distinguished service professor at the University of Chicago. Recipient of a dozen or more honorary doctorates of law or honorary professorships. It's hard for an economist to be more distinguished than that!

And it's a distinction he seems willing to share with asbestos, tobacco and other companies who have found themselves on the defensive in lawsuits.

The 69-year old Mr. Heckman has testified for asbestos companies and has assisted the defence of Philip Morris in a few important large tobacco trials in the United States, (Falise 2000, Blue Cross 2001, Department of Justice, 2005). In regulatory issues, also, he has produced opinions to help forestall plain packaging of cigarettes. (United Kingdom, 2012)

And yet, despite this 14-year relationship with tobacco clients (and a billing rate he revealed today to be US$2,300 an hour!) Mr. Heckman says that the industry has never paid for any of his research.

He positively bristled at the suggestion in this year's Surgeon General's Report that he was a "consultant researcher" to the tobacco industry.

(I will leave to others to assess his written direct testimony at the DOJ trial  and the subsequent paper he published on the topic same topic  - Exhibit 21320.5 - against the Surgeon General's conclusion. In making that assesment, one might want to consider that the co-authors of that paper - Mr. Frederick Flyer and Ms. Colleen Loughlin - were also in the courtroom this afternoon. The three are senior mounts in Compass Lexecon's stable of professionals "with testifying experience.")

The Expert Economist and the Straw Man 

Even speaking at a fast clip, and lingering only briefly on his accomplishments, Ms. Glendinning took well over an hour to allow Mr. Heckman to present his background before she asked Justice Riordan to qualify him as an expert in "economics, econometrics and determinants of causality." 

It was then the turn for plaintiff lawyer, Bruce Johnston, to point to reasons why the judge might want to doubt the value of Mr. Heckman's contribution. He immediately honed in on the way that Mr. Heckman's report (Exhibit 21320.1) was aimed at evaluating whether "[plaintiff expert witness on marketing] Dr. Richard Pollay has provided a reliable basis for concluding that a causal relationship exists between tobacco company advertising and aggregate smoking."

The problem, as Mr. Johnston saw it, was that such a topic had not been addressed in Mr. Pollay's report -- a fact that had been confirmed when Mr. Pollay testified last January.

Was this not an example of Mr. Heckman using a Straw Man to misrepresent an argument he was going to rebut?

Justice Riordan looked at the ceiling as the two men tangled over the meaning of straw-man. Mr. Heckman denied that his report was intended to suggest that Mr. Pollay had said there was a causal relationship. "I did not say he did. I said I was asked whether he had provided a reliable basis. I was asked to talk about the issue." 

Not long after, however, Mr. Heckman conceded that Mr. Pollay had never addressed causality. "I completely agree with you." He said his report was not directly linked to Mr. Pollay's except "to the extent that he talks about advertising and I talk about advertising."  

Like a professional wrestling match: a foregone call

Many months ago, another expert witness had been accused of going further in rebuttal than the reports he was challenging. On that occasion, the plaintiffs withdrew much of Robert Proctor's report, in anticipation of having sections struck down by the judge.

For a short while in an otherwise tedious day, we were on the edge of our seats. Would Justice Riordan be asked to apply the same sauce to the gander?  If so, would Mr. Heckman be finished by lunch?

The realpolitik at this stage of the trial demands that there are no more excuses for Appeal Court delays. The result -- no ruling against the defendants -- was thus a foregone conclusion.

So in an I'm-just-going-through-the motions tone, Mr. Johnston objected that that Mr. Heckman's testimony was irrelevant, as it did not respond to any evidence from the plaintiffs. And with a lets-get-this-over-with air, Justice Riordan said that he saw "enough of a connection, and would allow the report to come in."

Once over lightly 

Given Mr. Heckman's stature, one might have expected him to have been given a lengthy exposition, and the opportunity to drill down on some of his findings, as others have. (His main conclusions are appended at the end of this post).

But Ms. Glendinning seemed surprisingly anxious to not let her witness linger too long in front of Justice Riordan. She frequently cut short his answers, or otherwise tried to rush him through her set list of questions. The day adjourned with a full hour left on the clock, during which time her witness would have had a captive audience. So much left unexplained!

The brevity came with a mixture of the overly simple and the overly simplified.

In the morning, Mr. Heckman was asked to explain ideas that by this point in the trial are more than well understood, and which seemed almost insultingly basic. Justice Riordan, who received his own Masters in Economics only four years after James Heckman, certainly does not need to have the Scientific Method nor the role of the OECD explained to him!

James Heckman explains the Scientific Method
to Justice Brian Riordan - Exhibit 21321

In the afternoon, by contrast, the thinking behind Mr. Heckman's conclusions was rapidly skimmed over, where more detailed explanations would have been expected.

A dense table of calculated results was shown, for example, while the witness was merely asked to state his conclusions that there was no real relationship between advertising and smoking prevalence, or between the marketing of "light" cigarettes and the rate at which smokers quit.

I was reminded of the rapid display of the empty magician's sleeve before the dove was produced.

Endogeneity

One of Mr. Heckman's drive-home messages was unmeasured phenomena can lead to mistaken conclusions.

He used the colourful if somewhat dated/distasteful analogy of trying to answer the research question "Does reading Playboy lead to reduced homosexuality?"  One had to consider endogeneity, he said -- or the likelihood that those who read Playboy might already have a predisposition towards heterosexuality.

Endogeneity also played a role when society adopts measures like bans on tobacco advertising. This happens when social attitudes towards smoking were unfavourable and is thus also connected with fewer people smoking.

Smoking rates in Canada were already declining when health measures were adopted he said -- likely the results of an unmeasured social attitude, and not the measures themselves.

James Heckman's view of events in tobacco control
in Canada as tied to smoking rates. 

He applied the same concept at the individual level.

Endogeneity is a factor when considering the child who wears a T-shirt printed with Joe Camel, or who collects a tobacco-branded ashtray. This young person likely has a predisposition towards smoking -- one caused by factors other than industry behaviour.

"There is very little evidence that advertising has an effect – but the price of cigarettes, or whether or not parents are smokers, or the environment and peer effects are major determinants - even the education of the child."  Even genetics could predetermine smoking.

As if to underline that it is defects in the individual that leads to smoking, and not the actions of those who sell cigarettes, he several times characterized smokers as people who adopt "risky behaviours. "Those people who smoke marijuana, who drink, who participate in risky behaviours - they are more likely to also smoke." 

Heterogeneity

As other defence expert witnesses have before him, Mr. Heckman underscored that it was not appropriate to think of smokers as a homogenous class of individuals. 

The differences among people -- heterogeneity -- had to be considered before drawing any conclusions. "The 'representative individual' is a fiction. [It] disguises variability in individuals."

Earlier, he had cast smoking in the context of variance of non-cognitive abilities (a major focus of his usual research). It was not measurable brain power that predicted success, he said. Equally or more important were often unmeasured abilities, such as resilience and perseverance. People with these other characteristics did well in a host of ways - including having less chronic disease.

Mr. Heckman seemed to sincerely believe that it was not the way tobacco was provided in the market that led to what he called "the smoking decision". The factors he honed in on were personal strengths, parental background, education and peer values.  

The cross-examination of Mr. Heckman will take place tomorrow. On Wednesday morning, a motion to prevent the plaintiffs from asking for JTI-Macdonald president, Michel Poirier, to testify will be argued. In the afternoon, another industry marketing witness: Mr. David Soberman.

-----------------------------------------------

Some of Mr. Heckman's Conclusions

(Extracts from Exhibit 21320.1, 21320.2)

"[M]y main conclusion is that Dr. Pollay does not provide a reliable basis to support a causal inference between tobacco company advertising and aggregate smoking.

"Dr. Pollay’s analysis is based on a review of tobacco company documents, advertising/marketing theory, and his personal judgment on the impact of various marketing campaigns on consumer behavior. His work does not employ the rigorous analytical methodology necessary to establish the existence of a causal link between tobacco company advertising and the level of smoking in Québec above the level that would have existed absent this advertising. Dr. Pollay’s analysis does not provide reliable empirical support for the conclusion that tobacco company advertising was a causal factor in initiation, quitting or intensity of smoking decisions. As a result, his work does not provide reliable evidence addressing the narrower question of whether tobacco company alleged misconduct caused harm to the class.'

"I also conclude that tobacco companies have economic incentives to advertise, even absent any effect of such advertising on individuals’ decisions to smoke. Namely, I understand that plaintiffs have put forth the argument that a key incentive for tobacco companies to advertise is to increase aggregate cigarette consumption. This conclusion is conjecture without a reliable empirical basis, as firms have multiple economic incentives to advertise."

"My analysis of these Canadian data and other academic research efforts do not support Plaintiffs’ claim that the marketing of light cigarettes reduced cessation rates in Québec."

"I also considered the issue of whether any possible effects of Defendant tobacco companies’ alleged misconduct (including, but not limited to allegations of wrongful lights marketing and marketing to youth) would have differed significantly between class members. In my opinion, substantial heterogeneity among class members in important factors affecting smoking behavior provides support for differential impact. For example, the amount and frequency of smoking light cigarettes and other factors, such as exposure to marketing in Québec varied among class members. In addition, even if the exposure to the alleged misconduct were the same across class members (which it was not), the impact of such exposure would not be uniform across the proposed classes, which constitute a heterogeneous group of individuals who undoubtedly varied substantially with regard to time and place of initiation and continuation of smoking, socioeconomic circumstances, peer influence, genetic makeup and societal conditions such as the extent of public smoking restrictions and predominant attitudes regarding acceptability of smoking. Therefore, there is no reliable basis to conclude that impacts of the alleged misconduct on smoking behavior, even if such impacts existed, would be common among the class members.