Tuesday, 15 April 2014

Day 225: Hidden methods and reluctant answers

The cross-examination of James Heckman at the Montreal tobacco trial was likely more interesting than it felt.

Plaintiff lawyer André Lespérance and Bruce Johnston gave a sustained pounding to the report of this prominent American economist, raising a number of issues that by themselves could be the fodder for a lively debate.

But there was something that seemed rob the day of the dramatic tension it deserved. Was it the irritating way that Mr. Heckman quibbled over the simplest of questions, or refused to give clear answers? Is there a fin-de-siècle ennui as this trial sputters to an uncertain finish line? Was it Justice Riordan's apparent fatigue or disinterest with this witness?

For whatever set of reasons, this might be a rare day in the trial where the transcript is a more enjoyable read than sitting through the hearing. Too bad for the visitors from other plaintiff teams who were in court watching developments!

Filling in the blanks

James Heckman produced two econometric analyses for his clients, Imperial Tobacco and Rothmans, Benson and Hedges. One looked at "whether restrictions on advertising and warning label requirements on packages were followed by decreased youth smoking." (Expert report, part I, Exhibit 21320.1)

The other sought to evaluate whether "tobacco companies’ marketing of lights in Québec led consumers
to erroneously regard lights as a less risky alternative to full-flavored cigarettes, which in turn allegedly resulted in a lower level of quitting among proposed class members than would otherwise have been the case absent this alleged misconduct." (Expert report, part 2, Exhibit 21320.2).

In both cases his answer "no" came in the form of a table that would make sense only if one had some familiarity with regression studies. Explanation of the results, and the methods used to get there, were sparsely provided -- and even more parsimoniously presented during the introduction of this witnesses' opinions yesterday by Imperial Tobacco lawyer, Deborah Glendinning. Perhaps she thought his pedigree meant he didn't have to actually explain his methods
How James Heckman presented his results:
Advertising bans do not affect youth smoking

Fortunately for us observers, there was a lot of economic experience in the room that could help Mr. Heckman fill in these important missing blanks. (Both Justice Riordan and plaintiff lawyer, André Lespérance, completed graduate studies in economics before turning to law as a career. )

And so Ms. Glendinning's somewhat perfunctory reveal of her witnesses' conclusions was complemented by today's more detailed review of what did -- and more importantly, what did not -- go into Mr. Heckman's calculations.

This is not a graduate seminar

Although Mr. Heckman has court experience, he seemed to have difficulty understanding that the choice of topics was not his to make, and that someone else had the right to frame the questions.

Mr. Heckman quibbled with the use of some words. He didn't want to answer hypothetical questions. He wandered into the weeds of unrelated details. On several occasions he was directed by the lawyers - and also by Justice Riordan - to answer the question. Not once did I get the impression that it dawned on this witness that this style was not helpful to his cause.

On a handful of occasions, he was told that the point he wanted to make could be picked up by his lawyer at the end of the day. But when that moment came, Deborah Glendinning too gave it a pass. Again, she acted in ways consistent with wanting to minimize his time on the stand.

Criticisms but no answers

André Lespérance began the day by pointing to the relatively small attention given to tobacco issues in Mr. Heckman's long research career.  He really only published one paper on the topic (Exhibit 21320.5).

Mr. Lespérance pointed out that this paper had criticized the studies of other economists, and had recommended an improved "framework" to address what lead young people to smoke. Yet in the following 8 years, Mr. Heckman has never actually applied that framework to produce his own estimate.

This echoed the testimony last month of Laurentius Marais, Kenneth Mundt, and Bertram Price, who were long on criticism of public health epidemiology, but very short on their own contributions to the research questions that they thought others had improperly answered.

Under the hood

In listening to Mr. Heckman yesterday, and reading his report, I had identified a few flaws in his approach. But my short list was a fraction of the criticisms leveled against the report by Mr. Lespérance.

Over the course of the morning, the lawyer showed the witness many factors that should have been included in Mr. Heckmans study, but were missing. These included:

* not accounting for the addictive nature of cigarettes.
Mr. Heckman acknowledged that cigarettes were addictive, and said that there were many ways to model for addictive behaviour, some of which were controversial. But the only adjustment he had made to his analysis was to "control in a simple way by conditioning on previous smoking". He eventually agreed with Mr. Lespérance that addiction "is a background variable and it may play a role."

* his model discussed the 1989 Tobacco Products Control Act (after which corporate sponsorship advertising was permitted), but did not identify the more comprehensive restrictions of the 1998 Tobacco Act 
Mr. Heckman said the reason he highlighted the earlier law was because he was responding to a paper by an author whose methods he criticized. (Saffer and Chaloupka, Exhibit 21320.13). After a while, he said that he had modeled the later law in his sensitivity analysis - a part of his report that has not been shared with the court. He was asked to provide the court with that sensitivity analysis.

* his model did not reflect the fact that tobacco companies substituted direct advertising with sponsorship advertising in the period he studied.
When asked why he did not consider sponsorship promotion in his model, Mr. Heckman gave the quintessential economist's reply:: if sponsorship advertising had been as effective as direct advertising, the companies would have used it before. Therefore it could not be seen as an effective alternative.

* his model did not address the availability of cheaper contraband products in the period 1989 to 1994, even though his client companies had pleaded guilty to contraband offences in that period.
Mr. Heckman explained that it had been methodologically difficult to assess a "cigarette price index" in the period. He acknowledged that the lowered prices in this period "could be a factor" in prevalence not falling.

* his model did not include actual advertising expenditures, but only a "dummy variable" for whether an ad ban was in effect.
Mr. Lespérance showed him ACNeilsen estimates of advertising expenditure that were available on the same web-site Mr. Heckman's researchers had used to obtain other data. Mr. Heckman said it was not specific levels of advertising expenditures that would make a difference, "but the accummulated stock" of many years exposure to advertising. If he realized that acknowledging such lag effects undermined his own study methods, he did not show it.

ACNielsen estimates of tobacco advertising
expenditures 1987 - 2000
* his model did not look at the impact of advertising on perceptions of risk, or on the reception to health warnings.
Mr. Heckman was asked to consider whether advertising which associated smoking with health, activity and risk taking could influence the environment in which government information about smoking was being presented. He admitted that this too had not been put into his model. 

So what would he tell a health minister?

Mid-morning, Mr. Lespérance presented Mr. Heckman with the scenario where he was asked to advise a Health Minister who was considering responding to his conclusions by allowing cigarette advertising to return to television - including during prime time youth programming.  Would Mr. Heckman tell the minister that this was a safe thing to do, given his view that advertising did not increase smoking?.

The witnesses' discomfort with the question was answer enough. He tried to point out that this was an extremely unlikely scenario, given the tenor of the times, and that even tobacco companies were no longer in the business of advertising in this way.

But under force to answer, he admitted that it would "not be a good public policy initiative ... I think I would not want to run the risk."  He then identified a reason other than the potential impact of advertising to justify his qualms: a policy reversal by government, he said, would be received as "new information" and the subsequent changed perception of the harmfulness of smoking might lead to increased use.

Switching it up

On the plaintiffs' team, it is André Lespérance and Bruce Johnston who often conduct cross-examination. Usually they follow the good cop/bad cop pattern of gentle questions by Mr. Lespérance followed by more forceful attempts at admissions by Mr. Johnston.

Today, they broke this pattern. In the morning, Mr. Lespérance had responded to Mr. Heckman's intransigent answers by speaking unusually sharply (for him) to the witness. Yet in the afternoon, the admissions that Mr. Heckman was volunteering seemed to encourage Mr. Johnston to keep things in a nodding-along tea-party mode.

The Compass Lexecon Team

Although Mr. Heckman has only ever published one paper on tobacco advertising (Exhibit 21320.5),  and  it greatly resembles the opinion he was paid to present to Justice Gladys Kessler,

His co-authors were Fredrick Flyer and Colleen Loughlin, who are senior executives of Compass Lexecon, a company which provided several consultant witnesses for the tobacco industry during the U.S. Department of Justice Trial, including Mr. Heckman. (Exhbit 1740R)

Mr. Johnston seemed to think that it was a little unlikely that Compass Lexecon would have provided the service of reworking their court opinions into a paper for publication without support from their client, but Mr. Heckman repeated that he had not billed for his work on the paper. But he did admit that he "did not know for a fact" whether his co-authors had been paid.

Mr. Flyer and Ms. Loughlin were sitting in the court. Pointing to them, Mr. Heckman said that "they drafted the report" he was presenting to Justice Riordan, before correcting himself to say "they helped draft the report."

Rational economic agents who run tobacco companies

Mr. Johnston seemed to hit a sweet spot when he asked Mr. Heckman to comment on the behaviour of tobacco companies as "economic actors". Mr. Heckman uncrossed his arms and spoke openly about what companies could be expected to want to do to increase their profits.

He agreed that the companies would advertise to recruit new smokers, and would advertise to prevent smokers from quitting as long as the costs of doing so were not greater than the rewards. "If it were shown to be a profitable activity, I an sure they would pursue it."

He cast doubt, however, on whether such advertising activities would bear fruit. "If they did try, it has been a pretty miserable effort because the total marketing is shrinking."

Mr. Johnston elicited his agreement to many of the statements made by Rick Pollay regarding the execution of advertising activities -- that cigarettes were portrayed as part of the "good life", and that it "made economic good sense" to design adds to reassure and retain conflicted smokers.

Mr. Heckman was reminded that there was a time when some were encouraged to deny causality between smoking and disease. Mr. Johnston asked lightly whether he might not be in a similar role, denying causality between advertising and consumption. "I did not deny causality," said the economist. "I just said it hadn’t been proven."

Mr. Johnston pulled him into a personal endorsement of the lingering power of tobacco advertising. He showed an extract of Allan Brandt's book the Cigarette Century. (Exhibit 1544.2)  Mr. Heckman had not read the book, and was not familiar with the work of this Harvard historian. At first, he did not remember the Camel billboard in Times Square that Mr. Brandt discussed... and then the memory popped up.

Oh yes, he remembered -- "The one that blew smoke rings!"

Justice Riordan and light cigarettes

Earlier in the day, Mr. Heckman had been challenged about his choice of 1975 as the first period he used in studying whether light cigarettes had or had not affected quitting rates. Since this was the year in which cigarettes labelled "light" had first been used, his analysis did not allow a comparison with quit rates in pre-lights years.

Justice Riordan returned to this analysis with his short round of questions at the end of the day.

He pointed to the flat line of quit rates that Mr. Heckman had found for the years when the proportion of light cigarettes was growing. He wanted to know whether the quit rates should have been expected to increase in that time, given that there was growing concerns about smoking. "My intuitive reaction is to say that during that time, given those circumstances, the quit rates should have been rising. And yet you make a convincing portrayal that they were not."

Mr. Heckman's illustration of why light cigarettes
did not influence quit rates - Exhibit 21322
Mr. Heckman's answer was a gift to the plaintiffs. It was a logical possibility, he said, but he "didn't have access" to information on growing awareness.

The rules allow for questions which respond directly to the Judge's inquiry - and Bruce Johnston took advantage of the moment to show Mr. Heckman that his clients had indeed tabulated growing awareness of smoking as a health risk -- and that it had climbed steadily during this period. (Exhibit 62).

Deborah Glendinning had no questions for her witness today. Mr. Simon Potter, who represents his Philip Morris International-owned client, was not even in the room.

Well before 5, Mr. Heckman and his associates were trying to figure out whether they could catch a flight back to Chicago this evening. At $2,300 an hour, his two days before Justice Riordan would provided Mr. Heckman with more money than the median annual income of Canadians. It's a heterogenous world indeed!

Tomorrow the last of the industry expert witnesses will testify. Mr. David Soberman is an economist with the University of Toronto. Arguments will also be heard on a motion to block the return of JTI-Macdonald president, Michel Poirier