Paritosh Joshi: Heads, you win. Tails, I lose

01 Feb,2013

By Paritosh Joshi

 

The IRS is in a strange situation. If there are sharp changes in any statistic, it is accused of unspecified mischief. If there are no changes, it is pilloried for being inaccurate.

 

The criticisms usually come in these flavours.

 

1 Sudden, big moves: Publications launch new editions or prune existing ones on a regular basis. While there is no decision required when an edition disappears, the IRS needs to have a consistent view on incorporating a new edition into the study. Publishers clamour for inclusion no sooner than the edition goes to market. IRS takes the view that as a continuously 4-quarters moving total, it needs a whole year worth of data before the edition can be reported. This is not necessarily bad for the publication either. Basic statistics demand that if the readers actually picked up in fieldwork are below the ‘Normality’ threshold, they cannot be reported. A year’s worth of fieldwork gives every serious participant i adequate time to promote their new offspring so that it shows up in the study. Conversely, editions may sometimes be launched only tactically to preempt a competitor and may disappear once the short term objective is delivered. They certainly don’t belong in the study. Big moves happen when such editions go past the 1-year Rubicon and get reported.

 

2 Little or no change: This one usually stems from anecdotal observation. A publication may have mounted a particularly visible, or even successful marketing initiative leading to an apparently significant impact on its popularity. The IRS seems unimpressed when the next quarterly round emerges. Easy to explain. Let us assume that a particular saw as much as a 10% improvement in the brand’s performance vis-à-vis the preceding three quarters. If it had on an average, 100 readers in the previous three, it now has 110. This is what the maths would look like:

(100 x ¾) + (110 x ¼) = 75 + 27.5 = 102.5

In other words, the ‘smoothing’ effect of the Moving Annual Total reduces the large Δ of 10% to a small 2.5% perturbation in the final outcome.

 

3 Change in the wrong direction: Related to the previous observation, anecdote suggests an increase/decrease while IRS shows a decrease/increase. This is hard to explain without having some sense of the apparent capriciousness of Probability and Statistics. A simple random sample of adequate size yields convergent estimates of population parameters. However, samples can occasionally produce estimates that may have a wide variance from the underlying population statistics. These samples aren’t wrong. They just happen to be the outliers fully compliant with laws of probability. Such a sample will reveal estimates that are counter-intuitive but that doesn’t make them incorrect. If you never spot a estimate that seems to be out of kilter, you should be more worried about the reliability and/or integrity of a sample-based exercise than if you do, every once in a while.

 

4 Further analysis produces contradictions and conundrums: My response to this one? Don’t. The IRS reports only those numbers that pass the test of statistical propriety. When you start attempt to dice down whatever has been reported at minimal granularity, you are working with samples that fall below Normality and can no longer be used as consistent and convergent estimates of population behaviour. This, tragically, is practiced almost entirely in the breach by the alarmingly large number of strategists, planners and the like who appear to have no understanding of Statistics.

 

In exactly three months, we shall have another IRS release and notwithstanding these meek entreaties, the same rotten tomatoes will be hurled at it again.

 

Comes with the territory.

 

Paritosh Joshi has been a marketer, a mediaperson and a key officebearer on industry bodies. He is developing an independent media advisory practice. His column, Media Matrix, appears on MxMIndia, usually on Thursdays

 

Post a Comment 

Comments are closed.

Today's Top Stories
Videos