The First 9,387 Policies of freMTPL2 Are Not the Portfolio
TL;DR
The French motor portfolio freMTPL2 is in textbooks, scikit-learn’s examples and countless notebooks. Every one of its first 9,387 policies has a claim, against 3.7% of the rest. The block reports 28 times the claim frequency of the portfolio, and 97% of its policies have claims with no amount. It is not a sample of the portfolio, so it goes: that removes 27.5% of the claims but 1.1% of the claim cost. Three smaller checks also shape the model: short contracts claim four times as often per year, frequency drifts along the policy ID, and two vehicle ages are codes.
First of four posts on pricing freMTPL2, followed by large claims, the model comparison and the pipeline.
The data
freMTPL2 is a French motor third-party liability portfolio from the CASdatasets R package, published on OpenML as two tables: freMTPL2freq, one row per policy, and freMTPL2sev, one row per claim amount. Together they hold 678,013 policies from one accounting year. Each policy has risk features (driver age, bonus-malus, vehicle age, power, brand and fuel, population density, region), an exposure , the fraction of the year it was in force, and claims with total amount .
The quantity to price is the pure premium, the expected claim cost per policy-year, . It is a mean over policies, as are the claim frequency and the mean claim size it splits into. So every row has to be a policy of this portfolio, with the claims it really had.
The head block
A block of claimants
The policy table is sorted by IDpol, and every one of the first 9,387 policies, up to ID 24,951, has a claim. Outside this block 3.7% of policies do. If the block were a random draw from the same portfolio, the chance that all of its policies have a claim would be about .
Claims per policy-year by block of policy IDs in the raw data, log scale.
The block differs in everything else too:
- Frequency. 2.07 claims per policy-year, 28 times the 0.074 of the rest. On its own it lifts the portfolio frequency to 0.101.
- No amounts. 96.9% of its policies report claims that have no row in the claim table, against 0.09% of claimants outside it.
- A sharp edge. All of this changes at one ID, not gradually.
How the block got there isn’t documented: a claims extract appended to the portfolio, perhaps, or rows from another source. Either way, nothing in it can estimate a frequency or a pure premium.
What dropping it costs
So all 9,387 policies go, including the 293 that do have amounts, since they were selected for having claimed.
Share of each quantity that dropping the head block removes.
The block is small in everything the pure premium is made of and large in the claim count, because its claims are almost all claims without an amount: for the target, zeros that shouldn’t be zeros. The portfolio pure premium barely moves, from 167.1 to 167.5 per policy-year.
The alternatives don’t work. A flag for “in the block” can’t be set for a new policy. Setting the claim count to zero wherever there is no amount, as scikit-learn’s Tweedie example does, removes the inflated frequency but keeps about 9,000 policies that did claim as claim-free ones.
Three smaller checks
Exposure is part of the target
75% of contracts started or ended during the year. Under the usual assumption that claims scale with exposure, short contracts should claim about as often per year as long ones. They don’t:
Claims per policy-year by exposure band, relative to the whole portfolio, with the policy-years in each band below.
Short contracts are often contracts that ended because of a claim: the car was written off, or the contract cancelled. That is still no reason to make exposure a feature. A premium is quoted for a year before anyone knows when the contract ends, and a model priced at would charge everyone like the contracts that survived the year. So exposure enters only as a weight or a offset, the standard treatment in tariff GLMs, and the excess on short contracts stays a known limitation. Exposures above one year, 1,224 of them, are capped at one, as in the reference treatment of this data.
Frequency drifts along the ID
After the head block, claim frequency still falls steadily with the ID, from 0.089 in the first million to 0.063 above five million (first figure). IDs are probably assigned over time, and the risk factors explain little of the drift: a main-effects Poisson GLM’s observed over expected claims still fall from 1.18 to 0.88 across the ID blocks. So IDpol is never a feature, and the train/test split is random rather than by ID, which would mix a shift in base frequency into the model comparison.
Two vehicle ages are codes
Above 50 years no vehicle age has more than six policies, and none has an age from 86 to 98; then 99 and 100 hold 23 and 25. These vehicles are 46% diesel, like the portfolio’s 49%, which cars built a century ago would not be. So 99 and 100 are codes for “unknown”, set missing and later filled with the training median. The real thin tail stays: how it enters a model is a modelling choice, not a cleaning step.
Before pricing on public data
A dataset that has been used this often feels pre-cleaned. Two cheap checks found the block here: plot the target against the row order, which is the order someone wrote the file in, and check that the two tables agree about every policy. A block that differs in everything at once, with an edge at one ID, isn’t a segment of the portfolio; it is a trace of how the file was made.