Geeking Out: What Does COVID Vaccination Status Have to Do With Motor Vehicle Accident Risk?
Geeking Out: What Does COVID Vaccination Status Have to Do With Motor Vehicle Accident Risk?
Repeat the mantra: correlation is not causation, in any direction
And the answer is…. what do credit scores have to do with motor vehicle accident (MVA) risk?
(now the actuaries are nodding their heads. They know.)
The paper on “vaccine hesitant” folks in Canada having higher MVA risk
Here is a link to the paper: COVID Vaccine Hesitancy and Risk of a Traffic Crash
The abstract:
Background
Coronavirus disease (COVID) vaccine hesitancy is a reflection of psychology that might also contribute to traffic safety. We tested whether COVID vaccination was associated with the risks of a traffic crash.Methods
We conducted a population-based longitudinal cohort analysis of adults and determined COVID vaccination status through linkages to individual electronic medical records. Traffic crashes requiring emergency medical care were subsequently identified by multicenter outcome ascertainment of all hospitals in the region over a 1-month follow-up interval (178 separate centers).Results
A total of 11,270,763 individuals were included, of whom 16% had not received a COVID vaccine and 84% had received a COVID vaccine. The cohort accounted for 6682 traffic crashes during follow-up. Unvaccinated individuals accounted for 1682 traffic crashes (25%), equal to a 72% increased relative risk compared with those vaccinated (95% confidence interval, 63-82; P < 0.001). The increased traffic risks among unvaccinated individuals extended to diverse subgroups, was similar to the relative risk associated with sleep apnea, and was equal to a 48% increase after adjustment for age, sex, home location, socioeconomic status, and medical diagnoses (95% confidence interval, 40-57; P < 0.001). The increased risks extended across the spectrum of crash severity, appeared similar for Pfizer, Moderna, or other vaccines, and were validated in supplementary analyses of crossover cases, propensity scores, and additional controls.Conclusions
These data suggest that COVID vaccine hesitancy is associated with significant increased risks of a traffic crash. An awareness of these risks might help to encourage more COVID vaccination.
This conclusion is bullshit, and I will get into that in a moment.
The problem here, at its core, is the problem with all correlative studies. They’re not proving any cause-effect link.
I will explain by pulling out the stats from their own study, but I want to highlight somebody else’s analysis first: Dr. John Campbell (no (known) relation).
A video from a different Campbell
Here is a video from Dr. John Campbell (no relation to me, that I know of), who breaks down his own issues with the research paper:
Here is what he covers in the video, pulling from his own video description:
People that take covid vaccines are the sort of people who have less traffic accidents/ People who do not take covid vaccines are the sort of people who have more traffic accidents…. in Canada
Goes over the details of the sample in the study – time period, method, definition of vaccinated/unvaccinated (key – some of this is absurd given the point they’re trying to make in the study),
How they adjusted/controlled for other variables – what other variables
Potential effects on auto insurance pricing
Unvaccinated in Canada were excluded from mass transit
Vaccinated in Canada were more likely to be remote workers than unvaccinated
Age correlations
Pedestrians also were included in the sample (run over while they were walking on sidewalk? Passed out drunk on sidewalk?)
Excluded deaths, misclassifications of data samples
Data are not disclosed for others to check
Here are the links in his video description:
Norman Fenton on the pedestrian casualties in the data sample: A Study in Stupidity: does the covid vaccine really lower your risk of being in a car accident?
Igor Chudov: The Unvaccinated Had More Car Crashes… Because they Were the Ones Driving!
Dr. Clare Craig: Twitter thread on problems with the study — (she touches on one of the big problems I noticed — the small sample size leading to huge error bars — some of this had really low credibility, which is why they had a stupid “unvaccinated” definition.)
Forbes article from Aug 2021: Canada To Make Proof Of Vaccination Mandatory For Air And Train Travel
April 2021 COVID-19 Advisory for Ontario: A Vaccination Strategy for Ontario COVID-19 Hotspots and Essential Workers – shows many essential workers in Ontario not getting vaccinated
On the conclusion from the correlation result: complete bullshit
Let us pretend that all the weaknesses of the study noted above do not matter and the correlations are rock solid. I will explain below why I considered it completely believable.
The correlation result does not lead to this:
Conclusions
These data suggest that COVID vaccine hesitancy is associated with significant increased risks of a traffic crash. An awareness of these risks might help to encourage more COVID vaccination.
Why would that be?
All you have shown is a correlation.
Let us do a quick thought experiment: let us pretend that, by magic, all the people in the study were actually vaccinated for COVID. Do you think their risk level would have been lowered for motor vehicle accidents?
Well, maybe, if they got to use mass transit instead of having to drive (being barred from trains and planes because of their vaccination status). That is the only cause-effect link I can think of with vaccination.
Of course, one could also change that risk by simply allowing them on trains and planes. (UPDATE: I have been told the study was done before any transit ban in Ontario. It is irrelevant to my larger argument - which is that the vaccine has no direct connection to traffic risk. No matter the correlation. It has a connection to something else….which has a direct connection to traffic risk.)
Let’s look at the relative risk results they have for the various variables they tested:
Look at the “alcohol misuse” variable, which had the highest relative risk after adjustment (I will get to the other factors below).
If the people who were positive for alcohol misuse were gotten completely off alcohol and they didn’t drink it at all — do you think that would have a real effect on their risk level with respect to motor vehicle accidents?
Yes, it absolutely would.
There is a strong cause-effect link between alcohol abuse and motor vehicle accidents, even as a pedestrian. Walking while drunk (and definitely passing out on the sidewalk drunk in the dark, and getting run over) is a big danger.
But a simple correlation, no matter how strong the relative risk, is not telling you what the cause-effect link is. Changing the one thing you’re measuring will not necessarily change the cause-effect chain. You may simply be changing something outside that cause-effect chain and not changing the underlying risk itself.
If the COVID vaccination status is the result of a behavioral type that leads to more accidents, and you just change the vaccination status but not the behavioral type, you’re not going to change the accident risk level.
The type of people who were not vaccinated and had higher risk profiles as drivers would be unlikely to be convinced by a study like this anyway.
What’s the credit score connection?
So the actuaries probably know where I’m going with this when I mentioned credit score.
First off, insurers don’t use exactly the same score you see when you look at your “credit score”… There are special insurance-related credit scores, which look at specific items:
Insurance scores and credit scores differ. Credit scores predict credit delinquency while insurance scores predict insurance losses. Both are calculated from information in a credit report, such as outstanding debt, bankruptcies, length of credit history, collections, new applications for credit, number of credit accounts in use, and timeliness of debt repayment. Insurers or scoring agencies then calculate the insurance or credit score by assigning differing weights to the favorable or unfavorable information in the credit report. Information such as income, ethnic group, age, gender, disability, religion, address, marital status and nationality are not considered when calculating an insurance score.
Credit and insurance scores measure how well individuals manage their money—not how much money they make. And actuarial studies show that how a person manages his or her financial affairs is a good predictor of insurance claims. Statistically, people with a low insurance score are more likely to file a claim.
It’s also the likelihood of the amount of the claim — we care about total loss costs (frequency and severity, combined).
You may have seen inputs to your credit scores via a report. The difference for an insurance-related score is that it will use different weights on the inputs, and some of the items that go into the normal FICO score you see may not be used in an insurance score at all. Looking at one of mine, I see the following inputs:
Percentage of on-time payments for bills
Number of credit accounts (and nature – revolving consumer credit, HELOC, mortgage, auto loan, etc.)
Average age of accounts
Percentage of credit usage
Number of credit inquiries (how many new accounts am I trying to open)
And there are some other things where one can get more detail.
Insurers have known for decades that a score based on these inputs has been correlated with auto loss costs. Regulators are not happy with this result, and insurers have done lots of studies to support this result.
Here is a paper from 2016 on this phenomenon: Empirical Evidence on the Use of Credit Scoring for Predicting Insurance Losses with Psycho-social and Biochemical Explanations:
An important development in personal lines of insurance in the United States is the use of credit history data for insurance risk classification to predict losses. This research presents the results of collaboration with industry conducted by a university at the request of its state legislature. The purpose was to see the viability and validity of the use of credit scoring to predict insurance losses given its controversial nature and criticism as redundant of other predictive variables currently used. Working with industry and government, this study analyzed more than 175,000 policyholders’ information for the relationship between credit score and claims. Credit scores were significantly related to incurred losses, evidencing both statistical and practical significance. We investigate whether the revealed relationship between credit score and incurred losses was explainable by overlap with existing underwriting variables or whether the credit score adds new information about losses not contained in existing underwriting variables. The results show that credit scores contain significant information not already incorporated into other traditional rating variables (e.g., age, sex, driving history). We discuss how sensation seeking and self-control theory provide a partial explanation of why credit scoring works (the psycho-social perspective). This article also presents an overview of biological and chemical correlates of risk taking that helps explain why knowing risk-taking behavior in one realm (e.g., risky financial behavior and poor credit history) transits to predicting risk-taking behavior in other realms (e.g., automobile insurance incurred losses). Additional research is needed to advance new nontraditional loss prediction variables from social media consumer information to using information provided by technological advances. The evolving and dynamic nature of the insurance marketplace makes it imperative that professionals continue to evolve predictive variables and for academics to assist with understanding the whys of the relationships through theory development.
In the paper, they show from actual industry experience the very strong correlation between credit scores (a continuous variable) and loss costs (also continuous):
Figure 1 Average Incurred Loss for Policies Grouped by Credit Score Deciles.
Figure 1 shows the average incurred dollar loss for policies in each decile. Over the entire data set, the average loss per policy was $695, but for those policies in the lowest (worst) 10% of credit scores, this average loss was $918, whereas for the highest (best) credit score decile, the average loss per policy was $558. Thus, the average loss per policy is 64.6% higher in the lowest credit score decile than in the highest credit score decile. Regression analysis confirmed a statistically significant relationship between credit scores and incurred losses (p < 0.0001, adjusted R2 = 0.92)13 using the midpoint of the credit score interval versus the average incurred loss for the interval.
Now, you may wonder what insurance credit scores have to do with COVID vaccination status, but this is the point: a poor credit score is the result of a particular behavior that may come out of higher risk-taking in general.
While the article goes through a whole discussion of psychosocial and biological mechanisms for risk-seeking behavior, I will go with something a lot simpler. There is something called the Big 5 Personality traits, which have in common just that the five traits have good experimental support. These go by the mnemonic OCEAN (Openness to experience; Conscientiousness; Extraversion; Agreeableness; Neuroticism) — and the one I want to focus on is conscientiousness.
As per Wikipedia: Conscientiousness is:
a tendency to display self-discipline, act dutifully, and strive for achievement against measures or outside expectations. It is related to the way in which people control, regulate, and direct their impulses. High conscientiousness is often perceived as being stubborn and focused. Low conscientiousness is associated with flexibility and spontaneity, but can also appear as sloppiness and lack of reliability.
Hmmm, that sounds like… high conscientiousness would lead to a high credit score and getting vaccinated when you’re supposed to.
Low conscientiousness would lead to low credit scores and not getting vaccinated.
Risk-taking people take risks
Let’s go back to the original table of relative risks from the original study:
I circled “No COVID Vaccination”, “Male sex”, and “Depression”.
All three have about the same relative risk after adjustment (that’s a statistical adjustment to adjust for interaction effects… I’m just assuming they did all the stats properly, but I do not know this to be the case.)
Males do have higher loss costs and generally higher rates of risk for all sorts of things (definitely accidental causes of death, suicide, homicide, and all sorts of “hey, hold my beer, watch this!” types of dumbassery).
So, not having a COVID vaccination is just as dangerous as being male, when it comes to motor vehicle accidents.
Conclusion: this will encourage people not to be the male sex.
Oh, that’s not the proper conclusion, is it.
There were other items in there you can see that really expose problems with the study — being old and having dementia show very low relative risks because, guess what, the people with dementia weren’t allowed to go anywhere at that time. The old people stayed home.
I noticed this myself when I looked at what happened with motor vehicle deaths in the U.S. – they dropped a lot for old people, because the old people stopped driving during lockdowns.
And that last bit can give you another hint for the idiocy of this study – these correlations for the non-COVID vaccination characteristics would have been very different in 2019 for the old ages (not dementia, perhaps). It would be useful to see what these relative risks were like for these characteristics pre-pandemic, which would give you an idea of how weird the situation is.
Correlations are not necessarily stable.
Correlative studies are interesting, but shouldn’t be used to set policy (or premiums)
If you used this study to set public policy or premiums without deeper investigations, you would deserve all the ridicule coming at you for your idiocies.
This often happens in the political sphere, though on the insurance side, insurers have been burned too often by blips in trends that they know not to move underwriting and prices too rapidly on what may be a spurious result.
Indeed, in a few years, one might even see changes in correlations with vaccination status, in which people who are on their 20th booster shot really display high levels of neuroticism (which is probably neutral for auto loss costs) more than high levels of conscientiousness. It’s a possibility.
The authors know this
Let me be fair to the authors – they know the correlative study is just that:
A limitation of our study is that correlation does not mean causality because our data do not explore potential causes of vaccine hesitancy or risky driving.60 One possibility relates to a distrust of government or belief in freedom that contributes to both vaccination preferences and increased traffic risks.61 A different explanation might be misconceptions of everyday risks, faith in natural protection, antipathy toward regulation, chronic poverty, exposure to misinformation, insufficient resources, or other personal beliefs.62 Alternative factors could include political identity, negative past experiences, limited health literacy, or social networks that lead to misgivings around public health guidelines.63,64 These subjective unknowns remain topics for more research.
If they were honest, they might mention: “or perhaps there are other explanations that didn’t come to our minds, because we don’t know enough people to talk with, and our imaginations aren’t broad enough.” I can think of (and know) a lot more other explanations, because I’ve heard them directly from people who aren’t vaccinated.
And, my brothers and sisters in statistical reasoning, if you bring up this study as a reason to get vaccinated, you deserve to have a dead fish thrown at your head.
Weird, weak conclusions
Back to the paper:
Our findings have direct relevance by highlighting how injury risks have continued despite the COVID pandemic.
Despite? Why would accidental injuries stop because of the pandemic?
Primary care physicians who wish to help patients avoid becoming traffic statistics, for example, could take the opportunity to stress standard safety reminders such as wearing a seatbelt, obeying speed limits, and never driving drunk.
Yup, all these items have direct cause-effect relationships to motor vehicle accident incidence and outcomes.
The observed risks are sufficiently large that paramedics, emergency staff, and other first responders should be aware that unvaccinated patients are overrepresented in the aftermath of a traffic crash.
No, it’s a complete non-sequitur. It’s meaningless.
I’m sure they’re overrepresented among people who like to bungee jump, parkour, BASE jump, free climb, etc.
Think about it.
WITH YOUR MIND.
Together, the findings suggest that unvaccinated adults need to be careful indoors with other people and outside with surrounding traffic.
No.
There is nothing magic in the vaccines that change the result of your motor vehicle accident risks.
FOR CRYING OUT LOUD.
Ugh.
Anyway, yes, the correlation between COVID vaccine status in Canada and motor vehicle accident incidence is probably real, for certain values of “real”.
That realness may only be temporary, unlike the real correlation between (insurance) credit scores and auto loss costs. Insurers would likely be foolish to underwrite based on the COVID vaccine status, and it would be smarter to (continue to) use insurance credit scores, if only the regulators allowed them.
Yes, it’s a good idea not to drive like a maniac, no matter your vaccine status. Also, don’t sleep on the sidewalks; you could get run over.