The Future Of Actuarial Risk Modeling Is Smarter, Not Just Faster, Computing
Sanchet Pachpute is CIO of New York Life Insurance Company, where he leads platform modernization for Institutional Annuities.
gettyFor decades, actuarial modeling has operated under a constraint most of the business never sees: the sheer computational cost of understanding risk. Projecting and valuing a large insurance portfolio means running thousands of stochastic scenarios across a multi-decade horizon, accounting for mortality, lapse behavior, market conditions and regulatory reserve requirements at every step.
Done the traditional way, on a single machine working through calculations in sequence, that kind of analysis can take weeks. By the time the results land, the market conditions they were modeling have often already moved. This is a problem every carrier running sophisticated stochastic models eventually runs into, regardless of size or line of business.
In the past, how I’ve approached solving this is through high-performance computing: taking a workload built to run sequentially and rearchitecting it to run in parallel across a distributed cluster instead. In one case, a full valuation of a multibillion-dollar portfolio that took roughly 50 days of continuous sequential processing was rebuilt as a distributed workload and came down to roughly two hours. It was the same calculation and the same accuracy requirements spread across many machines working simultaneously, instead of one machine working through a queue. That’s close to a thousand-fold reduction in runtime, and it changes what the function can actually do day to day.
The architecture behind this kind of solution is, at its core, a straightforward idea executed with discipline: Break a massive computation into independent pieces, run them in parallel and reassemble the results efficiently. A coordinating process distributes scenarios across worker processes, each running its assigned share and streaming results back as it goes, with a final step combining everything into a portfolio-level answer.
None of the individual pieces are exotic. What makes this kind of system reliable in production is the discipline around it: fault tolerance so one failed worker doesn’t take down the whole run, careful memory management so dozens of processes aren’t duplicating the same reference data and verification steps that recheck a sample of results the traditional way. This confirms the parallel version hasn’t introduced errors nobody would otherwise catch.
The business impact shows up immediately in cadence. Valuations that used to run quarterly, because that was all the computational budget allowed, can run daily instead. Scenario coverage that used to be limited by time constraints can expand by an order of magnitude, which matters directly for tail-risk analysis. These include the rare, severe scenarios that conventional scenario counts tend to underrepresent simply because running more of them used to be computationally unaffordable. Going from 50 days to two hours doesn’t just save time; it converts actuarial modeling from a once-a-quarter batch report into an on-demand tool the business can actually iterate on.
Where this gets more interesting, and where I think most carriers haven’t caught up yet, is the next layer: using AI not to replace this kind of computing architecture, but to make it smarter about where it spends its cycles. Not every one of a thousand scenarios contributes equally to the final answer. Many cluster tightly around similar outcomes. Machine learning techniques can identify which scenarios are actually representative and which are redundant, letting a cluster spend its compute budget on the scenarios that materially affect the result rather than treating all of them as equally important.
Similarly, trained models can act as fast approximations for the most expensive parts of a calculation, with the full computation reserved for validation and for the scenarios where precision matters most. The computing architecture earns you the raw speed; applying AI on top of it earns you speed where it counts.
That second layer comes with a cost that’s easy to underweight in the excitement of the first. A faster model is only useful if you can still explain it to a regulator, and insurance is an industry where every material modeling assumption eventually has to be defensible, not just accurate on average. A surrogate model that approximates a reserve calculation needs validation against the full calculation on a meaningful sample, documentation of where it’s expected to be reliable and where it isn’t and a clear record of why it was trusted for production use. Skipping that step to chase speed is how a modeling shortcut turns into a regulatory finding 18 months later.
For technology leaders facing this same computational wall, the sequencing matters more than the headline speedup. Start with the distributed computing foundation and prove it against the existing process before trusting it. A 50-days-to-two-hours result only means something once the new system’s output has been validated against the old, slower one at scale.
Only after that foundation is solid should you layer AI-driven optimization on top, because AI applied to an unvalidated computation just produces unvalidated answers faster. Speed that your regulators and your own risk committee can’t fully stand behind isn’t actually an advantage; it’s a liability wearing the costume of one.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
