Built on three years of interpretability research, validated in collaboration with leading research and industry organizations.
The problem
Models can look right overall and still be wrong where it matters.
Financial institutions increasingly use predictive models trained on historical data to make decisions about credit, fraud, pricing, underwriting, customers, and risk.
But a model can have high overall accuracy scores while repeatedly making the wrong decisions in specific, hard-to-find parts of the portfolio:
- A credit model can decline good applicants who do not fit the usual profile.
- A fraud model can miss a particular kind of transaction.
- A pricing model can overprice one segment and underprice another because of an interaction between two factors.
These types of issues generally can't be found using current techniques, unless a team has exactly the right hypothesis – out of billions of possibilities. And at institutional scale, undetectable model errors can mean substantial losses, missed business, customer friction, or unmanaged risk.
What Halley does
Halley shows what the model learned, and where it is wrong.
Halley analyzes a trained model together with the data needed to evaluate it. It searches systematically for important learned relationships and concentrations of error, then returns the findings that matter.
Reveal learned behavior
See the interactions, thresholds, and relationships that materially affect the model's decisions.
Find hidden concentrations of error
Identify specific segments where the model is consistently overestimating, underestimating, approving, declining, flagging, or missing.
Understand the consequence
Where outcome data is available, assess how the discovered behavior relates to model performance, loss, missed opportunity, or customer friction.
Verify every finding
Each finding can be reproduced independently in the underlying data. Business leaders, model owners, and validators can work from the same evidence.
Halley works across all types of predictive models trained on structured tabular data, from logistic regression and tree-based models to XGBoost and neural networks.
Illustrative example
What a Halley finding looks like
A credit model performs strongly across the portfolio. Halley discovers that it systematically overestimates risk for a particular type of applicant when several otherwise unremarkable characteristics occur together.
The effect does not appear when those variables are reviewed individually, the team did not find it through hypothesis testing, and there was nothing for monitoring to pick up on as it had been there all along.
Halley shows:
- The segment it discovered
- The interaction driving the behavior
- How the model performs within that segment
- The scale and potential significance of the issue
- The underlying observations needed to verify it
- The options the institution may wish to test
Outcomes
Improve performance. Strengthen model governance.
The same analysis serves two institutional priorities.
Improve the decisions the model makes
Most models are not fundamentally broken. They are leaving performance on the table in places the usual checks do not see.
Halley helps teams:
- Uncover recurring sources of loss
- Identify good business being declined, mispriced, or overlooked
- Find fraud or risk concentrated in specific segments
- Reduce unnecessary customer friction
- Focus model improvement on the areas with the greatest potential value
Give validators stronger evidence
Halley complements existing model development, validation, and monitoring frameworks.
It helps model risk and validation teams:
- Discover limitations that were not identified in the documentation
- Examine behavior beyond aggregate metrics and predefined tests
- Identify populations where performance materially differs
- Challenge unexpected learned relationships
- Prioritize areas requiring further testing or remediation
- Verify findings directly in the institution's own data
Use cases
Built for the decisions financial institutions compete on.
Credit and lending
Find applicant and account segments where risk is being systematically overestimated or underestimated.
Reveal interactions associated with false declines, weak pricing, missed opportunities, or unexpected loss.
Fraud and payments
Identify transaction, merchant, channel, or customer segments where fraud is being missed or legitimate activity is being stopped.
Find combinations of factors behind persistent false positives and false negatives that portfolio-level metrics do not reveal.
Pricing and underwriting
Surface interactions among pricing and underwriting factors that contribute to systematic overpricing, underpricing, poor selection, or lost conversion.
The same capability can be applied to other high-value predictive models trained on structured tabular data.
Deployment
Designed to fit your existing environment.
- Deploy on-premises or in your private cloud
- Keep models and data inside your infrastructure
- Begin with read-only access
- Make no changes to production systems
- Work with your existing development, validation, and monitoring processes
- Use your current model stack, with no proprietary model format or vendor lock-in
- Reproduce every finding using your own data and standards
Research
Built on research. Designed for practical use.
Halley's technology grew out of three years of interpretability research and has been developed and validated through collaborations with researchers at MIT, Meta, UCL, and other leading organizations.
The research addresses a practical problem: the number of possible relationships and subgroups inside a modern dataset is far too large for expert teams to investigate manually.
Halley turns that research into a system financial institutions can apply to their own models, data, decisions, and outcomes.
“It would take us one postdoc year to analyze this… and you found something that we may never have found, that could be worth billions.”
– Senior scientist, U.S. national research center
Talk to us
See what Halley can find in one of your models.
Begin with one high-value model. Halley will show what the model has learned, where its performance is breaking down, and where there may be an opportunity to improve it.
