Who decides who deserves credit? The answer, long shaped by unclear rules and human-lead processes, has never been as objective as the industry would like to claim. Recently, with the surge of new technologies, AI promised to fix that, but it’s proving to be part of the problem, since its as biased as the ones feeding those LLMs.
AI systems in lending are typically trained on historical financial data, which is riddled with embedded bias. For example, Black and Hispanic borrowers in the U.S. have historically been denied mortgages at higher rates than white applicants, even when controlling for income and credit profiles, a recent study from the MIT showed. When these biased outcomes feed into machine learning models, the algorithms learn to replicate them, even if race is not explicitly included as a variable.
The problem
The root of the issue lies in what experts call “noisy data.” Minorities, low-income borrowers, and people in rural areas tend to have thinner credit files or nontraditional financial histories. These gaps mean that AI models trained on such data often make less accurate predictions for these groups. According to Stanford’s Human-Centered AI (HAI), this lack of representation skews results and leads to unfair risk assessments, exacerbating financial exclusion even further.
It gets worse when algorithms optimize purely for accuracy or profit without regard to fairness or macroeconomics. If an AI system finds that denying loans to people from certain ZIP codes improves its predictive accuracy, it will do so—even if those areas are historically marginalized. These “rational” decisions are deeply problematic from a social equity standpoint, and they’re often invisible to both developers and regulators.
The industry’s reliance on alternative data, like rental or utility payments, is often pointed out as a solution. But this too can be a double-edged sword. Not all consumers have equal access to these services, and inconsistencies in data collection can introduce new forms of bias. Moreover, regulators have yet to catch up with how to monitor and audit these unconventional variables effectively.
New solutions
Some fintechs are trying to address the issue. Companies like Zest AI and Upstart claim to use AI to expand credit access to underserved populations by including more nuanced data points and stress-testing models for fairness. However, independent audits of these systems are rare, and most firms consider their algorithms proprietary, making transparency difficult.
Despite the buzz around “ethical AI,” meaningful oversight is still lacking. Researchers argue that without interventions—such as requiring fairness metrics, bias audits, and interpretability features—the deployment of AI in lending could worsen the very inequities it claims to fix. Some even suggest that flawed AI models may mask discriminatory practices more efficiently than humans ever could.
The problem isn’t just with the algorithms, it’s also with the culture of innovation that prizes disruption over due diligence. Without more rigorous frameworks, the risk is that AI will continue to give a scientific gloss to fundamentally unfair practices. If credit access is a cornerstone of financial mobility, then fixing bias in lending AI is not just a technical challenge: it’s a societal imperative.
