What if every crucial business decision you made was shrouded in doubt about your data’s reliability? This unsettling scenario is a daily reality for many organizations. Trust in data analysis is paramount for informed decision-making, yet it’s often undermined by challenges that are not only human in nature. Technical factors that affect the data itself, such as collection, storage, and integrity are critical in how data is perceived and utilized. Where part one of this series explored the human significance of trust in data and the analysts who handle data, part two explores the more technical aspects that bridge data and trust.

Here is the link to Part 1 if you would like to go back and cover the more human challenges of trust with data.

Importance of Trust in Data and Analysis Revisited

The Unseen Gaps: A Bank’s Credit Review Process Unraveled

A seasoned wholesale lending manager at a prominent bank, prided herself on maintaining a rigorous credit review process. Her team’s thoroughness had consistently met the bank’s standards, ensuring that only the most qualified clients received loans. However, when internal auditors decided to conduct a comprehensive review of the credit process, leveraging the expertise of the bank’s data team, her confidence and trust in the data would be challenged.

The audit began as routine, but quickly revealed that all was not as it seemed. The auditors, partnering with their internal data team, began to uncover discrepancies that the lending manager’s team had missed. What initially appeared to be minor oversights gradually unraveled into more significant issues, which led to underlying flaws in the credit review process. While the bank had seen historical success with its wholesale lending unit, there have been some discussions around whether the bank was making the best lending decisions. Moreover, there have been social media posts of qualified customers getting denied credit for no apparent reason.

The first red flag the auditors identified was related to the quality of the data being used in the credit review process. The bank’s data team found that a significant portion of the financial data used to evaluate clients’ creditworthiness was outdated or incomplete. For example, financial statements from some clients were missing critical details, and several data entries contained errors that skewed the overall analysis. These data quality issues meant that the wholesale lending team was making decisions based on flawed information, leading to an inaccurate assessment of risk. The audit team recommended a modification of the lending software application to prevent an application from being complete when necessary information is missing from an application.

As the audit continued, it became clear that a lack of transparency during the credit review process exacerbated the problem. The data team discovered that the methodologies and criteria used to assess credit risk, based on the application data, were not clearly documented or consistently applied. This lack of transparency between the application data and the eventual credit review decision made it difficult for auditors to understand how certain credit decisions were made and why some high-risk loans had been approved. Without a clear audit trail, the process appeared opaque, leading to suspicions of inconsistency and potential bias in decision-making.

The data team also uncovered evidence of biased data and analysis within the credit review process. Historical data used to assess credit risk had been influenced by subjective judgments, applicant data, and regional preferences. For instance, loans from certain regions were more likely to be approved despite similar financial profiles to those from other regions, indicating a bias that was not aligned with the bank’s overall risk management strategy. Furthermore, the audit data teams uncovered that the existing credit review models were trained on data in past years that denied credit to minorities and special interest groups, even when they financially qualified, subjectively denying certain applicants. This bias skewed the risk models and led to a distorted view of the clients’ creditworthiness.

Another significant issue identified by the audit data team was the complexity of the data models used in the credit review process. The models, designed to evaluate a multitude of financial variables, were so complex that even experienced team members found them difficult to interpret. This complexity led to misinterpretation of the data, with some team members overestimating the creditworthiness of certain clients while underestimating the risks associated with others. The auditors noted that without proper training and clear guidelines, the risk of misinterpretation was too high to ignore.

Finally, the audit revealed inconsistent data governance practices across the bank’s various departments. While some departments adhered to strict data management protocols, others were lax in their data entry, storage, and validation practices. This inconsistency led to a situation where data from different sources could not be reliably compared or combined. For wholesale lending team, this meant that they were often working with data that had not been properly vetted or standardized, further undermining the reliability of their credit assessments.

The audit’s findings were a wake-up call for the wholesale lending manager and her team. The holes in the credit review process, exposed by the auditors and the data team, highlighted the critical need for improvements in data quality, transparency, and governance. The complexities of the current system and the biases in data analysis had created a perfect storm, leading to flawed credit decisions that put the bank at risk.

In response, the bank implemented a series of reforms aimed at addressing these issues. Data quality checks were intensified, transparency in decision-making was improved, and consistent data governance practices were enforced across all departments. Training programs were introduced to help staff better understand and interpret complex data models, and efforts were made to eliminate bias from the wholesale lending models by retraining them on more current data.

What Are The Technical Challenges That Cause a Lack of Trust in Data?

As this story illustrates, building trust in data can be a technical challenge. When a company’s data cannot be relied upon due to issues such as data quality or biases, trust can quickly erode among users of the data and business partners that rely on analysis to make key decisions. As mentioned in the story, the following 5 technical challenges are areas that should continually be addressed to allow trust with data:

How Can We Address These Challenges And Build Trust With Your Data?

To overcome these challenges and build trust in data analysis results, businesses consider the following strategies:

Conclusion

Trust in data is crucial for leveraging data analysis to its fullest potential. By addressing data quality issues, fostering transparency, mitigating biases, enhancing data literacy, and establishing robust data governance, organizations can build confidence in their data and the analysts who interpret it. Additionally, partnering with employees to understand their roles and fostering a collaborative environment can further strengthen trust in data analysis. Building a culture of trust in data is not just a technical challenge but a human one, requiring commitment and collaboration between data teams and their stakeholders. Implementing these strategies ensures better decision-making, drives innovation, and provides a competitive edge in the marketplace.

Thank you for taking the time to read through my two part series on Trust in Data and Data Analysis, please be on the lookout for more content!

Do your teams struggle to trust the data and analysis they use on a daily basis? Reach out via the form below so we can address this gap.

Image provided by OpenArt, link below:

OpenArt