A Case to Reconstruct the World of Business Applications with Data Governance By Kash Mehdi
Many industries worldwide deploy a wide variety of Business Applications (BAs) as a channel to offer services and, in return, they can capture, funnel, store and retrieve data which they can use for business activities such as reporting, predictive analytics, and other operations. Depending on the industry type, BAs can provide a wide array of… more
The post A Case to Reconstruct the World of Business Applications with Data Governance appeared first on The Informatica Blog – Perspectives for the Data Ready Enterprise.
Many industries worldwide deploy a wide variety of Business Applications (BAs) as a channel to offer services and, in return, they can capture, funnel, store and retrieve data which they can use for business activities such as reporting, predictive analytics, and other operations. Depending on the industry type, BAs can provide a wide array of services and offerings. A few cross-industry examples include:
- Insurance: Agents selling various types of insurance use BAs to retrieve product name, type, and plan details.
- Financial Services: Brokers use BAs to manage personal or commercial brokerage accounts to create customer portfolios (e.g., Customer Trust Fund), capture country of residence, and model potential tax liabilities.
- Healthcare: Workers use BAs to register incoming patients, capture medical history, and document procedures.
- Government: Border agents use BAs to fact-check visitors’ immigration status, reference visa documents, and verify background.
- Oil and Gas: Employees and managers use BAs to capture the supply demand of oil fields, monitor truck inventory, and confirm shipment transportation status.
- Higher Education: Student services staff use BAs to register incoming students, track programs, maintain curriculum schedules, enter grades and course completion certification, and verify transcripts.
These are just some examples—many other industries use BAs to execute relevant business activities.
The Challenges with Business Applications
Most BAs funnel manual data entries from operators. For most organizations, their data volumes and complexities continue to grow (and even double) every 12-18 months. This data growth presents unique challenges due to a lack of data understanding, consumer access, usage guidelines, relevance, and fitness of data for any specific purpose. The question then becomes: “How to create BAs that conform to commonly understood data fields, ownership, and the underlying infrastructure (i.e., the data pipeline that connects a vast ecosystem of disparate data sources, which in turn gets utilized by various business functions for daily operations)?” Finally, what’s the best way to distill a business-friendly user interface for adoption, keeping in mind cultural complexities and workforce data literacy?
BAs can lose touch and applicability over time, mainly due to changing technologies, varying user demands, and shifting market environments (as we’re seeing due to the COVID-19 pandemic). To address these gaps and others, organizations will undertake redesign and application consolidation projects such as cloud data lakes, data warehousing, ERP migration, and others. Because operators traditionally enter data manually in most BAs, this has usually led to the information becoming collated in several systems and applications over time. If left untouched, these data sources can turn into black box silos and lose their timeliness. Organizations that treat data as an enterprise asset have to act quickly to preserve decaying data value and insights.
One of the classic challenges many organizations face includes designing applications in environments that lack business stakeholder collaboration, input, and shared data understanding, which strains an organization’s ability to build practical business applications and alignment with business goals and objectives. Despite existing protocols for application development, not all organizations—including the top FinServ institutes—are equipped with acceptable data practices or have resources that can corroborate on application design, development, and implementation projects. Most redesign projects occur on an ad-hoc basis (e.g., ERP redesign and consolidation), further straining an organization’s ability to move in concert to create an effective data ecosystem. The result of such poor practices can lead to hardship when meeting business goals and outcomes. The COVID-19 pandemic has exposed many organizations where they continue to struggle when undertaking multi-cloud journeys. More and more companies in North America and throughout Europe, Middle East, and Africa are embarking on digital transformation projects in order to mobilize communities of data users and break the black box data silos.
How Better Data Practices Can Solve the Problem
One approach used by organizations includes establishing acceptable data practices and business enablement, which translates to a strong case for data governance, cataloging, quality, and privacy. For application development, data governance drives collaboration between business and technology groups when creating or modernizing BAs and cataloging to identify and tag the data ecosystem with appropriate standards for business context, quality, and privacy.
A typical application development process warrants cross-functional collaboration. The business stakeholders start by building a data dictionary or a business glossary to capture a list of lexicons, sometimes referred to as Application Data Elements (ADEs), to achieve shared data understanding. Once these ADEs are defined, the business stakeholders work closely with the product teams responsible for designing the User Interface and Experience (UI and UX). Finally, the engineering teams execute application development and implementation. The trio works to create business-friendly applications that support enterprise needs for effective data capture, storage, and retrieval for business operation needs while preserving optimal user experience and adoption.
One Global Financial Institution’s Story
A customer story from a London-based global financial institution comes to mind. Federal regulators requested significant documentation from this organization to better understand how the institution was managing customer risk and measuring the quality of its products and services. At the time of the regulators’ request, the organization needed an average of 5-6 weeks to respond to regulatory requests, which increased pressure for the C-suite executives who were responsible for the institution’s compliance mandates. Significant gaps in how the organization was measuring quality were identified. For the most part, there was no formal data quality function, and quality was being measured in different places via manual PLSQL scripts, custom programming hardcoded in applications, etc. These varying approaches made it difficult for the organization to have a holistic view of quality or of one type of scorecard/dashboard—a situation that became even more challenging if the regulator was asking about the quality of a specific type of product.
The organization hired a major Systems Integrator (SI) to undergo a data assessment. This assessment identified that the root cause of the regulatory delays as the inadequate application designs, along with many other gaps, including:
- Challenges to capturing data and alignment with organizational data needs
- Limited knowledge of who the data producers and data consumers were
- How users interacted with various business applications, and whether those applications met expectations
- A consistent process for creating a data inventory of elements and management.
- Information captured around a client’s tax attributes and knowing applicable attributes for U.S.-based vs. Europe-based business entities and understanding protections under applicable regulations such as GDPR and CCPA
- Shared data understanding and duplication. For example, data on “country of residence” was stored in three different places, with varying definitions
- Understanding of data flows and knowledge of the source and target information, including manual transformations
- Limited documented controls for data standardization, privacy, and protection
And the list of challenges and what wasn’t working kept growing. About 2 – 3 years earlier—before they hired the SI—the organization had gone through a year-long effort, and had at that time created an Excel spreadsheet (which got about 60% right) to document the current state of all data elements that were part of existing business applications. The business stakeholders were the ones facing the pain and living with the spreadsheet all the time. However, because no one had formally verified the spreadsheet or checked to see whether the data attributes were valid, as time passed, and changes continued to occur on business applications and ADEs, the year-long effort to build the spreadsheet was wasted as the information became stale over time.
A key highlight of the SI assessment revealed that the application development process had limited input from business stakeholders. While the business stakeholders maintained a data dictionary to capture Application Data Elements (ADEs), updates to the dictionary happened after the application implementation—when, in fact, it should have been the other way around (i.e., software changes should first include updates to the data dictionary with mandatory ADEs, and then follow suit for the application development). As the software application goes from QA, Development, and Production, the dictionary should move in tandem.
After the assessment, it was clear why many applications that had been there for 15-20 years failed audit requirements. The Financial Institution embarked on a journey to modernize its applications. Taking the SI recommendation and getting buy-in from the C-suite, they created a formalized data governance program to organize business, product, and engineering stakeholders. The program started the data strategy and operational tasks such as:
- Business: Alignment on common data understanding and ongoing review of ADEs
- Product: Consult with multiple personas to capture daily data needs, especially regarding how UIs need to be designed and experienced
- Engineering: Incorporating changes to the software applications and managing availability and minimizing disruptions resulting from cumbersome implementation cycles
A Strong Case in All Kinds of Industries
Similar examples exist in other industries where challenges emerge as a result of a flawed application development process, lack of acceptable data practices, and missed opportunities for collaboration with business stakeholders. And while organizations may realize the need to govern application design processes, they often only really experience ramifications later in the cycle. Alignment of business applications with real business needs can make or break a company. Ad-hoc fixes to legacy systems and applications ultimately add up to more significant challenges in the future, requiring organizations take immediate action when unable to meet business goals and outcomes. For robust and sustainable application development, it’s always an advantage for an organization to consider acceptable data practices such as data governance, enterprise catalog, quality, and privacy. The organization not only increases its ability to move all employees and leaders in concert to stay ahead of the competition but, more importantly, will fully benefit from data-driven innovations, especially during challenging times like the COVID-19 pandemic or whatever the future might unfold.