Data collection isn’t the problem: It’s what companies are doing with it

Data is a company’s most powerful asset. Yet, many businesses cannibalize this valuable asset by selling it to third parties when they should be using it to make their businesses stronger and more sustainable.

Nearly all digital businesses collect some type of data from their users, so there has been growing concern from privacy rights groups about how that data is used. Yet, data collection is not wrong in and of itself. It’s the why, how and what is done with it that matters most when it comes to building a profitable and sustainable business that simultaneously respects the privacy of its users.

In the majority of cases, there is no nefarious man behind the curtain collecting data for evil. Most companies rake in as much data as they can under the assumption that you never know when and how data might be useful at some point down the line.

Thankfully, this is starting to change, and data scientists at data-driven companies are leading the charge. Collecting data based on a vague hypothetical scenario indicates a lack of intuitive understanding of what kinds of data are actually important to have from users, but smart companies are rightly asking only for the data that is needed to provide products and services to the end users.

Making data work for you through AI and a data fabric

Instead of selling user data to make money, data-driven companies have opted to analyze this data to understand how to gain the most useful insights. Know Your Customer (KYC) initiatives are dependent on data, using artificial intelligence (AI) to analyze the information to uncover preferences that users might not be talking about in online reviews.

Companies like Pepsi are leading the way in using AI for consumer product development purposes, and digital businesses can and should follow suit. Online platforms that want to go this route should beef up their in-house capabilities by hiring more data scientists and AI experts.

In addition to helping improve customer experience by enabling better personalization and customization options, AI can assist in making the onboarding process smoother and seamless for products and services.

As data becomes more complex, companies are trying to make more efficient use of their troves of data by implementing a data fabric — an interconnected layer of data and processes that supports composite data and analytics, as well as their various components.

A data fabric lets companies reuse and combine different styles of data science, enabling them to reduce integration design time by up to 30%, deployment by up to 30% and support by as much as 70%. In addition, a data fabric allows firms to use existing skills and technologies from data hubs, data lakes and data warehouses, as well as introduce new approaches and tools for the future.

Companies that want to implement a data fabric should start by integrating machine learning algorithms into every level of data — from collecting the data to optimizing and cleaning it. They should use cloud technology and implement flexible configurations, unification and fast access to data. They will also need to understand their database orchestration processes and data flows and implement the end-to-end integration of their databases.

Fintechs and banks are using data fabric to protect data by managing the access to resources while also putting in place customizable and personalized product and service offers. Lloyds Banking Group, for example, uses a data fabric to analyze customer behavior and to improve its products and services.

Retail and grocery firms today use data fabrics to improve remote customer care services by analyzing customers’ requirements and demands, as well as logistics. Apple, for one, uses data fabrics to improve its customer care service and technology offerings. Even dating apps now use data fabrics to quickly access big data.

Using decision intelligence frameworks to optimize solutions

All companies collect data to improve their products and services to customers’ preferences and requirements. As a result, all companies should take a long and hard look at their data collection methods and motivations.

One way to do this is through decision intelligence (DI), which is a discipline that includes a wide range of solutions, including traditional data analytics, artificial intelligence and complex adaptive system applications. This intelligence is applied to individual decisions as well as decision sequences, grouping them into business processes and urgent decision-making networks.

Creating such structures allows organizations to get the information they need to stimulate business. Combined with the ability to lay out an overall data structure, engineering the analysis of solutions opens up new opportunities to rethink or redesign how a company can optimize these solutions to make them more accurate, reproducible and traceable.

Intelligence in decision-making can also be used to analyze data linked to the processes on a digital platform. For example, DI can help e-commerce businesses use data to understand the best way to redesign the user journey to minimize abandoned shopping carts.

Companies that want to use DI should have a clear vision and strategy for how the data will be used to improve products and services. They will need to hire or develop qualitative data scientists and experts and to organize stable data collection and data engineering processes. Finally, DI requires a company to correlate the collected data with hypotheses related to business goals.

Many data-driven companies currently use DI in their daily business. In the banking, finance and fintech industries, DI helps institutions analyze customer behavior, predict their requirements, solve issues, and customize products and services. For example, Morgan Stanley uses DI in its fund management platform and to improve decision-making for investments.

Retailers have employed DI to make decisions on pricing policies, predict customer behavior and optimize the supply chain. Amazon, for instance, uses both data fabrics and DI to optimize its supply chain. In healthcare, DI allows practitioners to analyze medical reports faster and enables doctors to more easily prioritize successful treatments.

Recognizing and predicting trends requires strategic data collection

Ideally, data-driven businesses should not be strategizing more than a year ahead; they should instead be analyzing and utilizing the collected data on a continuous basis. If this is baked into the workflow, changes to user wants and needs will be reflected in the data, and smart digital platforms will be able to see if they need to start pivoting or adjusting their offerings to users.

Companies need to be strategic in determining how much and what kind of data they collect. When companies collect too much data, much of it is irrelevant and simply makes it more difficult to sift through to find the relevant data. Therefore, developers should be key internal stakeholders in the evaluation of the quality of the data being collected for analysis.

When a business is conducting product-market fit research, the collected data should help to prove or disprove particular hypotheses that the company is testing, and it is up to the product and business developers to drive the data requests according to their hypotheses.

To further optimize the data collection process, companies should avoid building hypotheses and data collection methods based on the experience of their competitors. Determining product-market fit can only be accomplished when the data being collected specifically relates to the company’s own products and user base. Ultimately, ready-to-go tools — such as Power BI, Tableau or AutoML — in combination with data scientists’ skills at using Python, C# or MATLAB will help the company to use their data to make the best decisions.

Building a data-driven business without sacrificing user trust

Invading user privacy by collecting data just to sell it is an unimaginative waste of time and business intelligence, and it can irreparably damage a company’s relationship with its customers. Smart businesses are establishing relationships with users built on trust — trust that the data users are handing over will ultimately benefit them in the form of features and services that meet their ever-changing needs.

Building a data-driven digital business that respects user privacy doesn’t mean sacrificing profits. On the contrary, satisfying users by fully understanding their needs — through strategic data collection and the use of AI and decision intelligence — is the only way to ensure long-term profitability.