How to Use EDA to Become a Data-Driven Company
People engage in Exploratory Data Analysis (EDA) every day since many of our decisions are based on some form of data analysis. Before making a large purchase, like a car, we read reviews and research safety ratings. We test-drive different models. We look at our finances to see what we can afford. Then we use all that data to make a final decision. If we need a car loan, the bank does EDA on our finances, credit history, and job security to decide whether to grant us a loan, for what amount and at what interest rate. Like banks are doing with car loans, your business should be using EDA to harness your most profitable bottom line.
What is EDA?
Exploratory Data Analysis is a visualization concept defined by John Tukey in 1977. It’s the process of gathering information and putting it together—usually through visual models—in a way that makes it easier to understand and analyze.
The EDA process itself is usually based on simple techniques, including data plotting, statistics application, and generating visual representations of the data, so patterns and trends are more easily revealed and understood.
EDA is sophisticated because it determines both how to define a data set and also how to scrutinize it. If the wrong data is being analyzed, the process doesn’t matter. However, it is equally important if an incorrect analysis method is used, the quality of the data is irrelevant. EDA experts will develop an intuition for both what to look for and how to interpret what they see.
Crucially, EDA is meant to inform decisions by revealing patterns, not confirming or rejecting decisions made based on assumptions. It is the initial examination of data and should occur before any assumptions or conclusions are made to avoid faulty analysis.
When Tukey wrote his seminal book on EDA in 1977, it was an analog process. There were only a few pieces of information available to make decisions, and they required effort to gather into one place. With the advent of the internet, the data available has exploded, and potential EDA applications are now more efficient and significantly more complex.
With customers able to do a quick online search, it was no longer sufficient to advertise in the Yellow Pages or the newspaper; businesses had to adapt by building websites, optimizing search engine results, and responding to customers in various new ways and as quickly as possible. This had businesses scrambling to do their own EDA: what was the best way to build a website, and who should be hired to do it? What kind of online presence would best serve existing customers and attract new customers? The most forward-thinking businesses were also asking themselves this question: What data can and should we track online, and how can we leverage that data? That’s the foundation of business intelligence (BI).
When it comes to BI, EDA helps businesses with the critical initial investigation of data by tying together and making sense of the vast amount of information generated every day. This enables companies to leverage billions of data sets to uncover information patterns that human analysis may never see. EDA uses available data to reveal and predict trends for sales-based businesses, which informs decisions about how to market and sell one’s products and retain customers.
We’ve come a long way since the Wild West days of the internet exploding onto the scene in the 1990s, but businesses have not been able to slow down their adaptations to the ever-changing nature of our digital reality. The internet has made markets significantly more competitive, and the vast majority of businesses cannot afford to ignore the powerful applications of EDA if they want to stay relevant and viable in modern markets.
EDA vs. Reporting
Some companies may think they already have an EDA process in place, but what they are actually doing is generating reports. Reports can lead companies to chase their tails because they are not future-forward. Reporting shows what was relevant in the past, so actions taken as a result of reporting are retroactive and may not reflect the current or future market reality.
For example, most businesses run regular (monthly, quarterly, annual) financial reports. This is standard procedure to show a management team or board of directors what was spent and on what in a given time period, in addition to how much liquidity the business has. And while these static reports have real applications, they are not very useful to inform business strategy.
By applying EDA to financial information, a company is able to analyze its finances to reveal trends in past or present performance, which can then be used to inform future strategies. The most common elements of analyzing finances using EDA are horizontal analysis, vertical analysis, and ratio analysis.
- Horizontal analysis examines specific line items across a period of time (the longer—the better).
- Vertical analysis takes all line items and breaks them down as percentages of the whole.
- Ratio analysis uncovers statistical relationships, reveals strengths and weaknesses, and shows how well a business performed over a given period of time.
Financial analysis is incredibly important for any business, as is sales analysis. Again, most businesses run sales reports but don’t necessarily perform sales analysis. As with finances, sales reporting plays an important role, but it does not replace EDA. Sometimes reports reveal differences or anomalies, but without strategic analysis, what can be done with that information? How can it be used to translate into more sales, higher customer satisfaction, or better retention rates? Looking at trends to predict the future is a key element of successful EDA and gives companies the best chance of translating data into future success.
Why use EDA?
EDA is not a new practice. Larger businesses have been using EDA for years, and now more small and mid-size businesses recognize the critical role a streamlined EDA system can play in attracting and retaining customers.
By examining the data your business collects, initial EDA efforts let you manage that data efficiently and in a way that can confirm existing strategies or expose why or how they might not be working. Even better, once EDA is entrenched in business operations, it can reveal patterns before new decisions or strategies are implemented. Remember, EDA is meant to be an initial exploration of data.
Learning to use data analysis most effectively can require a cultural shift in your business approach and will require education and buy-in among all stakeholders. It should be an easy sell because when done right, EDA leads to improved customer retention and satisfaction and often translates into new customers and increased sales.
Take, for example, a produce supplier that does not track a customer’s reasons for returning or rejecting orders. When the customer calls to complain, a sales representative does everything they can to make the order right—sends a replacement order or processes a refund—but does not enter data about what was wrong with the order in the first place. As a result, the produce company cannot meaningfully review and address its customers’ biggest issues. Was the produce spoiled? Was it misclassified or mislabeled? Was the order simply filled incorrectly? If that information isn’t recorded and tracked, the business cannot use it to improve their services or their bottom line.
Now imagine that same produce company a year after one of its IT specialists gets EDA training, and the business implements an EDA plan. The data set revealed that the majority of customer complaints were related to fruit being overripe. The company has worked with its farm suppliers to implement new policies and practices about how the fruit is picked, packaged, and stored, and its warehouse team has new protocols for inspecting orders upon receipt and again before shipping them out. The sales team was trained to educate customers on which fruits should not be placed near each other—who knew that bananas emit a gas that ripens other fruits! The marketing team has created a set of infographics explaining appropriate storage temperatures and conditions. Complaints about overripe fruit have dropped by 64%, and many customers are placing bigger orders because they have more trust in the product and the business. This is a simplified example, but it is a powerful illustration of how EDA can reveal patterns that we may not be looking for and that can have a huge impact on sales once revealed.
EDA can also use outside data, like a third-party data set, to expose trends and patterns across the marketplace that your company can leverage to develop new marketing and outreach strategies and attract new clients. For example, a seasoned politician who wants to run for a higher office might purchase voter data from the state to find potential voters and donors. The candidate’s team can blend that data with their existing data from previous campaigns to create sophisticated models of the most passionate or generous voters to target.
How does EDA work?
EDA itself is a theory of analysis, and each organization must create its own EDA practices. This is a multi-step process.
- Get buy-in
All stakeholders must first agree that EDA is necessary. Then, with the help of an EDA expert, the company must decide what data should be collected. Successful EDA is powered by analyzing as many elements and dimensions of data as possible, so typically, a business decides to track as much data as possible.
- Collect and organize data
The data is collected from various sources, including customer relationship management (CRM) software and third-party data sets. It is then organized into a data management system, usually using Enterprise resource planning (ERP) software. Common ERP software solutions include Jet Analytics and TimeXtender. Many companies use SQL Server Integration Studio (SSIS), but it’s like taking a car versus a bus: SSIS will get the job done, but Jet Analytics or another dynamic ERP solution will get you where you want to go much faster and with more flexibility and ease.
- Extract data
An expert (or team) trained in the art and science of EDA identifies both the specific parameters of the data to extract as well as the lens through which to view that data for it to be analyzed in a productive way. The data set is then extracted into a software solution like Power BI or Jet Reports, and from there, it is just a few clicks to turn that data set into one or more dimensional models. A dimensional model is a user-friendly visual representation of a specific data set. When done properly, a dimensional model should be very simple to read and interpret, but as you can see, what goes into making a dimensional model is quite complex and requires EDA expertise.
- Analyze data
The dimensional models should reveal patterns and trends that can be used to develop new strategies; they may also reinforce existing strategies or reveal what isn’t working. These models can be re-created as often as necessary and can also be tweaked to use more, fewer, or different data points. The sky is the limit once a business commits to EDA.
What if a company has limited data to analyze?
A business can’t explore or analyze a data set if they didn’t track it to begin with. There is an adage among farmers that the best time to plant an apple tree is 20 years ago or today. The same is true with data collection and analysis. Some businesses may be kicking themselves for not starting sooner, but it’s never too late, and there is no time like the present.
With each customer or client, many elements can be tracked, and the detail of the data tracked significantly impacts EDA. Think of grading systems in college. When professors hand out Pass/Fail designations, it does not provide a full picture of how any given student did in class. Was a “Pass” closer to an A or a D? The same is true with customer data. It’s not enough to have their basic details; you also want to have a way to formally track their ordering or purchasing habits, including month, day of the week, and even time of day. Which sales representative did they talk to? When they reached out for support, which representative worked with them to resolve their issue? Did their purchasing habits change after that support ticket was resolved? These are just a few of the many, many data points that can and should be collected by any business wanting to implement a robust EDA program.
Implementing new data collection standards and processes can be done in a very short amount of time, and a useful data set will accumulate quickly. EDA experts can work with your staff to determine what data to track, how to track it, and which software solutions make the most sense for your business. If you start now, a year from now, you will have a rich data set to work with; if you don’t, a year from now, you may be falling behind your competitors.
What if a company has multiple data sets across various platforms?
It’s very common that a business will have multiple data sets across numerous platforms, none of which talk to each other. A business may use QuickBooks and SalesForce, both of which can provide rich data for analysis, but if those systems aren’t exporting into a blended data set, they cannot be used in conjunction with one another to allow for the most effective analysis.
Imagine a retail store that does not analyze customer trends. All of the data is there because door readers show how many people come in and at what time, and all sales are rung up in their Point of Sale (POS) system, but neither the door reader data nor the POS data is exported to a blended database, so no one is looking at that data meaningfully. This has tremendous implications for staffing levels, sales and promotions, and ordering practices. If the data is there, but no one is using it, what good does it do?
The solution is to build a blended data set that automatically exports data from all of your platforms into your ERP software of choice. This is simpler and more cost-effective than migrating data and makes data analysis much easier.
How to get started with EDA
If your business wants to do more with its data but you don’t know where to start, scheduling a call with an expert at Tigunia is an excellent jumping off point. If your business currently does some level of reporting but wants to take it up a notch with dimensional modeling and tying together data points, Tigunia is ready to hit the ground running on your behalf.
Smaller businesses may not know where to start or may think they don’t have the resources available to do EDA, but here at Tigunia, we are dedicated to providing right-sized solutions to its clients. We’re as big or as small as a client needs us to be, and we specialize in customizable solutions to fit your needs and budget. With Tigunia, you will get trustworthy, ego-free help from experienced staff who will guide the whole process until you’re an EDA expert.
Tigunia can help your business choose the ERP that is right for you, or we can work with whatever software you already use. We have a full ERP team ready to dive into any setup that we encounter. Once you’re ready to get started, we will export data from the SQL servers into your chosen ERP software, blend it into a data warehouse, and start building dimensional models.
Tigunia is committed to data protection, which is critical when exporting from one platform to another. Tigunia will merge your data securely and make sure you have easy, reliable access to all the data points you want to analyze.
Some of our clients have us manage all of their data reporting and analysis, but more commonly, we teach our clients how to do EDA on their own. Our experts can teach someone with database skills how to run Jet Reports or Power BI in about a day, including one-on-one instruction and job shadowing, and helping them learn what to look for and how to spot trends in data. Once we have taught your designated analyst how to do EDA, they become a resource for your company, able to run all your data analysis and train other staff. Once you’re up and running, we are always available for questions and to help strategize, or problem solve any issue that may present itself.
Using all available information and analyzing it with sophisticated tools is easier than you might think and provides a competitive edge that every forward-thinking business should use to their advantage or risk getting eclipsed by competitors. Contact Tigunia today to get started.