According to Statista, in 2010, the total of amount of data collected worldwide was 2 zettabytes – or to use a unit of measurement you might be more familiar with, that’s 2 trillion gigabytes. In 2024, that number is projected to hit 149 zettabytes. All that data isn’t kept, so IDC predicts that by 2025, the world’s accumulated datastore will be 175 zettabytes. According to Forbes, we (collectively) generate 1.7 megabytes of new data per person, per second. And here’s maybe the most interesting fact of all, according to IDC, less than 5% of that data will be analyzed.
So, why is any of that important in the world of the One Year, Thirty Minute Challenge? Companies who capitalize on the data available to their organization by –
- Identifying what parts of that data directly impact their financial performance
- Making meaning of that data with expert analysis
- Turning that analysis into actionable insights
- Changing organizational behavior based on those insights
- Measuring the financial impact of those changes
- Making additional changes based on those measurements
are seeing results. Here are a few examples from Tech Republic.
- Supply chain safety and theft detection enables companies, with help of item-placed sensors and business intelligence, to reduce in-transit theft rates of supplies from 50% to 4% and to detect when the environmentals or seals on shipment containers have been compromised.
- Logistics tracking and routing using business intelligence and machine-based data/sensors optimize delivery routes and driver habits creating fuel savings and better service.
- Collections work at companies is avoided by learning more about customers who are behind on their payments through big data aggregation and business intelligence that can predict who in good faith can pay their debts with a little help–and then helping these customers keep their purchases and keeping companies from having to write off defaults.
- Buying habits and preferences of consumers are better understood and lead to increased sales.
- Predictive maintenance enables urban tram systems to stay online, reroute traffic where necessary, and flash adviser alerts to customers over their mobile phones while repair crews are dispatched to replace faulty components before the components actually fail.
Big Data doesn’t just refer to just the volume of data available today, it encompasses the “4 Vs” of Big Data –
- Volume – Certainly volume is an important part of the equation. We have internal data from our CRM and ERP systems that tell us about vendor performance, product performance, customer behavior, employee performance and a host of other things. We have external data from social networks, online review sites and more. Because of the Internet of Things (IoT), we have data that originates not just from the actions of our employees or customers, but from inanimate devices connected to the internet. So, we can know the number of times a door opens and for who, the temperature inside a shipping container and when a client’s copier is low on toner.
- Variety – This data comes at us in multiple ways. Structured data from internal systems where we’ve controlled what is collected and how and unstructured data from external sources. We might get a text from a customer with a video of the dishwasher we just fixed showing us that it’s still doing what it was doing before we “fixed” it, a Google review, a Twitter DM, a reading from a sensor on our delivery van alerting us to a tire pressure problem and the list goes on.
- Velocity – If the previous two aren’t enough, maybe the most daunting is the speed at which it comes at us. Last minute’s data reporting that all is well, is superseded by this minute’s data reporting a problem on the factory floor or a customer unhappy with your product or service. Multiply those by the number of inputs (customers, employees, vendors, sensors) and it can seem overwhelming.
- Value – In actuality, this is the one that matters most. Of all the data collected by your organization, what really impacts financial performance, customer experience and employee wellbeing (ability to do their job effectively and efficiently)?
One more thing before we jump into this week’s exercise. Big Data requires different skills and tools than the traditional reporting you’ve pulled from your internal systems. First, because of the mix of structured and unstructured data, you’ll need a data management infrastructure that can manage both. Second, you need someone who can help you navigate this new world. You can hire a data scientist or you might opt for outsourcing this part of your work to a vendor specializing in Big Data Analysis. The most rudimentary Big Data analyses are looking for trends (each month for the past six months, distributors of our product in the Southeast have reported a stock out. Each month, it’s been earlier in the month than the month before), patterns (customers whose first purchase from us is product X never make another purchase, but customers whose first purchase is product Y have an 80% chance of being a repeat customer), and correlations (in the Fall, the first time the temperature dips below 50 degrees canned soup sales double and stay at that level until the first time the temperature hits 60 degrees in the Spring). A Data Scientist can help you start thinking in this vein. Third, in addition to the infrastructure tools to do the heavy lifting, you need visualization tools that help you easily see what this large amount of data is telling you. Even if the data scientist tells you everything you need to know, you want to roll this information out to everyone in the organization who can benefit from it. Good visualization tools will allow them to consume large amounts of information (and make sense of it) more easily.
For this week’s exercise, I want you to identify some problems or opportunities in the organization where Big Data-generated insights might make a difference. Here are some thought starters –
- More callbacks on service calls – Is it the same technician? Are they working on the same brand of equipment? Are replacement parts from the same vendor failing at a high rate?
- Inventory management is more challenging than it should be – Can you get access into distributor data so you can see when distributors are most likely to place a reorder? Is a single vendor slowing production with late or defective products?
- Customers seem uninterested in a new product or service – What is the factory defect rate on this product vs the defect rate on its predecessor? Have customers who purchased the product commented on social media regarding the product? Is it especially unpopular among your customers who purchase another product from you?
- We have too many employees during some shifts and not enough during others – Can you examine sales by hour for the same day of the week last week or the same week last year? Can you examine the nature of sales during each shift – selling a hand-dipped ice cream cone is more labor intensive than selling a bottled soft drink?
Take your list and contact a Big Data company for a consultation. See if it makes sense to do a pilot project.
Big Data is the foundation for technologies like Machine Learning – the improvement of computer algorithms through experience (people who bought this book also bought this book, powering your Amazon recommendations) and Artificial Intelligence – when a system “perceives its environment and takes actions that maximize its chance of successfully achieving its goals” (think Big Blue playing chess against a Grand Master, examining the chess board and making the optimal chess move).
Of all the technology assets in your company, data is the most important. It catalogs the past behavior of your employees and customers. And the best predictor of future behavior is past behavior. Don’t neglect the power of this asset to solve problems that have puzzled you for a long time.