The immortal words of the great Yogi Berra have an ever-increasing resonance for managers. Reliable forecasts of demand are foundational to any robust manufacturing or distribution plan. But it was not always thus, says Sam Tulip.
Once upon a time, (and not so long ago), many managers in many industries didn’t really need to bother much about demand forecasting. Buyers had a very limited choice, both of products and of suppliers. Demand could usually be relied on to outstrip supply, and production was planned around economic batch quantities and other measures of factory efficiency. If a customer wanted a new car, for example, they could choose from the limited selection that the car maker had chosen to supply to the dealer network.
Alternatively, if they really wanted the model with the twin Weber carburettors they could pay a deposit and join a waiting list and in the fullness of time, when enough firm orders had been taken, Longbridge or Cowley would condescend to assemble a batch.
Products had long lifecycles and obsolescence wasn’t a problem. Marketing was relatively crude and untargeted and consumer tastes changed at a glacial pace. And since the entire supply chain was awash with inventory at every point from supplier parts and materials, through factory work in progress, to finished goods stocks at wholesalers and retailers, manufacturing was well buffered from all but the most acute changes in demand. Meanwhile, actual sales data took weeks or even months to work through the system, so historical data was almost all that a firm had to work on. Fortunately, the old Met Office joke, pre satellite and supercomputer, that “statistically the most accurate forecast is that tomorrow’s weather will be much like today’s” was equally true in industry and commerce. So who needed demand forecasting?
The modern environment could hardly be more different. New products appear and disappear in the blink of an eye – the apparel industry has notoriously moved from two seasons a year to six, eight or more, but a similar trend can be seen in many other retail and business sectors. Marketing and promotion has become much more targeted and effective, but at the same time entirely new phenomena such as ‘social media influencers’ are making demand ever more volatile. Consumer spending is increasingly discretionary, rather than essentials, and buyers exercise their discretion in unpredictable, or at least unpredicted, ways. Inventory is no longer a valuable buffer: it is a cost and an obsolescence risk.
Says Hank Canitz, product marketing director at Logility: “Growing e-commerce sales, omni-channel fulfilment, and expanded consumer-focused offerings down to the individual customer have put tremendous strain on a company’s ability to accurately forecast demand. E-commerce sites have the ability to change content very quickly focusing attention on hot products overnight. Combined with the power of social media demand for products can exponentially grow or shrink.
“Unfortunately the time to source, produce and position products has not decreased in the same proportions as customer demand lead-times. Product replenishment lead-times are still considerably longer than customer order lead times so demand planning is a critical capability in today’s fast paced business environment”.
As Tommi Ylinen and colleagues at Relex Solutions summarise: “The benefits of good forecasting include the ability to attain excellent product availability with reduced safety stocks, minimised waste, as well as better margins. Further up the supply chain, good forecasting allows manufacturers to secure availability of relevant raw and packaging materials and operate their production with lower capacity, time and inventory buffers”.
Good forecasts are not just a benefit to supply chains, enterprises and consumers. Accurate forecasts are an essential tool in combatting societal scourges from excessive packaging and wasteful (less than full load) transport to food waste.
Fortunately, the science of forecasting has not stood still. The era of ‘big data’ gives forecasters far more information to work with, in close to real time. This is not just data from their own and their partners’ operations: complex data sets, covering topics from weather to local economic predictions are widely available. Information on what competitors are up to is also out there: for example analysis of aerial and satellite imagery for traffic flows in and out of shopping malls, business parks and logistics centres to estimate sales, production and inventory. The computational power to absorb and use all this data is cheap, and new techniques from multi-variant analysis to artificial intelligence and machine learning are making real contributions to forecast accuracy.
We look at some of these developments below; and next month we will see how firms are applying and benefitting from advanced forecasting methods in real life. But first, a few notes of caution. To quote Relex again: “Demand forecasts will always be inaccurate to some degree, and the planning process must accommodate this”.
Also, they say: “In some cases it may simply be more cost effective to mitigate the effect of forecast errors rather than invest in further increasing the forecast accuracy”, not least because if remaining forecast error is caused by essentially random variation any attempt to further increase forecast accuracy will be fruitless. In addition there may be other factors with a bigger impact on the business result” – they cite a food retailer with excessive spoilage of produce.
The problem was not with the demand forecast which was quite good: it was that too much stock was being carried in the store to ‘make the shelves look full’. The answer isn’t further data-crunching, it is a smaller shelf allocation. And of course there is the major caveat – more information doesn’t necessarily make for a better forecast. Much data will be irrelevant, some will be misleading, some just random noise, and of course correlation is not necessarily causation – a snare for machine learning.
Nonetheless, as Alexandra Sevelius, head of marketing at Relex, says: “Retailers can now apply new technologies such as AI and machine learning to understand how previously unconsidered factors such as weather may impact the demand and subsequent inventory requirements for products in individual locations. Software platforms continue to improve their support across multiple channels as well, as this grows increasingly important.
“Today’s technologies can manage complex decisions such as optimising parameters and recommendations at a detailed level, then deliver the areas requiring closer attention directly to users, allowing users to focus their attention where it’s most needed”.
One such technology is multi-variate demand signal management (MDSM). Hank Canitz explains: “MDSM facilitates data capture from multiple demand streams and translates these into demand data insights used to provide input for future planning activities, identify and pre-empt service disruptions, and generate measurable sales and profit growth”. This is distinct from traditional linear analytics which test just one variable at a time.
Another critical emerging best practice, says Canitz, is demand sensing. Demand sensing is the translation of market based demand information to detect short term buying patterns. As the Oliver Wight Organisation emphasises: “Demand sensing doesn’t replace demand management. Demand sensing is specifically for dynamically managing short-term demand by supplying accurate, real-time data during the two-week demand execution window. By leveraging analytics and the latest mathematical algorithms, it creates an accurate picture of changing demand, based on the current realities of customer behaviour, and assists the most effective execution of the demand plan. Naturally, success is dependent on automated, well- connected systems from cashpoint to manufacturer”.
Canitz claims that “the typical performance of demand sensing systems reduces near-term forecast error by 30 per cent or more compared to traditional time-series forecasting techniques”.
An evolving demand sensing trend, says Canitz, is the use of artificial intelligence to automate the process of analysing data to recognise complex patterns and to separate actionable demand signals.
Ian Lauer, senior principal solutions architect, supply chain, at Infor, describes machine learning and AI as “a huge shot in the arm. Software able to crunch the numbers from thousands and even millions of sensors, throughout a value chain, and then able to ask (and answer) “what if?” will arguably be more in charge of the business than the CEO”.
But AI isn’t just about recognising patterns in numerical patterns. “Used together, machine learning and natural language processing algorithms can be used to analyse information like social media text to determine the ‘Sentiment’ of the text and to predict the impact of that sentiment on demand,” says Canitz.
The potential of AI is clear, says Stephanie Duvault-Alexandre, a consultant at software provider FuturMaster. “Previously, retailers and manufacturers were only able to predict roughly the quantities of products to order to keep shelves fully stocked using (often out-of-date) inventory levels and historical sales data (usually going back a few years, at best). These days, AI can develop a much more accurate picture of exactly what types of products are likely to sell, by looking at multiple scenarios in real time (suppliers’ data, consumer behaviour, the weather etc.) and drawing on data from the internet. This means forecasting is no longer so much “stab in the dark” guess work.
“Machine learning is proving much more efficient at unravelling complex data quickly and meaningfully. For instance, retailers want to be able to cluster and identify who are their main customers – who are repeat purchasers, browsers, or so-called aliens. Or they might need to know which products are better to deliver last-minute; or which core lines should regularly be in stock.
“And with many retailers dependent on promotions, machine learning is more effective at clustering promotions based on looking at similarities and many more variables than is otherwise possible using traditional, linear-based forecasting techniques.
“AI-powered algorithms learn from a multitude of factors that are likely to influence buyer behaviour – including promotions, social media, or the weather – which then are used to more accurately manage inventory levels and replenishment. Not only will such advanced technology know when shelves are empty, but more importantly, it will predict what will happen next”.
Ian Lauer says: “Demand forecasting has truly came into its own as consumer preferences have shifted and more and more business must account for shorter lifecycles and greater personalisation. This has meant tightly controlling stocks of partly finished products, keen customer data for promotions and of course, an agile and fast supply chain that can deliver next or same day delivery.
“Traditional forecasting has strained under the weight of ever-increasing datasets. Businesses looking to spot the patterns that will yield success now realise they need to approach integrated business planning from a challenging perspective: the ability to ask ‘what if?’ anywhere in the business.
“The key word here is ‘anywhere’. Successful businesses are used to the changing patterns of demand from customers, but less well versed in pulling together commercial opportunity from shifts in the supplier community. Both up and down the demand chain, from suppliers to customers (and everywhere in between) there is the ability to find another link in the chain that refines planning and forecasting”.
So this is where advanced forecasting feeds into advanced planning (a forecast on its own is pretty worthless without a plan to exploit it). Canitz argues: “Sales & Operations Planning (S&OP) is evolving to Integrated Business Planning (IBP) which involves a much wider and deeper planning process across the extended organisation that aligns and synchronises all company planning. IBP goes a step further than S&OP by uniting volumetric and financial information into one flexible planning and decision support process for strategic and tactical planning horizons. It combines data from sales, marketing, production, procurement, transport and finance to create a powerful decision centre for all stakeholders.
“Because IBP involves multiple collaborative, cross-functional processes, it requires a technology solution specifically designed to accomplish these tasks within one holistic shared platform. This platform must provide collaborative workflow, configurable alerts, active messaging capabilities and powerful algorithms to streamline and facilitate plan development. Also crucial is the flexibility to view data in varying time horizons, and be able to display multiple years of history and out to ten years of projections. The ability to aggregate and disaggregate data allows users to analyse data and develop plans at the level appropriate to their positions, while staying synchronised with users planning at other levels of aggregation.
“Workflow, configurable limits, intelligent alerts, multi-variate segmentation, advanced analytics, and machine learning are becoming planning must-haves to automate the routine, focus planner attention on value-adding activities, and facilitate robust analytics and problem solving.”
Supply chain analysis and plans will only be as good as the data that they are based on, and the choice of analytical method depends critically on the nature, as well as the quality, of the data. Methods that work well with fairly complete data sets, for example ePOS data, may not be appropriate where data is incomplete – for example an online retailer may have an instant handle on orders for a new line, but not yet have any visibility of the likely returns rate. Sadly, there are theoretical reasons to believe that we will never have an ‘algorithm of everything’.
Canitz says: “Clean and consistent data will make or break efforts to digitise the supply chain, automate processes, optimise plans and responses, use artificial intelligence and machine learning, and move to higher levels of analytics maturity. Many ‘Big Data’ inputs that could be used by advanced supply chain capabilities do not originate nor reside in a company’s ERP systems. ERP master data management solutions will not support supply chain data needs. Without a supply chain master data management system, organisations may find themselves becoming data rich and information poor. Data is available, but the organisation can’t effectively use it in their planning and optimisation operations”.
Mark Hinds, global CEO at data sciences company Polymatica, emphasises that these issues are at least as relevant to logistics service providers. “Expectations for retailers and therefore their logistics partners to meet next day deliveries, provide traceability of an order from source to distribution, and offer a seamless returns system, are putting pressure on forecasting and planning. With order numbers growing, logistics companies must find new ways to address this using customer and internal data.
“Key to this is finding a new, more collaborative way to consolidate data sets stored throughout the business. Logistics organisations sit on a mountain of data, ranging from telematics information on routes and fuel, to customer data on how many orders are being delivered and whether extra services are required. Centralising this data is critical. The next step is making sure they are asking the right questions of their data to get the right insights. Analysing each step in this way will be key to logistics companies delivering within time limits, and helping retailers provide the end consumer with a great customer experience”.
This article first appeared in Logistics Manager, March 2019.
