A few years ago, I remember a meeting with Mark Palmer, now general manager of Progress Software’s Apama division (but in those days without his Apama hat) during which we discussed some clever technology Amazon had implemented to manage its newly created Marketplace operation.
The system could handle details of millions of second-hand titles offered by almost as many private sellers, and deliver all the relevant details to potential customers within seconds. It was all terribly leading edge, incredibly clever and had taken a couple of years’ to create.
Another meeting with Palmer last month focussed on technology capable of doing all this and more, only it would allow users to be up and running within weeks and the development costs would be little more than six figures rather than the seven for earlier projects.
Such speed and cost-effectiveness is, it seems, due to complex event processing (CEP).
Speaking volumes
‘CEP derives from the need to manage growing volumes of data combined across distributed computing systems,’ says Palmer. ‘By the 1990s the data emphasis was on business intelligence while service-oriented architecture (SOA) was transforming distributed operations, CEP has followed from these developments.’
Unlike traditional databases which handle static data, CEP engines are designed to cope with streaming events. Instead of being structured to manage pre-set formats and fields, a CEP system can respond to events and extract relevant information without the need for complex reprogramming. With SOA plumbing in place, information can be extracted from any source within the organisation and used to support future activities or needs rather than simply the scenarios and questions envisaged when the system was designed.
If this sounds rather esoteric and with little relevance to logistics or supply chains, consider one of Apama’s users – a logistics service provider in California. The company is using the technology to monitor truck locations in real-time, with the help of GPS, in order to re-schedule and re-route drivers as need be to meet Service Level Agreements (SLAs). The system monitors events and, using predictive analytics, can advise controllers well in advance if an expected sequence of events is likely to cause problems. Trucks can be rescheduled, customers alerted, or routes altered to meet vital SLAs all in real-time.
Elements of this type of solution have been around for some time but CEP now means that such systems are accessible to smaller players.
Progress acquired Apama in 2005. Like other CEP players, the operation was both comparatively new and derived from research at universities in the UK and US.
Supply chain specialists have talked about using predictive planning tools in real-time for years, and not just for meeting SLAs by amending route schedules. One scenario mooted by Unisys some five or more years ago was to link changing retail demand and the need to prevent stock-outs right through to delivery schedules and route plans. If stocks of merchandise began to run low, the theory ran, suppliers could be alerted and whoever happened to have a delivery truck in the vicinity with the needed goods could bid for the business and gain an extra sale.
The idea – a futurist discussion point rather than a commercial reality – was also suggested as a means for manufacturers to occasionally gain the upper hand in their sometimes fraught dealings with the big supermarket chains. A request for, say, bananas urgently needed at a store in Bristol or Barcelona could be made via a data pool. Suppliers – not necessarily existing preferred trading partners – could pitch for business on the basis that they had a shipment of suitable fruit within minutes of the store whereas an established supplier may only be able to offer 24-hour replenishment.
Five years ago, such a scenario was simply an idea and whether it would ever work or be commercially viable is still debatable, but today the technology at least exists for such concepts.
Dr. Penelope Ody is a regular columnist with SCS and is a retail market specialist
