But it’s all academic, isn’t it?

I do realise that this is a highly academic example: does it have any relevance to the ‘real’ world? I believe so. 

Remember that data is just data. The commercial world is adopting BI with open arms and is applying it to customer data, sales data and the like. But many companies could collect data in other ways. Production lines can be fitted with data sensors, lorries can be pinpointed, second by second, using GPS – all sorts of environmental parameters (temperature, humidity, gas levels etc.) can be tracked and monitored. It’s all data, ready and waiting to have commercially valuable information extracted. 

Transactional databases have traditionally stored only the data that supports the business process. That data allows us to confirm that “Yes, we did sell a toothbrush to Vivian Smith on the 12/09/07.” The brave new world of BI should give business people the opportunity to ask questions like “Do we sell more toothbrushes to males or females?” “What is the profile of a typical iPod purchaser in the north-west?” “What factors are causing stock to degrade more rapidly in warehouse X than Y?” For that we need to design transactional systems that collect far more data. But the devil is in the detail: what extra data should we collect? In one sense I can’t help you because it is impossible to know what data is required for a train-of-thought analysis until the analysis takes place. But I can tell you that there was very little data that we collected which turned out to be useless. We did indeed use the name of the paper manufacturer, the data when it was made, whether the name of the herbarium was printed or hand written…

So the bottom line is that, in the commercial world, there is huge potential value to be gained in designing our transactional databases to collect far more data than before. The skill is in deciding what extra data to collect but hey, we’re developers; we get paid to make intelligent judgements. Don’t we?