I would like to tell you the story about a project. This was a big data project. Streaming data was used to monitor and optimize production from a processing plant. The industry was in interesting times: competition from Asia and new players in the USA was squeezing margins. The company needed to cut several thousand positions.

The project had the following cast ot actors:

• A vice-president for operations with the latest in portable personal devices.
• An expert in simulation and analysis of the key processing equipment in the company’s value chain. He was really old: at least 50.
• An artificial intelligence research manager with a very expensive Mac.
• An operations support engineer who knew that this project could improve production by 2% and availability by about the same … and hoped that he could prove this to the vice-president for operations.
• A data scientist, who was very smart – he knew the latest about artificial intelligence, ontologies and Markov chains – and pitied anyone who couldn’t understand it like he could.
• An IT department who delivered a cloud-based set of standardized business services, with user self-service, a three month turnaround on change requests and an obstructive attitude to non-standard or technical computing.
• A 63-year old production planner, who had a spreadsheet that only he could run. This spreadsheet was used for all production planning in the plant. Consistency between assumptions in the spreadsheet and actual process behaviour was updated occasionally.
• An academic, who had a really cool algorithm that might be useful.
• A junior project manager who tried to tie all of this together and get everyone to pull in the same direction.

The project had a few false starts, with a huge challenge in communication between the AI people and operations. But it delivered in the end.

Anybody recognize this type of project?

This project happened nearly 30 years ago. The “personal portable device” was a Compaq Portable III “luggable lunchbox”, with a 20Mb hard disk, an orange plasma screen and a weight of 10kg. But it was an iPad for the Vice-President. The IT department’s standardized cloud was an IBM mainframe. But the project team and implementation challenges are the same now as they were in 1988. I was that junior project manager.

The more things change, the more they stay the same. Our project team was struggling with the same technical and organizational issues that capital-intensive industries still face today. Data-processing power has grown exponentially: this makes it possible to analyze big data sets. However, our capacity to extract value has grown at best linearly … and will continue to do so unless IT professionals help the business professionals, their customers – the geophysicists, petroleum technologists, operations engineers and logistics planners – with easy access to the right information to allow them to control (and the optimize) their corner of the world.