Big Data Technology Innovation versus Time to Value

VentanaResearch-Award-ReducedThe Ventana Research Technology Innovation Summit was held this week in Chicago, where VR CEO Mark Smith and BI analysts such as Tony Cosentino presented their perspective on Big Data technology adoption and the issues being faced by organizations as a result. This was not a conference about Hadoop, far from it, in fact the focus was more on how to best exploit Big Data as an asset and how this is being achieved across different software application areas. I would also mention that one of these areas was Operational Intelligence for which SQLstream picked up the award for Technology Innovation (see picture).

Time to value was a recurring theme. It struck me that the issues emerging from organizations were similar to pre-Big Data concerns, for example too many systems, too many data silos, poor quality data, technology not able to address business needs. What has changed is the scale of the problem. Big Data has elevated the classical data management challenges to new levels. More data, more data storage platforms, fewer standards, and resource-intensive solutions with high total cost of ownership. No wonder that time to value is an emerging theme as Big Data sails closer to mature enterprises.

We’re also experiencing a new phenomenon that means the time to value problem has to be addressed – the ‘Big Data Mindset’. This is perhaps the most important concept to emerge from the Big Data evolution. Not the technology, rather a step change in expectation as to the value locked in a data asset, and a willingness to explore new and innovative approaches in order to extract that value.

Organizations have seen the light, the art of the possible, and there’s no turning back from the Big Data Mindset. Organizations and are now seeking technology maturity, standardization and architecture stability. This is one reason why we’re seeing SQL emerge as the Big Data query language of choice. Not the SQL we learnt about in text books that was inseparable from an RDBMS. The new SQL is using the power of the query language to extract intelligence from any type of data (structured, unstructured or semi-structured) and across any data management platform (map reduce, NoSQL or streaming).

Reducing time to value requires standards and data management applications where the business logic is not locked in volumes of bespoke Java code. We require intelligent data management platforms that abstract away the underlying complexity, that scale up and out effectively (SQL as a parallel programming language), and deliver agile solutions that can be generalized and upgraded for new requirements rapidly and without system downtime.