Our Ventana Technology Innovation Award – The Insider’s View

Ventana Innovation Technology AwardVentana’s Technology Innovation Awards showcase “advances in technology that contribute significantly to improved efficiency, productivity and performance of the organization.” SQLstream’s IT Analytics and Performance award recognized SQLstream’s innovative technology and ability to optimize operational processes and systems. The award considered all aspects of SQLstreams technology and business approach, in particular:

  • Vision – bringing standards to the world of Big Data and delivering a platform for real-time operational intelligence and integration across all industries.
  • Technology innovation – massively scalable, streaming data management platform using standard SQL (SQL:2008) as the query language.
  • Operational process optimization – enabling business to eliminate latency entirely from their operational processes.
  • Business impact – delivering real-time, low latency applications with the lowest Total Cost of Performance of any Big Data technology.
  • Customer ROI – Delivering significantly lower cost solutions while impacting directly end customer quality of service and experience.

The SQLstream Vision – Standards, one platform and real-time

Big Data platforms and RDBMS both fall short as platforms for low-latency, high velocity data management. Storing data and continually polling to re-evaluate queries introduces unacceptable latency and cost (both operational and in lost opportunity). The few existing true real-time platforms are clustered in niche areas, requiring significant, specialized bespoke development, and systems highly tailored to specific applications, making them both expensive and very limiting.

The SQLstream vision is simple – to minimize the Total Cost of Performance for real-time applications by delivering a true real-time data management platform that:

  • Scales to the most extreme high velocity Big Data applications.
  • Supports multiple real-time applications on a single core platform.
  • Enables organizations to build sophisticated analytics with low TCO.
  • Complements and accelerates the performance of existing IT infrastructure through continuous integration of streaming data and analytics.

Technology Innovation – Streaming SQL Big Data platform

SQLstream is a 100% SQL-standards based streaming data management platform that turns high-velocity machine data into real-time operational intelligence before the data are stored. SQLstream is built on a massively-parallel, continuous query execution engine that delivers streams of valuable information to any consuming application or service. But why SQL? SQL is well defined and understood globally, it offers low cost and rapid development, and is ideally suited to dynamic optimization of streaming query execution over a massively distributed server infrastructure.

Operational Process Optimization

Streaming operational intelligence eliminates the gap between analytics and the operational processes that connect an organization with the outside world, its customers, consumers and partners. Eliminating latency from the key operational processes directly impacts the efficiency of an organization’s day to day execution. A streaming operational intelligence platform in particular offers the capability to integrate real-time, predictive analytics across and organization’s existing operational systems, and either drive automated actions or support faster, more effective decision-making through traditional visualizations and alerts.

Business Impact

Most industries today are looking to eliminate latency from their business operations. Some are further advanced than others, but all are looking to lower the cost of their operations, maximize potential growth opportunities and improve the quality of experience and the quality of the delivered service. The business impact is both internal, in terms of OPEX and CAPEX improvements, but also can be measured through improved quality of experience and service for our clients’ customers, consumers and partners. For example:

  • Cybersecurity – detecting the emerging advanced persistent threats requires the processing of vast volumes of typically log data in real-time, coupled with rules-based pattern detection, geo-spatial and location-based analysis.
  • Fraud prevention – from an enterprise perspective, particularly in the telecommuncations and M2M markets, the ability to detect and prevent fraudulent access in real-time is providing a major differentiator in terms of an ROI-led project.
  • Identity theft monitoring – has a direct impact from the perspective of the individual.
  • Smart Cities – automated, real-time management of facilities such as street lighting.
  • Intelligent transportation – accurate real-time travel time for travellers, and automated traffic infrastructure management to reduce congestion.

Customer ROI – Lowest Total Cost of Performance

Change that can be directly attributed to the deployment of SQLstream’s streaming operational intelligence platform can be measured in several ways.

First, the direct ROI of the project. For example, in Intelligence Transportation, SQLstream’s ability to deliver accurate, real-time traffic flow and congestion prediction from GPS data saves transportation agencies millions of dollars – in comparison with the high cost of deploying (less accurate) in-road and roadside sensor projects.

Second, OPEX and CAPEX reduction through improved operational efficiency. For example, Cornell Computing Centre was unable to find a log management tool that could scale to the volume and velocity of their log data. SQLstream provided a level of visibility into platform and application issues in real-time that they were unable to achieve using their own in-house development, or with any of the other log monitoring tools on the market.

And finally, improved customer satisfaction. For example, InfoArmor uses SQLstream as the platform for their real-time identity theft monitoring applications. SQLstream collects and analyzes a large number of different internet, log file and network data feeds in order to detect (and predict) patterns of identify theft. The existing environment was a common scenario – multiple different tools, applications and storage platforms, loose and bespoke integration, with the result that the entire IT environment was unable to scale to meet their business growth. SQLstream enabled them to centralize their real-time, streaming analytics on a single, future-proof and scalable platform, and to integrate streaming intelligence across the remaining systems, improving overall business support and performance.

And finally

Operational Intelligence is emerging as the next phase of Big Data, enabling true real-time performance from structured and unstructured machine data. Many have been surprised that Hadoop and similar Big Data platforms have not been as real-time as hoped, yet Hadoop was not designed for low latency applications. As with all new technology sectors, there are a number of different approaches vying for control. SQLstream’s streaming data management platform offers the most comprehensive set of technology attributes.

The design and implementation of the SQLstream streaming data management architecture is a new paradigm for processing real-time data. The technology offers a true platform – SQL – as the streaming analysis development language, ensuring:

  • Lowest TCO: SQLstream offers the lowest initial lifecycle TCO for real-time, low-latency applications. Please refer to the attached performance benchmark press release for details.
  • Speed of application development: SQL enables applications to be developed and deployed using a fraction of the code required to deploy a comparable Java-based or SQL-like streaming application.
  • High-velocity data support: SQLstream applications are sized on the basis of 250,000 to 500,000 records per second (depending on record size), and scales efficiently across multiple servers to deliver applications capable of processing millions of records per second.
  • Low-latency analytics: SQLstream supports latency requirements down to single digit milliseconds (where latency is measured from the time of data creation to the availability of analytics or results).

But perhaps the single largest contributing factor was the realization of the original idea. SQLstream’s core technology began due to a belief that real-time time applications required standards, a true platform, and high performance. Streaming technology, coupled with SQL, and the techniques used for multi-server scaleout were seen by SQLstream as the future and the only path available. The current release of the technology 3.0, is the realization of that idea.