Learning Setting the standards for streaming analytics

SQLstream Blaze is the only streaming analytics platform designed from the ground up for stream processing (record-by-record in true real-time), to be inclusive (integrating Fast and Big Data), accessible (so both analysts and developers can build sophisticated apps), easy to use (so apps can be built in minutes), and always on (deploy, scale, or kill running services without having to hard restart anything).

Real-time context

Business success relies heavily on taking the right action, at the right time, all the time. And actions are dictated by data.

But the Big Data save-first model fails to process and analyze data fast enough, and results are incomplete because they do not make use of real-time data. Similarly, other Fast Data technologies can’t include historical data in analysis fast enough, making the results either inaccurate or irrelevant. In both cases, business intelligence is not up-to-date, and cannot correctly inform true real-time, contextually intelligent applications and services.

  • Streaming analytics- definition

    STREAMING ANALYTICS is the acquisition and  analysis of all data available (live and historical) to make it actionable the moment it streams into the system.

    • Acquisition includes: capture, consolidation, filtering, aggregation, cleaning, repair, ingestion
    • Analysis includes: integration, filtering, parsing, enriching, transformation, windowing, correlation, time-based and geospatial queries
    • Actions include: visualizations, automated actions, continuous load

    BENEFITS

    • Best decisions: insight is collected from all the data available to a business (not just old, historical/stored) data) and delivered to decision-makers faster
    • Savings: using less hardware and skills
    • Real-time crisis control: responsive systems can detect threats, errors and health issues and address them before they happen
    • More opportunities: new applications, products and services can provide a solid competitive advantage
  • Stream processing- definition

    Stream processing is the real-time processing of data continuously, concurrently, and in a record-by-record fashion. SP treats data not as static tables or files, but as a continuous infinite stream of data integrated from both live and historical sources.

    BENEFITS

    • Accessibility: live data can be used while still in motion, before being stored
    • Completeness: historical data can be streamed and integrated with live data for more context
    • High throughput: high-velocity, high-volume data can be processed with minimal latency.
  • Using SQL for streaming analytics

    SQL is the standard for data management, because it is declarative (applications can be written in just a few lines of high-level queries), it is reusable and portable across data management platforms, and most importantly, distributive: queries can be automatically optimized and distributed by the underlying streaming analytics platform.

  • Query-then-(maybe)-store vs store-then-query

    Streaming applications treat data not as static tables or files, but as persistent, infinite streams. In database terms, instead of running a query on data collected in a database, streaming analytics uses queries that execute forever over the arriving data, avoiding the delays of traditional ETL processes, eliminating the need to ever re-execute queries, and reducing the amount of memory used to process millions of records per second.

    BENEFIT: The results are incrementally generated as a continuous operation, which provides a) true real-time results and b) the option to not store the data, therefore saving on costs and sheer processing power. 

  • Stream processing vs. microbatch

    Due to batching and intermittent query access, the microbatch design is vulnerable to data loss and vital delays. That creates the risk of incorrect, incomplete or irrelevant analysis, conditional fault tolerance and reliability, and out-of-order timing issues.

    BENEFIT: the record-by-record, distributed model employed by stream processing guarantees accuracy, completeness, and continuity.  Ad-hoc query access allows operators to update or create streaming applications in minutes, without interrupting execution.

  • Streaming analytics vs. Complex Event Processing (CEP)

    Streaming analytics is designed to process data streams with a high ratio of event throughput versus number of queries. In contrast, CEP utilizes event-by-event processing and aggregation often with large numbers of rules or business logic. CEP engines are optimized to process discreet business events- for example, to compare out-of-order or out-of-stream events, applying decisions and reactions to event patterns, and so on.

    BENEFIT: Streaming analytics presents lower latencies when volumes increase.

Sources

SQLstream Blaze streams data from live and historical sources, continuously and in real time.

Connectors & adapters

SQLstream Blaze can work with any data source (raw streams and historical data), in any format (unstructured and structured), to any destination (feeds, messages, databases) through a wide range of connectors and adapters built for streaming ingestion and analytics.

AAA

SQLstream Blaze connects to a wide range of technologies to attain streaming ingestion, streaming analytics, and live actions.

Third party libraries

Resources

  • Case Study

    Cloud | Streaming ingestion and analytics for automated ad service

    A robust advertising technology platform, Rubicon— also one of the biggest cloud and Big Data computing systems in the world—leverages over 50,000 algorithms and analyzes billions of data points in real-time to deliver the best results for sellers and buyers. And with over 65,000+ CPUs and 5.0 Petabytes of storage working at 100 gigabits data/second, they needed an efficient solution that would permit adaptive scaling without the added cost.

    Learn More
  • Case Study

    Smart cities | Streaming analytics for real-time traffic control

    The Roads and Maritime Services needed a streaming ingestion and analytics solution to solve the changing data puzzle including its network control systems, databases, data warehouses, and vast vehicle and road sensor network covering the entire Sydney metropolitan area. SQLstream built a Smart City solution that acquires and analyzes all this data in real time, loading the results into a live platform controlling traffic flow, public transportation, and incident response applications. At the same time, results are streamed into the RMS data storage solutions, where it is saved for later use.

    Learn More
  • Case Study

    Financial | Streaming analytics for cybersecurity

    InfoArmor, a data security company, employed SQLstream Blaze to power its real-time security intelligence platform servicing over 600 banks and financial institutions, thus improving its services and increasing its customer base capacity from 600k to over 10M. The SQLstream analytics solution runs continuous analytics on all data streams and automatically deploys user-authorization revisions and real-time alerts when network or user security is at risk.

    Learn More
  • Case Study

    Telco | Streaming IP Network and CDR Analytics for fraud detection, real-time rating and least-cost routing

    Veracity Networks needed to real-time the analysis of their CDR records and maximize the capability of their SIP access and VoIP platforms to boost operational efficiency and maintain market share. The SQLstream streaming solution provides continuous acquisition and analysis of network and call data, enabling powerful time-based and geospatial queries for instant reductions in churn rates and infrastructure costs, plus automated protection measures against fraudulent activities and network attacks.

    Learn More
  • Case Study

    Telco | Streaming integration for real-time 9-1-1 responses

    ECaTS, a Cloud-based 9-1-1 Data Analytics and Management Information System, used Blaze and the SQLstream Emergency Response StreamApp to deliver ECaTS Dashboard, a streaming analytics cloud solution that informs emergency responders on what is happening on the ground as it’s happening, live. The Dashboard can now capture, filter and analyze all emergency calls in California, and use the results to identify anomalies in 9-1-1 call patterns, isolate and monitor emergencies as they are developing, and manage team and ground response continuously and in real time. The implementation resulted in adaptive scalability (the system is now able to process country-wide data on a per-need basis) and response accuracy of less than 1s, proven to be vital in managing emergencies like natural disasters or the San Bernardino terrorist attack.

    Learn More
  • Datasheet

    Product | SQLstream Blaze

    SQLstream Blaze is a streaming analytics platform enabling anyone to create streaming applications in minutes, that deliver streaming ingest, analytics, and live action.

    Learn More
  • Datasheet

    Cloud | SQLstream/Amazon Kinesis Analytics

    The combination of Amazon Kinesis Analytics and SQLstream Blaze makes it easier than ever to cost-effectively ingest, analyze, and manage streaming data on and between cloud and on premises.

    Learn More
  • Datasheet

    Partnerships | Blaze integrates with Teradata Unified Data Architecture

    Businesses can react to their live data by processing each new record as it arrives in SQLstream Blaze, with the integration with Teradata enabling Blaze to join data at rest with arriving data in motion, and to maintain the accuracy of data at rest in the Teradata Database through continuous dataload and ETL.

    Learn More
  • Demo

    Streaming Analytics at the edge and in the cloud

    IoT applications, mobile devices, wearables, industrial sensors, as well as many software applications and services can generate staggering amounts of streaming data – sometimes TBs per hour – that need to be ingested, analyzed, and managed as the data arrive, to enable valuable business actions continuously and in real time. The combination of Amazon Kinesis Analytics and SQLstream Blaze offers the best of on-premises and cloud advantages.

    Learn More
  • Demo

    Teradata & SQLstream Blaze Unified Data Architecture

    The integration with Teradata enables SQLstream Blaze to join data at rest with arriving data in motion, and to maintain the accuracy of data at rest in the database through streaming ingestion.

    Learn More
  • Demo

    StreamLab

    SQLstream StreamLab is a drag-and-drop platform for building real-time dashboards and applications over streaming and time-series data, accessible to anyone (no need to code) and continuously (no need to ever interrupt execution).

    Learn More
  • Report

    The Forrester Wave™: Big Data Streaming Analytics, Q1 2016

    Forrester evaluates 15 leading streaming analytics vendors and places SQLstream in the leaders' circle.

    Learn More
  • Video

    Streaming ingestion | Webinar recording | Guests: Teradata and ECaTS

    SQLstream and our guests explore, explain, and demo streaming ingestion, continuous integration, and continuous load cases.

    Learn More
  • Video

    Streaming analytics | Webinar recording | Guest: Forrester, Inc.

    SQLstream and Forrester discuss and exemplify the benefits of streaming analytics.

    Learn More