Big Data Ingest for Data in Motion Just got easy

Increasing time to analytics by harnessing the value of data in motion has emerged as a key challenge for those adopting big data IT platforms. Organizations are finding it is taking too long to ingest data into Hadoop and other NoSQL storage-based analytics platforms, adding latency and slowing down the delivery of analytics to the business. It is taking too long to build and deploy the real-time processing applications for data in motion, and live data management in general is proving expensive and complex. Guavus SQLstream addresses these challenges, enabling organizations to be data-driven in real-time, eliminating latency, reducing time to analytics, and making real-time insights central to their business.

Click to play the demo video

With Guavus SQLstream’s interactive graphical interface, StreamLab, live data is discovered, analyzed and aggregated on the fly, and delivered as continuous ingest into Hadoop, data warehouses and other enterprise systems. No coding or data scientists are required. With Guavus SQLstream and StreamLab, new real-time processing apps for live data can be built and deployed in minutes.

Where do I need continuous ingest of data in motion?

Guavus SQLstream with StreamLab makes it easy to discover live machine data from just about any source, aggregate data on the fly, and to deliver the output as continuous ingest into your storage platforms. With Guavus SQLstream, you will be able to:

• Discover live data from log files such as syslog, clickstream data from websites, sensor data streams, data feeds captured over stream-oriented and message-oriented middleware such as Kafka and AMQP, from smartphones and devices, and from networks and service delivery platforms.
• Deliver continuous ingest of the live data into any number of destinations simultaneously, including big data storage platforms such as Hadoop and NoSQL platforms, RDBMS platforms such as Teradata, Oracle and SQLServer, and delivered over middleware platforms such as Kafka. Ingest data can be filtered, aggregated and enriched on the fly by Guavus SQLstream prior to delivery.
• Deliver streaming analytics and alerts from data in motion as it streams past. Guavus SQLstream can also deliver real-time analytics through live dashboards, working over the same live data streams. Analytics and alerts can also be delivered through continuous ingest into any of the target destination systems.

How does it work?

Unlike any other platforms for dataload and ETL, Guavus SQLstream is delivered with pre-built Agents for live machine data discovery, enterprise integration Adapters for continuous connections to storage and middleware platforms, and integrated real-time dashboards for the visualization of any streaming analytics generated from the flowing data as it streams past.

Real-time discovery Agents are lightweight Java objects that can be deployed on remote servers, or even embedded in devices and equipment directly. Each Agent transforms a data source such as log file into a live data stream (change data capture – effectively ‘tailing’ the data source for new records as they arrive). The Agent offers remote filtering and aggregation where network capacity and cost are issues, and provides local storage backup for resilience in the event of network failures.

And also unlike any other platform, Guavus SQLstream supports the graphical development of new real-time processing applications for data discovery and continuous ingest, but also supports custom development over data in motion using both ANSI SQL and Java if required.

Build real-time apps in minutes for continuous ingest

StreamLab is built on s-Server, Guavus SQLstream’s open standards-based platform for data stream processing, and integrated with s-Dashboard for real-time dashboards. StreamLab offers an intuitive, simple to use interface for building analytics and continuous ingest of data in motion. Through StreamLab, user can:

• Use the Sources and Destinations toolto connect input data sources and output platforms. Icons representing the each type of input source, such as log files, sensor feeds and middleware connections, and output platforms such as Hadoop and RDBMS data warehouses. are dragged and dropped onto the panel, with each offering additional configuration input screens based on the type of connection.
• Use the Guides to transform, filter and aggregate the live data being collected across the connected data sources. Guides enable live data to be viewed and formatted in real-time, with user actions being applied immediately and the stream format updated to reflect the user action. Guides include user options for adding and removing columns, changing data types, joining live streams, and applying analytics.
• Benefit from guided discover and formatting of data in motion using the Suggestions Tool. The Suggestions. StreamLab’s scrutinizer scans all incoming data, identifying characteristics such as common data formats, and individual data types such as timestamps and geospatial coordinates. The Suggestions Tool presents a contest sensitive list of helps and types to the user, which can be actioned by simply clicking on the suggestion.
• Save and deploy real-time processing apps in minutes. Once the discovery, formatting, external connections and live dashboards are built to the user’s satisfaction, the application is saved and then deployed using simply a web URL.

To see for yourself how StreamLab simplifies the building and deployment of real-time applications, click on the demo link above. This demo has been built using streaming GPS data from busses traveling on the road network around Sydney, New South Wales, Australia, and includes the building of a live dashboard for tracking bus movements, alerting on bus speed, and the continuous ingest of the streaming speeding analytics analytics into the Teradata Database. The whole application takes only ten minutes to build from scratch.