Streaming Sensor Data
Railroads have used track side readers to scan bar codes on the sides of freight cars since the 1970s. Such sensors provided real time tracking of goods as they made their way from the supplier to the delivery point. Retail businesses increased the use of RFID tags in the past 20 years to track goods through the manufacturing process. Since the Indian Ocean tsunami of December 2004 the public has become aware of deep water pressure sensors which sit on the ocean floor to detect tsunamis and are intended to generate warnings about potential disasters.
The cost of sensors has decreased significantly in recent years and as a result inexpensive sensors are present nearly everywhere in businesses. As the price of sensors decreases it becomes economically feasible to deploy thousands and even millions of sensors. Such sensors cumulatively generated huge volumes of data. Imagine placing a sensor capable of measuring temperature, humidity, sun light and air pressure sensor within each square kilometer in the state of Iowa to assist farmers in managing crop production. Now imagine each of those 145,743 sensors generating 100 bytes of data every minute resulting in a data volume of nearly 21GB per day.
There is much buzz about big data and the challenges of applying traditional database management tools to extract business value from such data. Fortunately, there is a better way – integrating real time data, as provided by sensors, with stream analytic processing, allows timely enterprise decisions in response to changing conditions.
Sensor Expo and Conference
In early June I attended the Sensors Expo and Conference held at the Rosemont Convention Center in Rosemont Illinois. My goal was to explore the intersection between SQLstream technology and sensors. The event describes itself as the “Only Industry Event in North America Exclusively Focused on Sensors and Sensor Integrated System”. I discovered a world, previously unknown to me, of sensors reporting pressure, flow, temperature, moisture, volume, motion, vibration, physical location, chemical vapors, weight, light, color, and more.
Many of these devices are lost-cost and also have very low power requirements. Those aspects are critical when one considers deploying thousands or millions of devices, especially in environmentally challenging locations. Sensors are generally capable of continuous reporting which leads to the challenges associated with “Big Data”.
Sensor Deployment
Considering sensors and the applications described by the various vendors in attendance at the expo and speakers at the conference, lead me to characterize sensor vendors and users based on their unique perspective. Raw Sensors are themselves devices that report state. For example a temperature sensor computes the ambient temperature at the location of the sensor itself.
Sensors are embedded into physical devices and industrial equipment as a way to locally control the device. For example a motion sensor may be used inside a device to ensure that the device is property oriented before use.
A set of sensors may be collected into a network so they report data in a unified manner. Wireless technology allows many of these networks to be low cost and use low power. Wireless network controllers receive data from the individual sensor and makes the sensor data available to the outside world over TCP/IP and utilize a simple web based reporting mechanism. The wireless network controllers are programmable which allow sensors to be added and configured. For example a set of wireless sensors may report the flow or pressure of liquid through a campus wide pipeline.
Of significance to SQLstream is the integration of wireless sensor networks into enterprises. Applications can do more than simply display the raw sensor data in a dashboard or set of flashing lights. Traditional applications might store the sensor data in a database to be queried later. With the power of SQLstream, applications can perform analytical analysis on the sensor data in real time without landing the data on disk and delaying the necessary decisions based on that data.
As I suggested earlier, sensor networks have the potential to inexpensively generate large amounts of data. I suggest that we look beyond the obvious use for such data. In an earlier example I imagined temperature, humidity sun light and air pressure data collected to assist farmers growing crops in Iowa. Can a retailer use the same data in order to predict the goods and in which quantity and style are expected to be purchased this evening by city dwellers in Des Moines? Sunscreen or home maintenance products?
Call Us
So I and the folks at SQLstream would like to know, “How are you embedding sensor data into your enterprise?” What kinds of business decisions are you making in real time or would like to make in real time? Comment here or email me.