Appendix A: Legacy Adapters and Agents

<< Click to Display Table of Contents >>

Navigation:  Integrating SQLstream Blaze with Other Systems >

Appendix A: Legacy Adapters and Agents

Previous pageReturn to chapter overviewNext page

SQLstream maintains a number of legacy adapters and agents that enable s-Server to interact with external systems.

Note: Unless you have used these in the past, we recommend using the Extensible Common Data Adapter and Agent instead of these legacy adapters and agents. See the topics Reading Data from RDBMS Sources, Writing Data to RBDMS Destinations, and Reading from and Writing to Other Sources/Destinations for more details.

Adapters

You use adapters at either the beginning of stream processing or the end, that is, at the point where a stream will be receiving data or at the point where a stream will be emitting data. Adapters implement foreign streams; they use an internal API to exchange data with SQLstream s-Server. (Code sections appear in the topic Example Code for Implementing Plugins. For a full adapter code example, see the topic Table Update Adapter in this guide.)

SQLstream s-Server's adapters conform to the SQL/MED object hierarchy, as follows:

Wrapper

Data wrappers are the SQL/MED standard's "container" for the Java plugin. The wrapper identifies the class file that implements the adapter. That class provides the main entry point for the Java plugin.

See the installing section in the Example Code section of this topic.

As the wrapper represents the adapter (the plugin), you only need one wrapper per installed adapter. However, you can have multiple wrappers with different names for the same adapter if desired.

Server

(also called
"foreign server")

A foreign server is an instance of the adapter. You can have multiple foreign servers based on the same wrapper, usually to allow for separate instances with distinct properties.

See the defining section in the Example Code section of this topic.

For example, the FileWriter Adapter writes flat files (XML or character-separated) and can create a pool of rotated output logs. Starting with the same wrapper, you might declare:

Server A with options suitable for a pool of XML files with certain naming conventions and log-rotation rules

Server B with options suitable for a pool of CSVs with their own naming conventions and log-rotation rules

Each foreign server can support multiple foreign streams.

Foreign stream

A foreign stream is an instance of a data stream associated with a particular external source or sink.

For example:

For the FileWriter Adapter, each foreign stream is associated either with a particular output file or with a pool of output files sharing a common base name.

For the TableUpdate adapter, each foreign stream is associated with a particular target table in an external relational database, into which updates are streamed.

For the TableReader adapter, each foreign stream is associated with a particular source table, from which new rows are streamed.

Out-of-the-Box SQLstream Plugins

The current list of SQLstream plugins (adapters and UDFs/UDXes) is summarized below. They are located in the SQLSTREAM_HOME/plugin directory. This list continues to grow. Please contact technical support for an updated list of adapters:

Extensible Common Data Adapter

Lets you read/write from a variety of sources, including CSV, XML, JSON, and various log file types over the file system, network or AMQP.

AMQP Adapter

Reads/writes data to AMQP (Advanced Message Queuing Protocol) topics and exchanges

(AmqpAdapter.jar)

JDBC Adapter

Mounts other instances of SQLstream or external DBs

(FarragoMedJdbc.jar)

FileWriter Adapter

Writes flat files (XML or character-separated) and can create a pool of rotated output logs \\ (FileWriter.jar)
(See also FileWriter Agent.)

Log File Adapter

Tails active log files, following log rotation, streaming new log lines as they appear

(LogFileReader.jar)

(See also Log File Agent.)

Network Socket Adapter

Receives data streamed from a client program using TCP or UDP. The adapter listens at a configurable port awaiting client connections. Incoming packets are streamed; by default, incoming TCP packets are streamed as multiple rows and incoming UDP packets are streamed one per row.

(NSAdapter.jar)

TableReader Adapter

Tails DB tables, streaming new table rows as they appear. (TableReader.jar)

Note: the functionality of this adapter has been replaced by the SQL/MED Plugin for JDBC. If you are new to s-Server, please use this plugin to connect with an external database. See the topic Writing Data to RBDMS Destinations for more details.

TableUpdate Adapter

Inserts/updates/upserts (merges) streaming rows into target DB table (TableUpdate.jar)

Note: the functionality of this adapter has been replaced by the SQL/MED Plugin for JDBC. If you are new to s-Server, please use this plugin to connect with an external database. See the topic Writing Data to RBDMS Destinations for more details.

RSS/Atom Adapter

Tails syndicated REST-style XML web feeds like RSS, Atom, TwitterXML

(WebFeed.jar)

Agents

SQLstream provides the following legacy client agents. With the exception of the Big Query Connector, all agents can be installed on either Linux or Windows. Linux agents run as shell scripts, while Windows agents run as cmd scripts.

A FileWriter Agent, for writing files from streams.

A Log File Agent, for parsing log files as they are generated and capturing useful data into fields and streams that can be turned into information-on-the-fly.

A Network Socket Agent ("NSAgent"), for writing to one or more SQLstream streams and optionally recording all incoming packets.

A Big Query Connector, for integrating SQLstream with Google Big Query for Continuous ETL.