Adding a CSV Sink

<< Click to Display Table of Contents >>

Navigation:  Using StreamLab > StreamLab Sinks Overview >

Adding a CSV Sink

Previous pageReturn to chapter overviewNext page

This sink writes data in CSV (Comma Separated Value) format to an output.

1.First, select a sink output type. You need to select one of these for CSV, JSON, and XML sinks. This could be a file, network socket, AMQP, or Kafka location.
2.Start by specifying the s-Server schema and stream names for the sink's stream. By default, StreamLab uses the project schema for the sink. You can also route guide data to the sink later by selecting the Routing Data to a Sink button in a guide.
3.Next,  fill in the list of columns and their SQL types. Click the + icon to add another column. Give the column a name, and select its type from the pulldown menu. You can use the Clipboard to copy column names and types from another form.
4.The character used to separate columns defaults to comma and can be changed via the Column Separators(s) field. You can also change the Character Encoding and decide if you want a CSV header containing the column names to be written.

sl_sink_csv

5.Use the output selector to choose an output and fill in the parameters to connect to that source. (This sink uses the Extensible Common Data Adapter to support writing to a number of different output media.) For more information on the options for ECDA, see the topic Writing to Files with the ECD Adapter and Agent in the Integration Guide.
6.Click the blue title to save the project. The title returns to white after you click it.
7.Click the Go Up arrow to exit the Add CSV Sink page.

sl_go_up_arrow

Local vs. Remote

sl_remote_agent_button In this mode the output is accessed directly from s-Server. Files are local to the s-Server host, and sockets and other network connections are relative to the s-Server host. When this mode is enabled, StreamLab submits your configuration for this sink directly to your instance of s-Server.

sl_local_adapter_button In this mode you can access sinks via a host running the ECDA agent. The agent is a Java application running on a host other than the s-Server host, configured to read data from your instance of s-Server. When this mode is enabled, StreamLab configures the stream that the remote agent will read from. To configure the remote agent,use the Export Properties button to get the contents of the properties file, and paste this into a new file on the remote agent server (for example, mysource.props).

sl_export_properties_sink

From a command shell on the remote host, run

%$SQLSTREAM_HOME/../clienttools/EcdaAgent.commondataagent.sh --output --props mysource.props.

Note: $SQLSTREAM_HOME refers to the installation directory for s-Server, such as /opt/sqlstream/5.0.XXX/s-Server.

Everything you've specified here to configure the sink will now apply within the remote agent, so file names will be local to that host, and sockets and other network connections will be relative to that host.