Writing to MongoDB

<< Click to Display Table of Contents >>

Navigation:  Integrating SQLstream Blaze with Other Systems > Writing Data Out of s-Server > Writing to Other Destinations >

Writing to MongoDB

Previous pageReturn to chapter overviewNext page

5_2_indicator Version 5.2 Feature

The MongoDB ECDA adapter writes batches of data to a MongoDB collection. Currently, the MongoDB ECDA adapter works with BSON only. In order to write to a MongoDB collection, you must first define a server object for a MongoDB server, as well as a foreign stream that defines the specific collection into which you are writing. The minimum credentials required to write to a MongoDB are a fully qualified URL and the name of the COLLECTION. Next, you set up and performing an INSERT into a foreign stream in order to write data to a MongoDB collection.

For more information on MongoDB URLs, see https://docs.mongodb.com/manual/reference/connection-string/.

For adapters, you configure and launch the adapter in SQL, using either server or foreign stream/table options. For agents, you configure such options using a properties file and launch the agent at the command line. Many of the options for the ECD adapter and agent are common to all I/O systems. The CREATE FOREIGN STREAM topic in the Streaming SQL Reference Guide has a complete list of options for the ECD adapter.

Note: Because of the nature of streaming data, you will need to set up a pump in order to move rows continually from an s-Server stream to another stream, file, Kafka topic, RDBMS table or other location. Pumps are INSERT macros that continually pass data from one point in a streaming pipeline to the other. A model for setting up a pump is provided below. See the topic CREATE PUMP in the s-Server Streaming SQL Reference Guide for more details.

Sample Code

CREATE OR REPLACE SERVER "mongodbWriteServer" TYPE 'mongodb'

FOREIGN DATA WRAPPER ECDA;

 

CREATE OR REPLACE FOREIGN STREAM "MongoWriterSchema"."MongoWriterStream"

(

"clientOS" VARCHAR(16),

"playerVersion" VARCHAR(32),

"sessionId" VARCHAR(64),

"clientIPAddress" VARCHAR(32),

"KPI" INTEGER

)

   SERVER "mongodbWriteServer"

   OPTIONS (

   "FORMATTER" 'BSON',

   "URL" 'mongodb://user:password@localhost:27017/db',

   "COLLECTION" 'stream',

   "DATE_FORMAT" 'mm/dd/yyyy',

   "TRANSACTION_ROWTIME_LIMIT" '0',

   "TRANSACTION_ROW_LIMIT" '70000');

 

CREATE OR REPLACE SCHEMA "Pumps";

SET SCHEMA '"Pumps"';

 

CREATE OR REPLACE PUMP "writerPump" STOPPED AS

--We recommend creating pumps as stopped

--then using ALTER PUMP "Pumps"."writerPump" START to start it

INSERT INTO "MongoWriterSchema"."MongoWriterStream"

SELECT STREAM * FROM "MyStream";

--where "MyStream" is a currently existing stream

 

To start writing data, use the following code:

ALTER PUMP "Pumps"."writerPump" START;