Global Error Stream

<< Click to Display Table of Contents >>

Navigation:  Streaming SQL Reference Guide > Error Handling >

Global Error Stream

Previous pageReturn to chapter overviewNext page

All errors that are logged to the trace log are also available in a system management stream called sys_boot.mgmt.all_trace. The latest (up to 1000) error messages are also available in a table called sys_boot.mgmt.all_trace. You can use these to create visualizations of errors; they can be accessed from sqlline or anywhere that has a JDBC connection to s-Server.

5_2_indicator

Version 5.2 Feature

As of version 5.2, you no longer need to use the preface SYS_BOOT.MGMT.

More information about the trace log can be found in the topic Using Trace Files in the Administrator Guide.  As described in that topic, you can control the level of trace messages -- error, informative, debugging -- is for each component of the server by modifying AspenTrace.properties. You can also change whether or not s-Server summarizes error messages (reports a periodic summary of errors), as described below.

To query the error stream, enter something like the following in sqlline:

select stream ROWTIME, * from sys_boot.mgmt.saved_trace;

 

This will produce a continuous stream of trace messages or errors as they occur. You can substitute specific columns for *. These are listed below. Columns are the same for both table and stream.

To query the error table, enter something like the following:

select * from sys_boot.mgmt.saved_trace;

 

to get the latest (up to 1000) trace messages or errors.

Error Stream Columns

Column

Column Type

Description

ROWTIME

TIMESTAMP

Timestamp for the error row itself.

error_time

TIMESTAMP

Time that error was logged. Differs from the rowtime of  the error row itself, which is slightly later than the time the error occurred. Also differs from the rowtime of the input row associated with the error (data_rowtime).

error_level

VARCHAR(10) NOT NULL

Java trace level, SEVERE, WARNING, and so on. See https://docs.oracle.com/javase/7/docs/api/java/util/logging/Level.html for details.

is_error

BOOLEAN NOT NULL

True for errors, false for other traces.

error_class

INTEGER

Code for error category: 0:other error, 1:ingest error, 2:egest error, 3:syntax error, 4:validation error, 5:planner error, 6:late rowtime, 7:calculator error, 8:other execution error.

sql_state

VARCHAR(5)

An alphanumeric error code. See Appendix B - List of Error Messages for more details.

error_name

VARCHAR(32),

Explanation of sql_state. See Appendix B - List of Error Messages for more details.

message

VARCHAR(4096),

Entire trace message.

thread_id

INTEGER

Number of the execution thread where the error happened.

session_id

INTEGER

Number of the client session where the error happened.

statement_id

INTEGER

The id number for the SQL statement where the error happened.

graph_id

INTEGER

graph_id and node_id identify which execution object had the error. See explanation of nodes and graphs below.

node_id

INTEGER

graph_id and node_id identify which execution object had the error. See explanation of nodes and graphs below.

xo_name

VARCHAR(128)

a unique name of the execution object

error_reporter

VARCHAR(256) NOT NULL

the name of the java logger that reported the error

error_sql

VARCHAR(256)

the SQL of the statement

error_backtrace

VARCHAR(4096)

if available, java backtrace at the point of the errot

data_row

VARBINARY(32768)

The input row that was being processed when the error occurred.

data_rowtime

TIMESTAMP

Rowtime of data_row

source_position_key

VARCHAR(4096)

Identifies the location in the input source, such as a log file or kafka topic.

pump_db

VARCHAR(128),

Database that includes the pump that runs the statement; null if not in a pump.

pump_schema

VARCHAR(128)

Schema that includes the pump that runs the statement; null if not in a pump.

pump_name

VARCHAR(128)

Name of the pump that runs the statement; null if not in a pump.

Summarizing Error Messages

By default, s-Server reports a periodic summary of errors, in order to avoid bloating the trace log. However, this means that the global error stream contains only the summary counts and a few sampled bad rows. If you turn off sampling, you get all the bad rows traced and reported to the error stream.

To adjust sampling, you can change properties in the aspen.properties file: For more information on modifying aspen.properties, see the topic Managing Java Parameters in the Administrator Guide.

When this property is set to false, s-Server reports every late row in both the trace log and error stream. When this property is set to true, true, s-Server reports the first event and then reports every 15 seconds after the first event.

aspen.trace.sampleLateRows=false

 

You can also specify that s-Server report every row that causes a calculator error. When this property is set to false, s-Server reports every row that causes a calculator error. When this property is set to true, s-Server reports the first calculator event and then reports calculator events every 15 seconds.

aspen.trace.sampleFailedRows=true

 

Stream-graphs, Nodes, and s-Server

s-Server consist of two layers:

1.A query engine that maintains a collection of SQL definitions and translates a SQL statement into a specified data-flow.
2.A data engine that executes and coordinates these data-flows.

The specification of a data-flow is called a physical query plan. The actual data flow is called a stream graph, which is a chain of stream nodes, each of which is a small software machine that applies some relational operation to its input data stream(s) to produce is output data stream(s). Nexus nodes are defined by a CREATE STREAM statement. They are freestanding nodes to which other stream graphs can connect and disconnect, in order to receive or deliver data.

nodes_stream_graph

In the most common case, a node has 1 input and 1 output, and so the graph is a linear chain of operations. But data flows can merge and split, and the graph of a new query can connect to the graphs of existing queries as they run.

Note: A node does not read data directly from its upstream neighbor. Instead, the upstream neighbor puts its output rows into a block of memory called a buffer, which it shares with the downstream neighbor. The upstream node writes rows to the buffer, and the downstream node reads rows from the buffer.

Because there are more stream nodes than CPU cores, all the nodes cannot actually run simultaneously. Instead, a software component called the scheduler manages the stream nodes. The scheduler runs nodes as needed in order to push data down the pipeline.

The scheduler decides when to execute a node based on its status, which consists of 1) its state (ready/sleeping/blocked/finished), 2) the availability of data for input or room for output, and 3) the rowtime of the next input row. The scheduler also keeps track of statistics for each execution of each node, measuring the time spent, the amount of data input and output, and so on.