| Internet-Draft | DataReel gxStream | November 2026 |
| Gaer | Expires 22 May 2027 | [Page] |
This document describes gxStream, the database block protocol of the DataReel C++ library, and the services carried on it: the DataReel database object server with its channel multiplex, remote access to database objects and vector searches, and gxObjRepl object replication between two independent object databases. gxStream runs over TLS with certificates on both sides on TCP port 2073, which is registered with IANA under the service name "gxs-data-port". The port was registered before a protocol description was required; this document provides that description, as implemented in DataReel 7.4. It is published for information and does not specify an Internet standard.¶
This is an Independent Submission. The protocol is defined by its reference implementation, the DataReel library, which is free software under the GNU Lesser General Public License version 3 or later (https://datareel.com). Where this document and the DataReel 7.4 source differ, the difference is listed in Appendix A.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 22 May 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document.¶
DataReel is a cross-platform C++ library with an embedded object database and TLS communication classes. Since 1999 it has had a block protocol, gxStream, that carries database blocks between programs: the same block header the database writes before every object in its files is sent on the network, followed by the block's bytes. TCP and UDP port 2073 were registered with IANA for this protocol as "gxs-data-port", "DataReel Database Socket".¶
DataReel 7 (2026) rewrote the protocol on TLS: no block is ever sent or accepted in cleartext, servers require client certificates by default, and every length received is checked before it is used. DataReel 7.3.1 added the database object server, which carries several services over one listener and a multiplex of independent channels. DataReel 7.4 added vector searches to the object service, a canonical block header on every platform, and gxObjRepl, which replicates objects (never file pages) from a primary node to a copy.¶
This document specifies:¶
DataReel 6.x, the legacy release line, can also carry gxStream blocks over plain TCP and over UDP. These legacy transports are described in Section 9 for completeness; they have no authentication or encryption and are NOT RECOMMENDED.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This document describes an existing protocol. The requirement words state what an implementation needs to do to interoperate with DataReel 7.4 and to keep its security properties.¶
Integer types are written u8, u16, u32 and u64 (unsigned, 1, 2, 4 and 8 bytes), i64 (signed, 8 bytes, two's complement) and f32 (IEEE 754 binary32). Unless a section says otherwise, every integer and float is sent in little endian byte order.¶
gxStream runs over TCP [RFC9293]. Servers listen on port 2073 unless configured otherwise, and clients connect to port 2073 unless given another port. A server MAY listen on IPv6 and IPv4 at once. Implementations SHOULD disable the Nagle algorithm on replication connections, since each frame is answered by a small acknowledgment.¶
Every gxStream connection MUST be protected with TLS. The TLS handshake starts immediately after the TCP connection is established; there is no cleartext phase and no STARTTLS command.¶
A failed handshake, failed certificate check or failed allow list check closes the TCP connection.¶
Every handshake, read and write SHOULD be bounded by a timeout. DataReel 7.4 uses 10 seconds for the handshake and 120 seconds for each block on the database object server, 30 seconds for the replication handshake and hello, and a configurable idle timeout (default 120 seconds) for replication frames.¶
Every block begins with a 32-byte header of four 64-bit little endian words:¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Check Word (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Status (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Result (64 bits) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The high four bytes of the Check Word and of the Status word are always zero. A receiver that finds a non-zero byte there has met a peer that sends the obsolete header with 32-bit words (a 24-byte or 16-byte header sent by DataReel builds older than 7.4 beta 5 on 64-bit Windows, or by 32-bit builds). It MUST close the connection and SHOULD report that cause; it MUST NOT try to read the block.¶
The block header is the one DataReel writes before each block in its database files; on the network the Result word carries what the file uses as a free list pointer.¶
The control code is one ASCII character:¶
| Code | Name | Meaning |
|---|---|---|
| 0x41 "A" | Add | The body is a new object to add. |
| 0x43 "C" | Change | The body names an object to change; an "S" block with the new contents follows. |
| 0x44 "D" | Delete | The body names an object to delete. |
| 0x4B "K" | Kill server | Header only. Asks the server to shut down. |
| 0x4C "L" | Close connection | Header only. The sender is done with the connection. |
| 0x52 "R" | Request | The body is a request; the reply is a block. |
| 0x53 "S" | Send | The body is data: a reply, a multiplex frame or a replication frame. |
| 0x54 "T" | Restart server | Header only. Asks the server to restart. |
| 0x57 "W" | Acknowledge | Header only. An acknowledgment; the Result word carries the result. |
Other code values are defined in the DataReel headers for database file use or for earlier designs and are not sent by the services in this document. A server MUST close a session that sends a code its service does not expect. "K" and "T" give a client control over the server process: a server MUST NOT act on them unless the client is authorized to administer the server; the DataReel 7.4 database object server does not act on them and closes the session.¶
Each receiver has a maximum block size. DataReel 7.4 uses 67108864 bytes (64 MiB) by default and 17825792 bytes (17 MiB) on the database object server. A sender SHOULD NOT send a block larger than the peer is known to accept: multiplex senders split messages into frames of at most 1 MiB (Section 6.3), and replication limits objects to 32 MiB and commits to 60 MiB.¶
The original gxStream exchanges use the control codes directly. The object service accepts them from DataReel 6.x style ("legacy") clients (Section 5.1).¶
The DataReel database object server carries several services on one listener. After the TLS handshake and the client certificate checks, the server reads the session's first block and gives the session to the first service that claims it:¶
A first block that no service claims closes the session. The first byte of a multiplex body is 0x4D ("M"); services that use "S" blocks for other messages MUST NOT start their first message with that byte.¶
To resist connections that never complete TLS, a server SHOULD run each handshake apart from its accept loop with a short timeout and a limit on concurrent handshakes (DataReel 7.4: 10 seconds, 16 handshakes), and SHOULD limit the number of authenticated sessions (DataReel 7.4: 4 by default). Only sessions that passed the certificate checks count against the session limit.¶
On a server configured with a legacy database:¶
The multiplex carries numbered channels over one connection. Each channel has its own order, flow control and errors, so one slow or failed channel does not hold up the others.¶
Each multiplex frame is one "S" block. Its body is an 18-byte frame header followed by the payload:¶
+--------+--------+--------------------+-------------------------+ | Mark | Kind | Channel (u32) | Sequence (u64) | | 1 byte | 1 byte | 4 bytes | 8 bytes | +--------+--------+--------------------+-------------------------+ | Flags (u32) | Payload (Length - 18 bytes) | | 4 bytes | | +-----------------+----------------------------------------------+
In payloads, "str" is a u32 length followed by that many bytes, and "bytes" is a u64 length followed by that many bytes. Every length MUST be checked against the remaining payload; a payload that is too short is malformed.¶
| Kind | Name | Payload |
|---|---|---|
| 1 | HELLO | str "GXDBMUX", u32 version (1) |
| 2 | OPEN | str service, str name, str mode, u64 credit |
| 3 | OPENED | u64 credit, bytes extra |
| 4 | REFUSED | str reason |
| 5 | DATA | service data |
| 6 | CREDIT | u64 bytes |
| 7 | ACK | u64 seq, u64 lsn |
| 8 | ERROR | str reason |
| 9 | CLOSE | (empty) |
A block that is not an "S" block, or whose body is not a frame (shorter than 18 bytes, a wrong mark, an unknown kind), ends the connection. An "L" block ends the connection.¶
Frames of one channel are delivered in the order they were sent; TLS over TCP provides that order. ACK frames are passed to the channel's consumer in order with its DATA frames; a service uses them to report that the data up to a DATA sequence number is durable at a log sequence number (lsn) of its own.¶
Flow control is per channel and counts DATA payload bytes only; control frames are not counted.¶
The object service gives remote access to the objects of the databases a server is configured with. A client opens a multiplex channel with service "object", the database name and mode "read" or "write". The server MUST refuse the open for a database it does not serve, for any other mode, and for mode "write" on a database that is read-only or is a replicated copy.¶
Each request is one message: a u8 operation followed by its arguments. Each reply is one message: a u8 status (0 success, 1 error) followed by the result on success, or by a str (u32 length and bytes) with the error on failure. An empty request message closes the channel with an error.¶
| Op | Name | Arguments | Result |
|---|---|---|---|
| 1 | ADD | bytes data (length at least 1) | u64 address |
| 2 | REQUEST | u64 address | bytes data |
| 3 | CHANGE | u64 address, bytes data | (empty) |
| 4 | DELETE | u64 address | (empty) |
| 5 | FIRST | (none) | u64 address, 0 if none |
| 6 | NEXT | u64 address | u64 next address, 0 if none |
| 7 | VECTOR_SEARCH | Section 7.2 | Section 7.2 |
| 8 | VECTOR_INFO | (none) | Section 7.2 |
A database configured with a vector index answers VECTOR_SEARCH and VECTOR_INFO on any channel, read only. The VECTOR_SEARCH arguments are, in order: u32 version (1); u32 k (1 to 1000); u32 ef (0 for the server default of 64, at most 4096); u32 dim (MUST equal the index's dimension); u32 flags (bit 0: apply the day range; bit 1: strict, a missing index file is an error; other bits MUST be zero); u32 tag_mask (bits 0 to 3; other bits MUST be zero); u32 lo[4]; u32 hi[4] (when bit i of tag_mask is set, tag i must be within lo[i]..hi[i]); u32 day_lo; u32 day_hi; and f32 query[dim]. The arguments MUST be exactly that long and every query value MUST be finite.¶
The VECTOR_SEARCH result is: u32 version (1); u32 count; u32 files skipped by their day range; u32 files missing; count entries of { i64 id, f32 distance, u32 tags[4], u32 file }, sorted by distance, then file, then id; u32 names; and names entries of { u32 file, str path } for the files the results name, with paths relative to the index manifest's folder.¶
The VECTOR_INFO result is: u32 version (1), u32 dim, u32 element type, u32 metric, u32 files, u64 nodes, u64 deleted, str spec.¶
A server SHOULD limit concurrent searches; DataReel 7.4 runs at most 4 at once and answers a search that waits more than 5 seconds for a slot with the error "busy".¶
gxObjRepl replicates the object databases of a primary node to a copy node. Only objects travel; each node keeps its own file addresses, free space and indexes. Objects are identified by 64-bit ids shared by both nodes. Replication is asynchronous.¶
The primary is the TLS client and the copy is the TLS server. The copy MUST require a client certificate from the configured certification authority with the configured subject common name; the primary MUST verify the copy's certificate and identity (Section 2.2).¶
Each replication frame is the body of one "S" block. The copy answers each frame with exactly one "S" block. The primary sends the next frame only after it has the answer. In this section a "str" is a u16 length followed by that many bytes.¶
The copy handles one primary connection at a time. It closes the connection, without an answer, when a frame breaks the rules below; it SHOULD record the reason where an administrator will see it (DataReel writes an alarm file).¶
The first frame on each connection is the primary's hello:¶
hello = u32 magic ; 0x3550524F ("ORP5" in byte order)
u64 epoch ; the primary's epoch
str link-map ; link fields, "DB:OFFSET:TARGET_DB,..."
str name ; the primary's node name
u8 role ; 1 (primary)
u64 last-seq ; the primary's last commit
u64 digest ; the running digest at last-seq
¶
The copy answers:¶
reply = u8 ok ; 1 accepted, 0 refused
u64 applied-seq ; the last commit the copy applied
u64 epoch ; the copy's epoch
u8 in-full-send ; 1 if a full send is under way
str name ; the copy's node name
u8 role ; 2 (copy)
u64 digest ; the running digest at applied-seq
str why ; the reason when ok = 0
¶
The copy MUST refuse (ok = 0, then close) a hello that is malformed or has another magic number; a hello received while the copy is itself a primary; a role other than 1; a name other than the configured peer name, if one is configured; an epoch lower than the copy's; a last-seq lower than the copy's applied-seq when no full send is under way; a malformed link map; or, when the copy has its own link map configured, a link map that differs from it or link fields that overlap the copy's index fields.¶
The primary MUST stop (and SHOULD raise an alarm) when the reply has ok = 0, a role other than 2, or a name other than its configured copy name. Unless the copy is in a full send or has applied nothing, the primary MUST also check the copy's history: applied-seq MUST NOT exceed the primary's last commit, and the copy's digest MUST equal the primary's digest at applied-seq. A primary that no longer knows its digest at that sequence MUST refuse the copy; an administrator then has the copy rejoin.¶
Having accepted the copy, the primary sends F_EPOCH (Section 8.3). The copy MUST refuse every other frame kind on the connection until it has received F_EPOCH. The copy saves the epoch durably on F_EPOCH, never at the hello, so that a primary that then refuses the copy cannot fence out the real one.¶
Each frame starts with a u8 kind:¶
| Kind | Name | Contents after the kind | Copy's answer |
|---|---|---|---|
| 1 | F_COMMIT | u64 seq, u64 epoch, u32 ops, ops | u64 seq applied |
| 2 | F_SNAP_BEGIN | u64 base seq, u64 epoch | u64 0 |
| 3 | F_SNAP_BATCH | u32 records, records | u64 0 |
| 4 | F_SNAP_END | u64 base seq, u64 digest, u64 count, u64 hash | u64 base seq |
| 5 | F_EPOCH | u64 epoch | u64 applied seq |
| 6 | F_SNAP_DLOG | u32 entries, entries of {u64 seq, u64 digest} | u64 0 |
An answer is an "S" block whose body is at least 8 bytes; its first 8 bytes are the u64 shown. A frame of an unknown kind is refused.¶
Each op is: u8 op (1 ADD, 2 UPDATE, 3 REMOVE, 4 PURGE), str database, u64 object id; ADD and UPDATE then carry u32 capacity (the record size the primary allocated), u8 moved (1 if the object moved to a new block on the primary), u32 length and that many bytes of object data. In the object data, every link field named by the link map carries the target's object id instead of a file address (0 for no link).¶
A database name consists of letters, digits, "_" and "-" only. The copy MUST:¶
REMOVE leaves a tombstone that keeps the object's id and address, so links to it still resolve. PURGE frees a tombstone.¶
When the copy is in a full send, or the primary no longer has the commits that follow the copy's applied-seq, the primary sends a full copy of its databases as of its last commit (the base):¶
The commits made on the primary after the base then follow as F_COMMIT frames.¶
The running digest is the 64-bit FNV-1a hash [FNV] (offset basis 14695981039346656037, prime 1099511628211) chained over the bytes of every F_COMMIT frame body in sequence order: it starts at the offset basis, and each commit continues the hash from the previous value over the whole frame body, from its kind byte to its last op. The primary computes it when it records a commit; the copy when it applies one. Each node keeps a log of (seq, digest) pairs for at least the last 1000000 commits.¶
The full send hash is the same FNV-1a function chained, from the offset basis, over the bytes of each F_SNAP_BATCH record and of each 16-byte F_SNAP_DLOG entry, in the order sent.¶
These digests detect divergent histories and lost, repeated or altered frames caused by faults. They are not cryptographic and do not authenticate the stream (Section 11.3).¶
A node starts at epoch 1. A takeover makes a copy the primary at its epoch plus one. Object ids that a primary issues carry its epoch in bits 40 and above, so a new primary never issues an id that an older primary may have issued in commits the new primary never received. Every copy that has saved the new epoch refuses an older primary. A former primary rejoins as a copy that waits for a full send, which replaces its databases.¶
Replication is a gxStream object stream and uses TCP port 2073. A copy listens on port 2073, and a primary connects to port 2073, unless configured with another port; a host that runs a copy and another gxStream service on the same address gives one of them another port. DataReel 7.4.1 makes 2073 the default of both settings; DataReel 7.4 had no default.¶
DataReel 6.x is the legacy release line (bug fixes and security updates only). Its gxStream and gxDatagram classes predate the TLS requirement of Section 2.2. Since DataReel 6.4 they send the same canonical block header as DataReel 7.4 (Section 3.1) and check received lengths before allocating memory. Both transports below are described so that implementations and network operators can recognize them; neither meets the requirements of this document.¶
A DataReel 6.x gxStream client uses TLS only when the program enables it. A DataReel 6.4 gxStream server refuses to accept clients unless it either uses TLS with client certificates and an allow list (as in Section 2.2) or the program explicitly opts out; a server that opts out without a server certificate exchanges blocks over plain TCP. Blocks on such a connection have no confidentiality, integrity or authentication. New deployments MUST NOT use plain TCP, and DataReel 7 does not support it.¶
The DataReel 6.x gxDatagram class sends blocks over UDP [RFC0768]: each block is one datagram holding the 32-byte block header, followed by one datagram holding the body (so a body is at most 65507 bytes). The control codes and exchanges of Section 4 apply. There is no retransmission, ordering, encryption or authentication: any host that can reach the port can send blocks, and a lost or reordered datagram leaves the two sides out of step. Receivers check the header form and the length against a maximum before reading the body.¶
This is the use for which UDP port 2073 was registered. gxDatagram is not part of DataReel 7, and implementations of this document do not use UDP. New deployments MUST NOT use gxDatagram on networks where any host is untrusted.¶
All traffic is protected by TLS (Section 2.2), and implementations refuse to send or accept blocks before the handshake completes, so there is no cleartext mode to downgrade to. The block header, the multiplex and the replication frames add no cryptographic protection of their own. Implementations should require TLS 1.3 where both peers support it. With TLS libraries that offer a hybrid key exchange such as X25519MLKEM768 (ML-KEM [FIPS203]), recorded traffic is protected against a future quantum computer; certificates remain RSA or ECDSA.¶
Servers authenticate clients by certificate (mutual TLS) and an allow list. Server certificates are checked against the server's name. Authorization is by service: a certificate in the allow list may use every database the server serves, in every mode the server's configuration allows for that database. A client allowed "write" access can change or delete any object of that database. Deployments that need finer authorization should run separate servers or databases per client group.¶
A server MUST accept a client-supplied object address only if it is the start of a live object of that database: the block header in front of it must be valid and not deleted, the block must lie inside the file, and the address must be a block boundary rather than a place inside another object, whose bytes a client may have written to look like a block header. DataReel 7.4 checked only the header's check word, so a client with write access could damage the database it may change and a client with read access could read deleted blocks; DataReel 7.4.1 makes the full check for writable databases (Appendix A).¶
A peer whose certificate the copy accepts is trusted with the stream. Such a peer can raise the copy's epoch, which fences out the legitimate primary, or replace the copy's data with a full send. The epoch, sequence and digest checks protect against operational mistakes (a stale primary, two primaries, a diverged history); they do not protect against a compromised key. Replication keys should be dedicated to replication, kept readable only by their owner (DataReel refuses a key file that group or others can read), and the copy's allow list should name only the primary.¶
Every length received is checked against the remaining data and against a maximum before memory is allocated. Servers limit concurrent handshakes, handshake time, sessions and concurrent vector searches, and the multiplex bounds each channel's queue with credit. An attacker who can reach the port can still occupy handshake slots for the handshake timeout, and a replication copy, which serves one connection at a time, can be delayed by a connection that stalls its handshake. The number of channels in one multiplex session is not limited in DataReel 7.4. Access to port 2073 should be limited by firewall to the hosts that need it, and servers used only locally should listen on the loopback address.¶
The "K" (kill) and "T" (restart) control codes let a client stop or restart a server process. They must not be honored for clients that are not authorized administrators; DataReel 7.4 servers do not honor them.¶
The legacy DataReel 6.x transports (Section 9) carry blocks in cleartext and, over UDP, without any peer authentication. Operators should block them at the network edge and replace them with TLS connections.¶
DataReel database files are not encrypted by the library; data is protected on the wire, not at rest. Block check words and optional CRC-32 checksums in the files detect accidental damage only.¶
IANA has assigned port 2073 for TCP and UDP in the "Service Name and Transport Protocol Port Number Registry" [RFC6335]:¶
Service Name: gxs-data-port Port Number: 2073 Transport: tcp, udp Description: DataReel Database Socket¶
This document asks IANA to update the Reference of the TCP assignment to this document, keeping the service name, port number, assignee and contact.¶
The UDP assignment is used only by the legacy DataReel 6.x gxDatagram transport (Section 9.2). This document asks IANA to update the Reference of the UDP assignment to this document as well, and otherwise to keep it unchanged while DataReel 6.x is maintained.¶
This document creates no new registry. The control codes, multiplex frame kinds, object service operations and replication frame kinds are maintained by the DataReel project with its source code.¶
This section follows [RFC7942]. The reference implementation is the DataReel C++ library [DATAREEL], release 7.4, on Linux (OpenSSL) and Windows (Schannel or OpenSSL). The DataReel 6.4.1 legacy release sends the same block header and interoperates with 7.4 over TLS. Files: include/gxheader.h, include/gxstream.h, src/gxstream.cpp, include/gxstls.h, src/gxstls.cpp, src/gxdbserver.cpp, src/gxdbmux.cpp, src/gxdbobject.cpp, src/gxobjrepl.cpp.¶
The DataReel 6.4 legacy release (6.4.1 and the 6.4.2 branch) implements the block header and the basic exchanges in its own gxStream class (include/gxstream.h, src/gxstream.cpp), with TLS, client certificates and an allow list as in Section 2.2, and also the legacy transports of Section 9 (src/gxdatagm.cpp for UDP). Its optional dr7features library links the DataReel 7.4 code for the object service client (vector searches) and for gxObjRepl replication, so a 6.4 program and a 7.4 program replicate to each other with the protocol of Section 8. The utils/objrepl/xver tests of DataReel 7.4 check 6.4 and 7.4 peers against each other.¶
Known differences between DataReel 7.4 and this document, to be resolved in a later 7.4.x release:¶
The gxStream protocol has been part of DataReel since 1999. This description was written from the DataReel 7.4 source code.¶