C++ Low-Latency HFT Architecture

A complete low-latency algorithmic trading pipeline, consisting of an exchange simulator, market data handler, limit order book, passive market maker strategy process with pre-trade risk checks, and a paper OMS.

Exchange Simulator

The exchange simulator generates and publishes market events over UDP multicast, running on three separate threads:

1. A configurable synthetic order generator publishes order requests to the exchange's matching engine, which publishes the corresponding market events to a lock-free SPSC ring buffer.
2. The exchange simulator's orchestrator reads these market events, passes them to a sequencer which encodes the messages into a custom binary protocol, stamps a sequencer number and header, and serializes the entire message. The orchestrator caches this encoded message on a thread-safe circular cache, and forwards it to the UDP broadcaster to send over UDP multicast.
3. The asynchronous, I/O-multiplexed TCP retransmit server handles client connections, retreives and corresponding cached packets, and transmits them over the TCP connection.

Market Data Handler

The market data handler retreives and processes market events from the exchange, running across four threads:

1. A real-time feed listener listens for packets broadcast by the exchange over UDP multicast, and directly places the packets from the kernel's buffer into a lock-free SPSC ring buffer.
2. The market data handler reads these packets, decodes their headers, passes them to a gap detector, and writes a retransmit request to a lock-free SPSC ring buffer if necessary. It then immediately writes the packet to the reorder buffer.
3. The TCP retransmit client spins on the lock-free SPSC ring buffer until it receives a request, then forwards this request to the exchange's retransmit server, receives and decodes the packet headers, and places them directly in the reorder buffer.
4. The market event publisher spins on the reorder buffer, waiting for the next packet's data. When available, it pulls this packet's data from the reorder buffer, decodes the market event, and pushes the market event to a lock-free SPSC ring buffer.

Strategy and OMS

Note

The strategy/OMS components are currently being developed.

The strategy/OMS places orders based on received data, running across three threads:

1. An order book engine that receives market events and reconstructs the state of the instrument's order book, as currently held by the exchange. The order book engine publishes the best bid and ask prices, protected with an atomic SeqLock.
2. The automatic trader process reads changes in the best bid and ask, and forwards information to a templated strategy process. This strategy enters orders it wishes to execute. The trader checks these orders with inline risk checks, before sending them to the exchange over TCP. Currently, no outgoing TCP gateway exists.
3. The order management system receives these orders, process them, calculates unrealized/realized PnL, and updates risk limits. It receives updates from the exchange to track order lifecycles, forwarding fills and other information to the trader, which forwards these messages to the strategy process.

Run and Execute

Warning

The exchange's TCP retransmit server uses kqueue to pool many connections on a single thread. This requires macOS or BSD and will not work on Linux. If you wish to run the repo in Linux, you will need to refactor the simulator and remove the retransmit server, or use epoll instead. Further, the pipeline makes use of several ARM64 instructions to optimize spin waiting on ARM systems, and is optimized for Apple Silicon (e.g. with thread affinity, using 128-byte cache lines to prevent false sharing on Apple's performance cores, etc.) and may not perform as well on other systems.

A Makefile has been provided for convenience:

1. Clone the repo.
2. Edit includes/config.hpp as desired. By default, LOGGING is enabled. You may need to edit the UDP MCAST interface or other network details.
3. Run make bin/exchange-market-data run ./bin/exchange-market-data. Alternatively, run ./bin/exchange to start the exchange, and ./bin/market-handler to separately start the market data handler.
4. Press CTRL + C to stop.