As enterprise networks scale to accommodate highly distributed workloads, traditional routing architectures face significant control-plane overhead and label scarcity. The Graphiant network addresses these challenges by implementing a scalable, Stateless Core architecture. By uniting Multiprotocol Label Switching (MPLS) with OSPFv3 Segment Routing (SR) and our patented Object Distribution Protocol (ODP, based upon MP-BGP), the architecture entirely relieves the core backbone of maintaining individual customer routing states or Virtual Routing and Forwarding (VRF) tables.
This document details the technical mechanics of this architecture, outlining the functions of network nodes: Edges, Gateways, Core nodes, and ODP servers. It considers the data-plane performance benefits of allocating per-prefix VPN labels at the edge and using Segment Routing Flexible Algorithms (Flex-Algo) to provide intent-based steering of traffic.
Introduction
The wide-area network is increasingly burdened by the necessity to maintain state across complex, distributed topologies. Traditional Provider Edge (PE) routers often experience control-plane exhaustion and label scarcity because they must maintain separate routing state for each tenant.
When a traditional Layer 3 VPN spans a core network, the blast radius of a routing update or anomaly scales linearly with the number of attached customers. As requirements grow to require continuous observability and telemetry, the core infrastructure inevitably becomes a bottleneck. The root problem is one of state: traditional networks require the backbone infrastructure to intimately understand attached networks.
Here we outline an alternative, decoupled architecture, invented and deployed in production at Graphiant. In this model, Customer Premises Equipment (CPE) edges execute all complex operations and overlay management. Meanwhile, the core remains entirely oblivious to customer prefixes, operating solely on OSPFv3 Segment Routing identifiers.
Background and Key Concepts
We shall assume the reader possesses a foundational understanding of MP-BGP, OSPFv3, and MPLS operations. The Graphiant architecture relies on several specific concepts:
ODP (Object Distribution Protocol): Graphiant’s control-plane protocol, built on MP-BGP, which operates in two distinct modes:
Inline ODP shares core topology to the Edge via MP-BGP (mapped internally to SAFI 242).
Off-path ODP servers act as BGP route reflectors, sharing customer reachability and routes directly between Edges (SAFI 128 for customer VPN routes; SAFI 241 for slice-loopback reachability through the core).
VPP (Vector Packet Processor): The high-performance dataplane technology used for fast packet forwarding.
Slice: An End-to-End point-to-multipoint IPsec interface dedicated to a specific enterprise to provide strict cryptographic and topological isolation.
VRF (Virtual Routing and Forwarding): A virtual routing domain used to separate and isolate different customer networks.
Hop-to-hop tunnels: a secure connection established between adjacent Graphiant network nodes, that primarily provides cryptographic authentication across the Graphiant network
Architecture and Solution
Node Roles and Functions
To achieve a decoupled, stateless architecture, the network relies on distinct device roles, each designed to perform specific control-plane or data-plane functions:
Edges (Customer Premises Equipment / CPE): Deployed at the customer site, Edges serve as the entry point into the Graphiant network. Edges maintain the customer VRF tables, execute high-performance dataplane features (Zone-Based Firewall, DPI, NAT), and handle the imposition of the MPLS label stack for source-based routing, before transmitting packets over Hop-to-Hop IPsec tunnels into the core.
Gateways: Deployed primarily in Provider Point of Presence locations attached to the Graphiant network, Gateways provide managed internet access and serve as integration points for external networks.
Core Nodes and Edge Terminators: The high-speed backbone. These nodes run OSPFv3 Segment Routing. They map the backbone topology and advertise their unique Node SIDs to the Edge devices via Inline ODP. Core nodes forward packets strictly based on outer MPLS labels and hold no customer routing state or VRF information.
ODP Servers: Centralised, off-path control plane nodes. Off-path means not in the core forwarding path for traffic. They act as BGP Route Reflectors to exchange customer CPE reachability information and cryptographic keys directly between Edge devices, thereby keeping the core free of knowledge of customer networks. While ODP servers function similarly to traditional BGP Route Reflectors, they incorporate advanced security mechanisms to guarantee strict multi-tenant isolation. By leveraging a centralised peer-node list provisioned by the controller, ODP servers maintain explicit awareness of each edge device's assigned slice role and slice membership. This topology intelligence is actively used to enforce export policies and validate inbound routing advertisements, to ensure that an edge device only advertises routes for its authorised slices. Furthermore, ODP servers utilise Route Target Constraint (RTC) to systematically limit the propagation of routes, ensuring that edge devices receive only the reachability information required for their permitted network slices.
Hop-to-hop tunnels
The Hop-to-Hop (H2H) tunnel functions primarily to cryptographically authenticate that packets received by a Core node genuinely originated from an authorised edge device, and vice versa. This mutual authentication strictly protects the Graphiant core backbone from unauthorised transit and spoofing.
Because tenant payload traffic is already fully encrypted end-to-end, applying redundant full-packet encryption across the underlay is both unnecessary and computationally inefficient. Instead, the H2H tunnel utilises a Graphiant Security Protocol (GSP) header to encrypt the first 64 bytes of the packet data. This generates the necessary cryptographic hash to securely validate the packet while maximising dataplane forwarding performance. Conversely, for control plane traffic, such as Inline ODP, the packet is fully encrypted to fully obfuscate routing state from the underlying transport network.
Core Routing with OSPFv3 Segment Routing
The Graphiant core is built on OSPFv3 with Segment Routing (SR) extensions. To minimise complexity and optimise convergence, OSPFv3 operates exclusively within Area 0 (the backbone), intentionally omitting Area Border Router (ABR) or Autonomous System Boundary Router (ASBR) functionality.
Every core node is uniquely identified by a Node Segment Identifier (Node SID) bound to a /128 IPv6 loopback interface. The architecture employs a single, static Segment Routing Global Block (SRGB) spanning from 100,000 to 199,999. This explicit design choice eliminates the need for dynamic SRGB negotiation and Label Distribution Protocol (LDP) overhead; the exact forwarding MPLS label is deterministically calculated by adding the Node SID to the SRGB base.
Because all core nodes natively support SR, the network relies entirely on Extended LSAs (RFC 8362), carrying Prefix SIDs using the OSPFv3 E-Intra-Area-Prefix-LSA type 0xA029. To optimise forwarding efficiency and enable dataplane QoS requirements, the egress node signals explicit null (IPv6 explicit null, label 2) rather than implicit null. This explicitly disables Penultimate Hop Popping: instead of the penultimate node popping the transport label, the egress receives a packet that still carries an (explicit-null) label and performs the final pop itself. Retaining the label at egress prevents sending our compressed IPv6 header directly over Ethernet and preserves the Traffic Class (CoS) bits for classification across Hop-to-Hop tunnels.
Bridging the Edge: Distributing Topology via Inline ODP
Customer Premises Equipment (CPE) must know how to steer traffic across the core without participating in the core’s OSPFv3 domain. Attached core nodes redistribute OSPFv3 information via Inline ODP and securely send it to the Edge. The two BGP speakers agree to process this using capability code 1 (MP-BGP) with AFI 16388 and SAFI 71. Internally, Graphiant maps this routing data to BGP SAFI 242 (ODP Inline/Core).
When redistributing OSPFv3 intoBGP, the Node NLRI (Type 1) uses a Protocol-ID of 6 to explicitly indicate to the Edge that the information was sourced from OSPFv3, and an Identifier of 0 to indicate the default L3 Routing Topology. Inline ODP distributes not just core reachability information for the default topology, but also for the Flex Algo topologies.
Putting it together
Let's put these concepts together to see how traffic steers from edge to edge across the core network.
Learning Core Topology via OSPFv3
Within the Graphiant core, Core Nodes and Edge Terminators use OSPFv3 Segment Routing to map the backbone topology. Every core node is assigned a unique Segment Identifier (Node SID) associated with its loopback address.Redistributing Node SIDs to the Edge
Because Edge devices do not run OSPFv3, their directly attached core node acts as a bridge. The attached core node redistributes the IGP link-state information (the SR Node SIDs) into Multiprotocol BGP. This core topology data is then sent down to the Edge device using Inlinbe ODP.Inline ODP Signalling Mechanics
The core node and the Edge establish the inlibne ODP exchange using MP-BGP capability code 1 with AFI 16388 and SAFI 71. Internally, we map this routing data to a proprietary BGP Address Family, SAFI 242 (ODP Inline/Core). Through this connection, the Edge learns two critical pieces of information:
The Node SIDs and metrics for all reachable core nodes in the Graphiant network.
The specific Node SID of its directly attached core node, which is explicitly signalled using “attachedness” identifiers within the ODP attributes (or an Opaque Node Attribute TLV).
Steering Traffic to the Receiving Edge
Inline ODP works in tandem with Off-Path ODP (which exchanges customer routes via centralised route reflectors) to achieve end-to-end delivery. When a remote receiving Edge advertises its customer networks via Off-Path ODP, it sets its directly attached core node’s SID as the “next-hop” to reach those networks.
When the sending Edge needs to transmit a packet to that remote site, it relies on the topology map provided by Inline ODP to find the correct Node SID for that destination core node. The ingress Edge calculates the correct MPLS label by adding the destination Node SID to the static Segment Routing Global Block (SRGB) base, and imposes this outer MPLS label on the packet.
Because the Edge handles this label imposition, the packet is steered seamlessly across the underlay. The intermediate core routers simply inspect the outer label and fast forward the packet directly to the receiving Edge’s attached core node, bypassing IP routing lookups and keeping the core entirely stateless.
To achieve end-to-end intent-based routing, path selection must extend beyond the core backbone to encompass the "last mile" egress links connecting the destination core node to the remote edge. The SLA parameters for these remote egress links—such as delay, jitter, and packet loss measured via continuous TWAMP probing—are communicated back to the ingress Edge via Off-Path ODP.
When the ingress edge dataplane encapsulates the flow, this specific pathing intent is embedded directly into the compressed IPv6 End-to-End (E2E) tunnel header. Upon receiving the packet, the egress core device reads this embedded information to deterministically steer the packet down the appropriate physical egress interface toward the remote Edge. This mechanism ensures that strict Application Quality of Experience (AppQoE) and traffic steering policies are enforced seamlessly end-to-end, preserving optimal path selection across the entire network without requiring the core to maintain discrete customer state.
Network Slicing and End-to-End Customer Isolation
To guarantee strict multi-tenancy, the architecture employs “Slices”—End-to-End point-to-multipoint IPsec interfaces securely connecting an enterprise’s Edge devices. Slices strictly protect enterprises from sending or receiving packets to or from unauthorised tenants, preventing cross-tenant attacks.
While Inline ODP handles core transport, customer traffic isolation within a Slice is strictly maintained via E2E IP and inner MPLS VPN labels. Unlike traditional Provider Edge routers, Edge CPEs do not face label exhaustion issues, as their VRF routing scale does not realistically approach the usable MPLS label space—the 20-bit label field provides 1,048,576 (1 << 20) values in total, of which the reserved values 0–15 are unavailable, leaving 1,048,560 usable labels.
There is a design choice & trade-off: Graphiant assigns a unique VPN label per prefix. This trades localised label space conservation for raw dataplane performance. Assigning a label per prefix entirely eliminates the need for a secondary IP lookup upon decryption at the receiving CPE. Customer routes are exchanged via Off-path ODP servers using Multiprotocol BGP SAFI 128, while reachability to the CPE’s slice loopback itself utilises SAFI 241.
Putting all the elements together, this is what a packet encapsulation looks like as it traverses the core network:
.webp)
Data Assurance and Intent-Based Steering with Flex-Algo
To provide granular control over core transport beyond the default shortest path, the architecture incorporates Flexible Algorithms (Flex-Algo). Instead of relying solely on the default Shortest Path First (SPF) tree, Flex-Algo computes a per-Flexible Algorithm SPF tree and maintains a per-FA local RIB.
Within the OSPFv3 domain, these capabilities are dynamically signalled. The network leverages the OSPFv3 Router Information (RI) LSA (function code 12) to carry SR node capabilities, SR algorithms, and Flexible Algorithm definitions (FAD). Concurrently, Application Specific Link Attributes (ASLA) are carried via Extended Router LSAs. These facilities allow custom core topologies based on IGP parameters. At the edge, BGP/ODP signalling maps customer traffic into these distinct FA topologies, programming MPLS forwarding entries directly from the RIB into the FIB to ensure traffic steering intent is preserved end-to-end.
High-Performance Dataplane
To deliver the described packet forwarding, Graphiant has a high-performance dataplane that can execute strict label imposition and extraction for line-rate performance.
Label Imposition (Outbound): When a packet enters the ingress CPE from the LAN, the dataplane resolves the route and imposes a 2-label MPLS stack. It pushes the inner MPLS VPN label (learned via SAFI 128) to identify the customer VRF, and then pushes the outer Core Node SID label (learned via SAFI 242) to steer the packet across the stateless core.
Label Extraction (Inbound): Upon arriving at the destination CPE from the multiplexed slice tunnel, the packet is encapsulated in the inner MPLS VPN label. Because multiple customer VRFs share the slice tunnel, the dataplane must execute IP features (Zone-Based Firewall, DPI, NAT) before final IP routing occurs. To achieve this, the dataplane performs a label lookup in the MPLS FIB to yield the IP VRF ID and the Load-Balance (LB) path. VPP then pops the label, writes the VRF ID and LB into the packet buffer’s metadata, and injects the packet into its IP input feature arc. This ensures that IP features can apply security policies in the correct customer VRF context, ultimately skipping the standard IP lookup by forwarding the packet directly.
Scaling and Performance Benefits of MPLS and Segment Routing
The mechanisms detailed combine to deliver scalability and forwarding performance that traditional Layer 3 VPN cores cannot match, by completely decoupling customer routing state from core transport infrastructure and shifting the computational work to the edge. The network eliminates traditional control-plane bottlenecks.
Traditional core networks suffer from state exhaustion as the blast radius of routing updates scales linearly with the number of attached tenants. Because Graphiant core nodes forward purely on a globally unique Node SID drawn from the static SRGB, they never hold customer prefixes or VRF tables. The blast radius of a routing update in the datapath, therefore, no longer scales with the number of attached tenants, and the deterministic label calculation eliminates the control-plane overhead of dynamic SRGB negotiation and LDP state maintenance.
At the edge, the per-prefix MPLS VPN label scheme maximises packet processing speeds. Whereas traditional Provider Edge routers often face label exhaustion and must aggregate labels per-VRF, an individual CPE’s routing scale does not approach the usable MPLS label space, so per-prefix allocation is comfortably accommodated. The design explicitly trades localised label space conservation for raw dataplane performance. By assigning a unique VPN label for every customer prefix, the receiving CPE can resolve both the target IP VRF ID and the forwarding from a single MPLS FIB lookup, bypassing the secondary IP lookup that conventional per-VRF aggregation requires. This guarantees that even heavily multiplexed slices can be processed at line rate without degrading performance.
Summary
The stateless core architecture fundamentally resolves the scaling constraints of traditional Layer 3 VPNs by explicitly decoupling edge intelligence from core transport. By utilising OSPFv3 Segment Routing and Inline ODP for underlay forwarding, alongside per-prefix MPLS VPN labels (Off-Path ODP) for strict tenant isolation, the core operates entirely free of customer state.
Glossary
ASLA: Application Specific Link Attributes.
CPE: Customer Premises Equipment, or Edge in Graphiant terminology
DPI: Deep Packet Inspection.
ET: Edge Terminator. A core routing node first-hop attached to a CPE.
FAD: Flexible Algorithm Definition.
Flex-Algo: Flexible Algorithm.
Gateway: A Graphiant node used for managed internet access, external service integration, and Extranet routing, providing essential NAT and firewall capabilities at the network edge.
Node SID: Node Segment Identifier.
ODP: Object Distribution Protocol.
PHP: Penultimate Hop Popping.
Slice: A secure, End-to-End point-to-multipoint IPsec interface dedicated to a specific enterprise.
SRGB: Segment Routing Global Block.
VPP: Vector Packet Processor.
VRF: Virtual Routing and Forwarding.
ZBFW: Zone-Based Firewall.