SONiC-DASH: VNET-to-VNET and More Networking Use Cases

In this blog post series, we will explore the emerging SONiC-DASH networking technology, including its use cases, how to integrate it with a DPU, and the specifics of DASH testing. Part 1 of our series describes DASH’s key use cases, with a closer look at VNET-to-VNET.

The need for a networking operating system for SmartNIC and DPU

With the growing amount of data being transferred, programmable hardware like a SmartNIC (Network Interface Card) allows for more efficient hardware use in compute and storage infrastructure, helping with offloading and the acceleration of the networking function. Designed for a limited number of specific deployments, a SmartNIC needs more supporting interfaces and functionality to target a larger market. Another challenge is ensuring compatibility through its integration with various controllers and orchestrators. 

In response, a new type of programmable hardware has now emerged, known as a DPU (Data Processing Unit) or an xPU (branded products by vendors). For a DPU with storage and HPC functionality, there are additional requirements like traffic encryption, data compression or tunneling. This drives a need for improved performance, along with the flexibility that only an abstraction level like Switch Abstraction Interface (SAI) can provide. This functional complexity creates non-trivial challenges for DPU management, which means that more dynamic software is necessary to sustain all these functions. This is where SONiC-DASH comes into play.

What DASH is and how it relates to SONiC

SONiC-DASH (Disaggregated APIs for SONiC Hosts) is a new open-source project initiated by Microsoft and spun off by the SONiC open-source community. DASH relies on a set of API and object models that describe network services for the cloud, offering greatly improved performance of any application running in the enterprise. Following an SDN approach to implementing the data plane, DASH helps accelerate flow processing. Fundamentally DASH provides an extension of the well-known SAI (Switch Abstraction Interface) which is intended for DPU technologies.  

DPU devices enabled with DASH can be smoothly and easily connected into SONiC-based data center networking infrastructure. As a SONiC subproject, DASH inherits its architecture and key components, enabling interoperability across different programmable hardware – APIs are standardized through SAI and a control plane through SONiC. As open software, it is flexible and scalable, allowing for its customization for various use cases. 

PLVision’s significant experience with SAI, SONiC open-source software, and other NOS enablement for various ASICs provided a jump-start for early DASH exploration and development. We will start this blog post series by reviewing DASH use cases.

DASH use cases

The spectrum of end-user needs for data center networking infrastructure is constantly growing, from simple load balancing of service endpoints through ensuring services via high availability, all the way to merging private and public infrastructure for running services and applications in a distributed manner. Such requirements place a large burden on the implementation of virtual network infrastructure, both in terms of design complexity and computational power required. This results in the need for more sophisticated switches and routers and utilizing server CPUs for building and operating virtual connectivity fabric. [1] 

DPU looks to be the key to solving this complexity by offloading networking functionality to semi-specialized ASICs and concentrating these resources at certain points of DC for shared usage. 

DASH optimizes cloud and enterprise services by leveraging DPU devices. The overall goal of DASH is to achieve up to 100x the performance of stateful connections with hardware offload using programmable technologies. Here is the list of initial use cases as defined by the Community [2]: 

• VNET-to-VNET – communication between two VMs deployed in different VNETs within the same Azure region includes the core DASH features such as routing, stateful ACL, and TCP state tracking, and the verification of performance properties like connections per seconds (CPS), flows, packets per seconds (PPS) and rule scaling. 
• VNET Peering – establishing connectivity (routing) between two or more VNETs within the same Azure region or across multiple ones. 
• High Availability (HA) – enabling redundancy at the DPU-based appliance level to keep virtual network infrastructure running during a fail-over. 
• Load Balancer – accelerating the connection between service endpoints hidden behind balanced VIPs by organizing a direct “Fast Path” channel. 
• Service Tunnel & Private Link – attachment of private service endpoints to a VNET (secure connection to a data center resource from/to an external network). 
• Encryption Gateway – encryption services for network traffic implementing security for VPN connections. 
• Express Route Gateway – optimizing the attachment of a private network to public data center infrastructure.

To run these use cases on a DPU, DASH places requirements on hardware datapath capabilities and defines two generic pipelines to be implemented: 

• Inbound – processes traffic entering a VNET. The DPU recognizes the direction as RX based on the incoming packet’s VNI, if it is not matched against any reserved VNI.

Figure 1. The inbound packet processing pipeline

• Outbound – deals with flows that are leaving a VNET. The TX direction is enabled based on the outgoing packet’s VNI that is matched against the reserved VNI. 

Figure 2. The outbound packet processing pipeline

Both pipelines have a great deal in common and can be easily implemented on modern programmable ASICs. The major functional components required for DASH to operate are: 

• VxLAN headers lookups 
• LPM (Longest Prefix Match) routing 
• IPv4 and IPv6 route entries 
• Stateful ACL (Access Control List) 
• TCP state tracking on flows 
• Telemetry and Monitoring

Diving into the VNET-to-VNET use case

The first use case designed by the DASH Community is VNET-to-VNET. It is rather synthetic, with a major focus on creating a simple scenario that leverages most datapath components and proves the overall design. This scenario helps verify a platform’s readiness for DASH implementation and reflects the general idea of DASH as an innovative approach for using DPU/SmartNIC for the cloud. We have already implemented this use case for one of our clients, so let’s talk about it in more detail. 

The goal of the VNET-to-VNET use case is to establish connectivity between two VMs running in different virtual networks, or in other words, in different L3 overlay subnets. The central part of the case is the SDN Appliance which is in fact a standalone server with a bunch of DPU cards installed in it and connected via PCI bus. Traffic to/from virtual machines running on other servers goes through the front panel network interfaces of DPU cards. Internally, the DPU works its magic and helps packets originated in one VNET reach their destination in the other VNET. To make this happen, a DPU should be capable of VxLAN tunneling lookups, as well as routing and applying ACLs. DPU datapath components are configured using DASH SAI APIs, which are an extension of SAI APIs used in the SONiC network operating system. 

Figure 3. VM to VM communication via DASH appliance

An SDN controller configures specific datapath entries on a DASH appliance and applies network policies: routes, ACL, NAT, etc. DASH APIs are called in this process to configure the DPU and push configuration entries to hardware. As a result, tunnels are established between the source VM and the appliance, and between the appliance and the destination VM. Internally, the DPU applies packet rules, performs routing, and transforms the packets. 

A final note 

Aimed at enabling programming hardware to achieve limitless networking, DASH benefits from SONiC’s architecture and key components and follows an SDN approach to implementing the data plane. It is a scalable, flexible open software with a high degree of interoperability, which is applicable for a number of use cases, ranging from basic DPU (aka NIC on a host) and SmartSwitch to high-performance network appliances. We’ve taken a closer look at the VNET-to-VNET scenario, as this is a starting point for leveraging all core DASH mechanisms and serves as a foundation for the implementation of other listed use cases.

DASH is part and parcel of PLVision’s experience in the development and integration of innovative software for networking devices. Our engineers are already involved with DASH and take part in the DASH community’s work. Among other things, we have extended the functionality of SAI Challenger, an open product for SAI testing and prototyping, initially developed by PLVision, by enabling it for DASH testing [3]. 

In the next post of our DASH series, we will explore the steps required for integrating DASH for a DPU, based on our extensive experience developing open NOS and DASH-enabled products in particular.