Optimal VNF Placement via Deep Reinforcement Learning in SDN/NFV-Enabled Networks

TLDR

Software‑Defined Networking and Network Function Virtualization enable scalable, cost‑effective deployment of virtual network functions, and their centralized control allows collection of rich device, traffic, and resource data that machine‑learning algorithms can exploit to optimize network performance. This work formulates the VNF placement problem in SDN/NFV‑enabled networks as a binary integer programming problem and seeks an efficient solution. We introduce DDQN‑VNFPA, a Double Deep Q‑Network based algorithm that explores the vast placement space, selects optimal VNF instances via a threshold‑based policy, and is evaluated through trace‑driven simulations on a real‑world topology. Simulation results demonstrate that DDQN‑VNFPA reduces Service Function Chain request rejection rates, improves throughput and end‑to‑end delay, shortens VNF instance runtime, and balances load more effectively than existing algorithms.

Abstract

The emerging paradigm - Software-Defined Networking (SDN) and Network Function Virtualization (NFV) - makes it feasible and scalable to run Virtual Network Functions (VNFs) in commercial-off-the-shelf devices, which provides a variety of network services with reduced cost. Benefitting from centralized network management, lots of information about network devices, traffic and resources can be collected in SDN/NFV-enabled networks. Using powerful machine learning tools, algorithms can be designed in a customized way according to the collected information to efficiently optimize network performance. In this paper, we study the VNF placement problem in SDN/NFV-enabled networks, which is naturally formulated as a Binary Integer Programming (BIP) problem. Using deep reinforcement learning, we propose a Double Deep Q Network-based VNF Placement Algorithm (DDQN-VNFPA). Specifically, DDQN determines the optimal solution from a prohibitively large solution space and DDQN-VNFPA then places/releases VNF Instances (VNFIs) following a threshold-based policy. We evaluate DDQN-VNFPA with trace-driven simulations on a real-world network topology. Evaluation results show that DDQN-VNFPA can get improved network performance in terms of the reject number and reject ratio of Service Function Chain Requests (SFCRs), throughput, end-to-end delay, VNFI running time and load balancing compared with the algorithms in existing literatures.