Controller placement in software-defined networks using reinforcement learning and metaheuristics

Mohammad Sadegh Sirjani; Ali Maleki; Amir Pakmehr; Maedeh Abedini Bagha; Ali Ghaffari; Ali Asghar Pour Haji Kazem

doi:10.1007/s10586-025-05331-y

Controller Placement in Software-Defined Networks Using Reinforcement Learning and Metaheuristics

Mohammad Sadegh Sirjani¹, Ali Maleki², Amir Pakmehr³, Maedeh Abedini Bagha^4,5, Ali Ghaffari^5,6,7, Ali Asghar Pour Haji Kazem⁶

¹Department of Computer Science, University of Texas at San Antonio, San Antonio, TX, USA

²Department of Computer Engineering, Shi.C., Islamic Azad University, Shiraz, Iran

³Department of Computer and Information Technology Engineering, Qa.C., Islamic Azad University, Qazvin, Iran

⁴Roshdiyeh Higher Education Institute, Tabriz, Iran

⁵Department of Computer Engineering, Ta.C., Islamic Azad University, Tabriz, Iran

⁶Department of Software Engineering, Istinye University, Istanbul, Türkiye

⁷Department of Computer Science, Khazar University, Baku, Azerbaijan

Cluster Computing, vol. 28, no. 10, art. 660 (2025)

Paper

Where Should the Controllers Go?

Software-defined networks move all control logic into controllers, so how many controllers a network has, and which switches each one manages, sets its latency, load balance, and energy use. Finding that placement is NP-hard, and prior work rarely optimizes all of these objectives together.

Abstract

Software-defined networking (SDN) has revolutionized network management by enabling dynamic control and optimization of network resources. A key challenge in SDN deployment is the strategic placement and assignment of controllers, which significantly affects network performance in terms of energy consumption, latency, and load balancing. This research addresses the controller placement problem by proposing a two-step method that combines enhanced reinforcement learning with an improved metaheuristic algorithm, termed Bedbug-GLA. In the first step, an irregular cellular learning automata model is developed to determine the optimal number of controllers required. In the second step, the Bedbug metaheuristic algorithm is employed to efficiently assign controllers to switches. Simulation results demonstrate that Bedbug-GLA achieves up to an 18% improvement in maximum controller load, a 69% reduction in congested controller overload, and a 20% decrease in energy consumption compared to state-of-the-art metaheuristic approaches, as evaluated on standard network topologies derived from real-world datasets.

Bedbug-GLA in Two Steps

Bedbug-GLA first learns how many controllers the network needs, then decides which switches each controller manages.

Step	Method	Output
Step 1	DCLA: irregular cellular learning automata with a GA-derived reinforcement signal	Number of controllers
Step 2	DBMHA: Bedbug metaheuristic with genetic operators and a chaotic initial population	Controller-to-switch assignment

Table 1. The two steps of Bedbug-GLA.

DCLA. Each cell of the cellular learning automata observes its neighborhood, selects an action, and adapts from feedback. A genetic algorithm supplies the reinforcement signal to speed up learning.
Choosing the number of controllers. DCLA sizes the controller set to the network's size, structure, traffic, and controller workloads.
DBMHA. Genetic crossover and mutation help the Bedbug algorithm escape local optima, and a chaotic map diversifies the initial population.
Multi-objective assignment. A weighted fitness function balances controller load, propagation latency, and energy usage.

Bedbug-GLA flowchart: find the optimal number of controllers with DCLA, then assign controllers with DBMHA using a chaotic initial population, GA and BMHA updates, and a global best — Figure 1. General flowchart of Bedbug-GLA: DCLA finds the number of controllers, then DBMHA assigns controllers to switches.

Real-World Topologies

Bedbug-GLA is evaluated on three topologies from the Internet Topology Zoo with 3, 4, and 5 controllers, against PSO, ALO, GEWO, and BMHA. Traffic between random origins and destinations varies from 0 to 500 packets.

Maps of the Colt Telecom network across Europe, the Internet2 OS3E network across the USA, and the IRIS network in Tennessee — Figure 2. The Internet2 OS3E, IRIS, and Colt network topologies.

Network	Extent	Location	Switches	Links
Internet2 OS3E	Country	USA	34	42
IRIS	Region	Tennessee, USA	51	64
Colt	Continent	Europe	153	177

Table 2. The details of the evaluated SDN topologies.

Parameter	Value
Initial population	30
Run time	30 iterations
Traffic	Random (0, 500)
e_base	0.1
e_dyn	0.001
e_trans	1
l_i	1
α, β (DCLA)	0.01
ε̂ (BMHA)	0.185
MinVar, MaxVar (BMHA)	0, 1
Search agents (ALO)	40
r1, r2 (PSO, GEWO)	Random in [0, 1]
c1', c2' (PSO)	1.7, 2
Inertia weight (PSO)	0.75

Table 3. Simulation parameters and constants.

Results

On Internet2 OS3E, averaged over 100 runs and 3–5 controllers, Bedbug-GLA has the lowest energy usage, maximum controller load, and congested-controller overload.

Bar chart of energy usage in the Internet2 OS3E topology for PSO, ALO, GEWO, BMHA, and Bedbug-GLA with 3, 4, and 5 controllers — Figure 3. Energy usage in the Internet2 OS3E topology with 3, 4, and 5 controllers.

Method	Energy (J)	Max controller load	Congested overload
PSO	2356.74	1493.33	393.33
ALO	2297.64	1357.00	257.00
GEWO	1957.51	1273.33	143.33
BMHA	1977.16	1266.67	166.67
Bedbug-GLA	1888.86	1221.00	121.00

Table 4. Averages over 3, 4, and 5 controllers in Internet2 OS3E. Lower is better.

vs.	Energy	Max controller load	Congested overload
PSO	19.85%	18.24%	69.24%
ALO	17.79%	10.02%	52.92%
GEWO	3.51%	4.11%	15.58%
BMHA	4.47%	3.61%	27.40%

Table 5. Bedbug-GLA improvement over each baseline.

Takeaways

Splitting controller placement into sizing (DCLA) and assignment (DBMHA) lets each step use the method that suits it.
Genetic operators and chaotic initialization keep the Bedbug search diverse and help it avoid premature convergence.
On Internet2 OS3E, Bedbug-GLA improves maximum controller load by 4%–18%, congested-controller overload by 16%–69%, and energy usage by 4%–20% over PSO, ALO, GEWO, and BMHA.
ALO reaches the lowest end-to-end delay; Bedbug-GLA stays comparable on latency while using much less energy, trading a little delay for a balanced multi-objective result.

BibTeX

@article{Sirjani2025,
  author={Sirjani, Mohammad Sadegh and Maleki, Ali and Pakmehr, Amir and Abedini Bagha, Maedeh and Ghaffari, Ali and Pour Haji Kazem, Ali Asghar},
  title={Controller placement in software-defined networks using reinforcement learning and metaheuristics},
  journal={Cluster Computing},
  year={2025},
  month={9},
  day={3},
  volume={28},
  number={10},
  pages={660},
  abstract={Software-defined networking (SDN) has revolutionized network management by enabling dynamic control and optimization of network resources. A key challenge in SDN deployment is the strategic placement and assignment of controllers, which significantly affects network performance in terms of energy consumption, latency, and load balancing. This research addresses the controller placement problem by proposing a two-step method that combines enhanced reinforcement learning with an improved metaheuristic algorithm, termed Bedbug-GLA. In the first step, an irregular cellular learning automata model is developed to determine the optimal number of controllers required. In the second step, the Bedbug metaheuristic algorithm is employed to efficiently assign controllers to switches. Simulation results demonstrate that Bedbug-GLA achieves up to an 18% improvement in maximum controller load, a 69% reduction in congested controller overload, and a 20% decrease in energy consumption compared to state-of-the-art metaheuristic approaches, as evaluated on standard network topologies derived from real-world datasets.},
  issn={1573-7543},
  doi={10.1007/s10586-025-05331-y},
  url={https://doi.org/10.1007/s10586-025-05331-y}
}