QTE-IoT: Q-Learning-Based Task Scheduling Scheme to Enhance Energy Consumption and QoS in IoT Environments

Scheduling Across Fog and Cloud

IoT devices generate tasks with very different needs. Fog nodes sit close to devices for low latency, while public and private clouds offer scale and security. Deciding where each task runs, and when, drives energy use and quality of service.

Abstract

As the proliferation of Internet of Things (IoT) devices continues unabated, the demand for efficient task scheduling mechanisms becomes increasingly critical. Task scheduling in the IoT is pivotal for optimizing resource utilization, minimizing latency, and enhancing the overall system's performance. This research proposes a novel method called QTE-IoT, standing for a Q-learning-based task scheduling scheme to enhance energy consumption and QoS in IoT environments. QTE-IoT commences by categorizing tasks into three classes: time-sensitive tasks, security tasks, and normal tasks. This classification is achieved using a multi-layer perceptron artificial neural network. Subsequently, time-sensitive tasks are offloaded to the fog layer and scheduled using the proposed African Vulture Algorithm combined with Q-learning, which we designate as QAVA. Security tasks are offloaded to the private cloud, while normal tasks are offloaded to the public cloud. For task scheduling in private and public cloud environments, QTE-IoT employs a proposed enhanced version of Artificial Rabbits Optimization integrated with the Q-learning algorithm, known as QARO. Additionally, the QTE-IoT method incorporates a monitoring agent to oversee resource workload, thereby preventing congestion and delays. Simulation results on instances of the HCSP benchmark dataset demonstrate that QTE-IoT outperforms other state-of-the-art methods in various performance metrics. QTE-IoT achieves significant improvements compared to other methods and algorithms, including a 6% to 12% reduction in energy consumption. Furthermore, QTE-IoT exhibits substantial improvements in load imbalance (42% to 79%), response time (25% to 40%), and deadline satisfaction (6% to 39%) compared to existing approaches.

QTE-IoT Pipeline

Tasks are classified first, then routed to the layer and scheduler that suit them, while a monitoring agent watches resource load.

• MLP-ANN classifier. Sorts incoming tasks into time-sensitive, security, and normal classes from 18 task features.
• QAVA. The African Vulture Algorithm combined with Q-learning schedules time-sensitive tasks on fog nodes.
• QARO. An enhanced Artificial Rabbits Optimization with Q-learning schedules tasks on private and public cloud VMs.
• Monitoring agent. Tracks resource load and, above a threshold, re-schedules cloud tasks or uses a greedy algorithm to offload fog tasks to another node.

Task Classification

An MLP-ANN with 18 input neurons, one per task attribute, sorts tasks into the three classes. It trains on 70% of the labeled tasks and is tested on the remaining 30%.

Q-Learning Hybrids

Each hybrid is benchmarked against Q-learning alone and its metaheuristic alone on 400 tasks over 50 independent runs. QARO uses 25 private and 25 public cloud resources; QAVA uses 50 fog resources.

Simulation Setup

Experiments use HCSP benchmark instances enriched to 18 features per task: 1000 labeled tasks (50% time-sensitive, 40% normal, 10% security), 200–600 tasks per run, 2000–6000 MIPS resources, and 10 repetitions per experiment in MATLAB R2019b.

Results

QTE-IoT is compared with ARO, AVA, MGWO, OSAPSO, WCLA+GA, and ETFC across 200–600 tasks and 30–50 resources. It posts the best average on every metric.

Takeaways

• Classifying tasks first lets each class run where it fits: time-sensitive tasks in the fog, security tasks in the private cloud, and normal tasks in the public cloud.
• Pairing Q-learning with metaheuristics speeds convergence: QARO and QAVA beat their standalone counterparts on makespan, energy, deadline violations, and load imbalance.
• The monitoring agent keeps load balanced by re-scheduling or offloading tasks before resources congest.
• On HCSP benchmark instances, QTE-IoT cuts energy by 6%–12% and response time by 25%–40%, and reaches over 99% average deadline satisfaction.

BibTeX

@article{GHAFFARI2025101247,
  title = {QTE-IoT: Q-Learning-Based Task Scheduling Scheme to Enhance Energy Consumption and QoS in IoT Environments},
  journal = {Sustainable Computing: Informatics and Systems},
  pages = {101247},
  year = {2025},
  issn = {2210-5379},
  doi = {https://doi.org/10.1016/j.suscom.2025.101247},
  url = {https://www.sciencedirect.com/science/article/pii/S2210537925001684},
  author = {Ali Ghaffari and Vesal Firoozi and Ali Maleki and Mohammad Sadegh Sirjani and Maedeh Abedini Bagha},
  keywords = {Internet of things, Metaheuristic, Reinforcement learning, Task scheduling},
  abstract = {As the proliferation of Internet of Things (IoT) devices continues unabated, the demand for efficient task scheduling mechanisms becomes increasingly critical. Task scheduling in the IoT is pivotal for optimizing resource utilization, minimizing latency, and enhancing the overall system’s performance. This research proposes a novel method called QTE-IoT, standing for a Q-learning-based task scheduling scheme to enhance energy consumption and QoS in IoT environments. QTE-IoT commences by categorizing tasks into three classes: time-sensitive tasks, security tasks, and normal tasks. This classification is achieved using a multi-layer perceptron artificial neural network. Subsequently, time-sensitive tasks are offloaded to the fog layer and scheduled using the proposed African Vulture Algorithm combined with Q-learning, which we designate as QAVA. Security tasks are offloaded to the private cloud, while normal tasks are offloaded to the public cloud. For task scheduling in private and public cloud environments, QTE-IoT employs a proposed enhanced version of Artificial Rabbits Optimization integrated with the Q-learning algorithm, known as QARO. Additionally, the QTE-IoT method incorporates a monitoring agent to oversee resource workload, thereby preventing congestion and delays. Simulation results on instances of the HCSP benchmark dataset demonstrate that QTE-IoT outperforms other state-of-the-art methods in various performance metrics. QTE-IoT achieves significant improvements compared to other methods and algorithms, including a 6% to 12% reduction in energy consumption. Furthermore, QTE-IoT exhibits substantial improvements in load imbalance (42% to 79%), response time (25% to 40%), and deadline satisfaction (6% to 39%) compared to existing approaches.}
}