Q-Learning-Based Task Scheduling for Low-Latency Edge Offloading in MEC Systems

Mobile devices increasingly handle computationally heavy activities including data processing, artificial intelligence, and virtual reality. Even with their advancements, mobile devices might not be able to do all of their duties locally and quickly enough because of their constrained computing capacity [1]. Mobile edge computing (MEC), which offers robust processing and storage capabilities closer to the end users, has emerged as a viable strategy in this area. In particular, MEC enables compute offloading, which has been shown to increase energy efficiency and application response time [2]. In order to shorten the time, it takes to execute a job, mobile devices can offload computationally demanding tasks to adjacent edge nodes with MEC. Additionally, for those activities that are time-sensitive, it can lower the percentage of dropped tasks [3]. There have been several recent studies on compute offloading problems in MEC networks with various goals, such as latency and energy efficiency. Nevertheless, in order to lower energy usage or task execution delay, these investigations look into compute offloading optimization strategies [4]. Both the horizontal compute task collaboration ignores the relationships between mobile edge nodes and the vertical collaboration between mobile cloud server nodes, mobile edge server nodes, and mobile devices [5].

Since the aforementioned offloading issue is NP-hard, several previous research has devised heuristics or approximation techniques. They do, however, mostly rely on precise analytical models or professional knowledge [6].

This work is open access and licensed under the Creative Commons CC BY 4.0 License.

Because of this, it takes a lot of human labour and experience to adjust these heuristics or analytical models to fit new situations. This process can be time-consuming and perhaps impractical given the complexity of MEC's system architecture and applications [7]. However, because of the distributed architecture, deciding where to offload is a major design difficulty. Thus, the primary challenge when creating a task scheduling serves as a tool for users in a cooperative MEC network to offload [8]. The experience of compute offloading can be significantly impacted by human mobility in addition to issues with job scheduling and resource management [9]. The offloading user's frequent location changes cause fluctuations in the practical distribution of computational and communication resources over time [10]. In particular, time-varying channel gains deteriorate the quality of service (QoS) between the mobile user and the MEC server; hence, maintaining the task offloading latency becomes crucial [11].

Machine learning (ML) is now seen to be a useful method for resolving a wide range of categorization issues. The authors have studied the machine learning approach to computation offloading on time-varying computing systems [12]. Although computation offloading to EC can improve battery life, response speed, and data security, it may be influenced by mobile users' movements and dynamic changes [13]. Data migration, reduced performance, increased latency, and extended processing times can all be caused by relocating operations between cloudlets [14]. Addressing the challenges of task scheduling, DL models predict resource needs and optimize allocation to mitigate delays impacting user experience [15]. Varying network connections and node-to-node data transfer rates, combined with diverse DNN-based activity demands, necessitate sophisticated resource management. Predictive modeling helps balance these variables for efficient operation [16]. Conventional compute offloading strategies transfer all work to distant clouds or mobile edges, but they cause a significant delay in data transmission. Thus, creating a schedule that works is essential, especially when there are several things being completed at once [17]. Generating and labelling data from the MEC network is difficult since standard supervised machine learning techniques require substantial labelled datasets for model training. A more effective technique is Reinforcement Learning (RL), which enables agents to learn optimum policies by interaction with the MEC network environment and does not require labelled data for training. Remission rate and content delivery optimization, offloading control, and micro services coordination are all done using RL. The use of classical reinforcement learning is limited in situations where the state or action space is extensive, as it stores learning outcomes in a Q-table. DRL, or deep reinforcement learning, can help with these problems [18].

Optimization techniques based on deep reinforcement learning (DRL) have drawn interest because they are more effective at scheduling strategies than conventional optimization algorithms [19]. But in intricate EC contexts, these algorithms encounter two significant difficulties. Initially, it is unable to effectively manage the varying fluctuations in the number of nodes and activities [20]. It need to properly reflect the dynamic environment and adjust to these changes in order to increase scalability. Second, actual EC scenarios with a high number of diverse nodes and jobs are not possible for the limited state sets and action sets that current DRL methods assume [21]. This might result in dimensional disasters since it necessitates processing massive volumes of data and direct information storage. As a result, efficiently storing all environmental data, conserving computer power, and making quick judgements present a significant obstacle. Hence a novel framework is proposed to schedule and offload a task in mobile edge computing.

The Main Contribution of the work is emphasized as

• ■    Propose a novel Q-Optimize OffloadPro Framework (QOOPF) for task scheduling and offloading in MEC environments, reducing latency and task failures caused by user mobility, limited network resources, and high VM utilization.

• ■    Introduce a OffloadPro Scheduling Method (OPSM) scheduling strategy using a critical path method for task prioritization and offloading, ensuring efficient task execution by assigning priority-based tasks to optimal computing entities, balancing VM utilization, and reducing communication overhead.

• ■    The Deep-Q-Driven Policy-Value Optimization (PVO) component employs a policy-value optimizer trained on large task datasets to adapt task offloading decisions in dynamic MEC environments. By adjusting task assignments across VMs, the optimizer supports improved resource allocation, reduces the likelihood of task failures, and helps to lower latency, thereby contributing to more consistent system performance.

2.    Related Works

The rest of the article is structured as follows: Section 2 examines relevant research from previous studies, Section 3 details the proposed method, Section 4 presents the simulation results and discussion, and section 5 concludes the article.

The necessity for efficient task scheduling and offloading solutions to deal with latency and efficiency issues has increased recently as a result of the explosive rise of edge computing. Real-time user demands, resource limitations, and changeable network circumstances are frequently excessive for traditional scheduling techniques to manage. The development of latency-aware intelligent task scheduling strategies that employ reinforcement learning, heuristic algorithms, and machine learning to reduce reaction times and improve resource usage has been the main focus of research. As a result of their versatility and capacity to learn the best task allocation strategies over time, Q-learning and other reinforcement-based techniques have become increasingly popular.

Robles-Enciso et.al [22] suggested using an innovative expansion of many layers of reinforcement learning (RL) techniques, known as ML-RL to address the edge layer Task Assignment Problem (TAP). To enhance performance in intricate and unpredictable scenarios, edge agents would be able to query a more knowledgeable upper-level agent. First, develop the task assignment process while considering the compromise between consumption of energy and execution time. There are still obstacles to overcome before neural networks may be used to improve Q-Learning, such as discrete variables and a low number of states.

Zheng et.al [23] suggested a workload scheduling method based on DRL with the aim of balancing the workload, decreasing the service time, and decreasing the failure rate of tasks. In the meanwhile, employ Deep-Q-Network (DQN) methods to deal with the workload scheduling problem's high dimension and complexity. Task failures can result from user mobility, limited network resources, and high virtual machine utilization. High VM utilization causes extended task wait times, while user mobility delays response times. Concurrent resource usage can deplete network resources, leading to task failures and packet loss.

Sellami et.al [24] examined a low-latency, energy-conscious computing job scheduling issue in an Internet of Things fog network that is defined by software. The issue with online scheduling and work assignments should first be formulated as a first step in a Deep Q-Learning under energy constraints. The one that follows, which is dependent on applications, aimed to minimize network delay while preserving battery life and energy efficiency. Next, in the Software-Defined Networking (SDN) enabled edge networks, offered a DRL technique for dynamic task scheduling and assignment given the task arrival process. It carried out extensive testing and contrasted the newly presented method with three of the most innovative deep learning algorithms (deterministic, random, and A3C agents). The A3C approach, deterministic greedy policy, and randomized exploration encounter difficulties with complicated tasks such as partially observable environments, delayed exploration, and lengthy intervals between actions and reward signals.

Jin et.al [25] presented a paradigm for fractional RL and demonstrated its convergence. Furthermore, created a model-free fractional DRL method in which every device uses the hybrid action space to schedule decisions without being aware of the system dynamics or the choices made by other devices. Its inability to use recurrent neural networks for multi-agent reinforcement learning, however, restricts its capacity to manage non-stationarity and accomplish socially optimum scheduling.

Zeng et.al [26] suggested a method for scheduling edge tasks that is based on an enhanced Double DQN, which is used to divide target Q value computations and action selection into two networks. The agent's experience replay unit gains a control unit in addition to a new reward function. To maximized experience data's value and boost learning effectiveness, the administration of experience data has also been altered. It is inefficient for reinforcement learning agents to typically learn from an ignorant state. In light of this, this work suggested a new particle swarm optimization technique that has a better fitness function and can produce ideal job scheduling solutions. The agent can pre-train network parameters with these optimized methods to achieve a higher degree of cognition. There is not enough research on process scheduling, which reduces the strategy's efficacy.

In summary, most existing methods either suffer from high computational overhead, converge only to local optima, or fail to handle dynamic MEC environments with user mobility and resource fluctuations. While reinforcement learning approaches such as DQN and Double DQN have shown promise, they struggle with task failures, high VM utilization, and scalability in large task scenarios. To overcome these challenges, we introduce the Q-Optimize OffloadPro Framework (QOOPF), which integrates the OffloadPro Scheduling Method (OPSM) for critical-path based priority assignment with a Deep-Q-Driven Policy-Value Optimizer (PVO). Unlike prior methods, our approach explicitly addresses task failure mitigation, dynamic adaptability, and efficient resource utilization in MEC environments.

3.    Proposed Methodology

Mobile Edge Computing (MEC) enables low-latency task processing by offloading computation from users to nearby edge servers. However, traditional task scheduling suffers from delays, VM overutilization, and mobility-induced failures. To address these issues, introduces the Q-Optimize OffloadPro Framework (QOOPF), a novel scheduling and offloading strategy designed for MEC environments (Figure 1). Efficient task scheduling is therefore essential to balance workload, minimize latency, and reduce task failures caused by limited resources and user mobility. The OffloadPro Scheduling Method (OPSM) selects offloading targets and assigns task priorities using a selection strategy based on the critical path method (CPM). The CPM identifies the longest dependent task sequence, ensuring minimum completion time. Tasks are then executed according to their assigned priorities, with high-priority tasks allocated resources first, thereby reducing communication overhead and balancing VM utilization. This holistic approach helps in managing high Virtual Machine (VM) utilization by balancing the load across different VMs and network resources. To adapt to dynamic environments, the scheduling model is trained on diverse task sets, with null tasks used when the actual task count is smaller than the training state space. A Deep-Q-Driven Policy-Value Optimizer (PVO) with actor–critical components are employed to refine both the policy and value functions, ensuring stable and adaptive task offloading decisions as conditions transformation. Overall, the QOOPF ensures task prioritization, efficient resource allocation, and optimized offloading decisions. This integrated approach minimizes latency, balances workloads, and enhances reliability in MEC systems.

EXECUTION LAYER

INPUT LAYER

*

Fig. 1. Block diagram of the proposed QOOPF Framework.

Figure 1. Block diagram of the proposed QOOPF framework for intelligent task scheduling and offloading in MEC. Mobile devices in the input layer generate computation tasks, which are prioritized by the OffloadPro Scheduling Method (OPSM) and optimized by the Deep-Q Policy-Value Optimizer in the middle layer. Tasks are then executed at either the Edge Server (MEC) or the cloud, depending on system conditions. Solid arrows denote task offloading flows, while dashed arrows represent scheduling and decision signals.

• 3.1.    System Model and Formulation of the Problem

  • 3.3.1.    Network Model

In MEC, imagine a network situation with one cloud server, a few edge servers, and a few mobile users. The numbers and denote the number of mobile users and edge servers in the network.

N = {1,2.....N})

M = {1,2,..M}J

The MEC servers' insufficient computational resources are thought to prevent them from completing all of the offloaded activities. Therefore, the primary emphasis of this study is the task scheduling problem of how the mobile user chooses the edge server for job offloading. Local computing is not considered because user frequently have significant computational demands that prevent them from being run locally.

A job T , is assigned to each mobile user i £ Nt This task is described as (1Тс С C , ), where IT_t is the quantity of input data needed for the task and C C , is the number of CPU cycles needed to complete the work. Every job T , can be offloaded to the cloud server or MEC servers, or it can be carried out locally by itself. The job is sent to the target edge server by the mobile user if it is to be completed on the MEC servers. Once the operation is completed on the edge servers, the user will receive the outcome. Consequently, it is presumed that every mobile user in this model has a task to be offloaded, represented as .

U T_t = ( C Co W Wi, D T™ ^ax)

The data amount of its offloaded work is indicated by DW_t (in bits), while the maximum permitted delay of its task is indicated by jy^max (in seconds). It should be noted that the job will be transferred to a cloud server that offers cooperative offloading of computations if the edge servers' processing capabilities are inadequate or if they are unable to fulfill the transmission power or energy consumption constraints. Indicate the offloading decision by using _, ∈ {0,1}. In particular, if , =1, then user I 's task T[ is offloaded to MEC server j for execution; if otherwise, , =0 indicate that the cooperative mode selection is , ∈ {0,1}. Using the cooperative MEC server j , let , =1 indicate if user i 's work T[ has been offloaded to the cloud server.

• 3.3.2.    Communication Model

Compute offloading allows tasks to be sent to either the cloud server or the MEC servers. The following is the communication model. Using the Shannon Theory, offloading to the MEC server is represented as Equation (3) by denoting the transmission rate between mobile user i and the MEC server j is represented by _,

, =    log . (1+" ⁽s^{)    ,} '- )

,               I        NPSqBW /

In this equation, _, represents the power used to transmit the job UT_t from the mobile user i to the MEC server j , BW represents the channel bandwidth, Pc represents the path loss constant, в represents the path loss exponent, &q represents the reference distance, d[ represents the distance between the mobile device i and the MEC server j , and NPS₀ represents the spectrum density of noise power. Transferring data to the cloud server involves first transmitting the work to the MEC server and then to the cloud server. Indicate _, as the MEC server j 's transmission rate when sending the mobile user i 's task T[ to the cloud server is given as Equation (4).

_p = ( d _d ■)Tq , , ,                                            (4)

TR 1, ’ =BWlog2  1+ NPS ВԜ where , is the transmission power used to send the job from the MEC server j to the cloud server. The distance between the cloud server and the MEC server j is represented by T[ , dj .

• 3.3.3.    Computing model

The work can be done locally, on cloud servers using compute offloading, or on MEC servers. The following is the computational model. Locally executed: when the work is carried out locally. The energy use and execution time are stated as equation (5) and (6)

ET} =                                                   (5)

, Local

Energy^ =     ( Ft , Local )                                           (6)

where В is the chip architecture-related coefficient and , is the local CPU frequency allotted for T[ . Once the job has been transmitted to the servers of the MEC for processing, it is said to have been executed on those servers. The energy consumption _, and transmission time _, are represented by (5) and (6)

TX i,j

ITi

Tr i,j

EC i,j =Tpi,j TX i,j

The MEC server's task computing energy consumption Energy^ ^, and task processing time ТРИМБС are represented in equation (9) and (10).

CC i                                                   (9)

TPT МЕС,C   _____ 1

,            Fi,j,MEC

Energy ■ , j  ,c=µCCi (F i, j ,MEC )2

pPT MEC =      + rjiprjiME C ,

_,    =    , + Energy^⁶ ,

Cloud server execution: When using edge server computation offloading to send the work to the cloud server for processing. Transmission energy usage      , and task transmission time , are represented by equation (13) and (14).

, =

, 1     TRi , j

TECt,j=    ,jTTXTi,j(14)

The cloud server's job processing time      ,    ^, is determined by equation (15)

rri prj-i cloud,c _(15)

,i=

1 Cloud

This study uses Edoud as a constant to represent the CPU frequency allocated to the jobs when they are offloaded to the cloud server. The following is the task's total processing time TPT^cl°^ud on the cloud server is given as equation (16)

3.3.4. Problem Formulation

-'Cloud i, i

=     + TTXTf f + ppp cloud ^,c

Examine the combined issues of resource allocation, job scheduling, and offloading decision-making. The MEC servers' and mobile users' CPU frequency assignment and transmission power allocation are included in the resource allocation. The goal is to reduce the overall work completion time. The objective (Equation. 17) is to minimize total task completion time subject to constraints on offloading decisions (Equation. 18), energy consumption (Equation. 19– 20), CPU frequency limits (Equation. 21–22), and transmission power bounds (Equation. 23–24). The decision variables are binary offloading indicators and continuous resource allocations, as summarized in Equation. 25.

/      Л V _ V min T=∑*ETlt (1-∑(At ,j+ Bt ,J))+∑At, jTPTi , j +∑Bi , jTPTi ,j   + i=l [     \    7=1              /   7 = 1                 7=1               ]

Subject to

M

∑( Ai , i + Bi , i )<1 ∀ i ∈ ℕ

7 = 1

(

1-∑(Л , , ⁺ Bi , , ))+∑( Ai , , ⁺ Bi , , ) EC_t , , ^≤

7 = 1               /    7 = 1

Ei , Local ∀ i ∈ ℕ

N                   M

∑( At , j Energy- , i , )+∑( Bi , jTECt , , )≤ ^Ei , MEC ∀ i ∈ м

1=1                       i=l

Fl ,L₀ cal^-^S^A  +^! J^^oc  ^{V ieM}

N

^(A ^F^ ,_MEC )< F^_c VjeM i=l

^( ^A u ^TP ij +В^р^ТТ^

7=1

N

^B_QTq^ < PP^c V e M 1=1

A tJ e {0,1}, B^ e {0,1}, P_{Pi J} > 0,Tq и > 0,F , ,Local >  0,F jmec >  0

where the optimization variables are % = {A ij, B ,, j, Tp t, j, Tq ,, j, Fi,lo cai F t,j,_MEC }. For mobile user i and MEC server j, denote E_{iiL0ca l} and Ej,_MEc as the value of optimizing energy consumption. The maximal CPU frequency for mobile user and MEC server is denoted by _, and _, . The maximizing transmission power is denoted by _, and ггрМах

, M E C ^.

4.    Q-Optimize Offloadpro Framework (Qoopf)

This section presents the proposed QOOPF, which includes the policy optimizer-based dynamic task offloading and priority assignment approach as well as the offloading target selection technique.

• 4.1.    OffloadPro Scheduling Method (OPSM)

• 4.2.    Deep-Q-Driven Policy-Value Optimizer

First, the edge server continuously checks for the presence and connection of mobile devices within its communication range, as the number of these devices varies dynamically. This guarantees that the mobile devices that may be able to transfer tasks to the edge server are always in view. Furthermore, service requests with variable quantities of related tasks are continually generated by mobile devices. The edge server collects the data volumes < C,S >  and processing requirements of these operations. The edge server can use this data to evaluate the viability and advantages of transferring work from mobile devices to other network computing units or to itself.

The edge server schedules tasks based on the task data and the existing workloads and computational capabilities of the available computing units. This entails deciding which tasks should be carried out locally on the mobile devices and which should be offloaded. Additionally, the tasks that are offloaded to the edge server or other mobile devices are given execution priorities. The edge server makes sure that high-priority activities on the critical route are completed on time by using the Critical Route Method (CPM), which finds the longest sequence of tasks that determines the shortest potential completion time. This method distributes the computational burden among computing units, reduces latency, and maximizes resource use. After calculation is finished, the results are combined and sent back to provide the mobile devices' final control choices.

Prioritizing and offloading activities to chosen targets with given priorities is the idea behind the Critical Task First Scheduler (CTFS) in a mobile edge computing (MEC) environment. This ensures effective scheduling and low latency. In order to establish the shortest amount of time needed to complete activities, CTFS uses the Critical Path Method (CPM) to find the longest chain of dependent tasks. In this configuration, a variety of data streams with varying degrees of urgency are continually generated by mobile devices. Using categories like "CRITICAL" (needing urgent processing), "SCHEDULE" (regular updates or periodic checks with moderate urgency), and "RECORD" (delay-tolerant data for cloud storage), the scheduler assigns jobs a priority. In order to ensure timely execution and response times, CTFS gives "CRITICAL" tasks the highest priority and offloads them to the edge server or nearby MEC nodes with available resources and moderate workloads. Tasks on the critical route are identified by CTFS using CPM, and they are scheduled first to maximize resource usage and minimize latency across the MEC environment. By scheduling less important jobs appropriately and ensuring that high-priority tasks are finished promptly, this method balances the load and improves network efficiency.

With the rapid growth of DL, training huge models for various tasks has become a prevalent concept. In addition to improving performance on existing tasks, these huge models show that they are capable of learning new ones. Training a model is the main goal of the project for mobile devices handling a lot of activities, motivated by this paradigm and striving for flexibility in dynamic contexts. With this method, the trained model may be used on a variety of mobile devices with different job needs.

Training Environment : To schedule tasks in this work, within a RL system, the edge server serves as the agent. The changing quantity of mobile devices, their service demands, and the computing resources that are available are just a few examples of the environment that the edge server learns from through reinforcement learning. This makes it possible for the model to respond to the shifting circumstances of mobile device networks with effective and flexible scheduling decisions.

States : The agents' operating environment's current circumstances are represented by the states. The states are represented in this work as a collection of environmental observations that are received by the mobile devices. P о e^e(f) is the position of the mobile devices and edge servers; In^m(t) is the information of the tasks created by each mobile device; С о m^m)t) is the communication overhead of each mobile device; and С о m^{m m} ( t) is the computation workload of each computing entity. As a result, each agent's state space may be represented by equation (26).

SS_t = {P о s^e(t), In^m(t),Com^m(t),C amp ^m ( t)}r (26)

Actions: Decisions made by the agents in reaction to the status of the environment at the moment are known as actions. In contrast to previous research, this study depicts the combination action as a collection of many actions that regulate each mobile's job scheduling behavior, such as the choices about priority assignment and offloading, which is expressed in equation (27).

A ct^ = { CA 1,...... ,CA %,......,CA *,.

CA1 = { a ^EP^ }

■, C ADo)^

where CA * represents the combination action for task к that was produced by mobile i (V i E {1,......,MD}), which includes the assigned execution priority P Pf ( Vi E {1,......, K} ) and the offloading decision ai E {0,1,2,3} .

Consequently, the action space is described using a two-dimensional coordinate system. Task offloading is represented by the x-axis, while priority assignment is shown by the y-axis.

Reward : Based on the agents' behaviors, the environment provides them with feedback, which is represented by the rewards. Usually, rewards are made to motivate the agents to perform tasks that result in favorable results, including reducing the time and energy required to complete requests.

The reward function in this study is described as follows, as equation (28).

Rew = ax

ExCt

---Чт + P<

MaxExCt

ExC

energy

MaxExC

energy

where and indicate the maximum and the agent has attained, and and ExC_energy stand for the predicted completion time and energy consumption of the service request, respectively. The constants a and p regulate the associated variable's weight.

Dynamic space : The scheduling approach is trained with a large number of jobs in order to develop adaptation to the dynamic environment. If the actual number of tasks is less than the number needed for model deployment, a null task is added to the state space after the model has been trained. With this strategy, the scheduling mechanism is guaranteed to stay environment-adaptive even in the dynamic scenario when task counts change. The definition of the task supplement scheme is given as equation (29), (30), and (31):

, f Tim< _<2,  if n=K

'P  \TI^2,  if nKK

Tjm _ [ m, 1 x2 L° m, 1 Cm, 2 - Cm,nlT                                      (30) Cm ,2 - Sm- , i......mmn T2™<2 = 1° m, 1 ...... ° m ,n Cm ,n+1 = 0 ......Стк = 01                          (31) ,   = 0 --  , = 0 where is a component of the state space that encodes task information , is the necessary number of tasks, and is the actual number of tasks in a single service request.

With two hidden layers and an output layer, the policy network is a neural network in the form of a Multi-Layer Perceptron (MLP). The Tanh activation function is used throughout, and two hidden layers include neurons (linear mapping). A discrete value is sampled by the output layer from the hidden layers' output, which is a categorical distribution produced by supplied logits in place of the softmax function, and this value is then converted into an action. On the basis of the given state, the policy network is utilized to forecast action. As a result, the policy network's input is environmental observation, and its output is task assignment.

• 4.2.1.    Policy-Value Optimizer

As demonstrated, in reinforcement learning, the actor and critic components cooperate to maximize the policy and value function. The policy тт _е , which dictates the agent's behaviors based on observable states, must be learned and updated by the actor. The value function ff^ which assesses the predicted cumulative benefit of existing in a specific condition, is learned and estimated by the critic. The fundamental concept of PVO is adding a "proximal" term to the goal function, which limits the policy update to remaining within a specific range of the prior policy. To preserve stability during the learning process, PVO makes sure that policy changes stay modest by preserving this proximity in mind. Figure 2 shows the architecture for Deep-Q-Driven Policy-Value Optimizer. The following five phases make up the Policy-Value Optimizer algorithm:

Mobile devices

Memory

Output

Fig. 2. Architecture for Deep-Q-Driven Policy-Value Optimizer.

Gathering trajectories: First, the environment's present policy is implemented to gather a collection of trajectories. These trajectories include states, actions, rewards, and other pertinent data.

Calculating Proxy aim: The PVO is based on Trust Region Policy Optimization (TRPO), which means that each policy update is monotonic and always better than the one before it. In TRPO, the objective function is symbolized by equation (32).

0 F_CPuer = E _t[m i n(P Rd) ) EE )]

The probability ratio, PR_t)в ), may be represented as follows as equation (33), where P P_{e t er} stands for the conservative policy iteration .

PR t (P)

(^МЮ ^П« о I _d ^t^t)

Where s_t stands for the sample weight of importance between the new policy 7Tg_new(u _£ |s _£ ) and the old policy n_g (u _t |s _t ). Maximizing 0F_CP , however, might result in a significant deviation and an unnecessarily big policy update. Thus, to guarantee stable policy updates, the proxy objective function utilized in this study is a revision to the goal function of the policy, which is shown by equation (34).

P0F_clip (0) = E _t [min(RR _t (0)E _t ,cl ip(RR_t(91 1 - e, 1 + f)^ _t )]

When the probability ratio is compared to a clipped version of the ratio, when the unclipped and clipped proxy objectives are the lowest, the objective function is defined, as seen in (32). One way to identify the clipped proxy aim is by equation (35),

1-e,    if RR_t(6)

cI ip (PR _t (0), 1 - e, 1 + e) = 1 + e,     if PR _t (0)>1 + e

(PR t (в),        о therwise

It prevents the policy update from departing too much from the previous policy by limiting it to a certain range [1 — e, 1 + e].

Optimization is carried out by updating the policy parameters using the calculated policy gradient. Stochastic gradient descent (SGD) is commonly used in PVO to update the parameters. To make the policy better over time, this procedure can be repeated several times. In particular, the policy is executed for T timesteps, after which an update is performed using the samples that were gathered. The advantage estimator AE is needed for this operation. An abbreviated form of Generalized Advantage Estimation (GAE) is the estimator employed as equation (36) in this study:

T-l

^E t = ^WA t_{+ k} fc=0

Where,

A_t = RR_t + уVar⁽(s_{t + r}) - Var⁽(s_t)                            (37)

where yand 8 are the future reward discount components to reduce the advantage function's instability, and AE_t is an estimation of the advantage function at timestep .

Iteratively repeating the process: New trajectories are gathered, the proxy objective is calculated, the policy gradient is determined, and the policy parameters are changed. By using the proximal term, this iterative process enables the policy to steadily improve over time. Algorithm 1 illustrates the PVO algorithm's training procedure.

Algorithm 1. The PVO algorithm's training procedure, which produces judgments on task offloading and priority assignment

Establish the policy's initial parameters в ₀

for i=1, 2, …… do

Gather an episode to relive memories for       = 1do for timestep = 1do

Examine the state   of the surroundings.

Depending on the observed situation, choose an action u_t .

Depending on the action chosen, the environment changes to a new state .

Depending on the new state, get a reward.

Use Equation (32) to calculate the advantage estimate AEt end for end for

All sampled data in the replay set should be cached.

Utilizing the collected data, update P 0 F_{c ; lp}.

Tin ^ Tin

°old anew end for

The Deep-Q-Driven Policy-Value Optimizer (DQPVO) Algorithm is shown as follows: All mobile devices producing different quantities of tasks should be taken into consideration to demonstrate how to schedule and carry out tasks depending on the developed DQPVO algorithm. First, using a device selection approach, a nearby mobile device is chosen as an optional task offloading target. Second, each mobile device's task offloading and priority assignment decisions are generated using the OPSM-based scheduling approach. Following these judgments, tasks are offloaded, and execution priorities are allocated in accordance with priority levels. Thirdly, a resource allocation mechanism is used to distribute resources across tasks throughout the task offloading process. Tasks on the same computing entity are then completed in priority order, ranging from high to low, following task offloading. The algorithm 2 describes the steps involved in the DQPVO-based energy-efficient and priority-aware task scheduling approach.

Algorithm 2. Process of the DQPVO-based Priority-Aware and Energy-Efficient Task Scheduling Method

The edge server hosts the trained model.

for timestep t =1 to T do

Determine the mobile devices (M) in the immediate area of the edge server.

for each mobile device m =1 to MD

Using the most recent observations, get the mobile device's status information ( SS]J ).

Based on the Device Selection Strategy, identify the mobile device SeM for device MD .

Determine how many tasks ( К ) were included in the service request that was sent from the mobile device ( MD ). for each task к =1 to К do

Using the PVO method, generate a task offloading decision and assign an execution priority to each task к . if task к is offloaded then

Using the Resource Allocation Strategy, distribute resources to к .

end if end for end for end for

5.    Result and Discussion

The experimental evaluation of the proposed Q-Optimize OffloadPro Framework (QOOPF) was conducted using the CloudSim simulation toolkit, which is a widely recognized and validated platform for modeling and analyzing cloud and edge computing environments. CloudSim allows controlled modeling of computational resources, task arrival rates, and network configurations, ensuring reproducibility and comparability with prior studies.

• 5.1.    Experimental setup

  The simulated MEC architecture consists of 14 edge servers, a single cloud server, and a variable number of mobile devices (300–2000). The cloud server is provisioned with 32 Virtual Machines (VMs), each having 800 GIPS CPU capacity, 64 GB RAM, and 1000 GB storage. Each edge server hosts 32 VMs with 10 GIPS processing power, 8 GB RAM, and 64 GB storage. Mobile devices (MDs) were configured with 2.6 GHz processing capability and 14 Mbps uplink transmission rate, closely reflecting realistic MEC deployment scenarios. To capture workload diversity, four representative MEC applications with different job durations, task sizes, and latency constraints were implemented in CloudSim. Task arrivals were modeled using stochastic processes to reflect realistic user request patterns. The evaluation was conducted over 1000 simulation episodes, each consisting of 100 time slots of 0.1 seconds. Training converged after approximately 2500 episodes (~3.5 hours), with the convergence curve stabilizing by 3000 episodes, ensuring reliable policy learning. The complete simulation parameters are presented in Table 1. By leveraging CloudSim, the proposed framework is tested in a standardized and reproducible environment, strengthening the credibility and comparability of the experimental results.

• 5.2.    Parameter Evaluation

Table 1. Evaluation Parameters.

Parameters	Values
No. of tasks	300
Edge server	14
No. of. VM	32
MDs Capacity	2.6 GHz
ENs Capacity	42.8 GHz
MDs Transmission size	14 Mbps
Task arrival rate	87.9 secs
Size of task	1000
CPU cycles of task	{0.197,0.297,0.397}×103
Task Success rate	0.8172
Utilization rate	0.1432
Activation	Relu
Classification	Softmax

Two important factors are taken into account when assessing the effectiveness of the proposed task scheduling and offloading paradigm for MEC as latency and utilization of resources.

Latency: The amount of time it takes for a job to be offloaded, processed, and then returned to the mobile device is known as latency. The latency formula takes into account a number of process steps, including processing, queuing, and transmission times, which is expressed in equation (38).

Ltot=Loffload +Lprocessing+Ltransmit +Lqueue

Where ^offload : The amount of time needed to transfer the work from the mobile device (MD) to the cloud server or chosen edge node (EN). This covers the duration of any control signals and data transmission. ^processing : The amount of time needed for the cloud server or edge node to process the job that was offloaded. This is dependent on the task's computational complexity and the edge server's resource availability. ^transmit :The amount of time needed after the job is finished to send the results back to the mobile device from the edge node or cloud server. ^queue : The amount of time spent waiting in line at the cloud server or edge node because of conflicting demands or scarce resources.

Fig. 3. Average Latency vs. Number of Jobs.

Fig. 4. Latency vs. Number of VMs.

The QOOP Framework, which uses a policy-optimizer-based method for dynamic task offloading and priority assignment, presents the average delay for different amounts of jobs handled in Figure 3. In order to prioritize activities and minimize latency for time-sensitive processes, this solution uses the critical route methodology. According to the figure, the average delay at 100 tasks is around 0.52 seconds. Latency drops to about 0.48 seconds when the number of jobs rises to 150, and it stays constant at 0.48 seconds for 200 jobs. The delay, however, increases once more to around 0.51 seconds when the number of tasks hits 250, then decreases to 0.48 seconds when the number reaches 300. The framework's ability to adjust its task offloading and prioritizing algorithms to balance compute load and reduce latency across different job volumes is demonstrated by these variations.

The QOOP Framework's average latency for varying virtual machine (VM) counts is displayed in Figure 4. The average delay reaches its maximum of around 0.60 seconds with 16 virtual machines. The latency is reduced to about 0.55 seconds when the number of virtual machines is increased to 20. The average latency drastically decreases to about 0.40 seconds when the number of virtual machines approaches 24. Nevertheless, latency increases once again to around 0.52 seconds at 28 VMs and then decreases to roughly 0.45 seconds at 32 VMs. The framework's capacity to dynamically modify task offloading and prioritizing algorithms in response to resource availability is shown in these latency fluctuations, which successfully balance computational load and lower latency across a range of virtual machines.

Resource Utilization : The ratio of the total resources utilized for processing and offloading activities to the maximum resources available during a certain time period is known as the resource utilization RU , as shown in equation (39).

=

^∑t=i ^t ^tot

where: RU denotes Resource utilization (expressed as a fraction or percentage); N indicates Total number of tasks being processed or offloaded; Ri represents Resources used by each task i (e.g., CPU cycles, memory usage, bandwidth for offloading); ^tot indicates the Total available resources on the edge server (e.g., total CPU capacity, total memory, total bandwidth).

Fig. 5. Resource Utilization Vs Number of Jobs.

Figure 5 shows the QOOP Framework's utilization of resources as the number of jobs changes. The data indicates that resource consumption rises when the number of tasks reaches 250, reaching a high of around 0.525. Utilization starts at around 0.495 at 100 jobs, increases gradually to about 0.515 at 175 jobs, and continues to rise slightly to 0.52 at 200 jobs. The resource usage, however, declines precipitously to about 0.495 at 300 jobs after peaking at 250 jobs. By effectively managing computational jobs, the QOOP Framework seeks to optimize resource allocation and avoid overload on edge computing resources. This trend shows how the framework dynamically adjusts to changing job volumes.

Figure 6 shows the QOOP Framework uses resources as the no. of. VMs changes. Variations in utilization of resources across varying no. of. VMs are seen in the graph. Resource consumption begins at 0.45 at 16 virtual machines. After then, it grows quickly, peaking at about 0.52 at 20 VMs. At 24 VMs, however, the usage drastically decreases to about 0.45. Utilization starts to recover as the number of virtual machines (VMs) keeps growing. It gradually increases to around 0.47 with 28 VMs and peaks again at roughly 0.52 with 32 VMs. This pattern points to the adaptive resource allocation of the QOOP Framework, which optimizes the utilization of virtual machines (VMs) while avoiding resource overload by dynamically modifying job offloading mechanisms based on available computing resources.

Fig. 6. Resource Utilization Vs Number of VM

• 5.3.    Comparative Analysis

A comparison study with prevailing methods is provided by evaluating the QOOP framework based on makespan and variance of completion time. This method emphasizes QOOP's potential for increased efficiency by demonstrating how well it optimizes job scheduling and lowers completion time variation.

• •    First Come, First Served (FCFS) [26]: This traditional heuristic algorithm schedules tasks in the order of arrival, offering simplicity but lacking adaptability in dynamic environments.

• •    Shortest Job First (SJF) [26]: prioritizes tasks based on their estimated execution time, aiming to minimize total waiting time but often struggling with real-time adaptability.

• •    Simulated Annealing PSO (SA-PSO) [26]: Combining Simulated Annealing with PSO, SA-PSO balances exploration and exploitation, enhancing task scheduling effectiveness in complex edge environments.

• •    Double Deep Q Network (DDQN) [26]: Double DQN mitigates overestimation in Q-learning by decoupling action selection and evaluation, leading to more accurate task scheduling decisions.

• •    Gradient-Driven Double Deep Q Network (GDDQN) [26]: GDDQN further refines Double DQN with an improved reward mechanism and experience replay management, enhancing learning efficiency and scheduling precision.

• •    Particle Swarm Optimization (PSO) [27]: PSO is a meta-heuristic algorithm inspired by swarm behavior, optimizing resource allocation by iteratively improving solutions based on individual and group best positions.

• •    Deep Q Network (DQN) [28]: DQN uses reinforcement learning to adaptively schedule tasks, learning through a neural network to select actions that maximize cumulative rewards.

• •    Long Short-Term Memory Deep Deterministic Policy Gradient (LSTM-DDPG) [29]: Integrates memory capabilities of LSTM with DDPG to handle sequential dependencies in task data. It improves stability over standard DDPG but is computationally intensive and slower to converge in highly dynamic environments.

• •    Deep Deterministic Policy Gradient (DDPG) [29,30]: A continuous action space reinforcement learning method that enables direct policy optimization. While effective in certain MEC scenarios, it often suffers from instability and poor convergence speed due to overestimation bias.

• •    Efficiency-Aware Adaptive Deep Reinforcement Learning (EADRL) [30]: Focuses on energy efficiency and resource utilization in task scheduling. It reduces energy consumption effectively but shows limited scalability when task loads increase rapidly.

• •    Double Deep Q-Network (Double DQN) [30]: Enhances traditional DQN by decoupling action selection and evaluation, reducing overestimation errors. While it achieves better convergence than DQN, its adaptability remains constrained under large-scale MEC scenarios.

• •    Server Real-Time Performance Deep Reinforcement Learning (SRP-DRL) [30]: Focuses on optimizing real-time server-side performance, ensuring lower latency and higher throughput. While it improves responsiveness, it requires high-quality server state data and may degrade when real-time information is incomplete or noisy.

• •    BEST-FIT [30]: A heuristic-based method that assigns tasks to the most suitable server based on available capacity. It offers simplicity and reasonable efficiency for smaller workloads, but lacks adaptability to dynamic MEC conditions with fluctuating task arrivals.

• •    Random [30]: Allocates tasks randomly to servers without considering workload or system state. Although

computationally lightweight, it leads to inefficient resource utilization and high makespan under practical MEC scenarios.

• •    Earliest Idle Time First (EITF) [30]: Prioritizes assigning tasks to servers expected to become idle first, aiming to minimize waiting time. While effective in reducing queue delays, it does not account for overall resource balancing, which may result in uneven server loads.

• •    Multi-UAV Assisted DDPG (MUAVDDPG) [31]: Designed for UAV-assisted networks, this method balances communication and computation resources. However, its UAV-centric design reduces general applicability in generic MEC systems.

• •    Asynchronous Advantage Actor-Critic (A3C) [31]: Employs asynchronous updates and policy/value

functions to accelerate training. It improves learning efficiency but can yield unstable convergence due to variance in parallel learning processes.

• •    Lyapunov Algorithm [32]: A mathematical optimization-based strategy that minimizes system drift and ensures queue stability. While effective in balancing energy and delay, it introduces high computational complexity and is less adaptive to sudden workload variations.

• •    Total Offload Execution Algorithm (TOEA) [32]: Executes all tasks via offloading to edge servers, reducing device-side energy consumption. However, it increases network energy costs and suffers from latency under heavy user loads.

• •    Total Local Execution Algorithm (LEA) [32]: Performs all computations locally, avoiding communication overhead. Although this reduces dependency on network conditions, it significantly increases device energy consumption as the number of users grows.

• •    Randomized Offload Execution Algorithm (ROEA) [32]: Assigns tasks randomly for offloading, offering a lightweight mechanism with minimal computational cost. However, its lack of intelligent scheduling cause’s imbalanced resource usage and suboptimal execution rates as users grow.

• •    Resource Allocation Strategy based on DQN (RASD) [32]: Utilizes reinforcement learning to optimize offloading decisions. It adapts well to dynamic environments and maintains high execution rates compared to heuristic methods, but learning overhead and convergence time limit real-time adaptability.

Fig. 7. Makespan Comparison for Various Scheduling Algorithms with Varying Task Numbers.

Figure 7 shows the makespan performance of many scheduling algorithms in comparison to the proposed QOOP architecture. These algorithms include FCFS, SJF, PSO, SA-PSO, DQN, Double DQN, and G-DDQN. Traditional algorithms like FCFS and SJF exhibit a sharp increase in makespan when the number of tasks grows from 100 to 300, reaching up to 1750 and 1500 seconds, respectively, at 300 tasks. PSO and SA-PSO perform better than FCFS and SJF, although they also show a noticeable rise. With makespan values of about 1000 seconds, deep Q-learning-based techniques like DQN, Double DQN, and G-DDQN show a very moderate increase. On the other hand, even with increased task volumes, the proposed QOOP architecture maintains a much smaller makespan of less than 500 seconds. By dynamically improving task scheduling in complex, high-load situations, the QOOP framework effectively maintains efficient task completion times.

Fig. 8. Comparison of Makespan for Different Scheduling Algorithms Across Varying Numbers of Virtual Machines (VMs).

Figure 8 shows how several scheduling algorithms, such as FCFS, SJF, PSO, SA-PSO, DQN, Double DQN, G-DDQN, and the suggested QOOP framework, perform over time as the no. of. VMs rises from 16 to 32. With a makespan of almost 1500 seconds at 16 virtual machines and continuing to hover around 1400 seconds as the number of VMs rises, FCFS exhibits the largest makespan of any setup. The makespan values of SJF, PSO, and SA-PSO range from 1200 to 1000 seconds, which are slightly better yet still show superior performance. Deep Q-learning-based techniques like DQN, Double DQN, and G-DDQN show a steadier decrease in makespan, keeping values close to 900 seconds as the number of virtual machines rises. Nevertheless, the suggested QOOP architecture performs better than any other approach. Its greater efficiency and flexibility in maximizing resource allocation and decreasing completion time in multi-VM scenarios are demonstrated by its noticeably decreased makespan, which stabilizes around 800 seconds as VM numbers increase.

Table 2. Comparison of Makespan (s) for Different Scheduling Algorithms

Methods		FCFS	SJF	PSO	SA-PSO	DQN	DDQN	G-DDQN	Proposed QOOP
Tasks	100	800	500	450	440	430	420	410	390
	150	1200	700	650	640	630	620	610	520
	200	1600	900	800	780	770	760	750	550
	250	1700	1300	1000	980	970	960	980	750
	300	1900	1400	1200	1100	1000	990	980	720
VMs	16	1650	1400	1390	1380	1270	1270	1260	445.88
	20	1500	1200	1100	1090	1080	1080	1070	578.987
	24	1890	1100	990	980	970	970	960	554.032
	28	1250	1000	980	970	960	950	950	430.805
	32	1200	990	980	970	960	950	950	608.542

The makespan (in seconds) of many scheduling methods for varying numbers of jobs and virtual machines (VMs) are compared in Table 2. In every circumstance, the proposed QOOP algorithm achieves the smallest makespan and continuously beats alternative approaches. When compared to other approaches, QOOP achieves a makespan of 390 seconds for 100 jobs. Similar to this, QOOP shows notable efficiency as the no. of. VMs increases. For example, 16 VMs, it takes 445.88 seconds, whereas DQN and DDQN take 1270 seconds. This illustrates the advantages of the proposed technique in terms of task scheduling optimization and execution time reduction.

Fig. 9. Comparison of Completion Time Variance Across Different Scheduling Algorithms with Varying Numbers of Tasks.

As the number of jobs rises from 100 to 300, figure 9 shows the variation in completion time for various scheduling algorithms, such as FCFS, SJF, PSO, SA-PSO, DQN, Double DQN, G-DDQN, and the proposed QOOP framework. Both FCFS and SJF show noticeably greater variability; as the number of tasks increases, FCFS rises over 200, indicating a large amount of unpredictability in task completion times. Its inadequate flexibility in dynamic contexts is highlighted by SJF's growing trend, which reaches about 150 at 300 assignments. When the work load grows, PSO, SA-PSO, DQN, and Double DQN still display discernible fluctuations, although their variances are smaller than those of conventional heuristics. G-DDQN has marginally superior performance, preserving a more consistent variance around 50. On the other hand, even as the number of tasks increases, the offered QOOP structure constantly maintains a variance below 20. This stability highlights the durability and efficiency of QOOP in dynamic scheduling circumstances by demonstrating how well it ensures consistent job completion times.

Fig. 10. Comparison of Completion Time Variance across Different Scheduling Algorithms with Varying Numbers of VMs.

The variance in task completion time for many scheduling and optimization algorithms such as FCFS, SJF, PSO, SA-PSO, DQN, DDQN, G-DDQN, and the proposed QOOP framework across various VM counts is shown in Figure 10. Even when the number of virtual machines varies, the QOOP architecture enables dynamic policy selection that lowers completion time variability. Across all evaluated virtual machine counts (16, 20, 24, 28, and 32), the suggested QOOP technique consistently achieves low variation, about 2, indicating consistent and dependable job completion times. The FCFS approach, on the other hand, has a substantially greater range, ranging from 50 to 85, which reflects its inability to adjust to variations in the availability of resources. A variance of about 5 is maintained by methods such as SJF, PSO, SA-PSO, DQN, DDQN, and G-DDQN, which outperform FCFS but still fall short of the low variability attained by the suggested QOOP architecture. Through dynamic scheduling in shifting virtual machine settings, the QOOP-based solution maximizes efficiency and dependability, minimizes completion time variance, and ensures steady task execution.

Table 3. Comparison of Completion Time Variance across Different Scheduling Algorithms.

Methods		FCFS	SJF	PSO	SA-PSO	DQN	DDQN	G-DDQN	Proposed QOOP
Tasks	100	47	2	2	2	2	2	2	0.512
	150	100	10	10	9	8	7	6	5.05
	200	130	40	38	37	36	35	34	21.89
	250	150	80	49	48	47	46	45	30.12
	300	220	110	52	51	50	49	48	30.03
VMs	16	90	16	15	14	13	12	9	4.58
	20	65	16	15	14	13	12	9	5.78
	24	90	15	15	13	12	10	9	5.54
	28	50	15	14	13	12	10	9	4.30
	32	55	15	14	13	12	10	9	6.08

The completion time variance for many scheduling algorithms, such as FCFS, SJF, PSO, SA-PSO, DQN, DDQN, G-DDQN, and the proposed QOOP approach, are compared in Table 3. QOOP exhibits the lowest completion time variance for different task counts (100–300), much surpassing both conventional and sophisticated approaches. Similarly, QOOP regularly achieves the minimum deviation across VM numbers (16–32), demonstrating its superior efficiency and versatility. The aforementioned outcomes demonstrate how well QOOP reduces completion time inequalities, guaranteeing improved performance and resource efficiency.

• ■    LSIM-DDPGBDDPG

• ■    MUAVDDPGBA3C

EADRL    ■ Double DQN

• ■    DQN       ■ Proposed

Fig. 11. Convergence Trends of Reinforcement Learning Models over Episodes.

Figure 11 highlights convergence comparison of reinforcement learning models including LSTM-DDPG, DDPG, EADRL, Double DQN, MUAVDDPG, A3C, DQN, and the proposed QOOP, over increasing training episodes from 0 to 3000. Initially, all models start with relatively low performance values, but as episodes increase, the proposed QOOP consistently achieves the highest values, indicating faster and more stable convergence. Models like MUAVDDPG and DDPG show moderate improvement, while traditional DQN and EADRL lag behind. Overall, the table highlights that QOOP outperforms other RL approaches in convergence speed and final performance across the training span.

Fig. 12. Comparison of average response time Vs number of tasks.

Figure 12 shows the comparison of average response time across different scheduling models as the number of tasks increases. It is evident that proposed QOOP consistently achieves the lowest response time at every task level, reducing latency more effectively than the benchmark models. For example, at 24,000 tasks, proposed QOOP achieves a response time of 0.18 ms, compared to 0.45 ms for DDQN and 0.37 ms for DDPG. This indicates that proposed QOOP can handle larger workloads more efficiently, minimizing the delay in task execution while maintaining scalability.

Fig. 13. Comparison of average success Rate Vs number of tasks.

Figure 13 highlights the task success rate performance under varying task counts. Across all workloads, proposed outperforms other methods by achieving the highest success rate, with values steadily improving as the number of tasks increases. For instance, at 24,000 tasks, proposed reaches a success rate of 0.88, while other models such as DDQN and DDPG only achieve 0.69 and 0.74, respectively. This clearly demonstrates that proposed QOOP provides a more reliable and stable scheduling approach, ensuring a higher proportion of tasks are completed successfully even under heavy loads.

Fig. 14. Comparison of average load variance Vs number of tasks.

Figure 14 shows average load variance, which reflects how evenly tasks are distributed among available resources. A lower variance signifies better load balancing. Proposed QOOP achieves the lowest variance across all scenarios, starting from 0.043 at 8,000 tasks and reducing further to 0.039 at 24,000 tasks. In contrast, DDQN and DDPG remain significantly higher at 0.055 and 0.051, respectively, for the same workload. This confirms that proposed not only reduces response time and increases success rates but also distributes tasks more evenly across edge nodes, leading to efficient resource utilization and system stability.

Fig. 15. Comparison of average response time Vs number of VM.

Figure 15 indicate that the average response time decreases as the number of virtual machines increases across all methods, showing that higher VM availability improves task execution efficiency. Among the compared algorithms, proposed consistently achieves the lowest response time, demonstrating its effectiveness in quickly allocating resources and minimizing delay. In contrast, heuristic-based methods like Random and BEST-FIT exhibit relatively higher response times, while learning-based baselines such as DDPG and DDQN perform better but still lag behind proposed model. This highlights the capability of proposed to optimize scheduling decisions, ensuring faster task completion under varying VM counts.

Fig. 16. Comparison of average success rate Vs number of VM.

In figure 16, the success rate results show a steady increase as the number of VMs increases, which is expected since more resources enable higher task acceptance. Proposed QOOP consistently outperforms all benchmark methods with the highest success rate across all VM counts, proving its robustness in handling dynamic workloads. While heuristic algorithms like Random and BEST-FIT achieve competitive performance, they fall short of proposed model, especially in higher VM scenarios. Deep reinforcement learning-based approaches such as DDPG and SRP-DRL also show improvements but still remain lower compared to proposed model. This demonstrates that the proposed QOOP provides more reliable task allocation, ensuring a higher proportion of successful task executions.

■ Proposed I EITF ■ Random BEST-FIT SRP-DRL DDPG eDDQN

Fig. 17. Comparison of average load variance Vs number of VM.

In figure 17, the load variance analysis highlights the efficiency of workload balancing across VMs. A lower variance indicates that tasks are distributed more evenly, preventing resource overloading or underutilization. proposed QOOP achieves the lowest variance across all VM counts, showing its ability to maintain well-balanced resource allocation. In comparison, methods like Random and DDQN exhibit significantly higher variance, leading to inefficient load distribution. Other learning-based approaches such as DDPG and SRP-DRL show moderate improvements but still do not match the consistency of proposed model. These findings confirm that proposed QOOP not only reduces response time and improves success rate but also ensures fairness and stability in resource utilization.

JO

4            б            8            10

Number of Users

• ■    Lyapunov иТОЕА LEA ■ Proposed

• Fig. 18.    Energy Consumption Comparison with Number of User Devices.

Figure 18 shows the energy consumption for different techniques, Lyapunov, TOEA, LEA, and the proposed model, based on the number of users. As the number of users increases, energy consumption rises across all methods, reflecting the additional computational and communication load. Among the existing models, Lyapunov consumes the least energy, followed by TOEA and LEA, which show progressively higher requirements. The proposed QOOP consistently shows the highest energy consumption, suggesting that it prioritizes performance, reliability, and accuracy at the expense of power efficiency. This reflects a common trade-off in system design, where enhanced service quality often comes with increased energy demand that must be carefully managed for scalability

■ Lyapunov иТОЕА LEA ■ Proposed

• Fig. 19.    Latency Comparison with Number of User Devices.

• 5.4.    Ablation study

Figure 19 presents the latency performance of four approaches, Lyapunov, TOEA, LEA, and the proposed model, under varying numbers of users. As the number of users increases, the latency grows steadily for all methods, which reflects the natural effect of higher user load on network delay. Among the existing methods, TOEA shows the lowest latency in the early stages, but its delay rises sharply after 10 users, while LEA increases more gradually. Lyapunov remains moderate throughout, showing better stability than TOEA and LEA at higher loads. The proposed QOOP consistently records the highest latency, indicating that while it may provide improved execution or reliability in other metrics, it does so at the cost of greater delay when user numbers scale up. This emphasizes the trade-off between performance gains and latency overhead.

Fig. 20. Execution rate Comparison with Number of User Devices.

Figure 20 presents the Successful Execution Rate (SER) comparison across different numbers of user devices for existing methods (LEA, TOEA, ROEA, and RASD) and the proposed model. It is evident that the SER decreases gradually for all models as the number of devices increases, reflecting higher execution challenges with larger user loads. Among the existing methods, RASD consistently outperforms LEA, TOEA, and ROEA, demonstrating better adaptability. However, the proposed QOOP achieves the highest SER values in all cases, maintaining strong reliability even with 25 devices. This highlights the superior efficiency and robustness of the proposed QOOP over existing approaches.

To evaluate the individual impact of the proposed framework’s core components, an ablation study was conducted by selectively enabling or disabling the OffloadPro Scheduling Method (OPSM) and the Deep-Q-Driven Policy-Value Optimizer (PVO). This analysis helps isolate the contribution of each module and demonstrates how their integration enhances overall system performance in terms of makespan, variance, latency, and resource utilization.

Table 4. Ablation Study of proposed QOOPF Variants on Task Scheduling Performance.

Variant	Description	Makespan (s) 300 Tasks	Variance	Avg. Latency (s)	Resource Utilization
Baseline (No OPSM, No PVO)	Simple Q-learning with random scheduling (no critical path, no policy-value optimization).	1200	95.4	0.72	0.42
QOOPF – OPSM only	Uses critical-path task prioritization without deep policy optimization.	920	60.2	0.58	0.48
QOOPF – PVO only	Uses Deep-Q Policy-Value Optimizer but without critical path scheduling.	810	42.7	0.52	0.50
Full QOOPF (OPSM + PVO)	Proposed integrated framework with critical path scheduling + deep policy optimization.	720	30.03	0.48	0.52

The ablation study table 4 highlights the contribution of each component in the proposed QOOPF framework. The baseline model, which excludes both OPSM and PVO, performs the worst, with high makespan, variance, and latency, showing inefficient task scheduling. Incorporating OPSM alone improves makespan and latency by prioritizing critical tasks but still results in moderate variance due to limited adaptability. Using only the PVO achieves lower variance and better latency by dynamically optimizing decisions, though it lacks efficient task sequencing. The full QOOPF model, integrating both OPSM and PVO, consistently achieves the best performance, delivering the lowest makespan (720 s), minimum variance (30.03), and reduced latency (0.48 s) with improved resource utilization. This confirms that both OPSM and PVO are complementary, and their integration is essential for optimal task scheduling and offloading in MEC systems.

• 5.5.    Discussion

• 99.77% and SJF by 74.91% in terms of reducing completion time variability. Reliable task scheduling is ensured by this performance consistency, which is essential for preserving effective resource use. The findings highlight the flexibility and optimization potential of QOOP, which makes it extremely useful for managing dynamic task offloading scenarios with a variety of resource types. Beyond task performance, the computational requirements of PVO must also be considered. Although inference introduces only ~3–5 ms of additional decision latency, training the model requires significant computational effort. While this overhead is acceptable in simulated MEC environments, future work should further optimize training efficiency and quantify inference scalability for deployment in large-scale real-time MEC systems.

6. Conclusion

The performance evaluation demonstrates that the proposed QOOP method to task scheduling and resource management achieves notable efficiency and consistency. With execution delay reductions of up to 63.16% when compared to more conventional algorithms like FCFS and 48.57% when compared to SJF in larger task situations, QOOP performs better than other approaches. QOOP consistently maintains significant processing time savings, even with different resource configurations. Furthermore, in large-scale offloading scenarios, QOOP outperforms FCFS by

In MEC environments, the proposed QOOPF Framework provides an effective solution for task scheduling and offloading, targeting high virtual machine utilization, reduced latency, and optimized resource management. At its core, the OffloadPro Scheduling Method (OPSM) applies a critical-path-based priority mechanism to allocate tasks to the most suitable computing nodes, while the Deep-Q-Driven Policy-Value Optimizer (PVO) ensures adaptive and intelligent offloading decisions. Together, PVO and Q-learning balance computational loads, minimize task failures, and enhance overall efficiency. Experimental results validate the effectiveness of QOOPF, achieving a makespan of 720 seconds and variance of 30.03 for 300 tasks, while scaling efficiently with VM counts, 445.88 seconds (variance 4.58) at 16 VMs and 608.54 seconds (variance 6.08) at 32 VMs. These outcomes demonstrate the robustness of QOOPF in sustaining performance across diverse task and VM configurations, confirming its potential as a practical and scalable approach for dynamic MEC systems. These findings demonstrate the efficiency of QOOPF in maximizing performance and resource utilization across different task and VM configurations. The framework requires considerable training time, which may affect responsiveness in dynamic MEC environments. It may also experience task priority drift under varying workloads and lacks mechanisms to manage edge node failures or mobility-related disconnections. Future work will focus on overcoming these challenges by incorporating meta-learning techniques to accelerate training and improve adaptability across varying task distributions. In addition, hybrid cloud–edge offloading strategies will be explored to ensure resilience and continuity in case of edge server overload or failure. These enhancements will strengthen QOOPF’s scalability, robustness, and practicality for deployment in real-world MEC systems.

All the Declarations and StatementsAuthors Contributions

All authors are equally contributed.

All contributors agreed and given consent to Publication.

Conflict Of Interest

They author declare that have no conflict of interest.

Funding Declaration

No fund was received for this work.

Availability of data and material

Data that has been used is confidential.

Ethical Declarations

This article does not contain any studies with human participants or animals performed by any of the authors.

Competing interests

On behalf of all authors, the corresponding author states that they have no competing interest.

Acknowledgment

The authors would like to thank the Deanship of Koneru Lakshmaiah Education Foundation, Green Fields for supporting this work.

Declaration of Generative AI in Scholarly Writing

None.

Abbreviations

The following abbreviations are used in this manuscript:

MEC - Mobile Edge Computing

QOOPF - Q-Optimize OffloadPro Framework

OPSM - OffloadPro Scheduling Method

PVO - Policy Value Optimization

CPM - Critical Path Method

VM - Virtual Machine

CPM - Critical Route Method

CTFS - Critical Task First Scheduler

MLP - Multi-Layer Perceptron

TRPO - Trust Region Policy Optimization

SGD - Stochastic Gradient Descent

GAE - Generalized Advantage Estimation

DQPVO - Deep-Q-Driven Policy-Value Optimizer

FCFS - First Come, First Served

SJF - Shortest Job First

SA-PSO - Simulated Annealing PSO

DDQN - Double Deep Q Network

SRP-DRL - Server Real-Time Performance Deep Reinforcement Learning

EITF - Earliest Idle Time First

MUAVDDPG - Multi-UAV Assisted DDPG

A3C - Asynchronous Advantage Actor-Critic

TOEA - Total Offload Execution Algorithm

LEA - Total Local Execution Algorithm

ROEA - Randomized Offload Execution Algorithm

RASD - Resource Allocation Strategy based on DQN

SER - Successful Execution Rate