open access

https://hdl.handle.net/2115/100410

Environment Construction of Remote Reinforcement Learning Using L4S Ultra-low Latency Network Architecture

日本語

Inverted Pendulum

2026-01-15

IEICE / 電子情報通信学会

IEICE Technical Report

125(320)

37-41

5

In recent years, research and development of systems that remotely learn and execute machine control, such as autonomous vehicle control and drone operation, has become increasingly active. In systems that apply remote reinforcement learning to machine control, data is frequently exchanged between actual devices and servers, and therefore network quality significantly affects control performance and safety. Recently, L4S (Low Latency, Low Loss, and Scalable Throughput), standardized as RFC9330, has attracted attention as a method for suppressing network latency. However, there are currently few reports on applying L4S to remote reinforcement learning.In this study, we construct a remote reinforcement learning environment on a high-quality network with L4S for fundamental evaluation of remote reinforcement learning. Specifically, we realize remote control of an inverted pendulum, which is one of the fundamental problems in control theory, using the network simulator ns-3 and DEFIANCE, a framework that enables remote reinforcement learning on ns-3. By placing a bottleneck link between the cart node and the agent node and applying L4S to the network, we constructed an environment that enables investigation of the impact on reinforcement learning when congestion occurs.

会議発表論文

Version of Record

Copyright ©2026 IEICE

2432-6380

AA12325711