Multi-agent Cable System
Reinforcement learning implementations for cable-driven parallel robots designed for agricultural monitoring and inspection tasks. The project features both single-agent and multi-agent approaches to optimize path planning and coordination in hydroponic farming environments.
Problem statements
We developed a two-robot cable-driven system, each equipped with a distinct sensor (RGB and hyperspectral), to collectively capture comprehensive plants data over time. We then implemented a reinforcement learning framework to optimize coverage strategies and inter-robot coordination.
Solution Strategies & Potential Limitation
Simulation Development
We built a multi-cable robot simulation environment with a real-time 3D renderer based on Open3D. The plants are randomly spawned at the start of each episode. The rendered environments are shown in the figure below. Once a plant is visited, it is colored green for visual distinction. Both the number of agents and the grid size are easily configurable.
Results
We choose PPO as the base policy. The training logs for both the single-agent and multi-agent settings are shown below. In the single-agent setup, the decreasing training loss and the explained variance approaching 1 indicate stable convergence and accurate value function estimation. In the multi-agent setup, shorter episode lengths, higher cumulative rewards, and lower policy entropy demonstrate improved efficiency and increased confidence in cooperative decision-making. As training progresses, agents reduce redundant movements when visiting plants and learn to adapt their actions in response to the behaviors of other agents.
The behavior for trained agents are shown below:
