scratchpad.md

Scratchpad

Scratch notes. Only really intended for me.

Conception

Ok so I want to build a project that achieves a few things:

• Is relevant to maritime autonomy.

USVs for days. Staying on the surface for this one. Maybe later venturing under the sea... under the sea, darling it's better down where it's wetter take it from me do do do do doo doo𝅘𝅥𝅮

• Refreshes me on ROS2.

It's been a minute since I've worked with ROS2; not being hands-on for a second means it's pretty rusty. I don't want to realise in e.g. an interview situation that I can't remember the specifics of something important.

• Gets me up to speed on more modern ROS2.

Foxy is where I last worked with it, which is long past EOL now; Lyrical is the latest release.

I imagine a bit's changed during F -> G -> H -> I -> J -> K -> L.

It's probably not a great idea to jump to the latest and greatest; industry is unlikely to be at the bleeding edge, and if something changes in significantly in say K, it could actually be adverse to refresh on that.

• Nails down specific versions.

Previous work has largely been project/role-specific and under NDA. So I don't have copies of source code or notes etc from it; I can't look back and say with certainty "it was Gazebo X" or "Blah Y".

Developing something now means I can more readily nail it to specific versions and specific architecture.

• Demonstrates autonomous vehicle understanding.

I've worked with autonomous vehicles; but not in maritime. Aside from much of it being under NDA and it being difficult to speak to specifics, water is a different beast just at surface level, let alone subsurface.

I can scream "I understand perception, I understand localisation, I understand blah" from the mountain tops as much as I like; but I don't have a lot that actually demonstrates it.

So I want to demonstrate that.

• Demonstrates simulation understanding.

As above.

• Is interesting and fun.

Because. That's why.

Development Rig

Modest development rig should be enough to run a reasonable simulation:

• AMD Ryzen 7 5700G.
• NVIDIA RTX 4060 Ti (8GB).
• 32GB RAM.
• SSD storage.
• Ubuntu 24.04.4 LTS.

I want to build the project as containerised (Docker). So anyone can run it, let's just avoid the naive "oh but it works on my machine" dilemma.

Google-fu

ROS2 Status

Checking the ROS2 releases:

• Foxy: released 2020, EOL 2023.
• Galactic: released 2021, EOL 2022.
• Humble: released 2022, EOL 2027.
• Jazzy: released 2024, EOL 2029.
• Kilted: released 2025, EOL 2026.
• Lyrical: released 2026, EOL 2031.

So Foxy and Galactic have been EOL for a while. Kilted hits EOL this year and obviously isn't a LTS so I'll skip that.

Leaving Humble, Jazzy, Lyrical.

Gazebo Status

The Gazebo I remember now seems to be called "Gazebo Classic", which was released as a numbered version. "Modern Gazebo" if you will seems to be released as "Gazebo <Codename>". I feel like the last version I used had a codename, but I also recall using a numbered release; so can't say exactly where I've been.

It seems there's been some significant architecture changes over the past few years too.

A few LTS versions are still under support per Gazebo Docs:

• Fortress: released 2021, EOL 2027.
• Harmonic: released 2023, EOL 2029.
• Jetty: released 2025, EOL 2031.

So seems Fortress, Harmonic and Jetty are the likely contenders. But it will largely depend which ROS2 version I go with.

Simulation Environment

After a bit of Googling and filtering through the AI guff, it seems that "Virtual RobotX" (VRX) is a potential sim starting point:

VRX is an open-source simulation environment designed as a free tool for students to test autonomous maritime robotics solutions.

It's a collaboration between RoboNation, the US Office of Naval Research and the Naval Postgraduate School, and is the basis of a virtual maritime autonomous competition that sits a level below a physical one. It's targetted at university-level teams, and generally seems like a good starting point for personal projects.

It adds the things a generic sim lacks for boats: ocean wave models that affect motion and sensor feedback, wind, 6-DOF surface vessel model, buoyancy, 3D LiDAR that interacts with the water surface, a non-linear thrust model. Basically wraps up the things that I would otherwise need to build to create the "world", and lets me focus on the autonomous unit.

Having a look at the VRX codebase (currently at commit 7609d1b), it notes:

- Code is now working with Gazebo Harmonic and ROS 2 Jazzy
- This is the recommended configuration for new users.
- Users who wish to continue running Gazebo Garden and ROS 2 Humble can still do so using the humble branch of this repository.

Latest VRX release is 3.1.2 (Nov 2025). And per release VRX 3.0.0 (May 2025):

Important

VRX Code is now working with Gazebo Harmonic and ROS 2 Jazzy

This is the recommended configuration for new users. Users who wish to continue running Gazebo Garden and ROS 2 Humble can still do so using the humble branch of this repository.

Which Versions

Now I'm not sure which is the best version to target here.

For ROS2 I'd lean towards Humble as it's the "oldest" LTS and so probably more industry-aligned. But Jazzy could be an option to consider too. Lyrical I think is just too new.

VRX seems like it's a really good sim starting point; having even semi-realistic dynamics would take me in the order of months to develop from scratch, and isn't the point of this project.

At the end of the day, what I want to focus on is "building something interesting", not "making sure it's the best possible version for imagined alignment". So I think that settles some of the versioning.

Additional Note

A specific company that I looked into uses Jazzy. Settles that.

Settled Software

• ROS2 Jazzy
• Gazebo Harmonic
• VRX 3.1.2
• Docker
• Runs on Ubuntu 24.04.4 LTS
• Everything developed in C++

Preliminary Overview

VRX takes care of the vessel model, thrust, sensors, world.

I want to develop an autonomous USV on top of that.

So broadly what I'll need to develop is:

• Navigation
• Perception

This breaks down into roughly:

• Localisation: where is the USV?
• Path planning: how does the USV get to X?
• Obstacle avoidance: how does the USV avoid crashing into something?
• Other vessels: how does the USV respond to other vessels (as opposed to static object avoidance)?

For now, I'm going to get it to follow a series of waypoints. It will start at a given point in the VRX environment, then navigate to waypoint A, then to waypoint C, then to waypoint D, etc.

So the problem is essentially: "From position A, how should I navigate to position B".

Localisation will be based on the sensors -- GNSS and IMU. Fusion with an extended Kalman Filter on top.

Broadly, I'm classifying obstacles as two kinds:

1. Static obstacles. Fixed landmarks, things protruding from the water (or only just beneath the surface), rocks, docks, etc.
2. Dynamic obstacles. Things that move, other vessels, people, animals.

Aside from the movement of the USV, I will need to consider other aspects of movement depending on the obstacle.

Now for static obstacles, I will likely need to consider the movement of the environment (largely wind, water). So even if my vessel is "stationary" as far as I can make it, it may still be subject to wind, waves, currents, and so on. This is dependent on how the VRX environment works, and I'll have to dig into it a bit more to ratify.

I will work out whether I only need to consider things that are above the surface of the water (e.g. rocks that stick out of the water), or things that are just below it too (e.g. rocks that don't stick out of the water, but are near enough the surface to crash into). Things above the surface of the water definitely need consideration; things below the surface depend on the specifics of VRX.

For dynamic obstacles, I will need to consider the movement of the environment and the movement of the obstacle. In the event the obstacle is another vessel, I will need to consider how to navigate in line with maritime law. So it might be worth classifying as a third kind (vessels), or simply responding to all dynamic obstacles in the same way. I'll dig into this a bit more later; but from the outset, regardless of what a dynamic obstacle may specifically be, I'm treating the USV as "lowest priority for right of way" -- it looks to navigate around absolutely anything else. This feels more aligned with general autonomous vehicles; don't expect a human to move out of the way, err on the side of safety.

Since a bit of this depends on the specifics of the simulation environment, I'll look to exploring the specifics of it a bit more, before further solidifying the design.

COLREGs

Specific maritime rules around navigation. This is basically the crux of how to legally navigate on water.

I'm interpreting this here so I can't be 100% certain on my interpretation. But it's a starting point for a simulated project.

Part A covers Rule 1-3:

• Rule 1 (application): all vessels on the high seas and connected navigable waters. Local authorities may make special rules for harbours, rivers and inland waters, conforming as closely as possible; navies may use additional lights and signals.
• Rule 2 (responsibility): nothing excuses neglecting the rules or ordinary seamanship; depart from the rules when necessary to avoid immediate danger.
• Rule 3 (definitions): vessel, power-driven, sailing, fishing, seaplane, WIG craft, not under command, restricted in ability to manoeuvre, constrained by draught, underway, in sight, restricted visibility.

Part B covers three sections:

• Section I Conduct of Vessels in any Condition of Visibility (Rule 4-10).
• Section II Conduct of Vessels in Sight of One Another (Rule 11-18).
• Section III Conduct of Vessels in Restricted Visibility (Rule 19).

In terms of developing an autonomous application on VRX, Part B is of particular relevance:

• Rule 5 (lookout): maintain a proper lookout by sight and hearing and all available means.
• Rule 6 (safe speed): speed appropriate to visibility, traffic, manoeuvrability, sea state.
• Rule 7 (risk of collision): use all available means, including radar; assume risk exists if in doubt.
• Rule 8 (action to avoid collision): action must be positive, made in ample time, and large enough to be readily apparent to the other vessel. Small incremental corrections are explicitly discouraged.
• Rule 9 (narrow channels): keep to the starboard outer limit; vessels under 20 m, sailing vessels and fishing vessels must not impede vessels confined to the channel; don't cross a channel if it impedes; overtaking in a channel requires sound-signal agreement; sound one prolonged blast at blind bends; avoid anchoring.
• Rule 13 (overtaking): the overtaking vessel keeps clear, regardless of vessel type.
• Rule 14 (head-on): both vessels alter course to starboard, passing port to port.
• Rule 15 (crossing): the vessel with the other on her starboard side gives way.
• Rule 16 (give-way vessel): take early and substantial action to keep well clear.
• Rule 17 (stand-on vessel): hold course and speed — but may act if the give-way vessel clearly isn't, and must act if collision can't be avoided by the give-way vessel alone.
• Rule 18 (pecking order): power-driven gives way to sailing, sailing to fishing, and everyone to vessels not under command or restricted in ability to manoeuvre.
• Rule 19 (restricted visibility): no stand-on/give-way distinction; every vessel proceeds at safe speed and avoids altering course to port for a vessel forward of the beam.

I think Rule 19 can be put aside for now. In a simulated environment, fog, smoke, rain, and other natural phenomena that restrict visibility can largely be controlled. I'd like to return to this later on; for the early stage, I think I'll go with high visibility in daylight conditions.

Part C is "Lights and Shapes", and Part D "Sounds and Light Signals". Using the provided vessel model, I'll want to check that it complies; and I might need to consider how to adopt them for identifying other vessels.

To begin with, I'm going to set some simulation constraints to allow adherence to the rules without having to account for absolutely everything:

Phase 1:

• Own-ship only.
• Static obstacles only.
• Open water, away from any channel or marked lane.
• Rules 2, 5, 6, 8 are in force.

Phase 2:

• Add another vessel.
• Same class as USV.
• Behaves as a compliant stand-on vessel.
• USV must give way.
• Rules 7, 13, 14, 15, 16 are added.

Phase 3:

• Add another vessel.
• Same class as USV.
• Behaves as the give-way vessel.
• USV must stand-on.
• Rule 17 added.

Phase 4:

• Add multiple vessels.

Phase 5:

• More complicated environments/visibility.

VRX 2023 Competition

I was just initially intending to just create a general autonomous USV. But looking further into VRX, the 2023 competition has a series of specific tasks.

Per the VRX 2023 Wiki:

• Task 1: Stationkeeping

Navigate to the goal pose and hold station. The best solutions will minimize the difference between the goal pose and the actual pose of the vehicle over the duration of the task.

• Task 2: Wayfinding

Navigate through each of the published waypoints, such that vehicle achieves, as closely as possible, the positions and orientations specified.

• Task 3: Perception

In this task, the vehicle remains in a fixed location and markers will appear in the field of view. The objective is to use perceptive sensors to identify the markers and report their locations.

• Task 4: Acoustic Perception

An underwater acoustic beacon broadcasts range, bearing and elevation indicating its position relative to the USV, with noise. The objective of the task is to navigate to the beacon (within 1 m) as quickly as possible.

• Task 5: Wildlife Encounter and Avoid

This task requires the system to track a heterogeneous set of moving animals representing animal life and plan an appropriate action according to the animal type. The system should plan and traverse a path that circles clockwise around platypus markers, circles counterclockwise around turtle markers, and avoids (i.e. remains at a distance of 10m from) crocodile markers.

• Task 6: Follow the Path

This task requires the system to traverse a channel marked by pairs of colored buoys, while avoiding obstacles.

• Task 7: Acoustic Tracking

In this task, the vehicle will track a moving underwater acoustic beacon while avoiding obstacles.

• Task 8: Scan and Dock and Deliver

Detect the dock and execute a controlled docking maneuver in the appropriate gate. The system should detect the color sequence emitted by the scan-the-code buoy, as this color sequence dictates the correct docking gate. Additional points will be awarded for vehicles that can successfully propel a projectile through one of the two holes in the placard at the head of the correct docking bay.

I think these make a nice set of goals to individually aim for and solve, mitigating scope.

Now to work out what I want the scope of this project to be.

Scope

There's a lot of directions I could take here.

I'm going to develop it in modular form, so I can iterate on it later etc.

v0.1 - Control stack on VRX

• Localisation: GNSS + IMU fusion via robot_localization.
• Guidance: line-of-sight, producing heading/speed setpoints.
• Control: PID heading and speed tracking.
• Allocation: force/moment to left/right thrust, with saturation and rate limits.
• Mission: executive holding the current objective (waypoints, stationkeeping), exposed as ROS 2 actions.
• Interfaces: message definitions; thruster hardware abstraction with a VRX backend.
• Infrastructure: Docker dev container, CI running colcon test.
• Benchmarks: VRX stationkeeping and wayfinding tasks, scored.

v0.2 — Model-based control

• System identification of the WAM-V 3-DOF model against VRX hydrodynamics.
• Control: NMPC via Acados (acados_vendor_ros2), replacing guidance + PID behind the same interface.
• Re-run benchmarks; quantify the delta.

Later

• Hand-rolled EKF replacing robot_localization.
• Perception (LiDAR/camera), obstacle avoidance as MPC constraints.
• COLREGs behaviours, multi-vessel encounters.
• Real thruster backend (PWM/CAN).

Development

v0.1 Plan

• Stand-up local environment.
• Build VRX and verify runs in Gazebo.

Packages:

• helm_msgs
• helm_localisation
• helm_guidance
• helm_control
• helm_allocation
• helm_hardware
• helm_mission
• helm_bringup

And a ThrusterInterface (VRX backend). This could theoretically be replaced by something real in the future.

Dev Notes

• Environment stood up.
• vrx_gz built via vrx_ws/build.sh
• Runs in Gazebo via vrx_ws/launch.sh
• Package skeletons created - skeleton.sh
• Basic messages: Setpoint.msg, ThrustCommand.msg
• Basic actions: FollowWaypoints.action, HoldStation.action

WAMV Thrusters:

• Topics: /wamv/thrusters/{left,right}/{pos,thrust}
  • data:float64
  • Test: ros2 topic pub /wamv/thrusters/left/thrust std_msgs/msg/Float64 "{data: 300.0}" -r 10
    • Success!
  • Writing VRX Thruster
  • Actually hold up. Better commit now.
  • NOW writing VRX Thruster.
    • Intellisense is having a fit. Might be time to VS Code into the container.
• VRX Thruster Node
  • Pubs to WAMV's thruster topics
    • Pos currently unused; look into later.
  • ros2 run helm_hardware vrx_thruster_node
  • ros2 topic pub /thrust_command helm_msgs/msg/ThrustCommand "{left: 300.0, right: -300.0}" -r 10
  • Works well enough for now.

Localisation:

• Using robot_localization for now.
• Having a squiz at the topics for relevant stuff: ros2 topic list | grep -E 'gnss|gps|imu|pose'
  • /wamv/pose
  • /wamv/pose_static
  • /wamv/sensors/gps/gps/fix
  • /wamv/sensors/imu/imu/data
• ros2 topic echo --once /wamv/sensors/imu/imu/data:

header:
  stamp:
    sec: 3145
    nanosec: 580000000
  frame_id: wamv/wamv/imu_wamv_link/imu_wamv_sensor
orientation:
  x: 0.00022668219081110075
  y: 0.005029954415099406
  z: -0.7931858312613371
  w: 0.6089588535032785
orientation_covariance:
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
angular_velocity:
  x: -0.02125
  y: -0.0005
  z: -0.67
angular_velocity_covariance:
- 8.099999831756577e-05
- 0.0
- 0.0
- 0.0
- 8.099999831756577e-05
- 0.0
- 0.0
- 0.0
- 8.099999831756577e-05
linear_acceleration:
  x: -0.035
  y: -0.08
  z: 9.905
linear_acceleration_covariance:
- 0.00044100001105107367
- 0.0
- 0.0
- 0.0
- 0.00044100001105107367
- 0.0
- 0.0
- 0.0
- 0.00044100001105107367

• ros2 topic echo --once /wamv/sensors/gps/gps/fix:

header:
  stamp:
    sec: 3223
    nanosec: 652000000
  frame_id: wamv/wamv/gps_wamv_link/navsat
status:
  status: 0
  service: 0
latitude: -33.722444715429596
longitude: 150.6736713061667
altitude: 1.268996267579496
position_covariance:
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
position_covariance_type: 0

• GPS is published at about 19 Hz per ros2 topic hz /wamv/sensors/gps/gps/fix. More than enough.
• Installing ros-jazzy-tf2-tools; adding to base image.
• ros2 run tf2_tools view_frames. Quick squiz of the frames.
• Found /ws/vrx_ws/src/vrx_urdf/wamv_gazebo/urdf/components/wamv_imu.xacro
  • Interesting; looks like ENU. Thought NED was maritime convention, but doesn't really matter.

/ws/vrx_ws/src/vrx_urdf/wamv_gazebo/urdf/components/wamv_imu.xacro:104:          <orientation_reference_frame>
/ws/vrx_ws/src/vrx_urdf/wamv_gazebo/urdf/components/wamv_imu.xacro:105:            <localization>ENU</localization>
/ws/vrx_ws/src/vrx_urdf/wamv_gazebo/urdf/components/wamv_imu.xacro:106:          </orientation_reference_frame>

• Mapping to compass heading: heading = pi/2 - yaw_enu
• Controller needs boats position; heading and velocity in local Cartesian frame
• Copying ekf.yaml, navsat_transform.yaml from $(ros2 pkg prefix robot_localization)/share/robot_localization/params/
  • Few minor changes. Will likely need tweaking. Let's see what it gives.
• Oddity with thrusters. Only one seems to be powered. Interesting.
  • As an aside: seems WAMV thrust is in Newtons; napkin math max is around 2353.
  • Seems to be a type conversion error occurred somewhere. Must've been something dodgy I passed it in the cli.
  • Cold start, works.
• Basic localisation functional.
• Putting that aside for a second.

Control:

• Basic PID to start with.
  • Discrete, not simulating continuous here.
• What does the controller want?

Allocation:

Two controls and path-following control: It is standard procedure to define a 2-D workspace (along-track and cross-track errors) and minimize the cross-track error by means of an LOS path-following controller; see Sections 10.3–10.4 and 12.2.8–12.2.9. Hence, it is possible to follow a path by using only two controls (surge speed and yaw moment). For a conventional ship this is achieved by using a rudder and a propeller only.

• Fossen: Thrust Configuration
  • This is a truly fascinating read.
  • τ = T(α)f = T(α)Ku (12.229)
  • T(α) = [t_1 , . . . , t_r] (12.230)
  • Assuming thrusters are fixed for now: α = α0 = constant; T = T(α0) (12.243)
  • τ = T(α)Ku = T(α)Ku -> Ku = τT^-1

So...

• Making it a bit easier for coding: u = tau x T^-1.
• Assuming fixed thrustybois:
  • No sway force; thruster can't push sideways: F_y = 0.
  • Surge force; entire thrust is along x: F_x = u_i.
  • Yaw moment

Alright that's another maths for today. Very interesting tbc, tbc.

References

• Fossen, 2011, "Handbook of Marine Craft Hydrodynamics and Motion Control", somethingth edition, some publisher.
• "Station-keeping control of an unmanned surface vehicle exposed to current and wind disturbances" (10.1016/j.oceaneng.2016.09.037)
• Fossen's Lecture Material for TTK4190