Orion & Diana - Autonomous OSINT Oil Spill Detection

Inspiration

#

In 2024, I was on my national team for the world’s premier robotics tournament, World Robotics Olympiad (WRO). Our challenge then was building a robot that was capable of autonomously cleaning up oil spills.

The real bottleneck wasn’t cleanup. It was detection.

Oil spills happen across millions of square kilometers of ocean every year. The Deepwater Horizon disaster killed 11 workers and dumped 4.9 million barrels into the Gulf of Mexico before responders even knew the scale. The Exxon Valdez took days to be fully understood. Countless smaller spills go unnoticed or are discovered weeks after the fact, when environmental damage is already done.

The detection problem still feels stuck in the 1980s. Environmental agencies still rely on:

• manual coastguard patrols,
• citizen reports and news media,
• periodic satellite imagery review,
• and reactive tasking, where somebody has to decide to look at a specific location at a specific time.

Meanwhile, the ingredients already exist. Sentinel-1 SAR and Sentinel-2 optical satellites image the Earth every few days. Machine learning can detect oil slicks well. Natural language processing can extract location coordinates from news articles in seconds. The hard part is integrating those pieces into one end-to-end system that actually works.

That was the starting point for Orion.

What it does

#

Orion is a marine oil-spill detection platform that combines automated OSINT with satellite imagery.

An autonomous agent named Diana continuously works through maritime reporting, extracts coordinates, and dispatches Sentinel-1 SAR and Sentinel-2 optical scans through Google Earth Engine. The pipeline classifies a full 3x3 spatial grid, combines SAR and optical verdicts, and surfaces the results in a live dashboard.

The part I care about most is that it does not pretend certainty where it does not have it. If one sensor is strong and the other is missing, weak, or structurally unreliable, I would rather mark the case as inconclusive than force a clean false positive.

How I built it

#

The architecture has three layers.

Diana

#

Diana is the autonomous OSINT layer. It is written in Jac and handles the article-side reasoning loop:

• scrape or load maritime incident reports,
• geocode them into coordinates and dates,
• decide whether the report is worth scanning,
• dispatch a scan job into Orion,
• narrate the result back into a dossier-style output.

This is where the project stopped being a one-shot watchdog and started becoming something that could actually run repeatedly.

Orion backend

#

The backend receives coordinates and date windows, orchestrates the imagery fetches, runs classification, and writes manifests for the dashboard.

The first version was a single-target watchdog. It took one rumor string, searched the web, extracted entities, fetched Sentinel-1 and Sentinel-2 imagery, classified both, and wrote a dossier.

The second version became SpatialSAR.

Instead of asking “is there oil at this exact point,” it builds a 3x3 grid around the target and evaluates each valid cell with:

• Sentinel-1 ascending SAR,
• Sentinel-1 descending SAR,
• Sentinel-2 RGB.

That means a single scan can involve up to 27 tile evaluations.

The scoring logic is intentionally simple:

for cell in valid_grid_cells:
    s1_mean = mean(cell.s1_ascending.confidence, cell.s1_descending.confidence)
    s2_score = cell.s2_rgb.confidence
    cell_score = max(s1_mean, s2_score)

winning_cell = argmax(valid_grid_cells, key=lambda cell: cell_score)

I did that on purpose. During a hackathon I wanted something I could inspect directly from manifests and tile outputs instead of building a fusion layer I would not trust anyway.

earth-watchtower frontend

#

The frontend is a React and TypeScript dashboard. I ideated the initial UI direction with Lovable and then modified the actual app behavior myself.

It has:

• a globe view,
• an anomaly feed with severity indicators,
• an inspection panel showing the winning tile and dossier,
• a full 3x3 tile gallery for SpatialSAR runs,
• and an operations panel that dispatches jobs and streams transcript events over SSE.

Without the dashboard, Orion was still a backend plus folders full of outputs. With it, the project finally felt usable.

Why the 3x3 grid exists

#

Satellite revisit gaps were a big deal and broke the naive version.

I kept running into the same issue described in maritime monitoring literature: if you ask for one exact point and one exact time, you can easily miss the slick geometry even when the event is real. The pass may be slightly offset. The reported coordinate may be rough. The oil may not sit exactly where the original report suggests.

So I stopped treating the problem like a single-point question and turned it into a local search problem.

The 3x3 grid approach does three things:

• gives the model a better chance of catching the actual slick geometry,
• exposes ambiguity instead of hiding it,
• and makes the system more robust when one tile is bad but a neighboring one is useful.

I also added land filtering because even a little coastline or land structure can fool the classifier into acting like it found oil.

Sensors and stack

#

• Jac for the agent graphs, walkers, and structured LLM calls
• Python for the backend orchestration and APIs
• Google Earth Engine for Sentinel-1 and Sentinel-2 fetches
• Sentinel-1 SAR for radar imagery
• Sentinel-2 RGB for optical confirmation
• FastAPI for the control plane and job endpoints
• React + TypeScript + MapLibre for the dashboard
• Featherless / Qwen 3 for the reasoning-heavy agent tasks
• A local ResNet classifier for tile-level oil-slick prediction

Challenges I ran into

#

Satellite revisit gaps

#

This was probably the main systems problem. The simple point-query approach was too brittle. The 3x3 search and multi-pass structure was the fix.

Dual-sensor consensus

#

RGB can look convincing while SAR is weak or unavailable. SAR can also be strong where RGB is unhelpful. I did not want to fake a clean answer by blending bad evidence into good evidence. The system keeps both visible and stays willing to call a case inconclusive.

Concurrent imagery fetches

#

Fetching 27 tiles sequentially was too slow, so I parallelized the fetches with a thread pool.

Tooling and workflow complexity

#

The project crosses agent logic, Earth Engine, classifier wrappers, FastAPI, manifest export, and frontend state. That sounds obvious after the fact, but getting all those layers to behave like one system instead of five disconnected demos was most of the work.

Accomplishments I’m actually proud of

#

• End-to-end OSINT plus orbital pipeline instead of a toy classifier demo
• SpatialSAR turning revisit gaps into a search problem instead of a failure mode
• Dual-sensor logic that does not hallucinate confidence when evidence is uneven
• A live dashboard that shows the anomaly feed, tile gallery, and dossiers in one place
• Diana acting as a real OSINT-to-scan dispatcher rather than a decorative chatbot

What I learned

#

• Satellite revisit gaps are real and they break naive spill detection fast.
• LLM dossier writing is useful when it explains uncertainty instead of pretending certainty.
• Jac is genuinely nice for this kind of agent control flow because the graph structure matches the workflow.
• Dual-sensor thinking should be standard here. I started out leaning too hard on radar, then realized the RGB side was often carrying more of the useful signal than I expected.

Media

#

Final demo video for Orion and Diana, covering the autonomous article-to-scan loop, SpatialSAR dashboard workflow, tile gallery, and dossier flow.

Limitations

#

• The consensus logic is intentionally simple.
• The classifier can still be confidently wrong on ugly edge cases.
• Some Sentinel-2 imagery has awkward black wedges because of real scene footprints.
• The geographic reasoning guards are pragmatic sanity checks, not elegant geospatial reasoning.
• Google Earth Engine cost and fetch volume become a real issue if this is run continuously at scale.

What’s next

#

Technically this is not just an oil-spill system. It is an OSINT plus satellite verification pattern that could be adapted for defense, intelligence, deforestation, ice monitoring, and other climate or geospatial search problems with changes to the fetch and classification layers.

The practical next steps are:

• let Diana run much longer and see what it surfaces,
• add alerting instead of keeping it dashboard-only,
• connect detections to official reporting databases,
• and make the long-run deployment story cleaner and cheaper.

That is the part I find most interesting now. The core loop works. The open question is what happens when it is left alive long enough to become useful.