Auro-Darshan: Every Gap Comes Back as a Label

Ten agents on one vision model sorted 4,900 hectares of Auroville into 33 land-cover classes. Where they had not been given enough to decide, they labeled anyway, and nothing in the output said so.

The previous piece told how those agents finished the map. This one is about why the last stretch was hard. The map before them was k-means over embeddings. It could sort land into piles; it could not be told what any pile meant.

Sat

Detail

80%

Legend

+−

Leaflet | © OpenStreetMap contributors

Loading land cover data...

Definitions Have to Come From Somewhere

What the vision model that replaced hand labeling added was not better eyes. For the first time, a method in this project could take instructions. Class definitions went into the prompt.

Ten readers, in parallel, each returned a label for a cluster's sample cutouts, and adversarial verifiers challenged what came back. It ran in rounds, each re-reading the clusters the last could not settle, now split finer.

The first time definitions had to be typed out, @restlessronin asked the obvious question, "Can we not look it up?" NRSC's national scheme was the place to start: fallow became land taken up for cultivation and rested a year or more, and the definition asks for positive evidence of cultivation, field geometry, bunds, plough lines, and for the absence of a crop. A bare rectangle on its own is not enough; the test is a paragraph.

Silence Comes Back as a Name

Nobody had written down what counts as evidence where nothing is obviously growing. On such ground the instructions steered readers to one catch-all, and the catch-all moved with the instructions: block grazing_land, and maintained_grass takes it; constrain that, and fallow does.

Roadside tree strips had no class of their own and went to fallow too. tree_lines arrived on 9 August, and a week later readers could say that no class fits. Both fired.

Eleven records raised the refusal, four on grazed village commons. Cluster c123 is the clean case: the reader wrote "closer to grazed commons than to a rested field", raised the refusal, and then labeled the cluster fallow, because grazing_land was blocked from its list. It saw the gap, said so, and gave the nearest label as required.

grazing_land was retired on 31 August, not assignable from imagery. A reader can refuse only a gap it can see.

What Never Reached the Reader

Two things. Each came back as an ordinary label.

The surroundings. Readers began with the patch itself and nothing around it; a 200 m window came later. The new definitions turn on what is around the thing: "amid casuarina", "an opening in canopy". Open cluster c174 at 200 m: a plain green field, and fallow is a fair read. Pull back to 800 m, same center: one parcel in a dense farmed mosaic beside a highway, harvested casuarina blocks next door. Readers now get 800 m as well. A rule applies only to what is in frame.

The season. Readers voted cluster c125 fallow: a dry irrigation-tank bed, pale and flat. The map this started from, built from a year of embeddings rather than one photograph, had it 65 percent water. A year sees the tank full; one photograph never does. The map's labels are now tabulated against every cluster, and where it says water, it is right.

Every fix sat upstream of the readers: a wider frame, a second map nobody asked them to read.

What the Human Is For

In late August @restlessronin read the readers' descriptions closely. They described what they saw more accurately than he had expected, and classified better than he could.

On cluster c67 he called a patch built_environment on the evidence of a red access track. The reader called it planted_forest at confidence 0.40, and the verifier upheld it. The definition of built_environment rules him out twice: too little artificial surface, no roofs through the trees. The readers had the definitions open; he did not.

With the definitions open he would be a third reader, not a second opinion. What is his alone lies outside them. He knows the ground: which tank fills, which commons are grazed. He decides whether two classes can be told apart from imagery at all, which is how grazing_land was retired. And he writes the rulings the readers then work from: a new class, a wider frame, an older map's water.

What was missing could only be said, not seen.