The first version of my LinkedIn company-jobs Actor looked like it worked. Point it at a company, get back every open role. The dataset had hundreds of rows, the fields were populated, the run finished green.
It was returning about three quarters of the jobs, and a different three quarters each time.
Worse, I had built change tracking on top of it — "these 12 roles opened since your last run, these 5 closed" — and that feature was manufacturing closures out of nothing. One test run reported 113 jobs closed and 117 opened, 75 seconds apart.
Nothing in the output distinguished this from a correct answer. That is the specific failure I want to write about, because "did my scrape actually finish?" turns out to be a question you can measure, not just hope about — using a method that ecologists use to count fish.
The public endpoint takes a start offset. I assumed start=0 gives you jobs 1–10, start=10 gives you 11–20, and so on.
I ran the same query twice and compared the returned job IDs: 3 pages deep: 33% of the IDs were different 10 pages deep: 61% of the IDs were different
So a straightforward loop — walk start from 0 upward until a page comes back short — does not collect a company's roster. It collects a sample of it. On a small company the sample happens to be everything. On a large one it isn't, and nothing tells you which case you're in.
The obvious one: change tracking. If run A samples 660 of 880 and run B samples a different 660, roughly 220 IDs are in A and not in B. Those get reported as closed. They were never closed. They were never absent — they just weren't in this draw.
The one I nearly missed: the basic listing was wrong too. Not the fancy feature — the core product. I was selling "every open role at this company" and shipping a list missing a quarter of it, with no indication that anything was missing.
My first attempt at "am I done" was the obvious heuristic: stop when four consecutive requests add nothing new.
Here's why that's not a measurement. Say the company has 700 roles and I currently hold 650. Each request returns 10 roles drawn from the pool. The chance that all ten are ones I already have:
Nearly half. Four in a row happens about 5% of the time at that exact point — and more often as you get closer. So the rule does fire eventually. It just fires at a completeness level that depends on the fraction you already hold, which is precisely the unknown you were trying to determine. The stopping rule is circular: it tells you you're done by assuming you're nearly done.
My second attempt went the other way: collect the roster twice and only accept it if the two passes match exactly. Rigorous, and useless. Two random samples of a large pool are essentially never identical. The condition never fired, so the feature never produced a number, so I had built an elaborate way of saying "I don't know."
The thing that unstuck me was reframing what those repeated draws are. They aren't retries. They are samples.
Ecologists count fish in a pond by catching some, tagging them, releasing them, then catching a second batch and seeing how many carry tags. Few tags means a big pond. That's Lincoln–Petersen:
I already had two samples — I just had to stop merging them into one bucket. So the Actor now splits its own requests into two sets, tracks them separately, and estimates the total from their overlap:
Then coverage is just what I hold divided by what I estimate exists, and "how many am I still missing" falls out of the same number.
Here is a real run against a company page, 800 requests, recording the unique jobs held at each point: