Finding the four corners of a page in a phone photo is the step that turns a picture of a document into a copy of one. Everything downstream — the perspective warp, the flattening, the PDF — is easy once you know where the page is.
We do that client-side, in a browser tab, on a grayscale copy of the photo scaled to a long edge of 640 px. The pipeline is the textbook one:
Two near-horizontal lines, two near-vertical lines, score every candidate quadrilateral, keep the best. Disclosure up front: this is from LensUp, which is ours. What follows is the part where the textbook pipeline quietly failed, and the one-line change that nearly doubled how often it works.
The adaptive edge threshold is the standard trick. Take the histogram of Sobel magnitudes, pick the 90th percentile, call everything above it an edge. It adapts to lighting, exposure and camera noise for free.
It also has a property nobody mentions: it is set by whatever has the strongest gradients in the image. On a photo of a document, that is not the page border. It is the letter strokes. Printed text is black-on-white at high spatial frequency — it produces the sharpest gradients in the frame by a wide margin.
The paper-to-desk step, meanwhile, is weak. Measured on SmartDoc 2015, a page lying on light wood clears the surrounding surface by roughly 14 luma levels. That is the signal you actually need, and it is an order of magnitude quieter than the text sitting in the middle of it.
So the adaptive threshold, doing exactly what it was designed to do, sets itself by the text and prices the page border out of the edge map. And it does this worst in the case you most want to work: when the user fills the frame with the page, so text dominates the histogram and there is barely any desk left to argue for a lower threshold.
An adaptive threshold with a hard ceiling is no longer fully adaptive, which felt wrong when we wrote it. It is also the single highest-value line in the file.
Capping the threshold gets the page border back into the edge map. It also lets a lot of rectangles in that are not pages — and the most dangerous one is a dark block of printed text, because it is genuinely a bright-surrounded rectangle with strong edges.
The first instinct is a contrast magnitude: require the inside of the quad to be meaningfully brighter than the outside. We had that set at 14 gray levels, which is the median paper-on-light-wood step from above.
That was the wrong knob. A threshold at the median of true pages rejects about half of the true pages. What actually discriminates is not how big the step is but which way it points:
The polarity rule is the actual test: a quad is a page only if it is brighter inside than outside, sampled at two depths, on all four sides. A text block fails that on the sides where more text continues past it. A page on a desk passes it everywhere.
There is still a case that beats local evidence: a large dark region of print can look like "the desk" if you only sample a few pixels out. So before applying a detection automatically — as opposed to just pre-positioning the handles — we probe farther outside, looking for the page continuing past that edge. If the page continues, the edge was not the page edge.
Three of the constants in that file are not computer vision. They are decisions about what to do when we are unsure, and they are the ones I would port to any similar project:
A quad with a side lying on the image border is rejected. If the "page edge" is the edge of the photo, the page is not fully inside the frame, and the honest default is the whole frame rather than a crop that silently cuts off whatever was outside it.
The largest well-supported quad wins. This sounds like a tie-breaker and is actually a correctness rule: it makes an inner rectangle — a photo printed on the page, a bordered table — lose to the page around it.
Below a quarter of the frame, a detection is only a proposal. It pre-positions the four draggable handles and waits. Auto-applying a small quad is how you produce the single worst outcome in this whole category: a confident crop that removes two thirds of somebody's passport.
That last one generalises. The result of this module is a proposal and never the truth; every handle stays draggable, and the caller decides from a confidence value whether to apply it silently. When detection fails outright it returns null, and null means keep the entire original image. A scanner that crops wrong is worse than a scanner that does not crop.