What to ask about one accuracy number: denominators and units for missed defects and false rejects, defect sample size, throughput, IntLearn Pro reports.
How should missed defects, false rejects and throughput be checked together when testing AI inspection?
A single accuracy number cannot decide an adoption. Look at missed defects, false rejects and throughput together, on the same test samples, the same decision criteria and the same evaluation unit, and write the denominator and sample size beside them. Before the test starts, agree on what counts as one item and where a defect begins.
If a test result comes back as a single number, "accuracy of X percent", ask four questions before copying it into a table.
Without these four answers, the same test can produce entirely different numbers. That is less because anyone is hiding something than because people move on without realizing they are counting different things.
Some places divide the false-reject rate by the total quantity inspected. That is not wrong, but with different denominators two results cannot be set side by side.
On the evaluation screens of IntLearn Pro, the same concepts appear under other names. Anomaly detection (IntAnomaly), which learns from normal images only, shows the missed-defect rate as FNR (MISS) and the false-reject rate as FPR (OVERKILL), and uses AUROC to show how well it separates normal from defective. The report for detection (IntDetection), which finds locations, uses precision (the share of what it reported finding that was correct) and recall (the share of actual targets it found). Subtracting recall from 1 gives the missed-defect rate in that unit. Precision, however, has "what was judged defective" as its denominator, so it cannot be read as a false-reject rate.
The same test gives different numbers depending on the unit counted.
Suppose one product is imaged from four directions. If only one of the four images of a good part is wrongly judged defective, that is one false-reject image out of four when counted per image, and one false-reject product when counted per product. That is why false rejects look better when counted per image. Shipping is decided per product, so the final comparison should be per product too, and the unit must always be written in the results table.
Make the decision criterion stricter and missed defects go down while false rejects go up. Loosen it and the reverse happens. In IntLearn Pro this criterion is the Confidence Threshold in detection or the decision threshold in anomaly detection, and the manual's advice is short: raise it if there are many false rejects, lower it if a lot is being missed. One threshold cannot reduce both at once.
Which side hurts more is the company's call. For a part where a single defect turns straight into a customer claim, you accept false rejects and reduce missed defects. If false rejects keep stopping the line and the re-inspection staff cannot keep up, you go the other way. That second situation is why IntAnomaly has a separate training scenario called "Low overkill".
So the following results tables are hard to use for an adoption decision: a table with a missed-defect rate but no false-reject rate at the same criterion; a table with a false-reject rate but no count of defect samples; a table with a processing time but no way to tell whether that time includes imaging and the reject signal.
If a test used only good parts, the denominator of the missed-defect rate, the number of actual defects, is 0. The missed-defect rate is then not "0%" but "cannot be calculated".
Even if every defect sample was found, with few samples you cannot say the true missed-defect rate is 0. By the "rule of three" used in statistics, when all n defects are found, the upper limit of the missed-defect rate at the 95% confidence level is roughly 3 ÷ n.
The table above is an example showing the calculation method, not a performance figure for any product. Source: Hanley JA, Lippman-Hand A, JAMA 1983;249(13):1743-1745.
A result of "not one missed" on 30 defects is closer to "the missed-defect rate probably does not exceed 10%". The harder defect samples are to collect in your process, the more you need to keep this difference in mind when reading the number.
They happen without anyone intending them. When preparing a test, check these five first.
Remember, too, that the ratio of good to defective parts in test samples differs from real production. How many false rejects will occur on the line has to be recalculated using actual production volume.
Illustrative example: if you inspect 10,000 parts a day and the per-product false-reject rate is 1%, about 100 parts a day must be rechecked by hand.
In the end, the false-reject rate is re-inspection staff time. When you set a missed-defect target, set a false-reject target with it.
The inference time for one image is only part of throughput. The 2.47ms on the IntLearn Pro product page should be read the same way. It is an average under one condition (segmentation engine, per 1024×1024 image, KIPT test), and one cycle on a line adds imaging, image transfer, preprocessing, merging results from several cameras and the reject signal.
Do not look only at the average; look at the slowest case. The reject device has to receive its signal within a set time, and one late signal lets a defect straight through. Write roll materials as line speed (m/min) and width, discrete parts as parts per minute, and do not mix the two units. Also check that the image size and number of cameras in the test match the actual installed configuration.
It may seem that the numbers are fixed once the test is over, but in practice changing the threshold just once changes both missed defects and false rejects. If you changed a threshold or retrained in IntLearn Pro, you must export the .dx model file for the line again and build a new inspection package (.izpkg) for the change to reach the line. If the numbers you saw on screen differ from the results on the line, check this first.
When you change the decision criteria, record who changed them, when and why, and check again with samples. If the golden-set gate is on when you export an inspection package, a package whose decisions on the reference images have shifted is not exported. The deployment registry records each exported package with its SHA-256 and keeps the history of where it was applied and rolled back by site, so if the numbers differ one day you can trace what changed that day.
How to choose the samples for a test is covered in the injection-molded sample preparation guide.
The tables and example numbers in this article are there to show calculation methods and are not the performance of INTELLIZ products. Only the 2.47ms (segmentation engine, per 1024×1024 image, KIPT test) is an external test value published on the product page. Test results, too, hold for the samples and conditions tested, so they need to be checked again if a new defect type, a new color or a change in lighting appears.