Unbiased pass@k
A model produced n samples for a task and c of them passed. passAtK(n, c, k) estimates the probability that at least one of k samples drawn from the n passes, using the unbiased estimator
pass@k = 1 − C(n − c, k) / C(n, k)
Return 1 when n − c < k (every draw of k must include a pass). Compute it as a product, not with factorials — n can be large. Results are compared after rounding to six decimals.
passAtK(10, 3, 1) -> 0.3 passAtK(10, 3, 5) -> 0.916667
Hints
Hint 1
C(n − c, k) / C(n, k) is the chance that k draws without replacement all fail. Written as a product it is Π over i from 0 to k − 1 of (n − c − i) / (n − i).
Hint 2
Handle n − c < k before the product: some factor would be zero or negative there, and the answer is 1.