1 Kings 1:45

“And Zadok the priest and Nathan the prophet have anointed him king in Gihon: and they are come up from thence rejoicing, so that the city rang again.” Problem Restatement:

We have a binary‑classification model that produces a continuous confidence score \(s\in[0,1]\) for each instance. The model is evaluated on a test set that is split into three subsets with the following sizes and desired positive‑rate quotas:

• Training size \(p\), desired positive rate \(k/p\)
• Validation size \(q\), desired positive rate \(k/q\)
• Test size \(n\), desired positive rate \(k/n\)

All test‑set elements are ordered by descending confidence. We must assign a binary label \(a_i\in\{0,1\}\) to each element \(i\;(1\le i\le n)\) such that:

• Positive‑rate constraints \(\displaystyle\sum_{i=1}^{p} a_i = k\frac{p}{n},\quad \sum_{i=1}^{q} a_i = k\frac{q}{n},\quad \sum_{i=1}^{n} a_i = k\)
• Ordering constraint \(a_i \ge a_{i+1}\) for all \(i\)

The objective is to minimise the total number of label flips (positions where the binary label differs from the previous one). Because of the ordering constraint, feasible labelings consist of a single transition from 1 to 0.

Mathematical Formulation:

Introduce the cumulative label function \(C(i)=\sum_{j=1}^{i} a_j\). The ordering constraint implies the existence of a cut‑off index \(t\) such that:

• \(a_i = 1\) for \(i\le t\)
• \(a_i = 0\) for \(i>t\)

All constraints can be rewritten in terms of \(t\):

• Training block lower bound \(t \ge \displaystyle\frac{k\,p}{n}\)
• Validation block lower bound \(t \ge \displaystyle\frac{k\,q}{n}\)
• Overall count \(t = k\)

Because \(t\) must be an integer, the feasible set is

\(t \in \Bigl\{\,\bigl\lceil \max\bigl(\tfrac{k p}{n},\tfrac{k q}{n}\bigr)\bigr\rceil\ \bigr\}\).

The training block over‑fills its quota because the lower‑bound interpretation is used; exact equality for all three blocks is possible only when the quotas are mutually consistent, i.e.

\(\displaystyle k\frac{p}{n}\le k\frac{q}{n}\le k\) and \(k\frac{p}{n},\,k\frac{q}{n},\,k\) are integers.

When consistency holds, the cut‑off \(t^{\star}\) yields a single flip, which is optimal.

Sensitivity:

The cut‑off index depends linearly on \(k\): \(t^{\star}(k)=\bigl\lceil \max(\tfrac{k p}{n},\tfrac{k q}{n})\bigr\rceil\). Updating the labeling after a change in \(k\) requires only this arithmetic operation ( \(O(1)\) ).

Implementation Sketch (Non‑Code):

• Step 1 Compute the required overall positive count \(k\) (if known) or treat it as a design variable.
• Step 2 Evaluate the lower bound \(L = \max\!\bigl(\tfrac{k p}{n},\tfrac{k q}{n}\bigr)\).
• Step 3 Set the cut‑off index \(t = \lceil L\rceil\).
• Step 4 Assign labels \(a_i=1\) for \(i\le t\), \(a_i=0\) otherwise.
• Step 5 (Optional) Verify the three count constraints by summing the first \(p\), first \(q\) and all \(n\) entries.
• Step 6 Report the coefficient table (step function) and the number of flips (= 1).

All operations are scalar arithmetic; no optimisation library is required.

Discussion of Trade‑offs & Extensions:

• Exact vs. relaxed labeling The current formulation forces a strict step‑function (binary) guaranteeing a single flip. A linear‑programming relaxation would produce a fractional ramp that, after rounding, could introduce multiple flips.
• Multiple flip allowance Disallowed by the ordering constraint. Relaxing monotonicity leads to a binary knapsack with a flip‑penalty, solvable by dynamic programming or mixed‑integer linear programming.
• Robustness to noisy confidence scores The solution uses only the ranking, not the score magnitude. Adding a secondary objective to maximise the sum of confidences of chosen positives pushes the cut‑off as far left as possible, yielding a lexicographic optimisation.
• Scalability Current solution runs in constant time for any \(n\). Introducing additional constraints (e.g., per‑group quotas) would require a full integer program and a solver.
• Verification Simple arithmetic checks of the three cumulative counts can be embedded as unit‑tests in a production pipeline.

Potential Pitfalls & How to Avoid Them:

• Non‑integer quotas The quotas \(k\frac{p}{n}\) or \(k\frac{q}{n}\) may be fractional. Round the quota up for the lower‑bound constraint and down for the upper‑bound constraint, or use a relaxed LP and then round the solution while checking feasibility.
• Inconsistent quotas If \(k\frac{p}{n}>k\) (or similarly for \(q\)), the problem is infeasible. Verify the consistency condition \(\frac{p}{n}\le1,\ \frac{q}{n}\le1\) before solving.
• All positives or all negatives The “single‑flip” objective becomes degenerate (0 flips). Detect this edge case early and return the trivial labeling.
• Tie‑breaking when scores are equal Apply a deterministic tie‑breaker (e.g., original index order) before computing the labeling.

Summary of the Final Solution:

• Compute the overall number of positives \(k\) (or treat it as a design parameter).
• Determine the minimal feasible cut‑off \(t^{\star}= \bigl\lceil \max(\tfrac{k\,p}{n},\tfrac{k\,q}{n}) \bigr\rceil\).
• Label the ordered list \(c_i = 1\) for \(i\le t^{\star}\), \(c_i = 0\) for \(i> t^{\star}\).
• Validate that the three cumulative counts meet the required quotas (or exceed them if only a lower bound is imposed).
• Report the coefficient table (step function) and the number of flips (= 1).

The algorithm runs in constant time and satisfies the required constraints whenever they are mutually consistent. It provides a minimal‑flip labeling that respects the ordering of confidence scores and the prescribed prefix‑set quotas.