Neural-spline optimization framework for robust document image binarization

Abstract

This paper presents Neural-Spline Optimization (NSO), a structured learning framework for document image binarization in which a compact tensor-product B-spline surface replaces a fixed analytic fusion function within an explicit thresholding rule. Instead of learning a dense pixel-to-label mapping from scratch, NSO learns a low-dimensional decision surface over page-level intensity descriptors while preserving the analytical structure of the underlying binarization operator. The paper is focused on the computational and methodological aspects of learning this surface, and on the manner in which controlled adaptivity should be introduced around it. Three closely related variants are considered within a common formulation: direct global spline optimization, spline-guided pixel-domain refinement, and bounded residual learning in control-point space. The main new contribution is the residual control-point variant, which introduces data-driven updates directly in the spline parameter space while preserving a single global and exportable decision surface. The mathematical formulation includes spline construction, differentiable relaxation of the hard decision rule, structural objective design, and deployment-oriented regularization. The experimental analysis is concentrated on optimization behaviour, interpretability, and structural changes under a fixed public protocol. The resulting framework provides an interpretable and computationally compact alternative to purely pixel-based deep binarization models, with a balanced trade-off between accuracy, robustness, and deployment simplicity rather than uniform maximization of all evaluation measures.

Keywords

Document image binarization b-spline surface structured optimization interpretable learning metric-driven learning

1. Introduction

Document image binarization converts a grayscale document into a foreground–background representation suitable for optical character recognition, layout analysis, archival restoration, and subsequent document understanding. The task remains challenging when documents contain non-uniform illumination, bleed-through, stains, faded ink, textured paper, or scanning artefacts. Classical global thresholding methods are often too rigid for such degradations, whereas local adaptive methods rely on assumptions that need not be valid uniformly across the page.^1,2 Recent deep-learning systems, including convolutional, adversarial, and gated architectures, achieve strong benchmark performance on degraded document images.^3–9 However, these systems typically depend on dense pixel-level prediction, high-capacity neural architectures, and only limited interpretability.

A useful observation is that many successful binarization rules are not arbitrary pixel mappings. They are explicit decision operators that compare local statistics with thresholds modulated by page-level characteristics. Such rules have interpretable parameters, deterministic behaviour, and low deployment cost. Nevertheless, when supervised data are available, structured rules are often replaced by end-to-end convolutional predictors. This paper follows the opposite direction: the decision rule is retained explicitly, and only a compact decision manifold that parameterizes it is learned.

The starting point is a prior public statistical-fusion formulation for document binarization,¹⁰ in which a fixed analytic logistic-type function combines page-level text and background intensity descriptors and modulates a thresholding rule. That prior work also introduced the MCHAM metric and reported external benchmarking on public DIBCO datasets. The present manuscript does not claim those contributions as new. Its original contribution is computational and methodological: the fixed analytic fusion function is replaced by a learnable tensor-product B-spline surface, and the learning process is arranged so that the final decision logic remains explicit, auditable, and exportable.

Within this framework, three closely related NSO variants are studied. Model I optimizes the spline control-point heights directly and does not use a neural network. Model II retains the spline as a global prior but introduces a spline-guided U-Net for pixel-domain refinement. Model III, which is the main new contribution of this paper, predicts bounded residual updates directly in control-point space. This residual strategy preserves a single global decision surface while enabling controlled adaptation and faster convergence. The central scientific question is therefore not whether a larger network can improve binarization, but where adaptivity should be introduced when the objective is to preserve structured decision semantics.

The contribution of this paper can be summarized as follows.

A structured learning formulation for document image binarization in which the core decision rule remains explicit and only the fusion surface is learned as a low-dimensional tensor-product B-spline manifold.

A unified mathematical treatment of surface construction, differentiable relaxation of the hard binarization operator, structural objective design, and deployment-oriented regularization.

A systematic comparison of three model families that differ only in the point at which adaptivity is introduced: direct spline optimization, spline-guided pixel-domain refinement, and bounded residual learning in control-point space.

A residual control-point strategy that performs neural adaptation directly in spline parameter space while preserving one global, interpretable, and exportable decision surface.

A self-contained explanation of how the present contribution extends the earlier public statistical-fusion baseline and how the same public benchmark protocol is reused to isolate the incremental effect of learnable spline-surface optimization.

Accordingly, the purpose of the proposed framework is to study how explicit decision surfaces can be optimized while preserving interpretability and deployment simplicity.

The remainder of the paper is organized as follows. Section 2 positions the work in relation to document binarization, structured learning, and spline-based modeling. Section 3 states explicitly how this manuscript differs from the earlier public statistical-fusion work. Section 4 defines the problem and the page-level descriptors. Sections 5 and 6 introduce the spline decision surface and the corresponding binarization operator. Section 7 summarizes the evaluation criteria and the training-time relaxations. Section 8 presents the three NSO variants, and Section 9 gives the associated training and inference procedures. Optimization dynamics and interpretability are discussed in Section 10. Section 11 describes the public benchmark protocol, implementation details, and sensitivity analysis. Section 12 analyzes the results through pixel-level structural changes and qualitative comparisons. Section 13 discusses strengths, limitations, and practical implications, and Section 14 concludes the paper.

2. Related work

Document image binarization has progressed from global thresholding through local adaptive methods to hybrid and deep-learning formulations. In global approaches, one threshold is estimated from the full image histogram, as in Otsu-type methods. Such techniques are attractive because they are simple and deterministic, but they are highly sensitive to background variability. Local adaptive methods such as Sauvola-style, Niblack-style, Nick-style, Su-style, and regression-based variants estimate a threshold from neighborhood statistics and therefore better tolerate illumination changes, yet they often depend on window size, heuristic parameters, and assumptions about local stationarity.^1,2,11,12 Recent reviews emphasize that strong degradations, bleed-through, and cross-domain generalization remain unresolved challenges despite decades of progress.¹³ A large body of work on document image binarization has been developed and evaluated within the ICDAR and IJDAR research community, particularly through the DIBCO and H-DIBCO competitions.^14–23 These benchmarks have driven the development of both classical and learning-based methods, including adaptive thresholding techniques, hybrid statistical models, and more recently fully convolutional and gated architectures that achieve state-of-the-art performance on degraded documents. For example, fully convolutional approaches have demonstrated performance exceeding competition-winning methods on multiple DIBCO datasets, highlighting the effectiveness of dense pixel-wise learning for binarization tasks.

Learning-based binarization methods treat the task as image restoration or segmentation. Convolutional and adversarial models such as DeepOtsu, CT-Net, DE-GAN, and GDB have shown strong benchmark performance, especially on degraded historical documents.^3,4,6,8 Earlier convolutional formulations also demonstrated the usefulness of learned document-specific features and restoration-oriented training.⁵ Additional hybrid directions combine classical preprocessing cues with learned refinement, for example through multichannel inputs or restoration-oriented pipelines.^7,24,25 The prior statistical-fusion study¹⁰ belongs to the structured, non-end-to-end side of this landscape: it retained an explicit analytical rule, introduced the MCHAM metric, and benchmarked the fixed logistic fusion formulation on public DIBCO data.

A second line of relevant work concerns structured and interpretable learning. In this literature, the goal is not only to improve predictive quality but also to preserve explicit constraints, meaningful parameters, and auditable behaviour. Hybrid analytical–learning systems, constrained optimization strategies, and rule-guided visual models have been explored in integrated engineering contexts.²⁶ Early work by Hung²⁶ demonstrated that neural learning can be coupled with structured optimization rather than treated as pure unconstrained backpropagation. Related studies investigated geometric and local-decision mechanisms in probabilistic neural models,²⁷ as well as fuzzy and logic-oriented interpretable neural systems.²⁸ These studies motivate the idea that learning can improve a system without fully replacing its explicit decision structure.

Spline-based models are relevant because B-splines provide smoothness, compact support, and low-dimensional parameterization. Spline representations have long been used in geometric modelling, approximation, and parameter fitting, including low-dimensional geometric reconstruction and point-set fitting.²⁹ In neural and hybrid settings, splines have been used as compact learnable function classes and smooth activation or representation mechanisms.^30–32 In control and system identification, spline-based neural models have been combined with optimization and dynamic structures, including non-uniform rational B-splines (NURBS)-based Hammerstein identification,^33,34 B-spline neural identification for linear-dynamic systems,³⁵ B-spline controllers in power electronics,³⁶ and adaptive B-spline neural networks for robust control.³⁷ In geometric deep learning, spline kernels define continuous convolutions on irregular domains,³⁸ while physics-informed spline parameterizations have been used for continuous-field approximation under partial differential equation (PDE) constraints.^39,40 Most of that work uses splines to represent signals, fields, or latent transformations. Our use of splines is different: the spline does not synthesize the output document image. Instead, it parameterizes a low-dimensional decision surface that modulates a classical binarization rule.

This distinction is important. In NSO, the primary optimization target is not a dense output map but a global decision manifold over page-level descriptors. Neural modules, when present, are subordinate to this explicit manifold. They either refine the pixel output under spline guidance or predict bounded residuals directly in control-point space. Architecturally, our pixel-domain refinement model is related to U-Net-style segmentation backbones⁴¹ and lightweight U-Net variants,⁴² while the control-point residual model is conceptually aligned with context-prior residual aggregation strategies such as CP-Net.⁴³ It is also noted that attention and multiscale fusion mechanisms are widely used in dense image models, including document-oriented enhancement pipelines,⁴⁴ whereas older image-analysis work also explored structured local-context representations and region interactions.⁴⁵ In contrast, the final artefact in NSO Models I and III remains a single global decision surface rather than a collection of image-specific neural responses.

Recent work has emphasized the importance of hybrid analytical–learning systems and structured decision models. For example, combining rule-based reasoning with deep learning has been successfully applied to visual detection tasks,⁴⁶ while segmentation networks have been used in industrial inspection contexts.⁴⁷ Explainable machine learning frameworks⁴⁸ highlight the importance of interpretability in engineering applications, while compact and structured image representations⁴⁹ demonstrate the benefits of low-dimensional modeling. Learning under explicit constraints⁵⁰ and structured optimization under limited supervision⁵¹ further support the use of constrained learning formulations. Fusion-based learning architectures⁵² and hybrid explainable AI frameworks⁵³ illustrate the trend toward combining analytical structure with data-driven adaptation. In contrast, the proposed NSO framework parameterizes the decision process as a compact spline surface, preserving interpretability while enabling controlled adaptivity. These works collectively support the relevance of hybrid analytical–learning systems within engineering applications, providing context for the structured decision-surface learning approach proposed in this paper.

3. Relation to prior public work

Because the manuscript is closely related to the earlier public statistical-fusion study, the distinction is stated explicitly. The earlier public work introduced three elements that are not new in the present paper:

the fixed analytic logistic-type fusion function that combines page-level text and background descriptors;

the MCHAM metric for structural, human-aligned assessment of binarization quality;

the public external benchmark protocol based on DIBCO datasets and comparisons against classical and deep baselines.

The present paper contributes a different methodological layer. It replaces the fixed analytic fusion function with a learnable tensor-product B-spline decision surface and studies the optimization of that surface under the constraint that explicit decision semantics are preserved. The new methodological questions are: How should the decision surface be parameterized? How can the hard thresholding rule be relaxed for gradient-based learning? Should adaptivity be introduced directly in the spline parameter space or indirectly in the pixel domain? What is gained, and what is lost, when neural flexibility is restricted to bounded residual updates in control-point space?

Accordingly, the paper should be read as a structured-optimization extension of the earlier public statistical-fusion baseline, and not as a replacement for that baseline. The public benchmark protocol is reused here precisely to isolate the incremental effect of learnable spline-surface optimization under a fixed evaluation setting. This design allows a controlled methodological study while avoiding confounding changes in datasets, metrics, or preprocessing. In particular, the present work does not aim to replace the earlier formulation, but to investigate how learnable spline-surface optimization alters optimization dynamics, interpretability, and deployment properties under the same experimental protocol. The residual control-point formulation is the primary new element enabling controlled data-driven updates within the same low-dimensional parameter space.

4. Problem formulation

Let a grayscale document image be

I : Ω \subset Z^{2} \to [0, 255], Ω = {1, \dots, H} \times {1, \dots, W},

(1)

with

H, W \in N

denoting the image dimensions.

The binarization output is

B (x, y) \in {0, 255}, text = 0, background = 255.

(2)

As in the earlier public statistical-fusion baseline, we assume per-page estimates of representative text and background intensities

t \in [0, 255], b \in [0, 255] .

(3)

These quantities are treated as fixed page-level descriptors. The page-level descriptors $(t, b)$ are computed using the same procedure as in the prior statistical-fusion baseline,¹⁰ where $t$ and $b$ represent representative foreground and background intensity estimates derived from the global intensity distribution of the document.

In this paper, they are not learned jointly with the decision surface, because such a procedure would entangle descriptor estimation with optimization of the explicit decision rule and would make interpretation less transparent. The descriptor estimation strategy is therefore regarded as part of the fixed protocol inherited from the public baseline. Sensitivity of the method to descriptor estimation is discussed as a limitation in Section 13.

Local statistics are computed on a neighbourhood $N (x, y)$ :

\begin{aligned} A (x, y) & = \frac{1}{| N (x, y) |} \sum_{(u, v) \in N (x, y)} I (u, v), \end{aligned}

(4)

\begin{aligned} M (x, y) & = median {I (u, v) : (u, v) \in N (x, y)} . \end{aligned}

(5)

In the earlier public baseline, the descriptors $(t, b)$ were fused through a fixed analytic function. In the notation used throughout this paper, that prior surface is

\begin{aligned} f (t, b) & = m_{t b} + \frac{1 - m_{t b}}{1 + m_{t b}^{2} \exp (m_{t b} + \frac{c_{p} - t}{m_{t b} t + ε})}, \end{aligned}

(6)

where

c_{p} = 255 / 2

is the pivot intensity,

m_{t b} = max (t / 255, 1 - b / 255)

balances the two descriptors, and

ε > 0

is a small numerical constant.

The present paper keeps the surrounding decision rule but replaces the fixed analytic surface $f (t, b)$ by a learnable tensor-product B-spline surface $S (t, b)$ whose control-point heights are optimized from data. This is the central computational change by which NSO differs from the earlier statistical-fusion formulation.

For deployment transparency, only the control-point heights are learned. Spline degree, knot vectors, and control-point locations remain fixed. This preserves the physical meaning of the descriptor domain $(t, b) \in [0, 255]^{2}$ , avoids warping the coordinate system, and allows deterministic export of the learned surface as a compact lookup table. Figure 1 illustrates the initialized spline decision surface, its control-point grid, and the corresponding discretized surface representation.

Figure 1.

Initialization of the NSO decision surface. The fixed descriptor domain is preserved, while the control-point heights are optimized during training. (a) Initial 3D decision surface over $(t, b)$ . and (b) Initial B-spline control-point grid. and (c) Discretized surface heat map.

5. Spline-based decision surface

5.1. Tensor-product B-spline surface

A decision surface is defined

S : [0, 255]^{2} \to [0, 1],

(7)

mapping page-level descriptors

(t, b)

to a scalar modulation parameter. In implementation, the surface is evaluated on the integer descriptor grid

{0, \dots, 255}^{2}

Let ${B_{i}^{(k)}}_{i = 1}^{M}$ be B-spline basis functions of degree $k$ over an open-uniform knot vector $u$ , and let ${B_{j}^{(k)}}_{j = 1}^{N}$ be the corresponding basis functions over $v$ . The tensor-product surface is

S (t, b) = \sum_{i = 1}^{M} \sum_{j = 1}^{N} Z_{i j} B_{i}^{(k)} (\tilde{t}) B_{j}^{(k)} (\tilde{b}),

(8)

where

Z \in R^{M \times N}

contains the trainable control-point heights and

\tilde{t} = t / 255, \tilde{b} = b / 255.

(9)

The surface is continuous on $[0, 255]^{2}$ , but for training efficiency and deterministic inference it is discretized as

S [t, b] = S (t, b; Z), t, b \in {0, \dots, 255},

(10)

producing a lookup table

S \in R^{256 \times 256}

5.2. Basis construction

For degree $k$ and $M$ control points, the open-uniform knot vector is

u = \underset{k + 1}{\underset{⏟}{0, \dots, 0}}, \frac{1}{M - k}, \frac{2}{M - k}, \dots, \frac{M - k - 1}{M - k}, \underset{k + 1}{\underset{⏟}{1, \dots, 1}},

(11)

and similarly for

v

. Basis functions are constructed by the Cox–de Boor recursion

\begin{aligned} B_{i}^{(0)} (t) & = {\begin{cases} 1, & u_{i} \leq t < u_{i + 1}, \\ 0, & otherwise, \end{cases} \end{aligned}

(12)

\begin{aligned} B_{i}^{(k)} (t) & = \frac{t - u_{i}}{u_{i + k} - u_{i}} B_{i}^{(k - 1)} (t) + \frac{u_{i + k + 1} - t}{u_{i + k + 1} - u_{i + 1}} B_{i + 1}^{(k - 1)} (t), \end{aligned}

(13)

with the convention that a term is zero when the denominator is zero. In implementation, basis matrices for the full integer descriptor grid are precomputed and cached. This allows repeated surface evaluation during training while keeping the spline parameterization computationally light.

6. Binarization as an explicit decision operator

The hard decision rule follows the earlier statistical-fusion baseline and remains unchanged at inference time. The only change introduced by NSO lies in how the modulation factor is obtained.

6.1. Hard decision rule

For a page with descriptors $(t, b)$ , let $S (t, b) \in [0, 1]$ denote the learned surface value. Define

\begin{aligned} T_{1} (x, y) & = S (t, b) M (x, y), \\ T_{2} (x, y) & = A (x, y) . \end{aligned}

(14)

The hard binarization rule is

B (x, y) = {\begin{cases} 0, & I (x, y) < min (T_{1} (x, y), T_{2} (x, y)), \\ 255, & otherwise . \end{cases}

(15)

This rule preserves the semantics of the earlier public baseline: local median and local mean still define two adaptive thresholds, and the page-level descriptors still influence the decision through one scalar modulation factor. NSO changes the source of that factor from a fixed analytic surface to a learned spline surface.

6.2. Differentiable training-time relaxation

The hard operator in Eq. (15) is not differentiable because it contains both the minimum operator and a binary threshold. During training only, smooth surrogates are used that preserve the same logical structure. The resulting soft output is denoted $\hat{B} \in [0, 255]$ , whereas all reported evaluation results correspond to the hard binary output $B \in {0, 255}$ . Detailed formulas are collected in Appendix A.

7. Metrics and training objectives

7.1. Evaluation metrics

As in the earlier public baseline, performance is evaluated by hard binarization outputs using three complementary measures:

F-measure (FM), balancing precision and recall for text pixels;

Negative Rate Metric (NRM), penalizing false positives and false negatives symmetrically;

Mean Chamfer-like Distance (MCHAM), a structural discrepancy measure that captures stroke continuity and global text geometry more directly than pixel counting.

The present paper does not introduce MCHAM; it adopts the measure from the prior public work because MCHAM is the most natural structural objective for learning a compact decision surface. What is new here is the use of differentiable MCHAM-driven optimization inside the NSO framework.

7.2. Training-time surrogates

FM, NRM, and MCHAM are not directly differentiable. During training, NSO therefore uses soft surrogates derived from the soft output $\hat{B}$ . These surrogates are optimization devices only. They are never used for reporting benchmark numbers. Models I and III are trained with the structural discrepancy term ${MCHAM}_{soft}$ , whereas Model II combines it with binary cross-entropy and an explicit smoothness penalty on the spline control grid.

8. Three NSO model families

All three model families share the same hard decision operator and the same spline parameterization. They differ only in the location at which learnable adaptivity is introduced. This common structure enables a controlled methodological comparison rather than a loose comparison of unrelated models.

Figure 2 summarizes the information flow and optimization target of the three NSO variants.

Figure 2.

Overview of the three NSO model families. All variants share the same explicit spline-driven decision rule and differ only in where adaptive learning is introduced.

A minibatch consists of

{I_{n}, G_{n}, M_{n}, A_{n}, t_{n}, b_{n}}_{n = 1}^{B_{mb}},

where

I_{n}

is a grayscale patch,

G_{n}

is the corresponding ground-truth mask,

M_{n}

and

A_{n}

are the local median and local mean images,

(t_{n}, b_{n})

are page-level descriptor indices, and

B_{mb}

is the minibatch size. For efficiency, the spline surface is evaluated once per iteration on the full descriptor grid to form

S \in R^{256 \times 256}

. Each patch then samples

s_{n} = S [t_{n}, b_{n}] .

8.1. Model I: direct global spline optimization

Model I updates only the spline control-point heights $Z$ . No neural network is used. The XY control-point coordinates remain fixed from the initialization surface, while a single global trainable height grid is shared across all pages and patches. The loss is purely structural:

L_{I} = \frac{1}{B_{mb}} \sum_{n = 1}^{B_{mb}} {MCHAM}_{soft} ({\hat{B}}_{n}, G_{n}) .

(16)

This model is the simplest expression of the central NSO idea: a compact global decision manifold is learned directly from task-level structural feedback.

8.2. Model II: spline-guided pixel-domain refinement

Model II keeps the global spline surface as a structured prior but adds a pixel-domain U-Net, following the general encoder–decoder design introduced in U-Net⁴¹ and later lightweight variants for document-oriented image processing.⁴² The patch input is

X_{n} = [\frac{I_{n}}{255}, \frac{s_{n}}{255}, \frac{M_{n}}{255}, \frac{A_{n}}{255}] \in [0, 1]^{4 \times H \times W} .

A U-Net produces logits

L_{n} = f_{θ} (X_{n})

, and the soft output is

{\hat{B}}_{n} = 255 (1 - σ (L_{n})) .

The training objective is

\begin{aligned} L_{II} = & \frac{1}{2} L_{BCE} (L, G) + \frac{1}{2 B_{mb}} \sum_{n = 1}^{B_{mb}} {MCHAM}_{soft} ({\hat{B}}_{n}, G_{n}) \\ + R_{smooth} (Z), \end{aligned}

(17)

where

R_{smooth} (Z)

penalizes high-frequency variation in the control-point grid through finite differences.

Model II is included for methodological completeness. It indicates the effect of combining spline guidance with dense pixel-domain correction. The model is more flexible than Models I and III, but that flexibility also weakens direct interpretability of the final deployed system. It is therefore used as a methodological control for assessing what is gained and lost when adaptivity is moved from the spline parameter space to the pixel domain.

8.3. Model III: bounded residual learning in control-point space

Model III preserves a single global spline surface while introducing controlled adaptivity through residual updates in control-point space. A base surface $Z^{base}$ is maintained, and a lightweight residual predictor inspired by context-prior residual aggregation ideas⁴³ receives

X_{n} = [\frac{I_{n}}{255}, \frac{M_{n}}{255}, \frac{A_{n}}{255}, \frac{s_{n}^{base}}{255}], s_{n}^{base} = S^{base} [t_{n}, b_{n}] .

The residual network outputs a bounded control-grid update

Δ Z_{n}

, and the residual is aggregated into one global update

Δ Z_{global} = \sum_{n} w_{n} Δ Z_{n}, Z^{step} = Z^{base} + Δ Z_{global} .

The weights

w_{n}

define the patch-aggregation strategy used to combine per-patch residual updates into a single global control-point update. In practice, different aggregation schemes can be employed. A simple baseline is uniform averaging,

w_{n} = 1 / B_{mb}

, where all patches contribute equally; this strategy is used as the primary configuration in the experiments reported in this paper. Alternative strategies, including a ground-truth-free ink-density proxy derived from image statistics and ground-truth text-fraction weighting, are evaluated in Section 11.7, where the latter serves only as a training-time reference and is not applicable in deployment. Ground-truth-based weighting is included solely as a training-time reference and is not required by the deployed system. Deployment-compatible aggregation strategies rely exclusively on image-derived statistics.

The loss is again purely structural:

L_{III} = \frac{1}{B_{mb}} \sum_{n = 1}^{B_{mb}} {MCHAM}_{soft} ({\hat{B}}_{n}, G_{n}) .

(18)

The three variants are intentionally closely related and are not intended as independent models, but as controlled configurations for studying where adaptivity should be introduced within the same decision-surface framework.

This model is the principal new contribution of the paper. Unlike pixel-domain refinement, it introduces adaptation directly in the same low-dimensional parameter space that defines the deployed decision surface. As a result, the final artefact remains a single coherent global rule.

8.4. Summary of optimization targets

Model I learns only the spline control points. Model II learns a U-Net and the spline jointly. Model III learns a base spline surface together with a residual predictor that operates in control-point space. In Models I and III, deployment requires only the learned spline surface; no image-specific parameters or neural outputs are needed at inference time.

9. Algorithms

The following algorithms summarize the training procedures for the three NSO variants. At inference time, the trained surface is evaluated once to obtain $S$ , the descriptor-indexed scalar $s = S [t, b]$ is sampled, and the hard rule in Eq. (15) is applied. Model I deploys only $Z$ , Model III deploys only $Z^{base}$ , while Model II additionally requires the U-Net to produce the final binarization output. In Model III, the neural component is used only during training to optimize the spline control points and is not part of the deployed system.

10. Optimization dynamics and interpretability

Unless otherwise stated, all results in this section correspond to the baseline spline configuration with a $10 \times 10$ control grid and spline degree $k = 2$ .

The optimization behaviour of the three model families can be understood by inspecting the action of gradients on the spline control grid and, when present, on the neural parameters. The main advantage of Models I and III is that the decision surface is explicit at every stage. Training therefore modifies a compact global object rather than a hidden collection of dense pixel filters.

10.1. Low-dimensional optimization in Models I and III

In Model I, the mapping

Z ⟶ S (t, b) ⟶ \hat{B} ⟶ L

is low-dimensional and well conditioned. The surface depends linearly on

Z

, basis functions have compact support, and each control point influences only a local region of descriptor space. This encourages smooth global adaptation rather than unstable patch-specific behaviour.

Model III preserves the same geometry but augments it with bounded residual updates. Each patch proposes a residual grid in the same control-point space, and those proposals are merged into one global update. This accelerates adaptation while preventing uncontrolled drift into dense pixel-specific corrections. In other words, Model III aims to improve convergence without giving up the core interpretability of Model I.

10.2. Interaction of structural and pixelwise supervision in Model II

Model II is intentionally different. The spline provides only one conditioning signal among four input channels, while the U-Net supplies the main representational power. The binary cross-entropy term acts at every pixel and can therefore dominate the structural term during early training. As a result, local corrections may appear that are helpful for pixelwise accuracy but not directly reflected by a smooth global decision surface. This tension is the main reason Model II needs explicit spline smoothness regularization.

10.3. Interpretability

Interpretability in NSO is not claimed in a generic or purely rhetorical sense. In Models I and III, the final deployed object is the discretized spline surface $S$ together with the control-point grid that induces it. This object can be visualized, archived, compared across training runs, and exported as a lookup table for deterministic inference. Such traceability is substantially harder to obtain from a purely pixel-domain segmentation network. The present paper therefore treats interpretability as an operational property of the learned artefact, not merely as a post hoc explanation device.

Based on the behaviour in Figure 3, the remainder of the parameterization study focuses on Model III, which constitutes the primary methodological contribution of this paper and provides the most relevant compromise between adaptive capacity and a single deployable global surface.

Figure 3.

Optimization dynamics of the three NSO model families under the baseline spline configuration ( $10 \times 10$ , $k = 2$ ). The top row visualizes deviations of the control-point grid from initialization; the bottom row shows the corresponding training trajectories of the soft evaluation measures. (a) Model I: control-point evolution, (b) Model II: control-point evolution, (c) Model III: control-point evolution, (d) Model I: metric trajectories, (e) Model II: metric trajectories and (f) Model III: metric trajectories.

10.4. Spline parameterization study on Model III

The following analysis focuses on Model III in order to isolate the effect of spline parameterization from differences among model families. We vary only the control-grid resolution and spline degree while keeping the remaining training protocol fixed. This analysis addresses the question of how much capacity the spline decision manifold requires before the representation becomes either too rigid or unnecessarily flexible.

Figure 4 visualizes the final learned control-point update $Δ Z$ across six spline parameterizations. Coarser grids constrain the update to broad global trends, whereas finer grids allow more localized variation. The comparison between $k = 2$ and $k = 3$ shows how higher-order splines can redistribute the update more smoothly across neighboring control points. The quantitative effect of these choices is summarized in Table 1.

Figure 4.

Final control-point updates $Δ Z$ for Model III under different spline grid resolutions and polynomial degrees. Columns increase the control-grid resolution, while rows compare spline degree. Visualizing the learned update in control-point space makes the effect of spline parameterization more explicit than the corresponding surface plots, which remain visually similar across settings. (a) $6 \times 6, k = 2$ , (b) $10 \times 10, k = 2$ , (c) $14 \times 14, k = 2$ , (d) $6 \times 6, k = 3$ , (e) $10 \times 10, k = 3$ and (f) $14 \times 14, k = 3$ .

Table 1.

Sensitivity to spline parameterization.

Degree $k$	Grid size	FM	NRM	MCHAM
2	$6 \times 6$	0.815	0.115	0.0506
2	$10 \times 10$	0.820	0.110	0.0510
2	$14 \times 14$	0.814	0.117	0.04944
3	$6 \times 6$	0.815	0.115	0.0506
3	$10 \times 10$	0.812	0.119	0.0495
3	$14 \times 14$	0.782	0.148	0.0463

Boldface indicates the best result in the corresponding metric column.

11. Experimental setup

11.1. Public benchmark protocol and relation to the earlier baseline

The experiments follow a standard public benchmark protocol in order to isolate the effect of learnable spline-surface optimization under controlled conditions. The evaluation corpus consists of the DIBCO and H-DIBCO benchmark editions from 2009 to 2019,^14–23 which include degraded handwritten and printed document images with publicly available ground truth and are widely used for comparative evaluation in the ICDAR/IJDAR community.

The DIBCO corpus has been widely used to report external comparisons against classical thresholding methods and recent deep-learning baselines such as DE-GAN and GDB, together with FM, NRM, and MCHAM evaluation.^6,8,10 In the present paper, representative external methods are recomputed under the same fixed benchmark protocol in order to provide a consistent reference for the NSO variants. The objective is not to perform an exhaustive state-of-the-art comparison, but to isolate the methodological effect of replacing the fixed analytic fusion surface with a learnable spline surface under identical experimental conditions. All datasets, preprocessing assumptions, and evaluation criteria therefore remain unchanged, ensuring that differences in performance can be attributed solely to the proposed spline-based optimization framework.

All reported results are computed from hard outputs $B$ using standard metric definitions. Training-time soft relaxations are used only for gradient-based optimization. All datasets used in this study are publicly available through the official DIBCO and H-DIBCO benchmark releases, typically distributed via ICDAR and ICFHR competition repositories. This supports reproducibility of the experimental protocol, provided that the same benchmark releases, patch extraction procedure, descriptor estimation, and preprocessing steps are used. The objective of the present study is not to establish a new state-of-the-art benchmark result, but to isolate the methodological effect of replacing a fixed analytical fusion function with a learnable spline decision surface under identical experimental conditions.

11.2. External benchmark comparison

To provide external context for the internal NSO comparison, Table 2 summarizes representative results for classical, hybrid, and learning-based binarization methods evaluated on the same DIBCO benchmark series.

Table 2.
Comparison with representative classical and state-of-the-art document binarization methods on the DIBCO 2009–2019 datasets.

Method FM NRM MCHAM

Improved Sauvola 0.762 0.084 0.062

Gatos 0.727 0.140 0.058

Otsu 0.751 0.090 0.067

Nick 0.755 0.094 0.065

Su 0.724 0.079 0.095

Wan 0.637 0.081 0.104

GDB 0.884 0.075 0.025

DE-GAN 0.769 0.036 0.048

NSO initial surface 0.829 0.105 0.047

Method	FM	NRM	MCHAM
Improved Sauvola	0.762	0.084	0.062
Gatos	0.727	0.140	0.058
Otsu	0.751	0.090	0.067
Nick	0.755	0.094	0.065
Su	0.724	0.079	0.095
Wan	0.637	0.081	0.104
GDB	0.884	0.075	0.025
DE-GAN	0.769	0.036	0.048
NSO initial surface	0.829	0.105	0.047

All results are computed under the same evaluation protocol as the NSO models.

The table provides an external reference for the NSO variants under the same benchmark protocol, including direct comparison with widely used state-of-the-art methods such as GDB and DE-GAN. Although the primary focus of this paper is the internal comparison of spline-learning strategies, this summary enables positioning of the proposed approach relative to both classical thresholding techniques and modern deep-learning-based binarization systems.

Classical methods achieve moderate performance but remain sensitive to degradations, whereas deep models such as GDB and DE-GAN provide strong pixel-level accuracy. The initial NSO surface yields competitive results across all three metrics, indicating that the structured decision formulation captures essential characteristics of document binarization.

The objective of this work is not to uniformly outperform existing methods, but to investigate how a compact, interpretable decision rule can be optimized through a learnable spline surface while preserving deployment simplicity.

11.3. Dataset composition and reproducibility summary

Table 3 summarizes the benchmark composition and patch statistics used throughout the experiments.

Table 3.
Benchmark datasets and patch extraction statistics.

Dataset Number of images Number of patches

DIBCO 2009 9 11

DIBCO 2010 7 122

DIBCO 2011 8 97

DIBCO 2012 12 300

DIBCO 2013 16 542

DIBCO 2014 10 182

DIBCO 2016 10 256

DIBCO 2017 20 610

DIBCO 2018 10 266

DIBCO 2019 20 738

Total 122 3124

Dataset	Number of images	Number of patches
DIBCO 2009	9	11
DIBCO 2010	7	122
DIBCO 2011	8	97
DIBCO 2012	12	300
DIBCO 2013	16	542
DIBCO 2014	10	182
DIBCO 2016	10	256
DIBCO 2017	20	610
DIBCO 2018	10	266
DIBCO 2019	20	738
Total	122	3124

Per-page descriptors $(t, b)$ are computed once for each document and attached to all patches extracted from that page. Patch extraction uses a regular $256 \times 256$ grid with stride $256$ , resulting in non-overlapping patches in the interior of each image. At image boundaries, patches are shifted to remain within the valid domain, and edge replication is applied when necessary. This ensures that no border content is discarded while preserving consistency of local statistics near image boundaries.

11.4. Implementation details

The reference implementation uses tensor-product B-splines with degree $k = 2$ , a default $10 \times 10$ control-point grid, and a cached $256 \times 256$ surface table. Training is patch-based with minibatch size $16$ on CUDA hardware and $1$ otherwise.

Optimization uses AdamW with cosine warm restarts and gradient clipping. Model I optimizes a global control-point grid, Model II jointly optimizes the spline surface and a 4-channel U-Net, and Model III optimizes a base spline surface together with a lightweight residual predictor operating in control-point space.

Unless stated otherwise, all experiments use this reference configuration.

11.5. Aggregate comparison among NSO variants

The results in Table 4 indicate that the three NSO variants exhibit different trade-offs between structural consistency and pixel-level accuracy. Model I provides a stable global solution with minimal model complexity, while Model II improves NRM through pixel-domain refinement at the cost of increased model complexity and reduced interpretability. Model III achieves comparable structural performance to Model I while enabling faster adaptation through residual updates in control-point space.

Importantly, the optimization does not uniformly improve all metrics relative to the initial surface. This reflects the nature of the structural objective, which prioritizes geometric consistency over direct pixel-level accuracy. The results should therefore be interpreted as demonstrating controlled adaptation within a constrained decision model rather than uniform metric maximization.

Table 4.
Aggregate comparison among NSO variants.

Model FM NRM MCHAM Time per epoch

Initial surface 0.829 0.105 0.047 –

Model I 0.819 0.111 0.051 31s

Model II 0.828 0.094 0.055 35s

Model III 0.819 0.110 0.051 30s

Model	FM	NRM	MCHAM	Time per epoch
Initial surface	0.829	0.105	0.047	–
Model I	0.819	0.111	0.051	31s
Model II	0.828	0.094	0.055	35s
Model III	0.819	0.110	0.051	30s

11.6. Sensitivity to spline degree and control-grid resolution

Model III is evaluated under different spline degrees and control-grid resolutions while keeping the training protocol fixed.

The $10 \times 10$ , $k = 2$ configuration provides the best FM and NRM, while the $14 \times 14$ , $k = 3$ configuration gives the lowest MCHAM but substantially degrades FM and NRM. This indicates that increasing spline capacity can improve the structural distance measure, but may also reduce pixel-level classification accuracy. The default $10 \times 10$ , $k = 2$ setting is therefore retained as the most balanced configuration. These results indicate that increasing spline capacity improves flexibility but does not guarantee consistent gains across all evaluation measures. The default configuration therefore represents a compromise between expressiveness and stability, supporting the design goal of a compact and interpretable decision surface.

11.7. Model III aggregation analysis

To evaluate the role of patch aggregation in Model III, we compare three weighting strategies under the same training protocol. The first strategy uses uniform averaging, where all patches contribute equally to the global control-point update. The second strategy introduces an image-based proxy, referred to as the ink-density proxy, which assigns higher importance to patches with higher foreground likelihood. The third strategy weights patches according to ground-truth text fraction and is included as a reference.

The results in Table 5 show that all aggregation strategies produce very similar performance. Uniform averaging provides a simple and stable baseline, while the ink-density proxy offers a lightweight data-driven alternative. The ground-truth-based weighting serves only as a reference and does not lead to consistent improvements.

Table 5.
Effect of aggregation strategy in model III.

Aggregation strategy FM NRM MCHAM

Uniform averaging 0.819 0.110 0.051

Ink-density proxy 0.818 0.110 0.051

Ground-truth text fraction 0.817 0.113 0.050

Aggregation strategy	FM	NRM	MCHAM
Uniform averaging	0.819	0.110	0.051
Ink-density proxy	0.818	0.110	0.051
Ground-truth text fraction	0.817	0.113	0.050

11.8. Runtime, footprint, and deployment summary

Table 6 summarizes the runtime characteristics and deployment footprint of the three NSO variants.

Table 6.
Runtime and deployment footprint.

Model Trainable parameters Stored artefact CPU inference (ms) GPU inference (ms)

Model I 100 control-point grid $1416 \pm 108$ $1474 \pm 151$

Model II 13.39M neural network + control-point grid $2464 \pm 64$ $1613 \pm 175$

Model III 97.6k control-point grid $773 \pm 78$ $983 \pm 30$

Model	Trainable parameters	Stored artefact	CPU inference (ms)	GPU inference (ms)
Model I	100	control-point grid	$1416 \pm 108$	$1474 \pm 151$
Model II	13.39M	neural network + control-point grid	$2464 \pm 64$	$1613 \pm 175$
Model III	97.6k	control-point grid	$773 \pm 78$	$983 \pm 30$

Inference time is measured per full image on a representative DIBCO sample. Boldface indicates the lowest inference time among the compared models.

For Model III, the trainable-parameter count includes the residual predictor used during training, whereas the stored deployment artefact consists only of the final control-point grid. Model III achieves the lowest inference time despite using a learning-based training procedure, confirming that its deployment reduces to a lightweight spline evaluation without any neural network overhead. The runtime results in Table 6 show that Model III achieves the lowest deployment cost among the investigated variants while preserving the explicit spline decision structure.

12. Results: Binary interpretation of structural changes

Because the present manuscript focuses on the internal behaviour of the three NSO variants, representative samples are analysed by comparing the binary initial result, the model output, and the ground truth at pixel level. Four mutually exclusive change categories are counted: correct removals of background artefacts (CR), incorrect removals of true text (IR), correct recovery of missing text (RT), and incorrect foreground additions (IA). These counts complement FM/NRM/MCHAM by making the nature of structural changes directly interpretable.

Table 7 reports the corresponding pixel-level change categories for the representative Model III parameterization example.

Table 7.
Pixel-level structural changes for model III under different spline parameterizations relative to the initial binarization (figure 5).

Model III configuration CR IR RT IA

$6 \times 6, k = 2$ 3815 (47.3%) 4217 (52.3%) 1 (0.0%) 25 (0.3%)

$10 \times 10, k = 2$ 3498 (51.2%) 3305 (48.4%) 2 (0.0%) 26 (0.4%)

$14 \times 14, k = 2$ 917 (54.5%) 556 (33.0%) 75 (4.5%) 136 (8.1%)

$6 \times 6, k = 3$ 3815 (47.3%) 4217 (52.3%) 1 (0.0%) 25 (0.3%)

$10 \times 10, k = 3$ 1951 (59.4%) 1260 (38.3%) 16 (0.5%) 60 (1.8%)

$14 \times 14, k = 3$ 1268 (58.3%) 775 (35.7%) 40 (1.8%) 90 (4.1%)

Model III configuration	CR	IR	RT	IA
$6 \times 6, k = 2$	3815 (47.3%)	4217 (52.3%)	1 (0.0%)	25 (0.3%)
$10 \times 10, k = 2$	3498 (51.2%)	3305 (48.4%)	2 (0.0%)	26 (0.4%)
$14 \times 14, k = 2$	917 (54.5%)	556 (33.0%)	75 (4.5%)	136 (8.1%)
$6 \times 6, k = 3$	3815 (47.3%)	4217 (52.3%)	1 (0.0%)	25 (0.3%)
$10 \times 10, k = 3$	1951 (59.4%)	1260 (38.3%)	16 (0.5%)	60 (1.8%)
$14 \times 14, k = 3$	1268 (58.3%)	775 (35.7%)	40 (1.8%)	90 (4.1%)

CR = correct removals, IR = incorrect removals, RT = recovered text, IA = incorrect additions.

The structural-change analysis reveals a clear dependency on spline parameterization for the considered sample. Coarse grids ( $6 \times 6$ ) produce a high proportion of incorrect removals, indicating loss of true text due to limited model flexibility. Intermediate configurations ( $10 \times 10$ ) achieve a more balanced behaviour, while higher-resolution grids introduce text recovery but also increase incorrect additions. This example is intended as a representative case illustrating differences in structural behaviour; aggregate conclusions are drawn from the full benchmark evaluation rather than from a single sample.

Figure 5 complements the quantitative sensitivity analysis by showing how the same document responds to different spline parameterizations within Model III. Coarser grids tend to enforce broader global corrections, whereas finer grids offer more local flexibility. The visual comparison also helps assess how spline degree influences the smoothness and spatial distribution of corrections.

Figure 5.

Qualitative comparison of binarization outputs under six spline parameterizations in Model III. The example corresponds to a patch extracted from a handwritten document in the DIBCO 2017 dataset. The first row shows the original patch, ground truth, and initial binarization. The second and third rows correspond to spline degrees $k = 2$ and $k = 3$ , respectively, while columns increase the control-grid resolution. (a) Original, (b) Ground truth, (c) Initial, (d) $6 \times 6, k = 2$ , (e) $10 \times 10, k = 2$ , (f) $14 \times 14, k = 2$ , (g) $6 \times 6, k = 3$ , (h) $10 \times 10, k = 3$ and (i) $14 \times 14, k = 3$ .

Table 8.

Metric values for representative qualitative examples.

Sample	Configuration	FM	NRM	MCHAM
Handwritten document (DIBCO 2017)	Model I ( $10 \times 10$ , $k = 2$ )	0.852	0.094	0.0187
	Model II ( $10 \times 10$ , $k = 2$ )	0.839	0.072	0.0249
	Model III ( $10 \times 10$ , $k = 2$ )	0.851	0.099	0.0179
Parameterization sample (Model III, Figure 5)	$6 \times 6, k = 2$	0.819	0.148	0.0652
	$10 \times 10, k = 2$	0.836	0.132	0.0686
	$14 \times 14, k = 2$	0.856	0.086	0.1064
	$6 \times 6, k = 3$	0.819	0.148	0.0652
	$10 \times 10, k = 3$	0.857	0.097	0.0950
	$14 \times 14, k = 3$	0.857	0.089	0.1026

The first block reports results on a full handwritten document from the DIBCO 2017 dataset, from which the patch in fig. 5 is extracted. The second block shows the effect of spline parameterization in model III evaluated on that patch.

Table 8 complements the visual comparison of Figure 5 by reporting quantitative metric values for the same representative examples. For the full handwritten document, all three models achieve comparable FM scores, while Model III provides the lowest MCHAM value, indicating consistent structural reconstruction of text components. Model II shows slightly lower FM but achieves improved NRM, reflecting stronger suppression of background noise at the cost of minor text degradation.

For the parameterization sample, the results illustrate that spline resolution influences the behaviour of Model III. Coarser grids ( $6 \times 6$ ) lead to higher NRM and reduced structural consistency, whereas increasing the grid resolution improves FM and NRM. However, the finest grids ( $14 \times 14$ ) do not consistently reduce MCHAM, suggesting that excessive flexibility may introduce structural distortions despite improved pixel-wise accuracy.

The qualitative analysis confirms that spline parameterization directly influences the type of structural correction performed by Model III. Coarse grids tend to remove background noise at the expense of text loss, whereas finer grids enable text recovery but introduce additional artifacts. Intermediate configurations provide the most balanced behaviour, consistent with the quantitative evaluation.

Figure 6 illustrates a qualitative stress test on a manually degraded printed document. Because no ground-truth mask is available, the example is not used for FM, NRM, MCHAM, or CR/IR/RT/IA computation. Its purpose is to illustrate visible differences between global structure preservation and local pixel-level adaptivity under severe degradation.

Figure 6.

Challenging printed-document stress example with manually introduced degradations intended to mimic severe real-world acquisition and preservation artefacts. The first row shows the clean printed text sample and the corresponding degraded input affected by stains, ink marks, wrinkles, burn-like damage, shadows, and uneven illumination. The second row shows the binarization outputs obtained by Model I, Model II, and Model III. Since no ground-truth mask is available for this manually degraded example, it is used only as a qualitative stress test and is not included in the quantitative benchmark evaluation. (a) Clean printed text sample, (b) Degraded input with stains, ink marks, wrinkles, burn damage, shadows, and uneven illumination, (c) Model I output, (d) Model II output and (e) Model III output.

13. Discussion

13.1. What the present paper establishes

The primary objective of the experimental analysis is methodological rather than exhaustive benchmarking, namely to compare alternative mechanisms for introducing adaptivity within the same spline decision-surface framework.

The present results should be interpreted as demonstrating controlled adaptation within a constrained decision model rather than establishing a new state-of-the-art binarization method. The observed gains are moderate, which is expected because the study isolates one methodological change within an otherwise fixed framework: replacing a fixed analytic fusion surface with a learnable spline surface.

The three model variants illustrate how the position of adaptivity affects the final behaviour. Direct spline optimization enforces global consistency, pixel-domain refinement increases local flexibility, and residual control-point learning provides a compromise between these two extremes. From an engineering perspective, the main observation is that useful adaptation can be introduced in a low-dimensional parameter space while preserving an explicit decision rule.

The page-level descriptors $(t, b)$ are inherited from the fixed protocol of the earlier public statistical-fusion baseline and are therefore treated as external inputs in the present study. Alternative foreground and background estimation procedures were not investigated because the primary objective was to isolate the effect of spline-surface optimization under a fixed evaluation protocol. Similarly, ground-truth-based weighting was included only as a training-time reference for analysing residual aggregation behaviour, while the primary experiments use uniform aggregation and remain compatible with deployment-oriented strategies that rely exclusively on image-derived statistics. The robustness of the framework under alternative descriptor-estimation and aggregation strategies remains an important direction for future work.

13.2. Positioning of the approach

The proposed framework should be viewed as a structured optimization extension of an explicit binarization rule, not as a replacement for high-capacity deep-learning methods. Its purpose is to study how a compact and interpretable decision surface can be optimized while maintaining deterministic inference and deployment simplicity.

This positioning is important because the strongest deep models are designed primarily for maximum pixel-level accuracy, whereas NSO emphasizes transparency, compactness, and controlled adaptation. The approach is therefore most relevant in settings where the deployed artefact should remain inspectable, lightweight, and reproducible.

13.3. Computational complexity and deployment

NSO is attractive from a deployment perspective because the learned object in Models I and III reduces to a small control grid or, equivalently, a precomputed lookup table $S \in R^{256 \times 256}$ . Inference is therefore limited to descriptor lookup and evaluation of the hard thresholding rule. No GPU is required, and no image-specific neural forward pass is involved.

Model II does not share this advantage, because it requires deployment of a U-Net in addition to the spline surface. Consequently, in engineering settings where auditability, runtime predictability, and compactness of the deployed artefact are important, Models I and III are the more practical variants. This distinction is reflected in Table 6, where Model III achieves the lowest inference time while requiring only the learned control-point grid at deployment.

In practical document-processing pipelines, the proposed method can be used as a preprocessing stage for OCR or document analysis systems. Since Models I and III rely only on a compact lookup table at inference time, they are suitable for large-scale digitization workflows and resource-constrained environments where neural-network deployment may be impractical.

13.4. Limitations

Several limitations should be acknowledged. First, the page-level descriptors $(t, b)$ are assumed to be fixed and externally estimated. The present work does not investigate alternative descriptor estimation strategies nor quantify sensitivity to descriptor errors. Second, spline hyperparameters are not exhaustively explored, although sensitivity experiments over spline degree and control-grid resolution are included.

Third, the manuscript focuses on grayscale document binarization and does not explicitly address several challenging real-world scenarios, including colour manuscripts, severely warped documents, or highly heterogeneous multi-script layouts. In such cases, the global descriptor formulation or the spline decision surface may require additional adaptation or preprocessing.

Fourth, although the study is evaluated on standard public benchmarks and includes representative external comparisons, the emphasis remains on internal comparison of spline-learning strategies rather than exhaustive benchmarking against all recent binarization systems. Finally, OCR-level downstream evaluation is outside the scope of this work and remains an important direction for future research.

A further practical limitation concerns the patch-aggregation strategy in Model III. Uniform aggregation is adopted as the primary configuration, while the additional aggregation experiments illustrate that alternative weighting schemes may influence convergence and final behaviour. An explicit ablation study of alternative descriptor-estimation strategies was beyond the scope of the present work. Evaluating the robustness of the learned decision surface under different foreground/background estimation procedures remains an important direction for future research.

13.5. Future directions

Natural extensions of this work include descriptor-robust training, broader exploration of spline parameterizations, and OCR-level evaluation of the deployed decision surfaces.

In addition to the gradient-based optimization used in this study, nature-inspired metaheuristic approaches represent an interesting alternative for future extensions of the framework. In particular, algorithms such as harmony search and simulated annealing have been successfully applied to complex non-convex optimization problems in engineering contexts.^54,55 These methods do not rely on differentiability and may therefore offer advantages in exploring the global structure of the objective landscape, potentially reducing sensitivity to initialization and local minima. Within the NSO framework, such approaches could be applied directly to the optimization of spline control-point parameters or integrated into hybrid schemes that combine global metaheuristic search with local gradient-based refinement.

Another important direction is to replace the current patch-aggregation mechanism in Model III with a fully learned or unsupervised weighting strategy. Future work may also explore explainable AI perspectives by treating the learned spline control grid and its optimization trajectory as explicit explanation objects, for example through control-point sensitivity analysis, descriptor-space saliency, and visualization of residual updates.

14. Conclusion

This paper has introduced Neural-Spline Optimization (NSO) as a structured learning framework for document image binarization, in which a fixed analytic fusion function is replaced by a learnable tensor-product B-spline surface while preserving the explicit decision rule. In this way, the learning process is shifted from dense pixel-level prediction toward a compact and interpretable representation of the decision mechanism.

Three closely related model variants have been studied within a unified formulation. Model I performs direct optimization of the global spline surface, Model II combines spline guidance with pixel-domain U-Net refinement, and Model III introduces bounded residual learning in control-point space. Among these, the residual control-point formulation constitutes the primary methodological contribution, as it enables data-driven adaptation directly within the parameter space that defines the deployed model.

The results indicate that structured decision-surface learning can provide a practical alternative to purely pixel-based approaches, particularly in scenarios where interpretability, stability, and deployment simplicity are of primary importance. The proposed framework is not intended to compete with high-capacity deep models in terms of absolute performance; rather, it provides insight into how structured and interpretable decision rules can be optimized using modern learning techniques.

Overall, the NSO framework demonstrates that meaningful adaptation can be achieved within a low-dimensional parameter space while preserving an explicit and analyzable decision structure. This perspective may extend beyond document binarization to other application domains in which the objective is not only predictive accuracy but also model traceability and controlled behaviour.

Footnotes

Acknowledgments

This work was supported by the Croatian Science Foundation under Grant HRZZ-IP-2022-10-8856. The authors also acknowledge the support of the Institutional Research Project ATASYS (IP-UNIST-06).

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

A Differentiable Relaxations

References

Gatos

Pratikakis

Perantonis

. Adaptive degraded document image binarization. Pattern Recognit 2006; 39: 317–327.

Park

. Robust regression for image binarization under heavy noise and nonuniform background. Pattern Recognit 2018; 81: 224–239.

Schomaker

. Deepotsu: Document enhancement and binarization using iterative deep learning. Pattern Recognit 2019; 91: 379–390.

Schomaker

. Ct-net: Cascade t-shape deep fusion networks for document binarization. Pattern Recognit 2021; 118: 108010.

Pastor-Pellicer

España-Boquera

Zamora-Martínez

, et al. Insights on the Use of Convolutional Neural Networks for Document Image Binarization, 2015. pp.115–126.

Souibgui

Kessentini

. De-gan: A conditional generative adversarial network for document enhancement. IEEE Trans Pattern Anal Machine Intell 2022; 44: 1180–1191.

Suh

Kim

Lukowicz

, et al. Two-stage generative adversarial networks for binarization of color document images. Pattern Recognit 2022; 130: 108810.

Yang

Liu

Xiong

, et al. Gdb: Gated convolutions-based document binarization. Pattern Recognit 2024; 146: 109989.

Zhao

Shi

Jia

, et al. Document image binarization with cascaded generators of conditional generative adversarial networks. Pattern Recognit 2019; 96: 106968.

10.

Ćurković

Samardžić

. A statistical fusion approach to document binarization with human-aligned quality evaluation. 2025. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5413901. SSRN preprint.

11.

Sauvola

Pietikäinen

. Adaptive document image binarization. Pattern Recognit 2000; 33: 225–236.

12.

Singh

Roy

Singh

, et al. A new local adaptive thresholding technique in binarization, 2012.

13.

Bataineh

Tounsi

Zamzami

, et al. A comprehensive review on document image binarization. J Imag 2025; 11: 133.

14.

Gatos

Ntirogiannis

Pratikakis

. Icdar 2009 document image binarization contest (dibco 2009). In: 2009 10th International conference on document analysis and recognition, 2009, pp.1375–1382. IEEE.

15.

Ntirogiannis

Gatos

Pratikakis

. Icfhr2014 competition on handwritten document image binarization (h-dibco 2014). In: 2014 14th International conference on frontiers in handwriting recognition, 2014, pp.809–813. IEEE.

16.

Pratikakis

Gatos

Ntirogiannis

. H-dibco 2010 - handwritten document image binarization competition. In: 2010 12th International conference on frontiers in handwriting recognition, 2010, pp.727–732. IEEE.

17.

Pratikakis

Gatos

Ntirogiannis

. Icdar 2011 document image binarization contest (dibco 2011). In: 2011 International conference on document analysis and recognition, 2011, pp.1506–1510. IEEE.

18.

Pratikakis

Gatos

Ntirogiannis

. Icfhr 2012 competition on handwritten document image binarization (h-dibco 2012). In: 2012 International conference on frontiers in handwriting recognition, 2012, pp.817–822. IEEE.

19.

Pratikakis

Gatos

Ntirogiannis

. Icdar 2013 document image binarization contest (dibco 2013). In: 2013 12th International conference on document analysis and recognition, 2013, pp.1471–1476. IEEE.

20.

Pratikakis

Zagori

Kaddas

, et al. Icfhr 2018 competition on handwritten document image binarization (h-dibco 2018). In: 2018 16th International conference on frontiers in handwriting recognition (ICFHR), 2018, pp.489–493. IEEE.

21.

Pratikakis

Zagoris

Barlas

, et al. Icfhr2016 handwritten document image binarization contest (h-dibco 2016). In: 2016 15th International conference on frontiers in handwriting recognition (ICFHR), 2016, pp.619–623. IEEE.

22.

Pratikakis

Zagoris

Barlas

, et al. Icdar2017 competition on document image binarization (dibco 2017). In: 2017 14th IAPR International conference on document analysis and recognition (ICDAR), 2017, pp.1395–1403. IEEE.

23.

Pratikakis

Zagoris

Karagiannis

, et al. Icdar 2019 competition on document image binarization (dibco 2019). In: 2019 International conference on document analysis and recognition (ICDAR), 2019, pp.1547–1556. IEEE.

24.

Akbari

Al-Maadeed

Adam

. Binarization of degraded document images using convolutional neural networks and wavelet-based multichannel images. IEEE Access 2020; 8: 153517.

25.

Bernardino

Lins

da Silva Barboza

. A quality, size and time assessment of the binarization of documents photographed by smartphones. J Imag 2023; 9: 41.

26.

Hung

Adeli

. A parallel genetic/neural network learning algorithm for mimd shared memory machines. IEEE Trans Neural Netw 1994; 5: 900–909.

27.

Ahmadlou

Adeli

. Enhanced probabilistic neural network with local decision circles: A robust classifier. Integr Computer-Aided Eng 2010; 17: 197–210.

28.

Pedrycz

. Logic - oriented fuzzy neural networks. Int J Hybrid Intelligent Syst 2004; 1: 3–11.

29.

Ćurković

Vućina

Ćurković

. Enhanced 3D parameterization for integrated shape synthesis by fitting parameter values to point sets. Integr Computer-Aided Eng 2017; 24: 241–260.

30.

Fakhoury

Speleers

. Exsplinet: An interpretable and expressive spline-based neural network. Neural Netw 2022; 152: 332–346.

31.

Romagnoli

Ratchford

Klein

. Building hybrid b-spline and neural network operators. In: 2024 IEEE 63rd conference on decision and control (CDC), 2024, pp.3611–3617. IEEE.

32.

Vecci

Piazza

Uncini

. Learning and approximation capabilities of adaptive spline activation function neural networks. Neural Netw 1998; 11: 259–270.

33.

Hong

Chen

. The system identification and control of hammerstein system using non-uniform rational b-spline neural network and particle swarm optimization. Neurocomputing 2012; 82: 216–223.

34.

Hong

Iplikci

Chen

, et al. A model-based pid controller for hammerstein systems using b-spline neural networks. Int J Adaptive Control Signal Process 2014; 28: 412–428.

35.

Folgheraiter

. A combined b-spline-neural-network and arx model for online identification of nonlinear dynamic actuation systems. Neurocomputing 2016; 175: 433–442.

36.

Deng

Srinivasan

Oruganti

. A b-spline network based neural controller for power electronic applications. Neurocomputing 2010; 73: 593–601.

37.

Zhang

Zhao

Guo

, et al. An adaptive b-spline neural network and its application in terminal sliding mode control for a mobile satcom antenna inertially stabilized platform. Sensors 2017; 17: 978.

38.

Fey

Lenssen

Weichert

, et al. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 2018,pp.869–877. IEEE.

39.

Wang

Romagnoli

Mowlavi

, et al. Physics-informed deep b-spline networks, 2025.

40.

Zhu

Liu

, et al. A best-fitting b-spline neural network approach to the prediction of advection–diffusion physical fields with absorption and source terms. Entropy 2024; 26: 577.

41.

Ronneberger

Fischer

Brox

. U-net: Convolutional networks for biomedical image segmentation, 2015.

42.

Zunair

Hamza

. Sharp u-net: Depthwise convolutional network for biomedical image segmentation. Comput Biol Med 2021; 136: 104699.

43.

Wang

Gao

, et al. Context prior for scene segmentation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2020, pp.12413–12422. IEEE.

44.

Jiang

Neri

Xue

, et al. Generalized attention mechanism to enhance the accuracy performance of neural networks. Int J Neural Syst 2024; 34(12): 233.

45.

Hamza

Krim

, et al. A multiscale approach to pixel-level image fusion. Integr Computer-Aided Eng 2005; 12: 135–146.

46.

Luelmo

SPD

Garcia-Espinosa

Montemayor

, et al. Combining deep learning methods and rule-based systems for automatic parking space detection. Integr Comput Aided Eng 2025; 32: 97–108.

47.

Marchi

Fornasier

Miorin

, et al. Segmentation networks for detecting overlapping screws in 3D and color images for industrial quality control. Integr Comput Aided Eng 2025; 32: 244–257.

48.

Macas

Garrigós

Martínez

, et al. An explainable machine learning system for left bundle branch block detection and classification. Integr Comput Aided Eng 2024; 31: 43–58.

49.

Łazewski

Cyganek

. Highly compressed image representation for classification and content retrieval. Integr Comput Aided Eng 2024; 31: 267–284.

50.

Liu

Rong

Neri

, et al. Deep deterministic policy gradient with constraints for gait optimisation of biped robots. Integr Comput Aided Eng 2024; 31: 139–156.

51.

Malkova

Amini

Denis

, et al. Neural architecture search for radio map reconstruction with partially labeled data. Integr Comput Aided Eng 2024; 31: 285–305.

52.

Zhou

Fan

Neri

. A spatio-temporal fusion deep learning network with application to lightning nowcasting. Integr Comput Aided Eng 2024; 31: 233–247.

53.

Basheer

Islam

Alwaheidi

, et al. Large language model based hybrid framework for automatic vulnerability detection with explainable ai for cybersecurity enhancement. Integr Comput Aided Eng 2026; 33: 38–54.

54.

Siddique

Adeli

. Harmony search algorithm and its variants. Int J Pattern Recognit Artif Intell 2015; 29: 1539001.

55.

Siddique

Adeli

. Simulated annealing, its variants and engineering applications. Int J Artif Intell Tools 2016; 25: 1630001.

Neural-spline optimization framework for robust document image binarization

Abstract

Keywords

1. Introduction

2. Related work

3. Relation to prior public work

4. Problem formulation

5.1. Tensor-product B-spline surface

6.1. Hard decision rule

7. Metrics and training objectives

7.1. Evaluation metrics

7.2. Training-time surrogates

8. Three NSO model families

9. Algorithms

10. Optimization dynamics and interpretability

10.1. Low-dimensional optimization in Models I and III

10.2. Interaction of structural and pixelwise supervision in Model II

10.3. Interpretability

11.1. Public benchmark protocol and relation to the earlier baseline

11.2. External benchmark comparison

Table 3. Benchmark datasets and patch extraction statistics. Dataset Number of images Number of patches DIBCO 2009 9 11 DIBCO 2010 7 122 DIBCO 2011 8 97 DIBCO 2012 12 300 DIBCO 2013 16 542 DIBCO 2014 10 182 DIBCO 2016 10 256 DIBCO 2017 20 610 DIBCO 2018 10 266 DIBCO 2019 20 738 Total 122 3124

11.5. Aggregate comparison among NSO variants

Table 4. Aggregate comparison among NSO variants. Model FM NRM MCHAM Time per epoch Initial surface 0.829 0.105 0.047 – Model I 0.819 0.111 0.051 31s Model II 0.828 0.094 0.055 35s Model III 0.819 0.110 0.051 30s

11.7. Model III aggregation analysis

Table 5. Effect of aggregation strategy in model III. Aggregation strategy FM NRM MCHAM Uniform averaging 0.819 0.110 0.051 Ink-density proxy 0.818 0.110 0.051 Ground-truth text fraction 0.817 0.113 0.050

13.1. What the present paper establishes

13.2. Positioning of the approach

13.3. Computational complexity and deployment

13.4. Limitations

13.5. Future directions

14. Conclusion

Footnotes

Acknowledgments

Funding

Declaration of conflicting interests

A Differentiable Relaxations

References

Table 4.
Aggregate comparison among NSO variants.

Model FM NRM MCHAM Time per epoch

Initial surface 0.829 0.105 0.047 –

Model I 0.819 0.111 0.051 31s

Model II 0.828 0.094 0.055 35s

Model III 0.819 0.110 0.051 30s

Table 5.
Effect of aggregation strategy in model III.

Aggregation strategy FM NRM MCHAM

Uniform averaging 0.819 0.110 0.051

Ink-density proxy 0.818 0.110 0.051

Ground-truth text fraction 0.817 0.113 0.050