AFTD Auto-Formalizing Theoretical Domains

relu_network_zero_of_ivt

theoremverified

If a feedforward ReLU neural network of width $k$ evaluates to a non-positive value at 0 and a non-negative value at 1, then by the intermediate value theorem it has a zero in the unit interval $[0, 1]$.

Statement

theorem relu_network_zero_of_ivt {k : ℕ} (net : ReLUNetwork k)
  (h0 : evalNetwork net 0 ≤ 0) (h1 : 0 ≤ evalNetwork net 1) :
  relu_network_has_unit_interval_zero net
Source
folklore
Verified
25 Oct 2026
Axioms
Classical.choiceQuot.soundpropext
Built on 4
  • evalNetworkEvaluation of a feedforward ReLU neural network of hidden layer width $k$ on a real input $x$, which forms an initial width-$k$ vector by…
  • relu_network_has_unit_interval_zeroA feedforward ReLU neural network net of width $k$ has a zero in the unit interval $[0, 1]$ if there exists some real number…
  • relu_network_eval_continuousFor every natural number $k$ and every feedforward ReLU neural network net of width $k$ with rational weights and biases, the evaluation…
  • ReLUNetworkA feedforward ReLU neural network from the real line to the real line with hidden layers of width k, specified by rational affine input…

Read back from the Lean

For any natural number $k$ and any ReLU network net with layer width $k$ (having rational weights and biases), if the evaluation of net at $0$ is at most $0$ (evalNetwork net 0 ≤ 0) and the evaluation of net at $1$ is at least $0$ (0 ≤ evalNetwork net 1), then net has a zero in the unit interval $[0, 1]$ (i.e., relu_network_has_unit_interval_zero net).

Written by a model that saw only the Lean, never the English above. If the two disagree, that is worth a challenge.

The proof, explained

The theorem is proved by applying the intermediate value theorem to the network evaluation function on the interval [0, 1]. Because every feedforward ReLU neural network defines a continuous function on ℝ, its restriction to [0, 1] is continuous. Since the network evaluates to a non-positive value at 0 and a non-negative value at 1, zero lies between the two endpoint evaluations. The intermediate value theorem then guarantees that zero is in the image of [0, 1], providing a point x ∈ [0, 1] where the network evaluates to zero.

  1. The goal is to find a point x ∈ [0, 1] at which the neural network evaluates to 0.

    Unfolding the definition `relu_network_has_unit_interval_zero` restates the theorem as the existential statement ∃ x ∈ [0, 1], evalNetwork net x = 0.

    line 5

      unfold relu_network_has_unit_interval_zero
  2. The network evaluation function is continuous on the unit interval [0, 1].

    By `relu_network_eval_continuous`, the function computed by the ReLU network is continuous on all of ℝ. Restricting any globally continuous function to a subset, here [0, 1], yields continuity on that subset.

    lines 6–7

    uses relu_network_eval_continuous

      have hcont : ContinuousOn (evalNetwork net) (Set.Icc (0:ℝ) 1) :=
        (relu_network_eval_continuous net).continuousOn
  3. The value 0 belongs to the interval [evalNetwork net 0, evalNetwork net 1].

    The hypotheses h0 (evalNetwork net 0 ≤ 0) and h1 (0 ≤ evalNetwork net 1) together mean that 0 lies in the closed interval between the network's values at the endpoints 0 and 1.

    line 8

      have hmem : (0:ℝ) ∈ Set.Icc (evalNetwork net 0) (evalNetwork net 1) := ⟨h0, h1⟩
  4. The value 0 belongs to the image of [0, 1] under the network evaluation function.

    Since 0 ≤ 1 (verified by norm_num), the intermediate value theorem for closed intervals (`intermediate_value_Icc`) implies that the image of [0, 1] under the continuous function evalNetwork net contains the interval [evalNetwork net 0, evalNetwork net 1], and hence contains 0.

    line 9

    uses intermediate_value_Icc

      have himg := intermediate_value_Icc (show (0:ℝ) ≤ 1 by norm_num) hcont hmem
  5. There exists a point x ∈ [0, 1] with evalNetwork net x = 0, proving the theorem.

    From 0 ∈ evalNetwork net '' [0, 1], we unpack a witness x ∈ [0, 1] such that evalNetwork net x = 0, which directly provides the witness required by the existential goal.

    lines 10–11

      obtain ⟨x, hx, hfx⟩ := himg
      exact ⟨x, hx, hfx⟩
relu_network_eval_continuous
A feedforward ReLU neural network computes a function that is continuous everywhere on ℝ.
intermediate_value_Icc
If a function is continuous on a closed interval [a, b] with a ≤ b, its image contains the closed interval [f(a), f(b)].

Written by a model (gemini-3.8-flash) from the Lean proof and the goal states Lean reported at each step. Lean verified the proof; this explanation is a reading aid and is not itself machine-checked.

Lean source view module on GitHub

/-- Zero existence via the intermediate value theorem for ReLU networks. -/theorem relu_network_zero_of_ivt {k : ℕ} (net : ReLUNetwork k)  (h0 : evalNetwork net 0 ≤ 0) (h1 : 0 ≤ evalNetwork net 1) :  relu_network_has_unit_interval_zero net := by  unfold relu_network_has_unit_interval_zero  have hcont : ContinuousOn (evalNetwork net) (Set.Icc (0:ℝ) 1) :=    (relu_network_eval_continuous net).continuousOn  have hmem : (0:ℝ) ∈ Set.Icc (evalNetwork net 0) (evalNetwork net 1) := ⟨h0, h1⟩  have himg := intermediate_value_Icc (show (0:ℝ) ≤ 1 by norm_num) hcont hmem  obtain ⟨x, hx, hfx⟩ := himg  exact ⟨x, hx, hfx⟩
See it in the graphDiscuss this resultChallenge it