Suppose
A
∈
R
n
×
n
A \in \mathbb {R}^{n \times n}
is invertible and we are looking for the solution of
A
x
=
b
Ax = b
. Given an initial guess
x
1
∈
R
x_1 \in \mathbb {R}
, we show that by reflecting through hyperplanes generated by the rows of
A
A
, we can generate an infinite sequence
(
x
k
)
k
=
1
∞
(x_k)_{k=1}^{\infty }
such that all elements have the same distance to the solution
x
x
, i.e.
‖
x
k
−
x
‖
=
‖
x
1
−
x
‖
\|x_k - x\| = \|x_1 - x\|
. If the hyperplanes are chosen at random, averages over the sequence converge and
E
‖
x
−
1
m
∑
k
=
1
m
x
k
‖
≤
1
+
‖
A
‖
F
‖
A
−
1
‖
m
⋅
‖
x
−
x
1
‖
.
\begin{equation*} \mathbb {E} \left \| x - \frac {1}{m} \sum _{k=1}^{m}{ x_k} \right \| \leq \frac {1 + \|A\|_F \|A^{-1}\|}{\sqrt {m}} \cdot \|x-x_1\|. \end{equation*}
The bound does not depend on the dimension of the matrix. This introduces a purely geometric way of attacking the problem: are there fast ways of estimating the location of the center of a sphere from knowing many points on the sphere? Our convergence rate (coinciding with that of the Random Kaczmarz method) comes from simple averaging, can one do better?