Every continuous open mapping of R into R is monotonic

I’m willing to bet that most students who have used Rudin’s Principles of Mathematical Analysis have encountered this problem:

15. Call a mapping of \(X\) into \(Y\) open if \(f(V)\) is an open set in \(Y\) whenever \(V\) is an open set in \(X\).

Prove that every continuous open mapping of \(\mathbb{R}\) into \(\mathbb{R}\) is monotonic.

Obvious?

Here is one “solution” that is fairly intuitive. It relies on finding a minimum or maximum and considering the image of a small neighborhood around that min/max:

Lemma 1. Let \(f:\mathbb{R}\to\mathbb{R}\) be a continuous open map. Then there cannot be three points \(p_1 < p_2 < p_3\) such that \(f(p_1),f(p_3) < f(p_2)\) (\(\wedge\) shape) or \(f(p_1),f(p_3) > f(p_2)\) (\(\vee\) shape).

Proof. Assume that the first case holds and let \(M=\sup f([p_1,p_3])\); since \(f([p_1,p_3])\) is compact, there is a point \(x\in[p_1,p_3]\) such that \(f(x)=M\). Since \(f(p_1),f(p_3) < f(p_2)\), we must have \(x \in (p_1,p_3)\). But then \(f((p_1,p_3))\) is not open, which is a contradiction. \(\square\)

Now we can prove the main theorem.

“Proof” of main theorem. If \(f\) is not monotonic, then there are three points as in Lemma 1. \(\square\)

The problem with this “proof” is the assumption: surely we must have a \(\wedge\) shape or a \(\vee\) shape, but how do we really prove its existence? Negating the definition of “monotonic” merely gives us the existence of four points \(a,b,c,d \in \mathbb{R}\) satisfying the following: \begin{align}
a < b &\quad\mathrm{and}\quad f(a) < f(b), \\ c < d &\quad\mathrm{and}\quad f(c) > f(d).
\end{align}

Then we use a purely combinatorial argument by a math.SE user:

If \(a=c\), use \(adb\) to form \(\vee\) or \(abd\) to form \(\wedge\).

If \(a=d\), use \(cdb\) to form \(\vee\).

If \(b=c\), use \(abd\) to form \(\wedge\).

If \(b=d\), use \(acd\) to form \(\wedge\) or \(cab\) to form \(\vee\).

Now we are left with the case when \(a,b,c,d\) are pairwise distinct.

If \(a < b < c < d\), use \(abd\) to form \(\wedge\) or \(acd\) to form \(\wedge\). If \(a < c < b < d\), use \(abd\) to from \(\wedge\) or \(acd\) to form \(\wedge\). If \(a < c < d < b\), use \(acd\) to form \(\wedge\) or \(cdb\) to form \(\vee\). If \(c < a < d < b\), use \(cdb\) to form \(\vee\) or \(cab\) to form \(\vee\). If \(c < d < a < b\), use \(cdb\) to form \(\vee\) or \(dab\) to form \(\vee\).

A different approach

Such a case-by-case analysis is not particularly enlightening, so is there a different approach? This is the best I could come up with when I was attacking the problem:

We first start with a lemma. Call a function \(f:\mathbb{R}\to\mathbb{R}\) locally monotonically increasing/decreasing if for every \(x \in \mathbb{R}\), there is a neighborhood of \(x\) on which \(f\) is monotonically increasing/decreasing.

Lemma 2. A continuous open map \(f:\mathbb{R}\to\mathbb{R}\) is monotonically increasing if it is locally monotonically increasing. The same applies if “increasing” is replaced with “decreasing”.

Proof. If \(f\) is not monotonically increasing then we can find points \(x < y\) such that \(f(x) > f(y)\). Choose a neighborhood \(U \subseteq (-\infty,y)\) of \(x\) on which \(f\) is monotonically increasing. \(f\) cannot be constant on \(U\), for otherwise \(f(U)\) would not be open. So there is a point \(x’ \in U\) such that \(f(x’) \ne f(x)\). If \(x < x'\) then \(f(x) < f(x')\) and we can form a \(\wedge\) shape, and if \(x' < x\) then \(f(x') < f(x)\) and we can again form a \(\wedge\) shape. Both cases contradict Lemma 1. \(\square\)

Using this, we have:

Proof of main theorem. Let \(f:\mathbb{R}\to\mathbb{R}\) be a continuous open map. Let \(A\) be the set of all points \(x\) such that \(f\) is monotonically increasing in a neighborhood of \(x\). Define \(B\) similarly, but with “monotonically decreasing” instead. These two sets are clearly open. They are disjoint, for if \(x \in A \cap B\) then there is a neighborhood \(U\) of \(x\) on which \(f\) is both monotonically increasing and decreasing, i.e. constant; this is a contradiction since \(f(U)\) is not open. If we can show that \(A \cup B = \mathbb{R}\) then by the connectedness of \(\mathbb{R}\) either \(A=\mathbb{R}\) and \(B=\emptyset\) or \(A=\emptyset\) and \(B=\mathbb{R}\), in which case we can apply Lemma 2 to conclude that \(f\) is monotonically increasing or decreasing. Suppose \(x \in \mathbb{R} \setminus (A \cup B)\). If there exists a point \(y > x\) with \(f(y) > f(x)\) then since \(f\) is continuous, there is a neighborhood \(U\) of \(x\) such that \(f(U) \subseteq (-\infty,f(y))\). But \(f\) fails to be monotonically increasing on \(U\), so we have a \(\vee\) shape that contradicts Lemma 1. By a similar argument, there cannot be a point \(y > x\) with \(f(y) < f(x)\). Therefore \(f\) is constant on \((x,\infty)\), which contradicts the fact that \(f\) is open. \(\square\)

An exercise

Prove Lemma 2 without assuming that \(f\) is continuous or open. Hint: if \(x < y\) then \([x,y]\) is compact.

Leave a Reply