Carrying on from the Danielson-Lanczos notes, we will discuss some interesting aspects of bit reversal here. We will consider all binary numbers that can be formed with \(N\) bits, calling it the set \(S(N)\). So,

where the subscript denotes the base of the set. This will be omitted further below where the distinction is obvious. Now denote by \(R_N\) the action of bit-reversal applied to \(S(N)\), i.e., \(R_N=\)Rev\((S(N))\). A key property of bit-reversal is that it is an involution: reversing twice returns the original number, so \(\texttt{Rev}(\texttt{Rev}(x))=x\) for all \(x \in S(N)\). This means every element either maps to itself (a fixed point) or belongs to a pair \(\{x,\texttt{Rev}(x)\}\) that swaps under reversal. Much of the structure that we discuss in this document follows from this fact.

There are some things to notice here. First, that some numbers reverse to themselves. \(R_2(3)\), \(R_3(2)\) and \(R_4(17)\) are some of them. By \(R_N(p)\) we mean the \(p\)-th element of the set with indexing starting from \(0\). Second, that every \(R_N\) can be formed from \(R_{N-1}\) by a simple procedure that has been explained in the earlier document on the Danielson-Lanczos method. To construct the bit-reversed array for \(N=p\), take the one for \(N=p/2\) and after each element \(i\), insert an index \(2^{N-1}\). We will rewrite the arrays given above but with the new elements colored red. The elements in black are carried over from the previous \(N\).

Cycles

Coming back to the first point, let us call the points that reverse to themselves cycles.¹ Also, denote the set of cycles of \(R_N\) by \(C_N\). Thus,

Cycles can be found by simply finding the symmetric binary numbers for a certain \(N\). We are interested in where these cycles occur in \(R_N\). To see this, we will color the cycles in blue in the sets below.

Obviously, \(R_N(2^{N-1})=1\) and \(R_N(1)=2^{N-1}\). The two trivial cycles are \(0\dots0\) and \(1\dots1\), which are the first and last elements. One can also spot easily the cycles \(100\dots001\) and \(011\dots110\). The former is two places before the “central” \(1\) and the latter right after it. See the elements \(R_N(2^{N-1}-2)\) and \(R_N(2^{N-1}+1)\) in the above sets. For example, \({\color{blue}{14}}\) and \({\color{blue}{17}}\) in \(R_5\), and \({\color{blue}{6}}\) and \({\color{blue}{9}}\) in \(R_4\).

How many cycles are present in \(R_N\)? By counting up to \(N=8\), we get the following table.

Let us denote the number of cycles in \(R_N\), or the number of elements in \(C_N\), as \(\mathcal{N}_N\). We may conjecture that if \(N\) is even, then \(\mathcal{N}_N=2^{N/2}\), and if \(N\) is odd, then \(\mathcal{N}_N=2^{(N+1)/2}\). This can easily be verified by looking at binary numbers in \(5\) and \(6\) bits. Take \(00100\). Finding a cycle under bit-reversal is equivalent to fixing the middle element — the 1 here — and folding one side onto the other. The elements to the left of the middle one can be chosen in \(2^{(N-1)/2}\) ways, and the middle element can be either \(0\) or \(1\). So, multiplying by two gives \(\mathcal{N}_{N\text{ odd}}=2^{(N+1)/2}\). For the even \(N\) case, consider \(001100\). Here it is merely a question of folding straight down the middle, so we only need to choose our bits on one side, giving \(\mathcal{N}_{N\text{ even}}=2^{N/2}\). Combining both odd and even cases, we can write the number of cycles for \(N\) bits as \(\mathcal{N}_N=2^{\lceil N/2 \rceil}\). This sequence \(\{2,2,4,4,8,8,16,16,\dots\}\) appears in the On-Line Encyclopedia of Integer Sequences (OEIS) as A060546. A consequence of the involution property is that the remaining \(2^N - \mathcal{N}_N\) elements pair off into 2-cycles, so the total number of 2-cycles is \((2^N - 2^{\lceil N/2 \rceil})/2\).

Distribution

How are the cycles distributed in \(R_N\)? We have already noted the locations of the trivial ones at the two ends of the set and the two close to the “central” \(1\). Let us write down the number of elements between consecutive cycles for the first few \(N\)’s, represented by Distances in the table below.

We see that the distances between consecutive cycles are symmetric about the middle distance of 2. It may be more useful to write the distances as a mere difference of their position indices, which would increase values in the table above by \(1\). We represent this distance by DiffIndex. The symmetric part to the right of the central distance is denoted by an ellipsis.

We are beginning to see some kind of structure. Apart from the central distance of \(3\), we see a doubling for each \((2k-1,2k)\) pair. Now look at the progression from \(N=3\) to \(N=5\) to \(N=7\). We see that we can take a mirror image around the central \(3\), multiply all distances by \(2\), reappend the central \(3\) at the end of that sequence to obtain the next set of differences. Thus, to get the \(N=7\) set, take the \(N=5\) one of \(4,6,4,3,\dots\). Mirroring about \(3\) gives \(4,6,4,3,4,6,4\). Doubling leads to \(8,12,8,6,8,12,8\). This gives the first set of distances. To generate the full set, we just need to mirror it after crossing the central distance of \(3\), giving \(8,12,8,6,8,12,8,3,\dots\). The same logic holds for going from \(N=4\) to \(N=6\) to \(N=8\).

Now look at the value of the first nontrivial cycle for a \((2k-1,2k)\) pair. You will see that there is a tripling. Why? What is the smallest cycle we can form after \(0\dots0\)? Writing down all possible cycles as we will show below for a couple of cases, we will see that it is the binary number with all zeroes except a one in the middle when \(N\) is odd and two ones in the middle when \(N\) is even. Their values in base \(10\) are \(2^{(N-1)/2}\) for the former and \(2^{N/2}\) for the latter. Taking the difference of these two numbers gives us \(3(2^{(N-1)/2})\), which is triple the one for \(N=2k-1\).

Generating cycles

Let us now write down the cycles for \(N=7\) and \(N=8\) in both binary and base \(10\). Look at the \(N=7\) table first. Note that the central bit in the binary numbers alternates between \(0\) and \(1\). For every consecutive \(0,1\) pair, we have the same 3-bit binary number — underlined in green in the tables below — which is mirrored about the central element to make the cycle. What is the value of this underlined binary in base \(10\)? It is shown in blue in the tables below. Now look at equation \eqref{r_blue}! These are exactly the values in our bit-reversed arrays \(R_N\)! For odd \(N\), we reproduce the array \(R_{(N-1)/2}\) and for even \(N\) we get \(R_{N/2}\). There is no central bit for \(N=8\) so the mirror is simply in the middle of the binary sequence.

The central bit in (a) is colored brown, the bits to the right are underlined in green, and the decimal value of the underlined binary is shown in blue. There is no central bit in (b).

The tables below show \(N=9\) and \(N=10\).

Cycles with decimal and binary values for N=9 and N=10.

We see that to construct cycles for any even \(N\), we just need the bit-reversed array \(R_{N/2}\). A suitable mirroring of the elements of this set leads to \(C_N\). We don’t need to search for all symmetric combinations of bits. Similarly for odd \(N\). And thus we can write the generator of cycles for \(N\) bits as \(R_{\lfloor N/2 \rfloor}\). The tables below show similar cycles for \(N\in\{3,4,5,6\}\).

More on distribution of cycles

Differences

In this section we will look at our data visually in the form of plots. First we will plot the DiffIndex sequence of distances between cycles. The figures below are plots of the DiffIndex sequence with the x-axis being the position of the distance value. We see our middle distance of \(3\) with the curve being mirrored about it. Looking at the values and the shapes makes the algorithm of arriving at the next pattern in the even and odd sets clear. For example, to obtain the curve for \(N=5\), take the points in the \(N=3\) curve, double their difference values, append the value 3 and take a mirror image about that point.

More structures can be seen if we plot these curves in the same graph. Specifically, we become more aware of some levels, which are shown as horizontal lines. The figure below shows the DiffIndex sequence for consecutive \(N\) values, followed by even \(N\) and odd \(N\) separately. In the even case, the levels are formed from the middle values of \(3\) and its subsequent doublings, so they are \(3,6,12,24,48,\dots\). In the odd case we have a more interesting situation. Look at the panel for \(N=3\) in the first figure above. We have the presence of the distance of \(2\) along with \(3\). Its doublings lead to another set of levels at powers of \(2\), but they are present only for odd \(N\).

The last figure above shows the sequences for odd \(N\) with the \(x\)-axis truncated to a factor of \(8\) for ease of viewing, which causes the curves for the top 3 \(N\) values to be cut.

Distribution of cycles

Here we will simply plot the binary number versus its bit-reversed value, both in decimal. The figure below shows this mapping for \(N=7\). Notice that logarithmic scaling to the base 2 has been used for both axes. Red dots signify our cycles, also called palindromes. The purple dots are the bit-reversed numbers. There is an immediately noticeable symmetry about the \(y=x\) line.

Another feature that can be observed is that with increasing value, the numbers near the top and right boundary seem to follow a curve. This is more evident in the next figure. We see what looks like a kite with curved edges. What is the equation of this edge? It is an easy matter to verify that for the top sheath it is

Interchange \(y\) and \(x\) for the equation for the sheath on the right. Thus for the top sheath \(x_k=\{1,3,7,15,31,63,127,\dots\}\). These are binary numbers of the form \(0\dots01\), \(0\dots011\), \(0\dots0111\) and so on. This is understandable if we consider their bit-reversals. To make large binary numbers we need as many ones as possible that multiply the larger powers of \(2\). So we should have \(y_k\)’s like \(10\dots0\), \(110\dots0\), \(1110\dots0\), etc. The points that make up this sheath are related to another interesting pattern that we will discuss later.

A diversion into primes

Any discussion involving natural numbers is inevitably bound to include primes. The table below gives the number of \(N\)-bit primes, i.e., the number of primes present in binary numbers of \(N\) bits. For example, with \(N=4\) we have the range in decimal as \([8,16)\) with the primes being \(11_{10}=1011_2\) and \(13_{10}=1101_2\) which reverse to each other. For \(N=5\) the range is \([16,32)\) and the set of primes is \(\{17,19,23,29,31\}\), which reverses to \(\{17,25,29,23,31\}\), giving us 4 reversals to prime numbers and 2 cyclic primes. All the sequences in the table may be described as follows and are given in the OEIS.

1. Total Primes: Number of \(N\)-bit primes — A374403
2. Rev to Prime: Number of \(N\)-bit primes that reverse to primes — A366910
3. Cyclic Primes: Number of \(N\)-bit primes that reverse to themselves — A117773

We immediately observe a periodic occurrence of zeroes in the sequence of cyclic primes. Barring \(N=2\), there are no cyclic primes for even \(N\).

Let \(y=f(x)\). Then \(y'=\frac{dy}{dx}=f'(x)\) and \(f^{-1}(y)=x\), wherever the inverse is well-defined and injective. Then,

Examples

Without knowing how to denote or name an inverse function a priori, we will use the designation \(\mathrm{inv}(y)\).

Example 1. Let \(y=e^x\). Then,

We call this integral \(\ln(y)+c\), with \(c\) being the integration constant. Now when \(y=1\), \(\mathrm{inv}(y)\), i.e., \(x\), is zero. And we know that \(y=1 \implies \ln(1)=0\). So, \(c=0\).

Example 2. Let \(y=\sin(x)\), \(x\in[-\frac{\pi}{2},\frac{\pi}{2}]\). Then \(f'(x) = \cos(x)\). We know \(\cos(x) = \sqrt{1 - \sin^2(x)}\). We take the positive root because cosine is non-negative on our domain. Substituting \(y = \sin(x)\) yields \(f'(x) = \sqrt{1 - y^2}\).

This integral defines the arcsine function, \(\arcsin(y) + c\). Here, \(y=0 \implies x=0\), and \(\arcsin(0) = 0\), so \(c=0\).

Example 3. Let \(y=\tanh(x)\). The derivative is \(f'(x) = \mathrm{sech}^2(x)\). Using the identity \(\mathrm{sech}^2(x) = 1 - \tanh^2(x)\), we can substitute \(y = \tanh(x)\) to express the derivative in terms of \(y\), yielding \(f'(x) = 1 - y^2\).

This integral defines the inverse hyperbolic tangent function, \(\mathrm{artanh}(y) + c\). Since \(y=0 \implies x=0\), and \(\mathrm{artanh}(0) = 0\), we find that \(c=0\).

We have a discrete Fourier transform of length \(N\).

\begin{equation}\label{main} F_k = \sum_{j=0}^{N-1} e^{2 \pi i j k/N} f_{j} \end{equation}

For now, we will only consider cases in which \(N\) is a power of 2 as this is where the approach is most clearly seen. Let us split this according to its even and odd components.

Here, \(F_k\) is the \(k\)-th component of the Fourier-transformed array. The sum that leads to it has been split into even and odd transforms of length \(N/2\), given by \(F_{k}^{e}\) and \(F_{k}^{o}\) respectively. \(W\) is a constant factor and is given by \(e^{2\pi i/N}\). Now suppose that \(N=2\). Then the sums in the second-last line collapse to one element each and are given by

\begin{equation}\label{eq2} F_k = f_0 + W^{k}f_1 \end{equation}

This is our DFT. For completeness’ sake, the elements of \(F\) are

Let us take \(N=4\). Then we can go further and create four Fourier transforms of length \(N/4\). We start from the second-last line of equation \eqref{eq1} and split each term into even and odd parts.

Notice how the power of \(W\) is the same as the index of the element of the original array \(f\) multiplying it. Also, \(W^{N}=W^{0}=1\). For \(N=4\), \(W=e^{i\pi/2}\). The complete \(F\) is as follows:

Let us do this one more time for \(N=8\).

Through three decompositions in halves, we have reduced our DFT calculation to 8 one-point transforms. The bookkeeping of which element in \(f\) corresponds to which odd-even subdivision is possible for small \(N\)’s, but it quickly becomes cumbersome. In equation \eqref{eq4} above, it is easy to go from the first line to the third, but to get to line 3 from line 2 is the point of bit-reversal.

Take the \(eeoeoeo\ldots oo\) pattern, reverse it and assign 1 to \(o\) and 0 to \(e\). This is the value in binary of the element in \(f\) whose one-point Fourier transform is represented as \(F_{k}^{eeoeoeo\ldots oo}\). For example, \(eoo \mapsto 110 = 6\). Thus, the bit-reversed data array \(f\), in index terms, for \(N=8\), would be \(\{0,4,2,6,1,5,3,7\}\). These bit-reversed elements are then combined in pairs (going backward in our equations) until finally the combined sum of halves results in the final FFT. So, the basic idea of an FFT algorithm is to first bit-reverse the original data by swapping elements and then to compute the subtransforms of length \(2,4,8,\ldots,N\).

To understand the above, take the original indices of \(f\) for \(N=4\) — \(\{0,1,2,3\}\) — in binary, i.e., \(\{00,01,10,11\}\). We can see that \(f_0\) and \(f_3\) bit-reverse to themselves, and \(f_1\) and \(f_2\) map to each other. Thus the bit-reversed array is \(\{0,2,1,3\}\). For \(N=8\), we have the indices of \(f\) in binary as \(\{000,001,010,011,100,101,110,111\}\). Here, \(f_0\), \(f_2\), \(f_5\) and \(f_7\) map to themselves, and \(f_1 \leftrightarrow f_4\), \(f_3 \leftrightarrow f_6\). This leads to the bit-reversed array given in the previous paragraph.

Suppose we wanted to construct the bit-reversed index array for \(N=16\). We would have to write the binary representation of every index, reverse it and then get the resulting array. This process quickly becomes cumbersome. But there is an easier way, inspired by the pseudocode presented in Johnson & Frigo². Consider the bit-reversed arrays for \(N \in \{2,4,8\}\).

Do we see a pattern here? To construct the bit-reversed array for \(N=p\), take the one for \(N=p/2\) and after each element \(i\), insert an index \(i+p/2\). We will rewrite the arrays given above but with the new elements in red. The elements in black are carried over from the previous \(N\). The \(N=16\) case is also shown.

This halving algorithm achieves a significant reduction in computational time complexity. Evaluating the DFT directly using equation \eqref{main} requires \(N^2\) complex multiplications, which is represented by a complexity of \(\mathcal{O}(N^2)\). As \(N\) becomes large — which is typical in audio processing, image analysis, and scientific computing — this quadratic growth becomes computationally prohibitive. In the Danielson-Lanczos method, we have \(\log_2(N)\) halvings of the data array and \(N\) operations to combine the subtransforms at each stage, giving us an \(\mathcal{O}(N \log_2 N)\) process. To illustrate the impact of this difference, for a dataset of one million points (\(N=10^6\)), the direct \(\mathcal{O}(N^2)\) approach requires a trillion operations, whereas the FFT requires only about 20 million — a speedup by a factor of fifty thousand.