This is a brief return to the topic of Irrational Sunflowers. The sunflower associated with a real number is the set of points with polar coordinates and , . A sunflower reduces to equally spaced rays if and only if is a rational number written in lowest terms as .
Here is the sunflower of of size .
Seven rays emanate from the center because , then they become spirals, and spirals rearrange themselves into 113 rays because . Counting these rays is boring, so here is a way to do this automatically with Python (+NumPy as np):
a = np.pi
n = 5000
x = np.mod(a*np.arange(n, 2*n), 1)
np.sum(np.diff(np.sort(x)) > 1/n)
This code computes the polar angles of sunflower points indexed , sorts them and counts the relatively large gaps between the sorted values. These correspond to the gaps between sunflower rays, except that one of the gaps gets lost when the circle is cut and straightened onto the interval . So the program output (112) means there are 113 rays.
Here is the same sunflower with the points alternatively colored red and blue.
The colors blur into purple when the rational approximation pattern is strong. But they are clearly seen in the transitional period from 22/7 approximation to 355/113.
How many points would we need to see the next rational approximation after 355/113?
What will that approximation be? Yes, 22/7 and 355/113 and among the convergent of the continued fraction of . But so is 333/106 which I do not see in the sunflower. Are some convergents better than others?
Finally, the code I used to plot sunflowers.
import numpy as np
import matplotlib.pyplot as plt
a = np.pi
k = np.arange(10000)
r = np.sqrt(k)
t = a*2*np.pi*k
plt.plot(r*np.cos(t), r*np.sin(t), '.')
This is a collection of entirely unoriginal remarks about Laplacian spectrum of graphs. For an accessible overview of the subject I recommend the M.S. thesis The Laplacian Spectrum of Graphs by Michael William Newman. It also includes a large table of graphs with their spectra. Here I will avoid introducing matrices and enumerating vertices.
Let be the vertex set of a graph. Write if are adjacent vertices. Given a function , define .
This is a linear operator (the graph Laplacian) on the Euclidean space of all functions with the norm . It is symmetric: and positive semidefinite: . Since equality is attained for constant , 0 is always an eigenvalue of .
This is the standard setup, but I prefer to change things a little and replace by the smaller space of functions with zero mean: . Indeed, maps to anyway, and since it kills the constants, it makes sense to focus on . It is a vector space of dimension where .
One advantage is that the smallest eigenvalue is 0 if and only if the graph is disconnected: indeed, is equivalent to being constant on each connected component. We also gain better symmetry between and the Laplacian of the graph complement, denoted . Indeed, since , it follows that for every . So, the identity holds on (it does not hold on ). Hence the eigenvalues of are obtained by subtracting the eigenvalues of from . As a corollary, the largest eigenvalue of is at most , with equality if and only if the graph complement is disconnected. More precisely, the multiplicity of eigenvalue is one less than the number of connected components of the graph complement.
Let denote the diameter of the graph. Then the number of distinct Laplacian eigenvalues is at least . Indeed, let be two vertices at distance from each other. Define and elsewhere. Also let for . Note that for all . One can prove by induction that when the distance from to is greater than , and when the distance from to is equal to . In particular, when and . This shows that is not a linear combination of . Since , it follows that is not a linear combination of . Hence the minimal polynomial of has degree at least , which implies the claim.
Let’s consider a few examples of connected graphs.
There are two connected graphs: the 3-path (D=2) and the 3-cycle (D=1). In both cases we get D distinct eigenvalues. The spectra are [1, 3] and [3, 3], respectively.
One graph of diameter 3, the path. Its spectrum is .
One graph of diameter 1, the complete graph. Its spectrum is . This pattern continues for other complete graphs: since the complement is the empty graph ( components), all eigenvalues are equal to .
Four graphs of diameter 2, which are shown below, with each caption being the spectrum.
The graph [1, 3, 4] has more distinct eigenvalues than its diameter.
The graph [2, 2, 4] is regular (all vertices have the same degree).
The smallest eigenvalue of graphs [1, 1, 4] and [2, 2, 4] is multiple, due to the graphs having a large group of automorphisms (here rotations); applying some of these automorphisms to an eigenfunctions for the smallest eigenvalue yields another eigenfunction.
[1, 3, 4] and [2, 4, 4] also have automorphisms, but their automorphisms preserve the eigenfunction for the lowest eigenvalue, up to a constant factor.
One graph of diameter 4, the path. Its spectrum is related to the golden ratio: it consists of .
One graph of diameter 1, the complete one: [5, 5, 5, 5]
Five graphs of diameter 3. All have connected complement, with the highest eigenvalue strictly between 4 and 5. None are regular. Each has 4 distinct eigenvalues.
14 graphs of diameter 2. Some of these are noted below.
Two have connected complement, so their eigenvalues are less than 5 (spectrum shown on hover):
1.382, 1.382, 3.618, 3.618
1.382, 2.382, 3.618, 4.618
One has both integers and non-integers in its spectrum, the smallest such graph. Its eigenvalues are .
Two have eigenvalues of multiplicity 3, indicating a high degree of symmetry (spectrum shown on hover).
1, 1, 1, 5
3, 5, 5, 5
Two have all eigenvalues integer and distinct:
1, 2, 4, 5
2, 3, 4, 5
The 5-cycle and the complete graph are the only regular graphs on 5 vertices.
This is where we first encounter isospectral graphs: the Laplacian spectrum cannot tell them apart.
Both of these have spectrum but they are obviously non-isomorphic (consider the vertex degrees):
Both have these have spectrum and are non-isomorphic.
Indeed, the second pair is obtained from the first by taking graph complement.
Also notable are regular graphs on 6 vertices, all of which have integer spectrum.
Here [3, 3, 3, 3, 6] (complete bipartite) and [2, 3, 3, 5, 5] (prism) are both regular of degree 3, but the spectrum allows us to tell them apart.
The prism is the smallest regular graph for which the first eigenvalue is a simple one. It has plenty of automorphisms, but the relevant eigenfunction (1 on one face of the prism, -1 on the other face) is compatible with all of them.
There are four regular graphs on 7 vertices. Two of them are by now familiar: 7-cycle and complete graph. Here are the other two, both regular of degree 4 but with different spectra.
There are lots of isospectral pairs of graphs on 7 vertices, so I will list only the isospectral triples, of which there are five.
Spectrum 0.676596, 2, 3, 3.642074, 5, 5.681331:
Spectrum 0.726927, 2, 3.140435, 4, 4, 6.132637:
Spectrum 0.867363, 3, 3, 3.859565, 5, 6.273073:
Spectrum 1.318669, 2, 3.357926, 4, 5, 6.323404:
All of the triples mentioned so far have connected complement: for example, taking the complement of the triple with the spectrum [0.676596, 2, 3, 3.642074, 5, 5.681331] turns it into the triple with the spectrum [1.318669, 2, 3.357926, 4, 5, 6.323404].
Last but not least, an isospectral triple with an integer spectrum: 3, 4, 4, 6, 6, 7. This one has no counterpart since the complement of each of these graphs is disconnected.
Regular graphs, excluding the cycle (spectrum 0.585786, 0.585786, 2, 2, 3.414214, 3.414214, 4) and the complete one.
Obviously, if and only if is a power of . It also looks like is always even, with powers of 2 dominating the list of its values. But does take on odd values, although this does not happen often: only 6 times in the first 100 integers.
[(n, pow(2, n, n)) for n in range(1, 101) if pow(2, n, n) % 2]
It is conjectured that the range of consists of all nonnegative integers except 1. Here is a proof that , provided by Max Alexeyev on the OEIS page for A036236
Suppose for some . Let be the smallest prime divisor of . Then . This means that the order of in the multiplicative group is a divisor of . But this group has elements, and the only divisor of that is smaller than is . Thus, the order of is , which means , which is absurd.
The fact that the smallest with is 4700063497 deserves to be mentioned here. But I would like to consider the most frequent values of instead. Experimentally, they are found like this:
from collections import Counter
freq = Counter(pow(2, n, n) for n in range(1, 100000000 + 1))
These are all powers of 2… but not exactly in the order one might expect. I repeated the experiment for the first integers with . The frequency order remained stable at the top. These are the most common 11 values, with their frequency (in %) listed for the aforementioned six ranges.
It seems that 32 and 128 are consistently less common than one might expect.
The most common value, 2, is contributed by primes and pseudoprimes to base 2. The value of 4 appears when with prime (but not only then). Still, the primes having zero density makes it difficult to explain the frequency pattern based on primality. A more convincing reason could be: when is even, we are computing and that is likely to be a power of . This boosts the frequency of the even (generally, composite) powers of 2 in the sequence.
scipy.integrate.quad is a popular method of numerical integration with Python. Let’s see how it chooses the points at which to evaluate the function being integrated. Begin with a simple example, the exponential function.
The blue dots indicate the evaluation points, their y-coordinate being the order of evaluation. So, it begins with x=0, continues with something close to -1, etc. The function, drawn in red, is not to its true vertical scale.
The placement of dots gives away the method of integration: it is the Gauss-Kronrod quadrature with 10 Gauss nodes and 21 Kronrod nodes, abbreviated (G10, K21). The Gauss nodes are included in the Kronrod nodes, and of course the function is not evaluated there again. The evaluation process is slightly out of order in that 0 is a Kronrod node but comes first, followed by 10 Gauss nodes, followed by 10 remaining Kronrod nodes. The process ends there, as the comparison of G10 and K21 results shows the necessary precision was reached.
The square root on [0, 1] is not such a nice function, so (G10, K21) does not reach the required precision at once. The interval is bisected again and again until it does.
Surely the cube root is even worse. Or is it?
The nodes are symmetric about the midpoint of the interval of integration. So for any odd function on an interval symmetric about 0 we get G10 = 0 and K21 = 0, and the process stops at once. To see the effect of the cube root singularity, one has to use a non-symmetric interval such as [-1, 2].
That’s still fewer subdivisions than for the square root: cancellation between the left and right neighborhoods of 0 still helps. Let’s look at smooth functions next.
Rapid oscillations forces subdivisions here. There are also other reasons for subdivision, such as the loss of analyticity:
Although exp(-1/x) is infinitely differentiable on this interval, the fact that it is not analytic at 0 makes it a more difficult integrand that exp(x). Finally, an analytic function with no oscillation which still needs a bunch of subintervals:
This is the standard example used to illustrate the Runge phenomenon. Although we are not interpolating here, numerical integration is also influenced by the function having a small radius of convergence of its Taylor series. After all, G10 can be thought of as degree-9 interpolation of the given function at the Gauss nodes (the zeros of a Legendre polynomial), with the formula returning the integral of the interpolating polynomial.
The code used to plot these things:
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import quad
f = lambda x: np.exp(x)
eval_points = 
a, b = -1, 1
quad(f_int, a, b)
n = len(eval_points)
t = np.linspace(a, b, 1000)
y = f(t)
yy = n*(y - y.min())/(y.max() - y.min())
plt.plot(t, yy, 'r')
plt.plot(eval_points, np.arange(0, n), '.')
The choice of piecewise polynomials of degree 3 for interpolation is justifiably popular: even-degree splines are algebraically awkward to construct, degree 1 is simply piecewise linear interpolation (not smooth), and degree 5, while feasible, entails juggling too many coefficients. Besides, a cubic polynomial minimizes the amount of wiggling (the integral of second derivative squared) for given values and slopes at the endpoints of an interval. (Recall Connecting dots naturally.)
But the derivative of a cubic spline is a quadratic spline. And one needs the derivative to find the critical points. This results in an awkward example in SciPy documentation, annotated with “(NB: sproot only works for order 3 splines, so we fit an order 4 spline)”.
Although not implemented in SciPy, the task of computing the roots of a quadratic spline is a simple one. Obtaining the roots from the internal representation of a quadratic spline in SciPy (as a linear combination of B-splines) would take some work and reading. But a quadratic polynomial is determined by three values, so sampling it at three points, such as two consecutive knots and their average, is enough.
Quadratic formula with values instead of coefficients
Suppose we know the values of a quadratic polynomial q at -1, 0, 1, and wish to find if it has roots between -1 and 1. Let’s normalize so that q(0)=1, and let x = q(-1), y = q(1). If either x or y is negative, there is definitely a root on the interval. If they are positive, there is still a chance: we need the parabola to be concave up, have a minimum within [-1, 1], and for the minimum to be negative. All of this is easily determined once we note that the coefficients of the polynomial are a = (x+y)/2 – 1, b = (y-x)/2, and c = 1.
The inequality ensures the suitable sign of the discriminant. It describes a parabola with vertex (1, 1) and focus (2, 2), contained in the first quadrant and tangent to the axes at (4, 0) and (0, 4). Within the orange region there are no real roots.
The line x+y=2, tangent to the parabola at its vertex, separates convex and concave parabolas. While concavity in conjunction with x, y being positive definitely precludes having roots in [-1, 1], slight convexity is not much better: it results in real roots outside of the interval. Here is the complete picture: green means there is a root in [-1, 1], orange means no real roots, red covers the rest.
Back to splines
Since the derivative of a spline is implemented in SciPy (B-splines have a nice formula for derivatives), all we need is a root-finding routine for quadratic splines. Here it is, based on the above observations but using built-in NumPy polynomial solver np.roots to avoid dealing with various special cases for the coefficients.
roots = 
knots = spl.get_knots()
for a, b in zip(knots[:-1], knots[1:]):
u, v, w = spl(a), spl((a+b)/2), spl(b)
t = np.roots([u+w-2*v, w-u, 2*v])
t = t[np.isreal(t) & (np.abs(t) <= 1)]
roots.extend(t*(b-a)/2 + (b+a)/2)
A demonstration, which plots the spline (blue), its critical points (red), and original data points (black) as follows:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import InterpolatedUnivariateSpline
x = np.arange(7)
y = np.array([3, 1, 1, 2, 2, 4, 3])
f = InterpolatedUnivariateSpline(x, y, k=3)
crit_pts = quadratic_spline_roots(f.derivative())
t = np.linspace(x, x[-1], 500)
plt.plot(x, y, 'kd')
plt.plot(crit_pts, f(crit_pts), 'ro')
Suppose we have real numbers and want to find the sum of all distances over . Why? Maybe because over five years ago, the gradient flow of this quantity was used for "clustering by collision" (part 1, part 2, part 3).
If I have a Python console open, the problem appears to be solved with one line:
>>> 0.5 * np.abs(np.subtract.outer(x, x)).sum()
where the outer difference of x with x creates a matrix of all differences , then absolute values are taken, and then they are all added up. Double-counted, hence the factor of 0.5.
But trying this with, say, one million numbers is not likely to work. If each number takes 8 bytes of memory (64 bits, double precision), then the array x is still pretty small (under 8 MB) but a million-by-million matrix will require over 7 terabytes, and I won’t have that kind of RAM anytime soon.
In principle, one could run a loop adding these values, or store the matrix on a hard drive. Both are going to take forever.
There is a much better way, though. First, sort the numbers in nondecreasing order; this does not require much time or memory (compared to quadratic memory cost of forming a matrix). Then consider the partial sums ; the cost of computing them is linear in time and memory. For each fixed , the sum of distances to with is simply , or, equivalently, . So, all we have to do is add these up. Still one line of code (after sorting), but a much faster one:
For example, x could be a sample from some continuous distribution. Assuming the distribution has a mean (i.e., is not too heavy tailed), the sum of all pairwise distances grows quadratically with n, and its average approaches a finite limit. For the uniform distribution on [0, 1] the computation shows this limit is 1/3. For the standard normal distribution it is 1.128… which is not as recognizable a number.
As , the average distance of a sample taken from a distribution converges to the expected value of |X-Y| where X, Y are two independent variables with that distribution. Let’s express this in terms of the probability density function and the cumulative distribution function . By symmetry, we can integrate over and double the result:
Integrate by parts in the second integral: , and the boundary terms are zero.
Integrate by parts in the other integral, throwing the derivative onto the indefinite integral and thus eliminating it. There is a boundary term this time.
Since , this simplifies nicely:
This is a lot neater than I expected: is simply the integral of . I don’t often see CDF squared, like here. Some examples: for the uniform distribution on [0,1] we get
and for the standard normal, with , it is
The trick with sorting and cumulative sums can also be used to find, for every point , the sum (or average) of distances to all other points. To do this, we don’t sum over but must also add for . The latter sum is simply where is the total sum. So, all we need is
Unfortunately, the analogous problems for vector-valued sequences are not as easy. If the Manhattan metric is used, we can do the computations for each coordinate separately, and add the results. For the Euclidean metric…
Given a sequence of numbers of length one may want to look for evidence of its periodic behavior. One way to do this is by computing autocorrelation, the correlation of the sequence with a shift of itself. Here is one reasonable way to do so: for lag values compute the correlation coefficient of with . That the lag does not exceed ensures the entire sequence participates in the computation, so we are not making a conclusion about its periodicity after comparing a handful of terms at the beginning and the end. In other words, we are not going to detect periodicity if the period is more than half of the observed time period.
Having obtained the correlation coefficients, pick one with the largest absolute value; call it R. How large does R have to be in order for us to conclude the correlation is not a fluke? The answer depends on the distribution of our data, but an experiment can be used to get some idea of likelihood of large R.
I picked independently from the standard normal distribution, and computed as above. After 5 million trials with a sequence of length 100, the distribution of R was as follows:
Based on this experiment, the probability of obtaining |R| greater than 0.5 is less than 0.0016. So, 0.5 is pretty solid evidence. The probability of is two orders of magnitude less, etc. Also, |R| is unlikely to be very close to zero unless the data is structured in some strange way. Some kind of correlation ought to be present in the white noise.
Aside: it’s not easy to construct perfectly non-autocorrelated sequences for the above test. For length 5 an example is 1,2,3,2,3. Indeed, (1,2,3,2) is uncorrelated with (2,3,2,3) and (1,2,3) is uncorrelated with (3,2,3). For length 6 and more I can’t construct these without filling them with a bunch of zeros.
Repeating the experiment with sequences of length 1000 shows a tighter distribution of R: now |R| is unlikely to be above 0.2. So, if a universal threshold is to be used here, we need to adjust R based on sequence length.
I did not look hard for statistical studies of this subject, resorting to an experiment. Experimentally obtained p-values are pretty consistent for the criterion . The number of trials was not very large (10000) so there is some fluctuation, but the pattern is clear.
P(L0.45|R| > 4)
Naturally, all this depends on the assumption of independent normal variables.
And this is the approach I took to computing r in Python:
import numpy as np
n = 1000
x = np.random.normal(size=(n,))
acorr = np.correlate(x, x, mode='same')
acorr = acorr[n//2+1:]/(x.var()*np.arange(n-1, n//2, -1))
r = acorr[np.abs(acorr).argmax()]