1What the method is for

Asking whether nearby places evoke similar emotions, as a correspondence between two distance matrices.

Suppose there are \(n\) objects, and that for every pair two kinds of "separation" have been measured. Write one as \(D^{(1)}\) and the other as \(D^{(2)}\). Both are \(n \times n\) symmetric matrices with zeros on the diagonal. The question is a single one: are pairs that are close in one also close in the other?

The naive move is to extract the upper-triangular entries into two vectors and take their correlation. As a statistic, that is in fact correct. The difficulty lies in the test. The \(n\) objects yield \(n(n-1)/2\) pairs, but these are not independent observations: the same object \(i\) recurs in \(n-1\) of the pairs. Applying a test of correlation that presumes an independent sample — the test based on the \(t\) distribution — overstates the effective degrees of freedom, and the type I error rate climbs well above the nominal level.

The Mantel test is a procedure for building a null distribution without destroying this dependence structure. It was originally proposed for detecting space–time clustering of disease (Mantel 1967).

2Definition of the statistic

A Pearson correlation between upper-triangular entries. Nothing more than that.

MANTEL STATISTIC
\[ r_M \;=\; \frac{\displaystyle\sum_{i<j}\bigl(d^{(1)}_{ij}-\bar{d}^{(1)}\bigr)\bigl(d^{(2)}_{ij}-\bar{d}^{(2)}\bigr)}{\sqrt{\displaystyle\sum_{i<j}\bigl(d^{(1)}_{ij}-\bar{d}^{(1)}\bigr)^{2}}\;\sqrt{\displaystyle\sum_{i<j}\bigl(d^{(2)}_{ij}-\bar{d}^{(2)}\bigr)^{2}}} \]

\(i<j\) … the upper triangle only. The matrix is symmetric, so the lower triangle repeats the same information and the diagonal is the constant 0
\(\bar{d}^{(k)}\) … the mean of the upper-triangular entries
\(r_M\) … takes values in \([-1,\,1]\), and does not depend on the units

There is nothing new in the statistic itself

The expression above is the definition of the Pearson correlation between two vectors. What the Mantel test contributes is not the statistic but the choice of distribution against which \(r_M\) is compared. In practice it is enough to pull out the upper triangle with np.triu_indices_from(D, k=1) and hand it to pearsonr.

3Why permute

The entries must not be permuted. What is permuted is the labels on the objects.

The null hypothesis is that there is no correspondence between the two distance matrices. It is expressed in the form: the labels attached to the objects may be interchanged. Choose a permutation \(\pi\) and apply the same \(\pi\) to both the rows and the columns of one matrix.

THE SAME PERMUTATION ON ROWS AND COLUMNS
\[ \bigl(D^{(2)\pi}\bigr)_{ij} \;=\; d^{(2)}_{\pi(i)\,\pi(j)} \qquad \pi \in \mathfrak{S}_n \]

There are \(n!\) permutations, too many to enumerate, so \(B\) of them are drawn at random as an approximation. For each draw, \(r_M^{(b)} = r_M\bigl(D^{(1)},\,D^{(2)\pi_b}\bigr)\) is computed, and the resulting distribution serves as the null distribution.

This is the heart of the method

Permuting only the rows, or only the upper-triangular entries, achieves nothing. Shuffling entries individually destroys what makes the object a distance matrix — the recurrence of the same object across many pairs, and the constraints that follow from the triangle inequality — and leaves one no better off than assuming an independent sample. Applying the same permutation to rows and columns breaks only the correspondence of which object is which, while preserving the internal structure of the matrix. That is precisely why the resulting null distribution carries the dependence structure with it.

One-sided or two-sided

If the question is only whether near things resemble one another, one side suffices. Onohara's implementation counts \(|r_M^{(b)}| \ge |r_M^{\mathrm{obs}}|\) and is therefore two-sided: negative correspondence — the farther apart, the more alike — is admitted as a detectable outcome. On the same data a one-sided test would roughly halve the p-value, so which was used has to be stated when reporting.

4The p-value and its floor

Fixing the number of permutations also fixes the smallest p-value that can be reached.

COUNT THE OBSERVED VALUE AS ONE DRAW FROM THE NULL
\[ \hat{p} \;=\; \frac{1+\#\bigl\{\,b \;:\; \bigl|r_M^{(b)}\bigr| \ge \bigl|r_M^{\mathrm{obs}}\bigr|\,\bigr\}}{1+B} \]

the \(+1\) in numerator and denominator … counts the observed value itself as one draw from the null distribution. Without it the p-value could come out as 0
\(B\) … the number of permutations

The floor is \(1/(B+1)\)

The numerator is at least 1, so \(\hat{p}\) can never fall below \(1/(B+1)\): with \(B=1999\) the floor is 0.0005, with \(B=9999\) it is 0.0001. A p-value resting on the floor must not be read as a point estimate. It says no more than that none of the permutations exceeded the observed value, and it does not distinguish a true p of 0.0004 from one of \(10^{-8}\). Claiming a smaller p requires increasing \(B\); there is no other route.

The default in the code, and the value actually used

Onohara's mantel_test takes \(B=9999\) as its default, but every reported run was called with perms=1999. The random seed is fixed at seed=42, so the same input reproduces the same p-value. The 0.0005 in the table below is exactly the floor for \(B=1999\).

5The partial Mantel test, and its limits

Removing the influence of a third matrix — though the validity of this extension has been questioned.

When a third distance matrix \(D^{(3)}\) is available, it is tempting to measure the correspondence with its effect removed. The idea is that of partial correlation: for the upper-triangular vectors \(v_1, v_2, v_3\), compute

A PARTIAL CORRELATION AS THE STATISTIC
\[ r_{12\cdot 3} \;=\; \frac{r_{12} - r_{13}\,r_{23}}{\sqrt{\bigl(1-r_{13}^{2}\bigr)\bigl(1-r_{23}^{2}\bigr)}} \]

The null distribution is built by holding \(D^{(1)}\) and \(D^{(3)}\) fixed and permuting only \(D^{(2)}\), rows and columns together. The p-value is counted as in the simple Mantel test.

This extension is reported not to hold the nominal level

The partial Mantel test exceeds the nominal type I error rate, particularly when the matrices carry spatial autocorrelation. Raufaste & Rousset (2001) exhibit cases in which the p-value does not represent the type I error rate; Guillot & Rousset (2013) demonstrate inflated error rates for both the simple and the partial test, and argue that the partial Mantel test fails to correct for the effect of spatial autocorrelation. The same point has been raised in the phylogenetic comparative setting (Harmon & Glor 2010).

A partial Mantel result therefore cannot be read as evidence that confounding has been controlled. The premise behind the partial-correlation expression above — that subtracting \(v_3\) linearly removes its influence — does not hold when the entries of the distance matrices are spatially correlated. Onohara's analysis runs this test alongside the others, but its standing is that of a supplementary check.

6How the method is used in this research

Three distance matrices — emotion, space and text — are built and their correspondences compared.

  1. Aggregating the emotion data　Responses to a survey on niche construction and environmental choice (4,424 rows) are aggregated into a table of place ID × 24 emotion categories. The categories are the eight basic emotions at three intensity levels each.
  2. Normalisation　Because respondents could select more than one option, the totals differ from place to place. Each row is converted to proportions summing to 1, so that the sheer number of responses does not enter the distance.
  3. Emotion distance　Cosine distance is taken on the 50 × 24 matrix of proportions, giving \(D_{\text{emotion}}\) (50 × 50).
  4. Spatial distance　Adjacency among subcategories is defined as a graph, and its shortest-path distance is taken as \(D_{\text{space}}\). This is a designed proximity structure, not a geographical distance computed from latitude and longitude.
  5. Text distance　The descriptive text for each place is passed through a Japanese BERT model (cl-tohoku/bert-base-japanese-v2, maximum length 128); the CLS vector is L2-normalised and cosine distance is taken, giving \(D_{\text{text}}\).
  6. Testing　Simple Mantel tests are run on the three pairings, followed by partial Mantel tests conditioning on the third matrix.
Comparison\(r\)\(p\)Reading
Emotion × space0.35950.0005p sits on the floor (\(B=1999\))
Emotion × text0.06000.4775no correspondence found
Text × space0.16850.0800not significant at the 5% level
Emotion × space ｜ text0.35510.0005partial Mantel; limits as in the previous section
Emotion × text ｜ space−0.00070.9940partial Mantel

The figures are quoted as they stand in the saved output of the notebook in the repository. All runs used \(B=1999\), seed=42, two-sided.

How the distance matrices are built determines the result

What deserves attention here is that \(D_{\text{space}}\) is not a measurement but a construct. How the adjacency among subcategories is drawn changes the distance, and with it the correspondence with emotion. A distance based on geographical coordinates would give different values. What the Mantel test answers is whether this definition of nearness corresponds to this definition of emotional similarity, and nothing beyond that.

The text is the stimulus that elicited the emotion

Respondents chose their emotions after reading the descriptive text for a place. \(D_{\text{text}}\) and \(D_{\text{emotion}}\) are therefore not two independent measurements: the latter was generated with the former as stimulus. Running a partial Mantel test conditioned on text is best understood as a check made in light of that structure.

A case in which the implementation silently returned a wrong value

In the first attempt at testing phase by phase, splitting the data into ten points per phase, the \(p\) for some phases was displayed as the floor value 0.0005. In fact the submatrices of \(D_{\text{space}}\) were constant throughout, the denominator of the Pearson correlation was 0, and \(r\) was undefined. pearsonr issued a ConstantInputWarning and returned NaN, but mantel_test did not detect it; the comparison abs(r) >= abs(r_obs) was therefore always false, the counter stayed at zero, and the floor p-value was printed.

Onohara found this and rebuilt the distances as shortest-path distances on the subcategory adjacency graph so that the submatrices would not be constant, then reran the phase-wise tests. A p-value resting on the floor is as much a sign of broken arithmetic as of strong evidence. In a permutation test one has to establish first that the statistic is defined at all.

7Sources and data

The analysis scripts and the data are public. The notebook runs from preprocessing through testing and visualisation in one pass, taking pivot_table_id.csv (place ID × emotion counts) and place_subcategory_neighbors.csv (the subcategory adjacency definition) as input.

References on the method

  • Mantel, N. (1967) The detection of disease clustering and a generalized regression approach. Cancer Research, 27(2), 209–220.
  • Raufaste, N. & Rousset, F. (2001) Are partial Mantel tests adequate? Evolution, 55(8), 1703–1705. doi:10.1111/j.0014-3820.2001.tb00689.x
  • Harmon, L. J. & Glor, R. E. (2010) Poor statistical performance of the Mantel test in phylogenetic comparative analyses. Evolution.
  • Guillot, G. & Rousset, F. (2013) Dismantling the Mantel tests. Methods in Ecology and Evolution. doi:10.1111/2041-210x.12018

Which perspective this method belongs to

The Mantel test treats the parameter as a fixed value and judges by how the statistic varies when the sample is rebuilt. It assumes no shape for the distribution, constructing the null directly by permutation rather than invoking a normal or \(t\) distribution. What it evaluates is the hit rate of a procedure under repetition, which places it squarely in Perspective 2 (frequentist) of the seven perspectives in equations.

This page sets out a method Onohara uses in her own research, written for readers in the same field. Every figure is quoted from the output of the public repository; no estimates or approximations have been supplied.
For how the methods sit relative to one another see the seven perspectives in equations; for which course session falls under which perspective see Methods & Mathematics.