1What the method is for

When the question is “what inherited from what”, but a tree cannot be assumed.

The input is a single distance matrix \(D\). For \(n\) objects, the separation between each pair has been measured. The question is whether there is a genealogical structure among these objects.

Onohara and Yano (2014) put this question to academic papers. In writing a paper, an author consults earlier work and, absent special reason, carries over its terminology and phrasing. If so, papers in the same field and on the same theme stand in a lineage to one another. From there one can ask whether the papers submitted to two journals form separate lineages in terms of vocabulary.

Fitting a tree here, however, would be too strong. A tree assumes that every object descends from a single parent, and there is no guarantee that papers inherit from one another the way organisms do. A phylogenetic network is the instrument for relaxing that assumption and leaving conflicting signal visible in the figure.

What contamination means here

The paper defines a phylogenetic network as “the result of phylogenetic estimation, shown in the form of a network that permits contamination”. Contamination, in the language of biological systematics, is admixture: the state in which several lineages merge (Yano 2012). A tree cannot represent this; a network draws it as a set of parallel edges, a box.

2Origin

Two lines, neighbor-joining and split decomposition, converged into one method.

  1. Neighbor-joining (Saitou & Nei 1987) Builds a tree from a distance matrix, at each stage joining the pair that minimizes the total branch length. The most widely used distance-based method of phylogenetic estimation.
  2. Split decomposition (Bandelt & Dress 1992) A theory decomposing a distance into a superposition of splits (bipartitions of the object set). When incompatible splits are supported at the same time, they can be drawn as a network rather than collapsed into a tree. The computation grows heavy as the number of objects rises, and fine structure is hard to resolve.
  3. Neighbor-Net (Bryant & Moulton 2003) Combines the two. Objects are ordered by the same agglomerative procedure as neighbor-joining, and only the splits compatible with the resulting circular ordering are retained. This holds the computation down while yielding a network that can be drawn in the plane even for hundreds of objects.

Application to culture

A phylogenetic method does not merely classify a set of events; it posits a genealogical relation behind them, and its mathematics was developed mainly in evolutionary biology (Nakao 2012). Applied to cultural material — manuscripts, linked verse, language, craft objects — it is called cultural phylogenetics; Yano (2012) applies it to the lineage of manuscripts of the linked-verse anthology Wakuraba. Onohara and Yano (2014) turn that framework on academic papers.

3Where it comes from mathematically

Approximate distances by a superposition of splits. Finding the coefficients is non-negative least squares.

For a split \(S = \{A, X \setminus A\}\) of the object set \(X\), define the split distance \(\delta_S\) to be 1 when \(i\) and \(j\) lie on opposite sides and 0 when they lie on the same side.

Write the distance as a superposition of splits
\[ d_{ij} \;=\; \sum_{S \in \mathcal{S}} \alpha_S\, \delta_S(i,j) \;+\; (d_0)_{ij}, \qquad \alpha_S \ge 0 \]

\(\alpha_S\) … the weight of split \(S\); drawn in the network as branch length
\(\mathcal{S}\) … the set of splits retained
\(d_0\) … the remainder no split accounts for (the split-prime residue)

Tree or network is decided by a property of the split set

When every pair of retained splits is compatible (one side of the one is contained in one side of the other), the representation is exactly a tree. When incompatible splits carry positive weight at the same time, drawing them as parallelograms (boxes) instead of collapsing them gives a network. The width of a box is the weight of that split.

Neighbor-Net first fixes an ordering of the objects around a circle and admits only the splits that form contiguous intervals in that ordering. Under this restriction the resulting network can always be drawn in the plane.

What determines the weights is least squares

\[ \min_{\alpha \ge 0} \;\sum_{i<j}\Bigl(d_{ij} - \sum_{S \in \mathcal{S}} \alpha_S\,\delta_S(i,j)\Bigr)^{2} \]

Minimize the sum of squared deviations between the observed distances and those reproduced by the superposition of splits. The only constraint is \(\alpha_S \ge 0\): a branch length cannot be negative.

This is the very equation at the centre of Perspective 1 (geometry). In regression the columns of the design matrix \(X\) were explanatory variables; here each column is the indicator function of one split. What is being minimized has not changed. What differs is only the choice of basis, and a non-negativity constraint on the coefficients.

\[ \underbrace{\min_{\beta}\;\lVert \mathbf{y} - X\beta \rVert^{2}}_{\text{regression}} \qquad\longleftrightarrow\qquad \underbrace{\min_{\alpha \ge 0}\;\lVert \mathbf{d} - \Delta\,\alpha \rVert^{2}}_{\text{split decomposition}} \]

\(\mathbf{d}\) … the upper triangle of the distance matrix, stacked into a vector
\(\Delta\) … a matrix whose columns correspond to the split distances \(\delta_S\)

4Definition and assumptions

The input is distance alone. Which is why how the distance is built decides everything.

  1. The input is a distance matrix only The contents of the objects are never seen. Build a distance from word frequencies, and from then on the distance alone determines the result.
  2. The existence of a lineage is not assumed No tree is fitted, so the structure “every object has exactly one parent” is not required. Conflicting signal remains as a box.
  3. Planarity is given priority Admitting only the splits compatible with a circular ordering is a constraint imposed to make the figure drawable, not something derived from the data.
  4. Branch lengths carry a scale Since the weight of a split becomes the branch length directly, figures come with a scale bar. Figure 8 of Onohara and Yano (2014) carries a scale of 0.01.

Branch reliability is measured by the bootstrap

The weight of a split says how long a branch is; it does not say whether that branch appears stably at all. To see that, the estimation is repeated on resampled data and the number of times the same branch appears is counted.

Onohara (2013) defines the value as follows: “this reliability expresses, as a proportion, when \(n\) runs of phylogenetic estimation are performed, how many times out of those \(n\) the branch configuration on the tip side of the branch bearing the figure occurs”. Throughout the dialect work \(n = 1000\) is used. The number written above a branch in a figure is this proportion, in per cent.

The number attaches to an individual branch. It is not a single figure expressing whether the network as a whole is correct.

5How it is used in this research

The same method is applied to the lineage of papers and to regional difference in dialect.

(A) Comparing two journals in terms of vocabulary — Onohara & Yano (2014)

The inputs, however, are built differently on the two sides. That discrepancy is taken up in section 6.

  1. The data on the Keiryō Kokugogaku side No digitized body text exists, so only the titles and keywords published on the society website were used. The years 1957 to 1987 are distributed in CSV.
  2. Bundling into one text Those titles and keywords were treated collectively as a single text, parsed morphologically, reduced to nouns, and turned into frequency data.
  3. Selecting words The 30 most frequent, down to where the rate of decrease stabilizes, were taken. This differs in procedure from the selection by standard deviation used on the Kōdō Keiryōgaku side, on the grounds that titles and keywords already contain only the important nouns.
  4. Building the matrix For those 30 words, the frequencies across the 30 papers of Kōdō Keiryōgaku were aligned, producing a table of 31 texts × 30 nouns. The thirty-first is reskk, the bundled Keiryō Kokugogaku material.
  5. Neighbor-Net A distance was built from this table and the phylogenetic network constructed (Fig. 8).

What was read from it

Branch lengths and branching show that the nouns frequently used in Keiryō Kokugogaku between 1957 and 1987 and those used in Kōdō Keiryōgaku from 1974 onward are comparatively distinct. The reading is that the two journals have handled different themes.

Some papers, however, extended from branches near reskk. The paper lists nine: 1974_03, 1978, 1982, 1985_02, 1990, 1995_01, 2003_01, 2003_02 and 2003_04.

The configuration is checked against outside information

On checking the authors of the papers positioned nearby, seven of them — 1974_03, 1978, 1982, 1985_02, 1990, 1995_01 and 2003_01 — turned out to be authors who had also submitted to Keiryō Kokugogaku.

This is an important step. Proximity in the network only means that the distances built from word frequencies are small; that is not in itself evidence of a lineage. Matching it against authorship, information not used in the analysis, supplies grounds for holding that the configuration is not accidental. In a method whose only input is a distance, external checks of this kind are what support the result.

Of the nine papers listed, though, authorship was confirmed for seven. No corresponding statement is made for 2003_02 and 2003_04. Not every case of proximity has been accounted for.

(B) Treating regional difference in dialect as a lineage

Comparing journals is only one application. Onohara has used the method continuously in dialect research, on the vocabulary of the upper Ibi River basin in Gifu Prefecture and on the accent of Shōdoshima in Kagawa Prefecture. Both the construction of the input and the object of reading differ from the case of papers.

  1. Borrowing the data of descriptive research Of the 200 Japanese dialect items of Hattori (1973), the 21 localities reported by Yamada (1978, 1980) are used. The design is not to collect material oneself but to re-analyse quantitatively a table already in print in descriptive research.
  2. Recoding to 0 and 1 A matrix is built with 1 where a locality has the phonetic or lexical form and 0 where it does not.
  3. Computing Gower distance The sequences of two localities are compared and the number of differing elements is divided by the total number of elements. Onohara (2013) works an example in which 6 of 19 elements differ, giving \(6/19 = 0.316\).
  4. Drawing the Neighbor-Net Using SplitsTree4.
  5. Assessing branches by bootstrap With 1000 replicates, Tokuyama Village separated from the rest at 86.1%, Imajō and Ikeda from the rest at 98.1%, and Tokyo, Kyoto and Kinomoto at 98.6% (Onohara 2012; Onohara, research paper).

What was read from it

Tokuyama Village stands apart from the other settlements on the network, and areas geographically close to one another are close on the network, while areas geographically distant sit apart on it. Some of the groupings on the network correspond directly to village units.

Further, the classification obtained is handed on as a class code to the next analysis, and random forest is used to extract the vocabulary that separates the settlements. That part is covered on the random forest page.

Two points in time, at the same locality, placed in one network

In the Shōdoshima case, the accent of nine localities surveyed by Nakai (1998) between 1993 and 1996, and the results of a survey at the same nine localities in 2012, are entered as separate taxa in a single network. “Ieura (1998)” and “Ieura (2012)” stand as separate leaves. This is not branching in a lineage but reading change over time as a phylogeny.

What supports the design is that Neighbor-Net is unrooted. Onohara (2013) writes that “since Neighbor Net is an unrooted phylogenetic network, it is appropriate for the analysis of Japanese accent, where it is unclear whether one may assume a common ancestral form, that is, a single proto-language”. With no root, there is no need to decide which is ancestor and which descendant. Placing two points in time from the same locality side by side raises no contradiction on the method’s side.

What can be read, however, is the configuration alone. Whether Ieura 1998 and Ieura 2012 are near or far is readable; which changed into which does not come out of the network. The direction of change is supplied from the side of linguistic argument.

Feeding the output of phylogenetic estimation into the next analysis

In “A mathematical analysis of dialect distribution data”, phylogenetic estimation is not the conclusion in itself but a step for producing a composite variable. Phylogenetic estimation is run on 47 basic vocabulary items across 18 localities; multidimensional scaling is applied to the inter-settlement distances it yields; and the first axis is reduced to a single variable. That variable becomes the response in a spatial error model whose explanatory variables are the number of households, population, paddy area, field area, koku yield and elevation.

The order is to compress the linguistic side into one number, then measure its relation to conditions outside language. The second half, the spatial error model, is covered on the linear mixed model and AIC page.

Ten years of files remain

This was not a one-off attempt. The working files hold 157 NEXUS files, sorted into directories by year from 2010 to 2019. The blocks break down into taxa, characters, distances, splits and networks — the procedure described in section 3, splitting a distance into splits and drawing it as a network, appearing directly as the structure of the files.

One file from 2010, for instance, holds a distance matrix over 110 taxa, from which 217 splits and a network of 218 vertices and 217 edges have been written out. The same file contains a CYCLE block recording the order in which the 110 taxa are arranged around a circle. The circular ordering described in section 3 is preserved as an intermediate result of the computation.

6Limits

The paper states its own reservation through the choice of method.

Whether a pattern of inheritance exists at all is unknown

The paper states: “since this analysis treats papers with two different lineages, those of Kōdō Keiryōgaku and those of Keiryō Kokugogaku, and since it is in any case unknown whether an exact pattern of inheritance such as is seen in organisms exists between earlier and later papers, it is considered desirable to examine them in the form of a network that permits contamination”.

Choosing a network rather than a tree is itself the statement of that reservation. The choice of method limits in advance the range of what may be claimed, which makes for an honest construction. Put the other way, this network cannot be read as saying that A inherited from B. What is readable is proximity in vocabulary, and the fact that this proximity does not fit into a tree.

Distance and position on the page carry no meaning

Of Figure 8 the paper states explicitly that “only the branch lengths and the manner of branching carry meaning”. Where a node sits on the page, or how long the straight line between two nodes is, must not be read.

The same caution accompanies the co-occurrence networks in the same paper (Figs. 6 and 7), where “only the thickness of an edge carries meaning; the length of an edge and the positional relations among nodes carry none”. Which quantity carries meaning differs from one network figure to the next. Interpreting apparent proximity without reading the legend leads to error.

The two inputs being compared are not aligned

Thirty of the 31 texts are individual papers, but the thirty-first, reskk, is 31 years of titles and keywords bundled into one text. The units do not match.

The nature of the data differs as well. The Kōdō Keiryōgaku side is body text (with English abstracts, references, equations and figure text removed); the Keiryō Kokugogaku side is titles and keywords only. Word selection differs too — by descending standard deviation on the one side, by descending frequency on the other. The procedures are not the same. The periods differ as well: 1957–1987 for Keiryō Kokugogaku, 1974 to the time of writing for Kōdō Keiryōgaku, overlapping only in part.

All of this is stated in the paper; nothing is concealed. But when the distance between reskk and a given paper is read off the network, how much of that distance comes from a difference in content and how much from a difference in how the data were built cannot be separated.

The set of admitted splits is fixed for the convenience of drawing

Neighbor-Net admits as candidates only the splits compatible with the circular ordering fixed beforehand. That restriction exists in order to obtain a network drawable in the plane; it is not derived from the data. Change how the ordering is determined, and the set of admitted splits may change as well. Reading the shape of the network as “the structure the data showed” requires keeping this design constraint in view.

A network is used because a tree is hard to fit to language

Biological descent is, in the main, vertical inheritance from parent to offspring, which is why a tree fits. In language, however, forms and tone patterns are borrowed from neighbouring areas as a matter of course. An element transmitted horizontally cannot be assigned to one parent or the other on a tree.

That is the reason for choosing this method. The capacity to permit contamination, described in section 1, is not an exceptional accommodation where linguistic data are concerned but the premise itself. Because conflicting signal can remain as a box, a distribution mixed by borrowing can be drawn without being collapsed into a tree.

Which also means that a large box is itself information. Ignoring it and reading off “roughly this tree shape” defeats the point of using the method.

A bootstrap value is not the probability of historical correctness

The figure 86.1% on a branch is the proportion of times that branch appeared when the data were resampled. It measures the degree to which the data at hand repeatedly show the same tendency, and no more.

It does not mean that the branching occurred historically with probability 86.1%. If the way the data were collected is biased, the biased structure will be reproduced stably and the value will come out high. A high value is evidence that the data are stable, not that the estimate is close to the truth.

The premises of glottochronology are deliberately not taken on

Using basic vocabulary for phylogenetic estimation belongs to the line of glottochronology that begins with Swadesh and was developed by Shirō Hattori. Onohara (2012), however, states explicitly that its premises are not adopted, setting out the problems as follows: “because glottochronology took the rate of linguistic change to be constant, and took as a premise that the two languages compared are cognate in the comparative-linguistic sense with no contact between them after divergence, there have been many cases in which observed linguistic phenomena and the theory of glottochronology fail to agree. For this reason, research deriving dates of lexical divergence from basic vocabulary has fallen out of favour”.

On that basis the use is restricted: “basic vocabulary tends to be avoided on the grounds that its use as glottochronology is problematic. Nevertheless, in the restricted use of extracting similarity among the vocabularies spoken in a region, effective results were obtained in the present analysis”.

In other words, no dates can be read from this network. Branch length is the extent of lexical difference, not elapsed time since divergence. Since the assumption of a constant rate has not been made, a quantity that depends on it must not be read.

Change how the distance is built and the branching changes

The input is a distance matrix alone. Which means that the same set of localities yields a different network under a different definition of distance. Across this body of research, Gower distance is used for nominal-scale data, the Jaccard coefficient for presence–absence data, and normalized compression distance between XML documents, according to the object.

The choice of distance is the analyst’s judgement, not something the data fix. The shape of the network is “the structure the data showed” and at the same time “the structure as measured by that distance”. The discussion of taking a compression distance as input is on normalized compression distance; for a different geometry over the same kind of data, see correspondence analysis.

7Sources and data

On the dialect research side

  • 小野原彩香 (2012)「揖斐川上流域の語彙に関する系統推定」情報処理学会研究報告 IPSJ SIG Technical Report, Vol.2012-CH-94 No.4 (26 May 2012). Affiliation at the time: Graduate School of Culture and Information Science, Doshisha University.
  • 小野原彩香「揖斐川上流域における語彙の類型と地理的分布」研究論文。The copy at hand carries no confirmable journal title, volume or year, so the bibliographic entry stops at the title.
  • 小野原彩香「方言分布データの数理的分析」. Affiliation: Graduate School of Culture and Information Science, Doshisha University. Likewise, the venue and volume could not be confirmed from the copy.
  • 小野原彩香 (2013)『数理的アプローチからの言語変化と外言語的要素との関わりに関する研究』同志社大学大学院 文化情報学研究科、2013年度。Doctoral thesis. Chapter 3 (case 1) covers the accent of Shōdoshima, chapter 5 (case 3) the basic vocabulary of the upper Ibi River basin, and chapter 8 (case 6) a multivariate analysis of basic vocabulary against extralinguistic factors.
  • The dialect data derive from the 200 Japanese dialect items of Hattori (1973) and from the reports of Yamada (1978, 1980) on the upper Ibi River basin, on Imajō and Ikeda in Fukui Prefecture, and on Kinomoto in Shiga Prefecture. The Shōdoshima accent data come from Nakai (1998) and from the 2012 survey. Networks were drawn with SplitsTree4.

The 2012 Shōdoshima survey was conducted in March 2012 as part of JSPS Grant-in-Aid for Scientific Research (A) 23242024, “Elucidating change in dialect distribution: grasping the actual state of variation and testing and constructing theory” (principal investigator: Takuichiro Onishi), led by Professor Shinsuke Kishie and the Japanese Linguistics Laboratory of Tokushima University.

References on the method

  • Saitou, N. & Nei, M. (1987) The neighbor-joining method: a new method for reconstructing phylogenetic trees. Molecular Biology and Evolution, 4(4), 406–425. academic.oup.com
  • Bandelt, H.-J. & Dress, A. W. M. (1992) A canonical decomposition theory for metrics on a finite set. Advances in Mathematics, 92(1), 47–105. sciencedirect.com
  • Bryant, D. & Moulton, V. (2003) Neighbor-Net: an agglomerative method for the construction of phylogenetic networks. Molecular Biology and Evolution, 21(2), 255–265. academic.oup.com
  • 矢野環 (2012)「『老葉』に対する系統学的アプローチ——宗祇による連歌の系譜」中尾央・三中信宏 編『文化系統学への招待：文化の進化パターンを探る』勁草書房。ISBN 978-4-326-10216-7
  • 中尾央 (2012)「文化の過去を復元すること」同書所収。
  • Gower, J. C. (1971) A general coefficient of similarity and some of its properties. Biometrics, 27(4), 857–871. jstor.org
  • Huson, D. H. & Bryant, D. (2006) Application of phylogenetic networks in evolutionary studies. Molecular Biology and Evolution, 23(2), 254–267. academic.oup.com (SplitsTree4)
  • Swadesh, M. (1952) Lexico-statistic dating of prehistoric ethnic contacts: with special reference to North American Indians and Eskimos. Proceedings of the American Philosophical Society, 96(4), 452–463. jstor.org
  • Hattori, S. (1973) Japanese dialects. In T. A. Sebeok (ed.) Current Trends in Linguistics, vol. 11: Diachronic, Areal and Typological Linguistics, pp. 368–400. The Hague: Mouton. degruyterbrill.com
  • 山田達也 (1980)「福井県今庄、池田方言及び滋賀県木之本方言の基礎語彙について——徳山村方言に関連して」『名古屋市立大学教養部紀要 人文社会研究』24、23–34頁。CiNii
  • 山田達也 (1978)「揖斐川上流域における基礎語彙について」『名古屋市立大学教養部紀要 人文社会研究』22、33–66頁。

The volume of Hattori (1973) is given as “Vol.1, no.11” in the reference list of Onohara (2012). The original is volume 11 of Current Trends in Linguistics, and the bibliographic form has been followed here. For Yamada (1978), an identical entry appears in the reference lists of the papers above, but it could not be confirmed in a bibliographic database.

In the reference list of Onohara and Yano (2014), Bryant and Moulton (2003) is given as “Molecular Biology and Evolution 21”. The paper was published online in 2003 and appeared in print in volume 21, issue 2, pages 255–265.

Which perspective this method belongs to

Neighbor-Net sets up no probability model. It approximates a distance by a superposition of splits and fixes branch lengths by minimizing the sum of squared reproduction error. It is least squares with a non-negativity constraint; the columns of the design matrix have merely become indicator functions of splits, and what is being minimized is no different from regression or principal component analysis. It belongs to Perspective 1 (geometry, least squares) in the seven perspectives in equations.

For the same kind of dialect data analysed from the opposite direction — treating the tree itself as a random variable — see Accentual class merger (Perspective 3). The two make a useful contrast: reduce to distances and draw, or write down a generative process and take a posterior.

This page sets out, for readers in the same field, a method Onohara uses in her own research. All figures and quotations are taken from what is printed in the sources; no estimates or approximations have been added.
For the correspondence analysis used alongside it in the same paper, see correspondence analysis; for the random forest used alongside it in the dialect research, see random forest; for a different construction of distance, see normalized compression distance; and for what receives the output of phylogenetic estimation, see the linear mixed model and AIC. The overall placement of methods is in the seven perspectives in equations, and which course sessions fall under which perspective is in Methods & Mathematics.