1What the method is for
Putting “which object goes with which category” into a single figure.
What is at hand is an \(I \times J\) contingency table \(N\). Rows are objects (papers, respondents), columns are categories (nouns, dialect items), and each cell holds a count. The question is a single one: which rows go with which columns.
The naive move is to compute a rate per row and line them up. But with 24 or 30 columns, no one grasps the structure by staring at the table. The dimension has to come down. Principal component analysis could be used here, but it places either the rows or the columns, not both. Correspondence analysis places rows and columns on the same plane at once. “This paper uses this word distinctively”, “this generation uses this dialect form” — such associations become readable as proximity between points.
That is why correspondence analysis sits at the centre of two of Onohara’s studies. The journal study puts 30 papers and 24 nouns on one figure; the dialect survey puts 809 respondents and the survey items on another.
2Origin
From reading contingency as correlation, to the French school of data analysis.
The starting point is Hirschfeld (1935), who posed the problem of assigning numbers to the rows and columns of a contingency table so that their correlation is maximized. In today’s terms, a discrete counterpart of canonical correlation.
Benzécri (1973) systematized this as L’Analyse des Correspondances, and it became a central instrument of the French school of data analysis. Its character is to render the geometric structure of the data directly as a figure, rather than to fit a model and test it; it reached the English-speaking world largely through Greenacre’s work. In Japanese, Adachi and Murakami (2011) present it as a continuation of principal component analysis, and Onohara and Yano (2014) cite that book when defining the method.
The definition given in the paper
Onohara and Yano (2014) describe correspondence analysis as “a visualization method that represents, in a low-dimensional space and at the same time, the proximity among objects, the proximity among the categories belonging to each variable, and the proximity between objects and each category, when several variables measured on a nominal scale have been observed on the same set of objects”. The weight falls on handling three kinds of proximity simultaneously.
3Where it comes from mathematically
A singular value decomposition under chi-square distance. A weighted least-squares approximation.
With \(n\) the grand total, form the correspondence matrix \(P = N/n\). Write the row sums as \(\mathbf{r}\) and the column sums as \(\mathbf{c}\), with \(D_r = \mathrm{diag}(\mathbf{r})\) and \(D_c = \mathrm{diag}(\mathbf{c})\). What correspondence analysis decomposes is the departure from what independence between rows and columns would give.
\(\mathbf{r}\mathbf{c}^{\top}\) … the proportions expected if rows and columns were independent
\(D_r^{-1/2},\,D_c^{-1/2}\) … weighting by the marginals; rarer rows and columns are treated as larger
\(\Sigma\) … singular values \(\sigma_1 \ge \sigma_2 \ge \cdots\), the square roots of the inertia on each axis
Total inertia is the chi-square statistic itself
The whole departure from independence is exactly what gets decomposed. The explained proportion of an axis, \(\sigma_k^2 / \sum_l \sigma_l^2\), says what share of that departure the axis picked up.
The distance in use is not Euclidean
Rows are first turned into profiles \(p_{ij}/r_i\) summing to one, then compared. Dividing each term by the column marginal \(c_j\) is what defines chi-square distance, and this is where it parts from principal component analysis.
The difference from PCA, written side by side
What principal component analysis maximizes is variance under Euclidean distance.
Both are minimizations of squared error in a low-rank approximation, and both are solved by the singular value decomposition (Eckart–Young). All that differs is what goes into the matrix and how it is weighted.
Principal component analysis decomposes the raw (or standardized) values; correspondence analysis decomposes the departure from independence, weighted by the marginals.
Correspondence analysis therefore extends the section “principal component analysis has the same idea behind it” in Perspective 1 (geometry) of the seven perspectives. Swap the design matrix and the weights; what is being minimized has not changed.
4Definition and assumptions
The input is a table of non-negative counts. The size of a row disappears.
- Non-negative counts The table has to make sense as a contingency table. It cannot be used on negative values, or on a table mixing quantities that mean different things.
- Row profiles Each row is rescaled to sum to one before comparison, so differences in the size of a row disappear. The length of a paper, or the number of respondents, does not enter the distance. Which also means it is the wrong tool when the quantity itself is the question.
- Axes come out in order of inertia The first axis is fixed as the direction picking up the most departure. What that axis means is not something the method answers.
- Dimension falls off continuously The loss from reducing to \(K\) dimensions is \(\sum_{k>K}\sigma_k^2\). To look at a two-dimensional figure is to discard the third axis onward.
5How it is used in this research
In two studies, the rows and columns hold entirely different things.
(A) Journal submissions (2014)
Rows are papers, columns are nouns. The study asks which topics an interdisciplinary journal has carried, and how they shifted.
- Selecting the corpus From the 481 papers of Kōdō Keiryōgaku (The Japanese Journal of Behaviormetrics) available on J-STAGE, 30 were selected in which “dialect”, “language”, “speech” or “text mining” appears in the title, keywords or body.
- Data cleaning English abstracts, end references, in-text reference information, equations, symbols, affiliation fields, text inside figures and tables, and acknowledgements were removed.
- Morphological analysis and counting Parsed with MeCab; noun counts alone tallied with MTMiner. The number of distinct words is 6,096.
- Normalizing for length Since papers differ in length, the frequency of each word was taken as \(tf_j/n_i\), dividing by the total noun count \(n_i\) of paper \(d_i\).
- Narrowing the word list The standard deviation \(s\) of this value was computed per word and sorted in descending order. The first 24 words, up to where the rate of decrease stabilizes, were kept.
- Correspondence analysis Applied to the 30 papers × 24 words table.
- Principal component analysis Applied to the same data structure, to look from another angle.
The reasoning is that a word whose use varies widely across papers expresses the differences among them. Put every word in, and words appearing equally everywhere take over the axes.
| Method | Axis 1 | Axis 2 | Axis 3 | Axis 4 | What could be read |
|---|---|---|---|---|---|
| Correspondence analysis | 14.45% | 13.01% | — | — | Papers cluster by genre |
| Principal component analysis | 20.54% | 12.1% | 10.27% | 9.38% | A temporal shift in first-component scores |
Figures quoted from the body of Onohara and Yano (2014). For correspondence analysis the paper does not report the explained proportion beyond the second axis, so those cells are left blank.
Why both were applied
The paper states that principal component analysis was run on the same 30 × 24 data “in order to consider the data from another angle”. The result is that the two methods showed different things.
In the correspondence analysis figure, papers appear at first glance to cluster by year. But when the original papers are divided by genre (speech, social survey, education, psychology, text mining, other) and overlaid, they turn out to cluster by genre, not by year. Special issues in 1995, 2003 and 2011 had carried several papers of the same genre, which is what made the clustering look temporal. What correspondence analysis picked up was the structure “different fields use different words”.
Principal component analysis, on the other hand, showed that when first-component scores are re-sorted by year, they turn from negative to positive from the 2000s onward. The positive side of the first-component loadings holds “evaluation”, “nature”, “scoring”, “task”, “person”; the negative side holds “word”, “phoneme”, “vocabulary”, “speech”, “language” — readable as a shift toward topics in education. This temporal change did not appear on the second component or below.
To be precise, though, the difference is not due to the methods alone. The correspondence analysis result was read with genre information overlaid, and the principal component result was read after re-sorting the scores by year. Which external information the low-dimensional coordinates are matched against changes what becomes visible.
(B) A dialect survey on the prefectural border (2015)
Rows are respondents, columns are dialect items. The survey asks how the use and comprehension of dialect divides across the old provincial border between Ōmi and Mino, in Maibara City (Shiga Prefecture) and Sekigahara Town (Gifu Prefecture).
- The survey A written questionnaire completed by an interviewer. It covered junior high school students in three school districts of Maibara (Kashiwabara, Kanan, Ibukiyama) and two of Sekigahara (Imasu, Sekigahara), together with adults resident in those areas. Adults are split into a parent generation (under 60) and a grandparent generation (60 and over).
- Scale 544 students and 329 adults took part. Valid responses number 809, a valid response rate of 92.6%. The survey ran from November to December 2015.
- Items For all 40 items, covering 106 words, respondents were asked whether they knew the form and whether they used it.
- Correspondence analysis Respondents and survey items were placed on the same figure (Fig. 2). It recurs throughout the booklet: by school district (Fig. 3), for east–west contrasting items (Fig. 4), for the similarity among areas (Fig. 6), for the usage rates of Ōmi-type and Mino-type forms (Figs. 8 and 9), and for the school dialect of students (Fig. 13).
What was read from it
In Figure 2, students, the parent generation and the grandparent generation separate cleanly from left to right, while respondents from Sekigahara concentrate above the origin and those from Maibara below it. Speech differs by generation, and it also differs between the two municipalities. The survey items line up in the same way: forms used by younger speakers to the left, forms belonging to older speakers further right.
The booklet introduces correspondence analysis as “a method that reduces multi-dimensional data to a low dimension, making the tendencies in the data easier to grasp”. It keeps technical vocabulary to a minimum and explains first how to read the figure: points close together mean similar answers.
6Limits
The figures read well, but what is being read is only a part.
The two-dimensional figure explains less than a third of the whole
The correspondence analysis in (A) gives 14.45% on the first axis and 13.01% on the second. Together, 27.46% — no more. Over seventy per cent sits on the third axis and beyond, which the plane does not draw. Principal component analysis gives 32.64% on the first two, and only reaches 52.29% once the fourth axis is added.
So two points being close in the figure means no more than “close in these two dimensions”. They may always be far apart on a discarded axis. Conversely, points far apart in the figure are not necessarily unrelated. A claim read off a configuration plot is guaranteed only within the explained proportion.
The meaning of an axis is supplied by the analyst, not by the method
What correspondence analysis returns is a set of axes ordered by inertia, and nothing more. It never says that the first axis means “new ↔ old”, or that the second means “Sekigahara ↔ Maibara”.
Figure 2 in (B) is annotated with “new ← → old” on the horizontal axis and “Sekigahara ↑ / Maibara ↓” on the vertical, but these are interpretations the analyst gave after seeing the configuration. The names were attached after confirming that generation and area — information known in advance — line up along the axes, so the order of reasoning is sound. What is hard for a reader to see is that the axis labels are an interpretation and not an output of the analysis. In material written for a general readership this deserves particular care.
Chi-square distance gives heavy weight to rare categories
Because \(1/c_j\) enters the definition of the distance, the rarer a column, the harder it pulls on the axes. A word occurring in a single paper, or a dialect item used by a single respondent, can end up fixing the first axis.
That (A) did not use all 6,096 words but narrowed to the 24 with the largest standard deviation is in part a response to this. The paper itself notes that “some have rather low counts. Since these do not exert a large influence in the analysis, it is desirable to exclude them in advance”. But where to cut is the analyst’s judgement: the threshold of 24 words was chosen by eye, at the point where the decrease in standard deviation stabilizes.
Losing the size of a row is both a strength and a weakness
Because of profiling, differences in the length of a paper or the number of respondents do not enter the distance. Being able to compare objects of differing extent is an advantage, but the quantitative information — that something is used a great deal — is dropped. If the question is what share of the whole a word accounts for, the correspondence analysis figure cannot answer it. That (A) also reports raw occurrence counts and IDF alongside, in Table 2, can be read as an arrangement that supplies what is missing.
7Sources and data
(A) Journal submissions
- 小野原彩香・矢野環 (2014)「雑誌『行動計量学』における言語に関する投稿論文の傾向——投稿論文の時系列変化と『計量国語学』論文との比較」岸江信介・田畑智司 編『テキストマイニングによる言語研究』ひつじ研究叢書〈言語編〉第121巻、pp. 1–20、ひつじ書房。ISBN 978-4-89476-695-2 (first printing, 3 December 2014)
Onohara, A. & Yano, T. (2014) Trends in language-related submissions to the journal Kōdō Keiryōgaku: temporal change and a comparison with Keiryō Kokugogaku. In S. Kishie & T. Tabata (eds.) Text Mining and Its Applications in Language Research, Hituzi Linguistics in English No. 121, pp. 1–20. Tokyo: Hituzi Syobo. - The material analysed is the set of papers from Kōdō Keiryōgaku published on J-STAGE. For Keiryō Kokugogaku (Mathematical Linguistics), the titles and keywords distributed on the society website (1957–1987, in CSV).
(B) The dialect survey on the prefectural border
- はびろネット『滋賀・岐阜県境を越えた方言ハンドブック』（「まいばら協働事業提案制度」採択事業、東西東西プロジェクト）
Habiro Net, A Dialect Handbook Across the Shiga–Gifu Prefectural Border. Published by a citizens’ group based in the Kashiwabara district of Maibara City, under the city’s collaborative-project proposal scheme. - According to the afterword, Habiro Net has run its “Tōzai Tōzai” project under that scheme since fiscal 2014, and the theme for fiscal 2015 was dialect. Sekigahara Town was considering a similar survey, so the two were carried out as a joint project, producing a questionnaire survey that crossed the prefectural border. The afterword records that Ayaka Onohara and Yoshihiro Sugisaki advised on the survey and carried out the analysis.
This booklet is not a peer-reviewed paper
(B) is a survey report for a general readership, issued by a local citizens’ group; it was not submitted to or refereed by an academic journal. It is built as something to read, including a dialect dictionary and columns, and it does not record the details of the analysis (software used, preprocessing, criteria for selecting axes). What is quoted here is the survey summary and the figures printed in the booklet, together with its text.
References on the method
- Hirschfeld, H. O. (1935) A connection between correlation and contingency. Mathematical Proceedings of the Cambridge Philosophical Society, 31(4), 520–524. doi:10.1017/S0305004100013517
- Benzécri, J.-P. (1973) L’Analyse des Données. Tome 2: L’Analyse des Correspondances. Paris: Dunod.
- 足立浩平・村上隆 (2011)『非計量多変量解析法：主成分分析から多重対応分析へ』シリーズ〈行動計量の科学〉9、朝倉書店。ISBN 978-4-254-12829-1
Adachi, K. & Murakami, T. (2011) Nonmetric Multivariate Analysis: From Principal Component Analysis to Multiple Correspondence Analysis. Tokyo: Asakura Publishing. - 古沢正三・芳賀敏郎 (1992)『多変量解析事例集 第1集』日科技連出版社
Furusawa, S. & Haga, T. (1992) Casebook of Multivariate Analysis, vol. 1. Tokyo: JUSE Press. Cited by Onohara and Yano (2014) for the definition of principal component analysis.
In the reference list of Onohara and Yano (2014), the subtitle of Adachi and Murakami (2011) is given as “from principal component analysis to correspondence analysis”, whereas the book’s formal subtitle reads “to multiple correspondence analysis”. The bibliographic form has been followed here.
Which perspective this method belongs to
Correspondence analysis sets up no probability model. It arranges the departure from independence in a matrix and fixes the axes by minimizing the squared error of a low-rank approximation. That the solution is given by the singular value decomposition is the same structure as in principal component analysis and k-means. Only the way distance is measured differs; what is being minimized does not. It falls squarely under Perspective 1 (geometry, least squares) in the seven perspectives in equations.
This page sets out, for readers in the same field, a method Onohara uses in her own research. All figures are quoted from values printed in the sources; no estimates or approximations have been added.
For the phylogenetic network used alongside it in the same study, see Neighbor-Net. The overall placement of methods is in the seven perspectives in equations, and which course sessions fall under which perspective is in Methods & Mathematics.