Technological innovation is a driving power for financial progress and social adjustments, and in specific, disruptive innovation has the potential to drastically change the present industrial buildings and market environments. From the attitude of formulating insurance policies and industrial methods, strategies of discovering such applied sciences earlier and assessing their impression objectively is a big problem. In the sphere of meta-science, qualitative evaluation is performed conventionally, however research on quantitative evaluation approaches by way of such means as huge knowledge evaluation are additionally progressing. Here, the writer will introduce approaches developed in latest years and will analyse research undertaken in Japan utilizing these approaches.
1. Which research modified the circulate of the science (Disruption Index)
The Disruption Index (D-Index) is an index which signifies the impression of a sure research paper on the quotation community after its publication, that’s, whether or not the paper is disruptive (“gives birth to a new trend”) or developmental (“deepens existing trends”). This index was developed by a examine crew of the University of Chicago in the United States, and others in 2019 and was revealed in Nature (Wu et al., 2019).
The D-Index is calculated as follows by categorizing research papers revealed after the publication of the focal research paper into three sorts:
- A research paper that cites the focal paper however doesn’t cite its references (Nsolely)
- A research paper that cites each the focal paper and its references (Neach)
- A research paper that doesn’t cite the focal paper however cites its references (Nrefonly)
[
mathrm{Dtext{-}Index} = frac{N_{mathrm{only}}−N_{mathrm{both}}}{N_{mathrm{only}}+N_{mathrm{both}}+N_{mathrm{refonly}}}
]
When evaluating current research based mostly on D-Index scores, research papers that created new paradigms in tutorial circles and research papers that considerably modified the traditional frameworks are extracted. The D-Index for fully disruptive research papers is 1 and that for fully developmental research papers is -1.
In the newest paper by Lin et al. (2025) that analysed a complete of 49 million research papers revealed between 1800 and 2024, a comparability between knowledgeable interviews and D-Index scores was performed. Papers that consultants evaluated as essentially the most disruptive in the world had been a paper on the elucidation of molecular construction of DNA by Watson & Crick (D=0.96), a paper on fractals by Mandelbrot (D=0.95), and a paper on deterministic nonperiodic circulate (butterfly impact) by Lorenz (D=0.81), all of which present excessive D-Index scores. On the opposite, a paper on the non-cooperative sport principle by Nash (Nash equilibrium), for instance, reveals a comparatively low D-Index rating (D=0.28). This paper by Nash developed the content material of a earlier paper on a sport principle by John von Neumann et al., which had been revealed seven years earlier than, and appears to be thought-about to have been developmental.
Let us transfer on to the analysis of Japanese research based mostly on the D-Index. Using the open knowledge revealed by Harvard University this February (Li et al., 2025), the writer ranked the highest 5 research papers based mostly on the comparability with the quantity of citations, an method typically used for evaluating research papers.
The most cited paper is the Akaike’s Information Criterion (AIC) by the late Hirotsugu Akaike, which has been cited 48,000 occasions. What is attention-grabbing is that two sorts of MEGA sequence, software program for creating developmental phylogenetic timber, by Prof. Koichiro Tamura (Tokyo Metropolitan University) et al. are ranked in the highest 5, however their D-Index scores are slightly low (0.371 for MEGA6 that’s ranked second and 0.06 for MEGA5 that’s ranked fourth). This displays the truth that these papers characterize developments based mostly on earlier research.
On the opposite hand, in phrases of the D-Index, a paper on photocatalysts by Mr. FUJISHIMA Akira is the highest ranked paper with a excessive rating of 0.998. Following this are the “Toyota production system and Kanban system” offered at a tutorial convention by Mr. SUGIMORI et al. of Toyota Motor Corporation and a paper on quarks and leptons by Prof. INAMI Takeo (Chuo University) et al. These two papers present extraordinarily excessive D-Index scores, though their numbers of citations had been smaller by an element of 10. They might be evaluated as research that current new ideas.
3. Developing new know-how coverage instruments
The use of any of the quantitative evaluation approaches launched right here might enable for the target extraction of disruptive applied sciences and revolutionary applied sciences from literature knowledge. Their use will enable policymakers to appropriately talk about which fields ought to obtain intensive assist and establish areas of threat.
As a sensible software, these approaches have come to be utilized by nationwide governments in setting their precedence areas and analysing dangers. In the United Kingdom, an initiative titled the “Metascience Novelty Indicators Challenge,” shall be hosted by the UK Research and Innovation (UKRI) this autumn. This is a high-visibility initiative geared toward in search of concepts on new indicators for figuring out novelty in research broadly from most of the people, with an award of 300,000 kilos (approx. 60 million yen). The initiative is co-hosted by Elsevier, which operates the research article database, RAND Europe, and the University of Sussex, and winners can even obtain assist from these co-hosts.
On the opposite hand, you will need to acknowledge the restrictions of quantitative indicators in order to keep away from an over-reliance on them. New applied sciences can’t be evaluated with the D-Index, and novelty ranges might range relying on how fields and phrases are chosen. Whether superior know-how trends might be captured by publication of research papers in the primary place additionally bears examination. Therefore, it’s essential to appropriately mix qualitative assessments and quantitative indicators in precise operation. How can we quickly confirm the disruptiveness and novelty of new applied sciences and make the most of them in insurance policies? We have to proceed growing our information of meta-science and selling its software and verification each domestically and internationally.
July 9, 2025
>> Original text in Japanese