A propósito de produção científica, aí abaixo pode-se ler uma boa reflexão sobre o mau uso dos chamados 'fatores de impacto'.
The hidden factors in impact factors: a perspective from Brazilian science
- Laboratório de Genômica Evolutiva e Biocomplexidade, Departamento de Microbiologia, Imunologia e Parasitologia, Universidade Federal de São Paulo, São Paulo, Brazil
Contrary to all current international recommendations on evaluation of academic achievement the evaluation of graduate programs in Brazil relies heavily on journal impact factors (Garfield, 2006; San Francisco Declaration on Research Assessment, 2013). The governmental agency CAPES from the Education Ministry monopolize this evaluation and pressure programs by the distribution of funding resources and departmental fellowships conditioned to adherence to a journal classification system called “Qualis” which is a discretization of the continuous distribution of journals ranking by their impact factors (Greenwood, 2007). In several institutions the graduate committee authorizes professors to act as thesis advisors only if in a certain period (e.g., 4 years) they publish at least one paper in a journal classified as “Qualis A2.” This classification has seven categories with decreasing impact factor ranges (A1, A2, B1, B2, B3, B4, B5, and C,http://www.capes.gov.br/
To investigate the “Qualis” effect we obtained journal citation data from Thomson Reuters for six journals in the Biological area and calculated, from raw data, the impact factors (mean), standard deviations, median, and mode. The journals selected are in the CAPES classification “Biology III” and correspond to the highest impact factor of A2 and B1, the middle impact factor of A2 and B1 and the lowest impact factors of A2 and B1. The A2 category includes 180 journals and B1 includes 255 journals.
These journals lie at the “exclusion zone,” in other words the frontier between A2 and B1 exactly where a professor may or may not be promoted to formal advisor status. As expected, the distributions of citations for these six journals follow power laws (Baum, 2011). As widely known power laws cannot be compared on the basis of their means (the impact factor) but preferably on their medians. Also, the standard deviations are larger than the means, showing the over-dispersed nature of the data and how the mean is meaningless in this case. In Figure 1 the journal ranking is depicted showing that the differences between journal means (impact factors) are significantly smaller than the standard deviations. This shows that the “Qualis” categories and journals considered are not statistically different. How can scientists and publications from graduate programs be compared on the basis of these measurements? Could this be, at least in part, responsible for the drop in citations of Brazilian research? As shown by others (Editorial, 2005) is this an effect of free-riding on power law outliers?
FIGURE 1
Figure 1. Statistics of impact factors of journals indexed in the Brazilian Qualis.BBActa, Biochemical and Biophysical Acta; JCM, Journal of Clinical Microbiology; BMC Mic., BMC Microbiology; Exp. Derm., Experimental Dermatology and Curr. Genet., Current Genetics; SD, Standard Deviation of the impact factor (error bars).
This shows yet another nationwide example of misuse of journal based metrics to evaluate individual scientists and its catastrophic consequences when, contrary to recommended by the International Mathematical Union (IMU Report, 2011), it is imposed by governmental agencies. In a country that is desperately trying to move from a peripheral scientific and technological condition to be among the major players, the journal impact factor based “Qualis” might be bad news.
References
Hermes-Lima, M. (2013). Methodologies of CAPES Evaluation. Available online at:http://cienciabrasil.
IMU Report. (2011). Report of the Journal Working Group of the International Mathematical Union. Available online at:http://www.mathunion.org/
Rughetti, S. (2013) Brasil Cresce em Produçãoo Científica mas Índice de Qualidade Cai (Brazil Grows In Scientific Production but Quality Index Falls). Folha de São Paulo, April 22nd. Available online at:http://www1.folha.uol.com.
San Francisco Declaration on Research Assessment. (2013). Available online at:http://www.ascb.org/
-----------------------------
Nenhum comentário:
Postar um comentário