Spanish researchers have developed a tool to measure academics’ knowledge, skills and abilities and improve research efficiency. In the past decade, despite steady growth in academic staff and investment, European universities have “not always reached the desired levels of research productivity”, write the authors of a study published in Science and Public Policyin August 2021. This suggests “inefficiencies in their management research policies”, they write.
To maximize the strategic resource of academic human capital, universities need a tool that can identify and assess the right staff attributes to match their academic goals, say the authors, from the Faculty of Business and Management at the University of Cadiz, Spain.
The tool they developed can be useful to address “some of the limitations of the traditional metrics of research competence, (which are) based almost exclusively on objective indexes built from citation counts”, says co-author and business management academic, Félix Guerrero-Alba.
“The model highlights specific underlying measures that are normally not included in traditional systems because of their psychometric difficulties,” he says.
Evidence-based tool
In order to identify relevant indicators for assessing academic human capital, the authors asked 62 research group leaders at Spanish institutions 8 targeted questions, such as, “In your opinion, what knowledge, skills, and abilities should a researcher have to develop his/her research activity efficiently?”
Based on the responses, follow-up queries and several rounds of discussions within the group, they reached consensus around 22 indicators. These were refined to statements in a survey tool, from “I have the necessary training in research methodologies and techniques” and “I know the most relevant publications in my scientific field” to “I have the ability to interact fluently with other researchers” and “I consider myself a creative person.”
Answers are rated on a five-point scale from “completely agree” to “completely disagree”.
The indicators describe five dimensions of academic human capital, according to the authors: research knowledge, research ability, alertness skills, work organization skills and criticism skills. Individuals can use their scores to self-assess, to guide them in developing their careers and identify areas where they need training, the authors suggest.
Principal investigators can use the tool to ensure a good balance and range of attributes in research teams, while institutions can use it as a “supplementary way of assessing potential promotions and conducting evaluations of academic researchers”.
It can also be used to design talent-management initiatives, the authors suggest. For example, it could be used to justify funding allocations within institutions, based on specific academic human capital requirements, and to compare researchers within and between universities.
Caution needed
“This study has codified some of the essential academic human capital factors, and this can help inform the development of institutional career development support for individual academics,” said James Cunningham, professor of strategic management at the Newcastle Business School in the United Kingdom, who has no connection to the research.
“This is critical, as the academic role and the demands on individual academics have changed significantly.”
Yet caution is warranted, says Alison Barnes, president of the National Tertiary Education Union in Australia, which represents higher education employees.
The adoption of new research-performance metrics, when quantitative minimums become pre-requisites for advancement, has in the past resulted in the inability of staff to access promotions and even termination for unsatisfactory performance, she says.
“So, in this context we are cautious about quantitative performance measures, how they are generated, and how they are applied to individual staff across diverse research areas,” says Barnes.
“That said, a system of this nature that measures skills rather than externally measurable outputs would be an improvement from our perspective.”
But to be an improvement, it would need to “completely replace existing measures”, rather than be imposed on top of them, Barnes said.
The model is yet to be used outside the University of Cadiz. More data are required to confirm its validity, including in different national contexts, and to determine the relationship between research performance and academic human capital scores, the authors said.
https://www.natureindex.com/news-blog/new-tool-assess-researchers-promotion-recruitment