University of Divinity

Name (Individual/Organisation)

University of Divinity

Responses

Q1. How could the purpose in the ARC Act be revised to reflect the current and future role of the ARC?

For example, should the ARC Act be amended to specify in legislation:
(a) the scope of research funding supported by the ARC
(b) the balance of Discovery and Linkage research programs
(c) the role of the ARC in actively shaping the research landscape in Australia
(d) any other functions?

If so, what scope, functions and role?

If not, please suggest alternative ways to clarify and define these functions.

The Act does not require substantive change. It is an instrument that essentially does what it is meant to do: constitute the ARC, define its core purpose, its core staff and accountability mechanisms, and set out its funding arrangements. Its key purpose is to set the boundaries between the functions and authority of the ARC CEO and the functions and authority of the Federal government (the Minister). A key principle of documents of this kind is that they be kept as simple as possible to enable flexibility of operation and evolution of function under the umbrella of the constituting document. In its current iteration the Act permits the incumbent government to use the ARC to support priorities as determined by that government. That is, the government sets the agenda and has discretion to influence how money is divided across schemes.

A key purpose of the Act should be to ensure that neither the Minister nor the CEO can pervert the purpose of the ARC by introducing ideological bias.The question is whether the clauses of the Act are sufficient to preserve the independence of the ARC, with particular regard to the Minister having veto over the decision-making of the ARC in regard to what projects are funded in a particular grant scheme round (determination of National Benefit). And whether the division of available money across schemes should be embedded fully in the Act. These are key points for the Review Panel to consider.

It is agreed that establishing a.-c. as defined above as guiding principles is important. The University does not believe that the Act is the appropriate place to do this. These matters are better defined and set out in the ARC Strategy document, adjusted periodically according to benchmarked best practice and refined on the basis of wide sector consultation.

Q2. Do you consider the current ARC governance model is adequate for the ARC to perform its functions?

If not, how could governance of the ARC be improved? For example, should the ARC Act be amended to incorporate a new governance model that establishes a Board on the model outlined in the consultation paper, or another model.

Please expand on your reasoning and/or provide alternative suggestions to enhance the governance, if you consider this to be important.

The functions of the Board as outlined in the Consultation document are desirable, especially if these functions are not currently within the remit of the Advisory Committee. It is especially important that there is a body that makes recommendations to the Minister regarding the appointment of the CEO.

It is strongly recommended that regardless of how governance is constituted (Advisory Committee or Board) membership be required to include a representative of the Creative Arts sector to ensure that the voice of cultural research, which is typically non-traditional, is represented. The contribution to national interest and the economy by that sector is significant and not recognised in current governance, which has a strong STEM and industry-focused bias. Indigenous representation in governance should also be required. That is, any advisory body should reflect the full diversity of the Australian research sector, inclusive of HASS and pure basic research disciplinary expertise.

Q3. How could the Act be improved to ensure academic and research expertise is obtained and maintained to support the ARC?

How could this be done without the Act becoming overly prescriptive?

While some more explicit wording could be introduced to the Act in this regard, how this is obtained and maintained is not appropriate to the Act.

Q4. Should the ARC Act be amended to consolidate the pre-eminence or importance of peer review?

Please provide any specific suggestions you may have for amendment of the Act, and/or for non-legislative measures.

Yes. The Act should be amended to consolidate the pre-eminence or importance of peer review and to require any intervention by the Minister in genuine and extraordinary circumstances to be reported to Parliament as outlined in the Consultation document. Full and open accountability to Parliament and the public by the Minister for any intervention should be made explicit in the Act.

Q5. Please provide suggestions on how the ARC, researchers and universities can better preserve and strengthen the social licence for public funding of research?

The National Interest Test and publication of the NIT statement for funded applications is a very basic and minimally useful means of preserving and strengthening the social licence for public funding of research. Its benefit is higher to researchers themselves in the case of HASS and pure basic researchers in that it challenges them to think intentionally about how their research benefits Australia and to learn to communicate this in plain language.

Engagement and Impact drivers set by the ARC are also helpful for producing narratives (success stories) that can be captured by the media.

For HASS disciplines there is a significant aspect to creating public awareness of the public benefit of research that is better achieved through the respective Academies conducting and publishing evidence-based research regarding the economic, social, and cultural benefit of their research, and through the Education Minister and Chief Scientist advocating publicly for the benefit to Australia of research across the full range of disciplines (in addition to STEM).

Q6. What elements of ARC processes or practices create administrative burdens and/or duplication of effort for researchers, research offices and research partners?

ERA and EI cycles have had a significant impact on UD, which, as Australia's smallest university, has a limited number of staff in its research office who are managing all aspects of ERA and EI reporting in addition to pre- and post-award.

The University commends the ARC for implementing measures in recent grant scheme rounds that have improved the administrative burden.

Q8. With respect to ERA and EI:

(a) Do you believe there is a need for a highly rigorous, retrospective excellence and impact assessment exercise, particularly in the absence of a link to funding?

(b) What other evaluation measures or approaches (e.g. data driven approaches) could be deployed to inform research standards and future academic capability that are relevant to all disciplines, without increasing the administrative burden?

(c) Should the ARC Act be amended to reference a research quality, engagement and impact assessment function, however conducted?

(d) If so, should that reference include the function of developing new methods in research assessment and keeping up with best practice and global insights?

a. Regarding ERA and research excellence:
It is useful to have research assessed by rigorous peer review. It may be argued that such assessment takes place in the academic publication process, as research outputs appear in peer reviewed outlets. Such a process is useful to individual researchers as it gives them specific feedback, leading to building excellence over time. It is also useful to institutions, as acceptance in a particular outlet can be recorded as quantitative recognition of research excellence.

A larger assessment, such as ERA, undertaken by a panel of discipline experts, may also have its uses. It allows institutions to benchmark their research outputs’ excellence with external reviewers, who may have a higher level of expertise than the peer reviewers accepting work for publication, as well as providing some nuance in degrees of excellence beyond acceptance/non-acceptance for publication, or pass/fail of TEQSA accreditation.

However, the main perceived value of a larger national assessment seems to be in comparison to other institutions. If universities maintain rigorous internal schemes to assess research excellence—which all inevitably do, both for TEQSA compliance and for internal evaluations and career progression—then they are already well aware of their own staff’s research performance. What each institution wants to know is how they compare to others.

The reasons for desiring such comparisons are well documented and fall at the intersection of perpetuating excellence and perpetuating economic sustainability or growth, in attracting partners, staff and students to the institution. (See, for example, Brankovic, Jelena, Leopold Ringel, and Tobias Werron. "How rankings produce competition: The case of global university rankings." Zeitschrift für Soziologie 47.4 (2018): 270-288; Horstschräer, Julia. "University rankings in action? The importance of rankings and an excellence competition for university choice of high-ability students." Economics of Education Review 31.6 (2012): 1162-1176; Musselin, Christine. "New forms of competition in higher education." Socio-Economic Review 16.3 (2018): 657-683.) Such motives are not necessarily suspect, but as this consultation paper well notes, ‘drive institutional comparisons and competition in ways that often lead to counter-productive duplication of expertise’.

UD is an institution dedicated to the study of the fields of divinity. Divinity is centred on the formal study of religion, religious practices, and religious beliefs, though our work intersects with “anthropology; archaeology; area studies; classics; cultural studies; economics; education; ethics; gender studies or women's studies; health studies; history; language(s); law; literature; media studies; natural sciences; philosophy; political science; psychology; sociology; social policy and social work; visual and performing arts, as well as our new focus on Indigenous Theologies and Studies (‘Subject Benchmark Statement for Theology and Religious Studies’, UK Quality Assurance Agency for Higher Education, 2007). We believe we are well equipped to assess research excellence internally in these disciplines, through our own staff, our close networks with discipline colleagues nationally and internationally, and the many discipline-specific publishers and journals that our researchers favour.

Divinity as a discipline may be disadvantaged in an ERA-like process, as assessment panel members may not be experts in the central Theology and Religious Studies areas of our work. We have provided feedback about this issue in past ARC consultations, and continue to affirm and echo the 2021 ERA Review Report in our concerns that ‘the world standard benchmark in ERA is not well-defined and may not be applied consistently between citation analysis and peer review disciplines’, and that there are unique ‘methodological and disciplinary-specific needs, such as equity between humanities and social sciences (HASS) disciplines and science, technology, engineering and mathematics (STEM) disciplines. It should also include Indigenous stakeholders to consider Indigenous research quality standards’. If any future assessment exercise were to focus on discipline needs rather than institutional comparisons, this may help address some of the inequities between disciplines. A scheme which reported on Australian research excellence broadly by FoR code, rather than by institution, may allow institutions to work together not only to advocate for rigorous processes to assess research excellence in their fields, but to achieve research excellence through cross-institutional research projects.

It is the position of the UD that any future exercise should focus on fostering excellence in its inevitable effects, rather than purporting to merely measure excellence yet in its effects stifling the research it claims to support. Such an exercise could be truly said to be needed by the Australian research community, rather than seen as an unnecessary burden.

The notion of a non-retrospective, i.e. prospective exercise is provocative, but raises questions about what such a scheme could look like. A process of awarding recognition for the viability of future research outcomes already exists in ARC grant programs. Awarding another kind of ‘score’ or recognition not attached to funding seems tenuous in its value.

We are aware that for some, the lack of a link to funding makes ERA of less value or necessity. However, simply linking ERA to funding would not be an adequate response. Until the deficiencies in the assessment process are addressed, linking an assessment exercise to funding would only further disadvantage peer review/HASS disciplines, which would disadvantage the UD more than any other Australian University.

Regarding EI and impact:
Measuring impact in the humanities is notoriously difficult.* The ‘narrow window’ identified in this consultation document disproportionately affects humanities disciplines as impacts such as social and institutional change take place over generations rather than calendar years. Any narrow-windowed assessment exercise is therefore bound to disadvantage UD and give an inaccurate picture of our true impact. *Hammarfelt, Björn. "Using altmetrics for assessing research impact in the humanities." Scientometrics 101.2 (2014): 1419-1430; Donovan, Claire, and Magnus Gulbrandsen. "Introduction: Measuring the impact of arts and humanities research in Europe." Research Evaluation 27.4 (2018): 285-286; Pedersen, David Budtz, Jonas Følsgaard Grønvad, and Rolf Hvidtfeldt. "Methods for mapping the impact of social sciences and humanities—A literature review." Research Evaluation 29.1 (2020): 4-2.

This is particularly important, as UD is essentially a University of Technology. We exist primarily for practitioners and end users in community, for whom real applied impact is the purpose of our research. This impact is no less crucial to the thriving of Australian communities than research in the medical sciences. However, unlike medical science, we can gather very little quantitative data, or even short-term case studies to demonstrate impact.

Any short-term retrospective assessment of impact would therefore be of little value in fostering genuine impact outcomes at UD, and in fact encourage short-term-results-driven activities, undertaken to provide data, rather than to achieve genuine impact.

The 2021 ERA Review Report Recommendation 8, ‘that beginning with EI 2024, universities be required to submit impact studies proportionate in number to Unit of Assessment size and/or capacity’, does not address the issue with timescales. The UD would welcome the opportunity to be recognised for its impact on end users, but such recognition would need to reflect the true timescale of impact in humanities disciplines upon society and culture.

b. Making the process of submitting research data to the ARC easier would benefit all concerned. We are aware of ARC investigations into software structures that would allow for rolling or annual submission of data previously submitted for ERA.

It is understood that for some Australian universities, a transition to such a system would be a large administrative task, but thereafter reduce the administrative burden. The UD has already built its research management infrastructure to map with government data submission requirements, including ERA. We could therefore transition to a regularly uploaded submission of data without much difficulty.

However, the use of the term ‘data-driven’ has varied in its apparent meaning throughout this consultation process. Making the data submission process more technologically streamlined is one thing; attempting to use data rather than human effort in actual assessment is another.

While data may be more efficiently utilised in the assessment of citation-based disciplines, drawing citations data from providers of the same, it is irrelevant to the assessment of HASS, and thus to the UD. Any attempt to move the research excellence assessment process away from peer review, and towards quantitative data analysis, would oppose all the recommendations of the 20-21 ERA Review Report. It would result in the stifling of quality HASS research in favour of numbers-driven cynical publication strategies. The UD is strongly opposed to any move in the direction of more ‘data-driven’ assessment of HASS research in Australia.

Q9. With respect to the ARC’s capability to evaluate research excellence and impact:

(a) How can the ARC best use its expertise and capability in evaluating the outcomes and benefits of research to demonstrate the ongoing value and excellence of Australian research in different disciplines and/or in response to perceived problems?

(b) What elements would be important so that such a capability could inform potential collaborators and end-users, share best practice, and identify national gaps and opportunities?

(c) Would a data-driven methodology assist in fulfilling this purpose?

UD values the idea of harnessing the ARC's capability to evaluate research excellence and impact and to apply this beyond the university sector, but refers to the concerns about a data-drive methodology raised under Q.8.

Q10. Having regard to the Review’s Terms of Reference, the ARC Act itself, the function, structure and operation of the ARC, and the current and potential role of the ARC in fostering excellent Australian research of global significance, do you have any other comments or suggestions?

The University suggests that the ARC consider expanding its definition of research excellence to include majority world and indigenous frameworks of research excellence in addition to its recognition of indigenous frameworks as an important field of research. In the field of Divinity, the majority world is the largest growth area globally which will have significant impact on how and in what ways research is produced in this field by the end of the 21st century.

Submission received

14 December 2022

Publishing statement

Yes, I would like my submission to be published and my name and/or the name of the organisation to be published alongside the submission. Your submission will need to meet government accessibility requirements.