Academy of the Social Sciences in Australia

Name (Individual/Organisation)

Academy of the Social Sciences in Australia

Responses

Q1. How could the purpose in the ARC Act be revised to reflect the current and future role of the ARC?

For example, should the ARC Act be amended to specify in legislation:
(a) the scope of research funding supported by the ARC
(b) the balance of Discovery and Linkage research programs
(c) the role of the ARC in actively shaping the research landscape in Australia
(d) any other functions?

If so, what scope, functions and role?

If not, please suggest alternative ways to clarify and define these functions.

The Academy strongly supports amending Section 3 (a) of the ARC Act to reflect the fundamental purpose of the ARC, including an unequivocal commitment to basic research, comprehensive support across all disciplines, and fostering research that traverses or transcends disciplinary boundaries. In addition, the Review should consider the broader context of our national research funding landscape and connect to parallel processes, including the Australian Universities Accord and the National Science and Research Priorities Refresh.

A fragmented national research funding landscape
Internationally, vibrant research and innovation systems are created when whole-of-government methods develop strategy and ensure coordination across agencies. In the last financial year, the Australian government invested $11.8B in research and development, dispersed across 202 programs and 13 portfolios. Australia has no whole-of-government policy for research investment and no central agency overseeing or administering research funding. Without a coordinated national ambition and strategy, the ultimate purpose of research investments lacks clarity, resulting in misaligned priorities. This fragmented approach means that many research investments lack the scale to provide the game-changing breakthroughs that Australia deserves and needs.

The Academy believes that Australia requires a future-focused, whole-of-system re-think of the national research and innovation landscape to ensure it is fit-for-purpose in driving innovation, economic prosperity, and social wellbeing. This would build on the recommendations of the Industry, Innovation and Sciences Australia Report Driving Effective Government Investment in Innovation, Science and Research, articulate a national purpose and objectives for research and innovation funding, and identify the most effective structures and administrative arrangements to deliver on these ambitions.

Role and purpose of the ARC
The role and purpose of the ARC should flow from this broader system-perspective and should reflect the critical position of the ARC in the research and innovation landscape. The ARC must obviously and increasingly support excellence across basic and applied research. It also must be responsible for national evaluations of research excellence and impact, as well as be a valuable source of advice on national research policy (see our response to Q8).

Critically for the social sciences, the ARC is the primary government funding mechanism for basic research in Australia. Over 80 per cent of ARC funding secured by the social sciences in 2021 came from the Discovery scheme. Social science projects that have been supported by the ARC have delivered significant long-term and transformative social, economic, educational, and cultural benefits to the Australian community. This fundamental purpose—to strongly and fully support basic research across disciplines—must be overtly recognised and safeguarded in the Act.

The Academy welcomes recent comments by the Prime Minister and Minister for Industry and Science recognising the role of government in valuing and funding basic research. This is consistent with evidence from Europe, the US, the UK and elsewhere that advanced, internationally competitive research and innovation ecosystems must have strong basic research foundations. Investment in basic research not only drives economic growth, productivity growth, international collaboration and innovation but has broader social benefits such as improved health, quality of life, and improvements in education.

Although basic research is a keystone of innovation around the world and it is a public good of immense value, Australian government investment has almost halved since the early 1990s. In contrast to this decline, elsewhere — such as in Denmark, Japan, the United Kingdom — funding has remained stable or increased. Australia’s decline in funding for basic research adversely affects our global positioning and competitiveness in multiple ways.

Finally, effective research on and solutions to the megatrends that will shape the next decades nationally and internationally require expertise and collaboration across disciplines. Comprehensive funding of all academic disciplines across the social sciences, science, engineering and technology, health and medical, and the arts and humanities as well as encouragement of interdisciplinary research are critical for a fit-for-purpose, future-focused research and innovation system in this country. The ARC’s remit for such support must be more clearly reflected in the Act.

Recommendation 1: The Australian Government undertake a future-focused, whole-of-system review of the national research and innovation landscape to ensure it is fit-for-purpose in driving innovation, economic prosperity, and societal wellbeing.

Recommendation 2: Amend section 3 (a) of the ARC Act to affirm the central purpose of the ARC as being: to secure Australia’s university research capability by supporting excellent research, promoting research innovation and application, supporting and enhancing the research workforce, supporting research collaboration (both national and international), ensuring research training, developing and maintaining world-class national research infrastructure, assessing research disciplines, and providing advice on research policy.

Recommendation 3: Amend the ARC Act to affirm the ARC as the primary funder of basic research in universities across all disciplines not covered by the National Health and Medical Research Council. This amendment should recognise the critical importance of basic research beyond commercial applications and encourage interdisciplinary and multidisciplinary research that addresses priority challenges and advances knowledge frontiers.

Q2. Do you consider the current ARC governance model is adequate for the ARC to perform its functions?

If not, how could governance of the ARC be improved? For example, should the ARC Act be amended to incorporate a new governance model that establishes a Board on the model outlined in the consultation paper, or another model.

Please expand on your reasoning and/or provide alternative suggestions to enhance the governance, if you consider this to be important.

The Academy supports the re-establishment of an ARC Board, with the composition and functions outlined in the Consultation Paper.

Trust in the appropriate use of public funding depends on a clear, reliable, and transparent system of governance and management. The current governance model is inadequate for the ARC to perform its function.

The ARC needs governance structures—specifically a Board and a CEO with specific objectives and appropriate expertise—to deliver its core purpose: providing advice and oversight to sustain Australia’s current and future university research capability in a dynamic, globally embedded research and innovation system.

The objectives of current ARC ‘designated committees’ and the CEO are too diffuse. The Act neither clearly recognises nor articulates the academic research leadership experience and expertise that committee members must hold and which the CEO needs to effectively govern and manage the ARC successfully.

Role of a re-established board
An ARC Board would have oversight of ARC performance on research funding, workforce support, research evaluation, and policy advice. Board members must have relevant expertise and experience to advise on the ARC’s goals, priorities, policies, and strategies in light of disciplinary developments, national and international priorities, emerging issues, and opportunities.

The Board should oversee the ARC’s whole of portfolio investment against an established portfolio evaluation framework and provide advice to the CEO on strategic and operational matters in relation to performance of this portfolio. The Board could also provide a high-level resource for the Department and Minister.

The Academy agrees that, to effectively advise the CEO, enhance the functioning of the ARC, and gain the trust of the university sector, members of a new Board must have high levels of disciplinary expertise and leadership across the spectrum of ARC disciplines; they also should be able to bring national and international perspectives on research and research funding. Members also must have clear authority in matters that bear directly on the ARC’s purpose, including research leadership, research evaluation, organisational performance evaluation, and industry-university engagement for advanced research and development.

Board composition
The Academy supports a particular requirement for Indigenous research leadership on the Board given the ARC’s stated commitment to Aboriginal and Torres Strait Islander Researchers (including the recent introduction of the 45-discipline code). Further, the Board composition should be considered within the context of the ARC Gender Equity Statement, ensuring diversity of background and approach.

In addition, the Board could include at least person who has high-level experience in counterpart organisations elsewhere in the world, such as one or more of the organisations represented on the Global Research Council, for international benchmarking purposes and breadth of outlook.

CEO role
The Academy affirms that an expert and experienced CEO is essential to the effective functioning of the ARC. For the ARC to address its primary purpose of sustaining university research capability, the CEO needs to be highly experienced in university research, including, ideally, in advanced disciplinary and research leadership. The academic research standing of the CEO is necessary for the ARC, ensuring that the person is held in high esteem across the university research sector. The requirement for such research standing should be established in the Act.

Recommendation 4: Amend the ARC Act to include a new governance model that establishes a Board, with the composition and functions outlined in the Consultation Paper. This amendment should articulate the expertise and experience expected of members of the Board and diversity requirements, including Aboriginal and Torres Strait Islander and gender representation.

Recommendation 5: Amend the ARC Act to articulate that The Minister must not appoint a person as CEO unless the Minister has considered the person’s record in research and management and is confident that: (a) they are highly experienced in university research with advanced disciplinary and organisational research leadership, and (b) will be held in high esteem across the university research sector.

Q3. How could the Act be improved to ensure academic and research expertise is obtained and maintained to support the ARC?

How could this be done without the Act becoming overly prescriptive?

The Academy suggests that the Act prioritise and enshrine academic and research expertise. Additional forms of expertise should also be acknowledged.

The ARC’s ability to support university research innovation and capability depends on the quality of expertise it draws upon. Where research excellence is a precondition for funding and research evaluation, academic expertise through the mechanism of peer review is the right expertise to assess this. The design of academic expertise and peer review into ARC positions and processes is a key strength of the current system and leverages significant good will within the academic community.

The Consultation Paper notes that the Act does not define high quality and is silent on where expertise can or should be sourced. For reasons of excellence and legitimacy in the university sector, The Academy recommends that the Act prescribe forms and levels of expertise for the ARC Board and the CEO and ensures that senior positions including Executive Directors of Disciplinary Clusters have demonstrable research excellence and leadership.

ARC funding programs and evaluations such as the Linkage Program and the Engagement and Impact Evaluation increasingly examine engagement and impact beyond disciplines and academia. Responsible research and innovation, as reflected in the Horizon Europe Program, the Indigenous Data Sovereignty movement, and the Consumer Reference Panel in the Medical Research Future Fund (MRFF), all indicate ways to build additional forms of expertise into research and innovation governance and ethical decision-making frameworks. Funding programs or evaluations intended to support or assess commercial application or apply in complex real-world settings, may need additional forms of expertise.

The ARC would also benefit from a Consumer Reference Panel (Enduser Reference Panel), modelled on the MRFF Consumer Reference Panel. Such a Panel would have an advisory role only, and its objective would be to strengthen public involvement and engagement in the implementation of the ARC’s function. This could include advising the ARC on how to better demonstrate the social value of publicly funded research and evaluation, when and how to involve end-users and the broader community in ARC processes, including people from typically unrepresented groups, and so on (see also our response to Q5).

Recommendation 6: Amend the ARC Act to enshrine the centrality of academic and research expertise and peer review in ARC governance and processes, and to allow for additional forms of expertise as required.

Recommendation 7: In the governance and management of the ARC, ensure that Executive Directors of Discipline Clusters and other senior positions demonstrate appropriately high levels of academic research excellence and leadership alongside other necessary skills and experience.

Recommendation 8: The ARC investigate establishing a Consumer Reference Panel to advise the CEO on promoting public engagement in ARC processes and activities.

Q4. Should the ARC Act be amended to consolidate the pre-eminence or importance of peer review?

Please provide any specific suggestions you may have for amendment of the Act, and/or for non-legislative measures.

The Academy supports the amendment of the ARC Act to consolidate the pre-eminence of peer review in the decision-making process. Should the Minister intervene in genuine and extraordinary circumstances, then a predictable, transparent, and informed process must be in place for those decisions.

Expert peer review is the gold standard for assessing research excellence and should remain the foundation of the ARC’s assessment and evaluation activities. Unlike other approaches, such as article citation metrics, expert peer review involves direct expert evaluation of quality.

Minister’s role
The Academy recognises that the Minister must comply with obligations under the Public Governance, Performance and Accountability Act 2013 and may prevent recommended projects from being funded under extraordinary circumstances. However, as the Consultation Paper notes, perceived or actual arbitrary intervention by the Minister can have significant negative consequences. Most directly, these include delegitimising the ARC and its processes as well as an associated perception that funding decisions are politicised and do not reflect research quality. Together, these undermine the social licence for public funding of research (as argued in our response to Q5) and ultimately undermine trust in social institutions, political, academic and otherwise.

One way to mitigate these consequences is for the Act to specify that ARC funding decisions are based on expert peer review, and that Ministerial intervention should only be exercised in extraordinary circumstances. The Act should further be amended to require that if a Minister seeks to intervene, then they are obliged to notify and justify that intervention to Parliament.

The Academy completely rejects the notion that a Ministerial perception or interpretation of national interest (whether precisely or imprecisely defined) is a basis for Ministerial intervention. Strong and effective research and innovation systems are founded on high quality research, especially basic research. Since expert review processes are the best way to determine research quality, an expert-based funding recommendation serves the national interest.

Recommendation 9: Amend the ARC Act to specify that funding decisions are based on expert peer review, with the exception of extraordinary circumstances in which the Minister might be required to intervene. In such circumstances, the Act should require that the Minister notify the ARC CEO and the Parliament of their intention and reasons.

Q5. Please provide suggestions on how the ARC, researchers and universities can better preserve and strengthen the social licence for public funding of research?

The Academy supports the intent of the National Interest Test (NIT) to demonstrate the societal benefits (health, economic, environmental, social and/or cultural) of the proposed research. Ensuring that publicly funded research delivers a public good is a fundamental purpose of our national research and innovation system.

However, the Academy considers that the NIT in its recent form is an incomplete and inadequate mechanism both for assessing national benefit of research and for communicating this benefit to the Australian public.

The Academy welcomes the ARC’s recent decision to simplify implementation of the NIT in upcoming NCGP schemes. However, we reiterate our concerns about the original (as well as new) implementation of the NIT and whether it preserves and strengthens the social licence for public funding of research.

The NIT has been a source of disquiet in the research sector and beyond since its introduction. It’s use, and perceived misuse, has generated perceptions that the ARC’s funding process is politicised and illegitimate, potentially damaging the reputation of the ARC and its programs. In addition, requests revise NITs, as well as related Ministerial decisions to not fund projects, have fallen disproportionately on some disciplines and cohorts in the social sciences, including Indigenous colleagues applying to the Discovery Indigenous scheme.

Weakness of the National Interest Test
The latest implementation of the NIT will ask Advisory Committees and peer reviewers to assess the societal benefits (economic, commercial, social, or cultural) of the proposed research beyond the academic community, yet the value of each funding application to the community has always been thoroughly considered as part of the peer review process prior to a recommendation to fund.

To overturn a recommendation based on the NIT undermines academic expertise and peer review in ARC processes. It lacks transparency, noted in our responses to Q3 and Q4, and undermines the social licence for research by allowing an inference that the ARC’s expert panels have not been making decisions with the national interest in mind. Social licence is granted, in part, through an assurance that our national research funding and evaluation systems are robust, consistent, and transparent.

The Consultation Paper notes that the NIT functions as a critique of the success with which that value has been conveyed in plain English for the benefit of the uninitiated member of the public. This the “Pub Test” approach appears to convey that expert, complex, or controversial ideas need to be immediately and easily understood and endorsed by all, or at least a particular group of, “everyday Australians”.

But social science research tells us that cultural value and social legitimacy are conferred via communication, which should be treated as a shared responsibility across government, funders, institutions, and individuals. Even in its current implementation, the ARC states that the audience of the NIT is the general public — a different audience than the Application Project Summary, whose audience is the applicant’s peers. But it is unclear if the broad general public actually read or care about NITs.

The Academy agrees that public funding for research depends not only on research benefitting society in the short-, medium-, or long-term, but also that society understands and appreciates these benefits, which take a range of forms. However, a NIT applied to individual projects in specific NCGP schemes cannot capture and convey the complex ways in which different kinds of basic, applied, and translational research across disciplines and timescales (as noted in our response to Q1), all serve the national interest and add value to our community.

As an alternative to project-by-project assessments and communication via the NIT, The Academy suggests that the ARC take a more wholistic, portfolio approach to investment and adopt practical ways to assess and convey to the community national benefits and return on investment of funded portfolios. This may be within and across NCGP schemes or within and across ARC Panels.

How to measure and convey value
A significant body of work from the social sciences could inform a comprehensive strategy on the value of public funding for research. This includes measures and models that predict social licence to operate, frameworks for formal and informal reporting of cultural value, and public views on the role of institutions that conduct research.

Social licence is a shared responsibility across government, funders, institutions, researchers, and participants who converge on a message of value.

Two practical ways to encourage this sense of shared responsibility would be to introduce a Consumer Reference Panel (see our response to Q3) and incentivise co-creation and participatory practices in research projects.

Recommendation 10: The ARC should recognise social licence as a shared responsibility across multiple stakeholders and develop a strategy to convey the value of public funding for research at the portfolio level, informed by existing social science research or new research by the ARC.

Q6. What elements of ARC processes or practices create administrative burdens and/or duplication of effort for researchers, research offices and research partners?

The Academy supports the efforts of the ARC to reduce administrative burden while maintaining robust and credible processes. The Academy notes that the trade-off between time spent preparing and reviewing applications and the ability of researchers to conduct research and respond to new opportunities, has a disproportionate impact on early and mid-career researchers, as well as socially vulnerable and female researchers.

Administrative delays, changes, uncertainty, duplication, onerous requirements, and overall burden associated with ARC applications, grants, and reports are problematic for the ARC and for universities. Not only do these burdens reduce participation inside and outside universities, but they also limit the ability of researchers, teams, and the sector to respond to new opportunities and innovations in the national interest.

Delayed funding calls or announcements, changing requirements or eligibility criteria, and other administrative burdens can have career-altering consequence for vulnerable, female, and early career researchers, undermining investment in Australian research capability for the future.

Recently released outcome statistics for the 2023 DECRA, Discovery, and Laureate schemes, for instance, indicated that women’s participation (40 per cent, 32.5 per cent and 27.5 per cent, respectively) fell below or well below parity despite attempts to match success rates.

Social science research reveals that COVID lockdowns, home schooling, and carer duties disrupted female researchers more than male researchers, as well as early career researchers more than more senior colleagues. People at the intersection of these systemic disadvantages are especially vulnerable to capricious timing, requirements, burdens, and outcomes of funding schemes. Other funders, such as the NHMRC, have developed strategic frameworks and specific measures (e.g., timing of grant opportunities, application requirements, instructions to assessors) to achieve gender equity.

However, any attempts to address administrative burdens should not forfeit the disciplinary nuance required to successfully attract, expertly assess, and reasonably fund the best research. For instance, attempts to reduce administrative burden via automation that involves ‘scraping’ data from databases that cover disciplines unequally will undermine the importance of research excellence in the NCGP.

Recommendation 11: Amend ARC processes to reduce administrative burden and uncertainty, drawing on lessons from other national and international funders that have effectively reduced grant preparation and review requirements to enable greater efficacy, agility, and inclusion.

Recommendation 12: Commission an independent evaluation of the ARC’s gender equity framework and develop an overarching strategy and targeted measures to improve participation of socially vulnerable and female researchers across its program portfolio.

Q7. What improvements could be made:

(a) to ARC processes to promote excellence, improve agility, and better facilitate globally collaborative research and partnerships while maintaining rigour, excellence and peer review at an international standard?

(b) to the ARC Act to give effect to these process improvements, or do you suggest other means?

Please include examples of success or best practice from other countries or communities if you have direct experience of these.

This Review is an opportunity to re-imagine and re-position the ARC as a world-class, leading-practice grant funding body.

The ARC should position itself as a funding body that encourages high risk/high reward research, invests in long-term projects and infrastructure across all disciplines, and builds national skills and capability via improvements to the NCGP.

The Academy believes that the excellence, agility, and international competitiveness of Australian research, innovation, and partnerships depends on a confident, transparent ARC, with processes aligned to purpose (see our response to Q1), especially regarding what, who, and how researchers are funded. Deviations from that clear purpose erode public confidence in the ARC specifically and research generally.

Take an international view
Other national and international bodies encourage more high risk/high reward proposals. A strategically balanced portfolio either within or across NCGP schemes could more actively evaluate risk appetite within ARC processes, ensuring that high-risk and high-reward proposals and projects are encouraged and balanced by more conservative, incremental proposals and projects. Whereas some ARC schemes might be best suited to “enhancement oriented” innovation, new schemes should be established to tackle complex social problems with novel and potentially disruptive research.

The Academy suggests that the ARC take a more international perspective on research to accelerate Australia’s and Australian researchers’ global involvement and influence.

The ARC could achieve this via better support for international collaboration, beyond simply covering the cost of short-term travel abroad by Australian-based researchers and/or short-term visits to Australia by foreign collaborators. For instance, provision could be made, with appropriate constraints, for funding/co-funding staff working outside Australia in the employ of a non-Australian university or research institute as part of an international research collaboration. This suggestion echoes the Swiss National Science Foundation’s International Co-Investigator Scheme, some UK Research and Innovation grants, which allow principal investigators to be based outside the UK, and a range of international and collaboration-oriented grants awarded by the Social Science Research Council in the US.

Take a long view
The Academy also considers that the ARC should better accommodate projects that represent longer-term investments. For example, an important form of social science infrastructure is longitudinal datasets, which provide increasing value as more waves of data are collected (e.g., The 45 and Up Study). Although larger schemes such as ARC Centres of Excellence allow for at least 7-year projects, the ARC should consider methods by which long term, social research infrastructure can be supported as a national research asset via stand-alone projects within Discovery, Linkage Infrastructure, Equipment and Facilities, and other schemes.

Over a long period of time, the ARC has invested significantly in Australia’s research workforce via a range of early career, mid-career, and senior research fellowships. This investment has shaped our workforce in expected and unexpected ways. Across these schemes, The Academy is concerned about calibration of years since PhD with competitive performance standards noting that outcome statistics for the 2023 DECRA round show that most successful applicants were more than 4 years post PhD.

There are a number of similar gaps in the ARC’s suite of fellowship support, and The Academy considers that policy and practice on funding research career pathways should be reformed to ensure that leading researchers are supported at all career stages in more flexible ways (e.g., from 1 to 5 years and from various sources or partners), which are better mapped to the purposes of ARC funding and the Australian Government’s stated aspirations to elevate our national workforce for the future.

Reduce inefficiencies
The Academy agrees with the Consultation Paper’s suggestion to consider a two-step EoI process, similar to methods used in EU schemes, to save academic and professional time on an inefficient system with high failure rates.

The Academy urges the ARC to rethink processes for allocating funding with the intention of increasing responsiveness and agility while decreasing application failure rates. Outcome statistics for the 2022 and 2023 Discovery rounds showed that 19 per cent and 18 per cent of applications, respectively, were successful. Across these two years, 4,618 teams were unsuccessful in this scheme alone, representing an enormous expenditure of researchers’ effort for no immediate return. It is estimated that Australian researchers waste 38,000 person days per year, or $24 million AUD on administrative bureaucracy related to grants and projects (see also our response to Q6).

A shorter project outline may be more effective with requests for detailed applications from those invited to stage two. The ARC would need to be mindful of balancing expert peer review in an EoI system without increasing workload further as well as without introducing perverse consequences that exacerbate system inequalities.

Recommendation 13: Amend the ARC Act to enshrine a portfolio-based approach to ARC investment with an explicit consideration of the balance or risk across the national program of funding and different timelines for investment delivery.

Recommendation 14: The current process review of the NCGP by the ARC should consider the purpose and objectives of each scheme with a view to accommodating funding of longer-term projects, social science infrastructure, and better support for the research workforce.

Q8. With respect to ERA and EI:

(a) Do you believe there is a need for a highly rigorous, retrospective excellence and impact assessment exercise, particularly in the absence of a link to funding?

(b) What other evaluation measures or approaches (e.g. data driven approaches) could be deployed to inform research standards and future academic capability that are relevant to all disciplines, without increasing the administrative burden?

(c) Should the ARC Act be amended to reference a research quality, engagement and impact assessment function, however conducted?

(d) If so, should that reference include the function of developing new methods in research assessment and keeping up with best practice and global insights?

The Academy recognises that reporting and assessment of research outcomes are valuable features that encourage trust and transparency as well as reinforce the social licence for research investment.

However, ERA has outgrown any value, especially in light of enormous and unsustainable costs for both the ARC and universities and in the absence of a link to funding. The Academy considers it essential that cost-effective, innovative, and sensitive evaluation approaches and measures be identified and implemented to evaluate the quantum, quality, impact, and value of Australian research, whether research directly funded by the ARC or more broadly.

As noted in the Consultation Paper, the validity and influence of national assessments depend on a clear sense of purpose. A balance between process costs and disciplinary sensitivity will result in better understanding of research strengths, scale, and opportunities. However, to determine the most appropriate approaches, data-driven or otherwise, the ARC should focus on: a clear set of evaluation questions, radically open data, a move from lag to leading indicators, and sensitivity to sector, disciplinary, and interdisciplinary strengths.

In particular, the ARC should ensure that any changes to evaluation practices and outcomes are co-created with the university sector and piloted to confirm they are workable and informative over the long-term.

Although the ARC routinely uses data it collects for process improvements and other insights, a radically open approach to data would enable more sophisticated insights into Australia’s research investment and return (see also our response to Q9). For instance, if all evaluation data are open to universities, then data analytics could grow in sophistication from reports of retrospective performance to comparisons with national or international benchmarks to return on investment analyses and to predictive modelling of systems and system effects.

Current ERA and EI exercises are limited in the sensitivity of their measurement. For example, ERA relies predominantly on lag, not leading, indicators of research success, such as citations. Grant applications to the ARC would offer a more leading indicator of potential research success and shift attention from the end to earlier in the research value chain. In EI, assessment again focuses on the end point of research, but defines impact in a very specific way and within timeframes that can miss the longer-range impacts of much social science research in particular, but basic research more generally.

The ARC should make all data at its disposal publicly available to prospectively inform research funding policy and practice as well as to “take the temperature” of the research enterprise both prospectively and retrospectively at multiple points along the value chain (see also our response to Q9).

As identified through previous reviews of ERA, the current process is narrowly focused on individual institutions and individual disciplines. This is at the expense of sector-wide considerations of disciplinary strengths and of interdisciplinary opportunities and success. This failure to capture the scale, maturity, and quality within and across particular disciplines in Australia limits the ability to prospectively direct investment to already strong, emerging, or important areas relative to the national interest.

Another well identified challenge in ERA is the complexity of reasonably and fairly measuring quality and success of citation-based versus peer review disciplines, leading to either methodological artifacts or a lack of sensitivity in ERA results. Rather than seeking a “one size fits all” approach for every discipline, the ARC should work with universities to define success and the best measures or proxies of this success across different discipline groupings when developing the next generation of evaluation methodologies

The Academy understands that the ARC is working to establish a modern, data-driven approach to ERA. If automated processes of evaluation in a refreshed ERA or a completely new assessment framework are introduced in place of more manual processes, then the ARC should identify a threshold for these methods in collaboration with universities. New methods could be run in rounds, as pilots involving a subset of disciplines each year much like course review is done at universities. The ARC also could consider an approach whereby the sector identifies common measures that differentiate all cases of research enterprises as well as distinctive, more nuanced, disciplinary-specific measures most appropriate for sub-sets of research enterprises.

The Academy considers that the ARC should either undertake or commission research into methodological choices, benchmarked to best national and international practice, in order to ensure that differences in absolute performance across disciplines are contextualised within policy advice and practice. This is especially important given worldwide conversations and concerns about the responsible assessment of research.

Recommendation 15: The ERA Transition group should identify a clear purpose for research assessment as part of ARC activities and a new model for assessment exercises centred on radically open data, leading indicators and processes that ensure sensitivity to sector, disciplinary, and interdisciplinary strengths.

Recommendation 16: Pilot new or revised assessment exercises (including ERA or EI), particularly the introduction of any automated processes, to ensure they are workable over the long-term and do not disadvantage any disciplines.

Q9. With respect to the ARC’s capability to evaluate research excellence and impact:

(a) How can the ARC best use its expertise and capability in evaluating the outcomes and benefits of research to demonstrate the ongoing value and excellence of Australian research in different disciplines and/or in response to perceived problems?

(b) What elements would be important so that such a capability could inform potential collaborators and end-users, share best practice, and identify national gaps and opportunities?

(c) Would a data-driven methodology assist in fulfilling this purpose?

The Academy recognises that the ARC holds a considerable quantum of valuable data on Australian research excellence and impact. This data could be better used to understand the research and innovation system, to support industry collaboration and engagement, and to improve the public understanding of and social licence for research (see also our responses to Q5 and Q8).

The Consultation Paper notes the ARC’s “evaluation capability”. However, it is unclear if this capability refers to the data, to ARC personnel with research and evaluation expertise in substantive areas relating to research and innovation policy, and/or to the broader university community that provides expert evaluation in partnership with the ARC.

The Academy is aware that considerable, related expertise exists elsewhere in government, including in the Department of Education and in the Department of Industry, Science and Resources, both of which have relevant policy responsibilities.

Opportunities for collaboration
Australia’s research and innovation system needs a root-and-branch review (in response to Q1 and Q10). Such a review gives considerable opportunity to explore ways to work collaboratively across government and the university sector and build a joint research capability, undertaking ongoing academic and policy research and evaluation. The Australian government has precedents for such cross-departmental policy research units that bring together cognate areas to allow more wholistic and powerful analyses of priority topics.

Fellows of the Academy have experience collaborating with units like this for policy-relevant research. For example, the Departments of Health, Social Services and Education combined under the previous Federal Government to establish the Social, Health and Welfare Analytical Unit (SHWAU), which undertakes research projects at the intersection of health, education, and social services and welfare. SHWAU leverages new integrated public-sector datasets drawn from across these portfolios and others (including the Australian Tax Office) in research that informs policy improvements, interventions, and further analysis. SHWAU was established under the “Data Integration Partnership for Australia”. In addition to bringing government policy analysts and researchers together on projects, SHWAU collaborates with university researchers on co-designed projects of mutual interest using sensitive government datasets.

The recent establishment of the Office for the National Data Commissioner and the Data Availability and Transparency Act 2022, which support sharing of Australian government data for research and policy, provide the legislative and administrative underpinnings of a new research policy unit. Such a unit might sit across the ARC, NHMRC, and other agencies within the Departments of Education and Science, Industry and Resources. ARC’s rich datasets could be linked to other public sector data in these and other portfolios. Working in partnership with university researchers, such a unit could be an engine for research, policy development, and intervention in the research and innovation system (see also our response to Q8).

One international model for such a unit is the Institute for Research and Innovation in Science (IRIS) at the University of Michigan. IRIS is a consortium of research universities who submit and use big administrative data to monitor, evaluate, and investigate university research, university-government relations, and public communications about the value of research. IRIS was developed to continue US government-university initiatives that measure the academic and non-academic impacts of university research.

Recommendation 17: The ARC should collaborate with relevant Australian government departments and the Office for the National Data Commissioner to develop a cross-departmental research unit that models, maps, and improves research and innovation in the university sector using public-sector data.

Q10. Having regard to the Review’s Terms of Reference, the ARC Act itself, the function, structure and operation of the ARC, and the current and potential role of the ARC in fostering excellent Australian research of global significance, do you have any other comments or suggestions?

The Academy advises that, supported by the Australian Government and the university sector, the ARC take this once in a generation opportunity to fully and frankly evaluate how it operates within the Australian research and innovation system.

This root-and-branch review must explicitly converge with parallel processes underway as part of the Australian Universities Accord and National Science and Research Priorities Refresh.

While the Academy notes that the ARC evaluates programs and schemes within the NCGP, we consider a comprehensive review of the purpose and the interrelationships across the entire NCGP would be timely. Such a review should develop a clear statement of the objectives of the ARC’s portfolio of investments (via its various programs and schemes) as well as update appropriate methods to monitor the ARC’s performance against these objectives at the portfolio level.

The Academy especially urges the ARC to examine whether its Fellowship schemes are serving their purpose. As discussed in our response to Q7, while there is clear evidence of the enormous and significant contributions of ARC Fellows over many years, concerns remain about eligibility, “bracket creep”, gender equity, and gaps in the suite of pathways. The ARC should ensure that the next iteration of workforce support is mapped to the purposes of ARC funding and the Australian Government’s aspirations to elevate Australia’s capabilities across basic, applied, and translational research.

Recommendation 18: The ARC should commission a comprehensive independent review of the purpose and the scheme interrelationships across the entire NCGP.

Submission received

14 December 2022

Publishing statement

Yes, I would like my submission to be published and my name and/or the name of the organisation to be published alongside the submission. Your submission will need to meet government accessibility requirements.