Anonymous #45

Name (Individual/Organisation)

Anonymous #45

Responses

Q1. How could the purpose in the ARC Act be revised to reflect the current and future role of the ARC?

For example, should the ARC Act be amended to specify in legislation:
(a) the scope of research funding supported by the ARC
(b) the balance of Discovery and Linkage research programs
(c) the role of the ARC in actively shaping the research landscape in Australia
(d) any other functions?

If so, what scope, functions and role?

If not, please suggest alternative ways to clarify and define these functions.

The balance of Discovery and Linkage research programs should be justified. Funding for discovery has stagnated or gone backwards. Whereas in my view their is too much funding in e.g. LIEF and not enough in Discovery Project funding. National infrastructure for research continues to improve - large, expensive equipment items based at individual univiersities which often do not have the proper resources to maintain or manage such facilities does not make sense. I would like to see more equipment placed in open access facilties to the benefit of all. ARC needs to invest more in people (particularly ECRs/MCRs) not equipment.

ARC should maintain a degree of independance from government interference, researchers are well placed to understand the impact and value of the research conducted by their peers. Politician's not so much.

The role of the ARC College of Experts should be clairified. The current system where the College of Experts have to score 50+ applications is simply not working. This is too much weight given to too few individuals who then are not obliged to give *any* feedback to the applicants. Frequently ARC college members are only able to spend a few mins per application compared to the detailed assessors who often spend hours per application. Yet the college scores carry the same weight as the detailed assessor scores, when often the detailed assessors have more knowledge and expertise of the particular area they are reviewing. College members have too much work and too much responsibility, the lack of transparency with the scores in not recieving any feedback from the College (particularly as their scores often seem to differ from the detailed assessors), is frustrating for many.

I would recommend removing the scores from the College entirely and having them only adjust the scores of the detailed assesssors when there is a breach of guidelines or a particularly compelliing rejoinder. The widespread mismatch between detailed assessor feedback and final rankings is often down to the ARC College scores not matching those of the detailed assessors. This has generated a lot of mistrust in the current system.

Q2. Do you consider the current ARC governance model is adequate for the ARC to perform its functions?

If not, how could governance of the ARC be improved? For example, should the ARC Act be amended to incorporate a new governance model that establishes a Board on the model outlined in the consultation paper, or another model.

Please expand on your reasoning and/or provide alternative suggestions to enhance the governance, if you consider this to be important.

I don't think another board is needed, in fact I believe the scope of the ARC College of experts should be significantly reduced to one of just 'quality control'. In the vast majority of cases the detailed assessor scores should simply stand. Please make things more simple then they currently are. Adding another board with a small number of 'discipline experts' is just going to generate more complications and administrative overhead.

I would strongly advocate to leave all of the descision making to the detailed assessors, it's not perfect and sometimes the quality of the feedback is limited, but this would be much better than the current system and at least there is some accountability for the detailed assessors to provide the applicant with feedback that they can use to improve their application. There is too much interference in the process at the moment and not enough respect for the time, effort, and knowledge of the detailed assessors who are really best placed to comment on the impact within their relevant FoR.

I would remove the College of Experts scoring step, and let the research community decide what should be supported.

Q3. How could the Act be improved to ensure academic and research expertise is obtained and maintained to support the ARC?

How could this be done without the Act becoming overly prescriptive?

I would significantly revise the scope of the ARC College of Experts to not include a mandate for individuals to assess large numbers (50+!) of applications which they simply do not have time for. We should be working to simplify the current Act.

Q4. Should the ARC Act be amended to consolidate the pre-eminence or importance of peer review?

Please provide any specific suggestions you may have for amendment of the Act, and/or for non-legislative measures.

Again, revisit the scope of the ARC College of Experts. Having their scores count for 50% of the final assessment lacks transparency, fails to acknowledge the fact that detailed assessors spend far more time with each application then ARC College members, detailed assessors are also often more knowledgable about a specific area that ARC College members. The lack of any written feedback from the ARC College on individual application scores that they have assessed (in addition to the detailed assessor feedback) is frustrating and engenders a lack of trust in the current system.

Q5. Please provide suggestions on how the ARC, researchers and universities can better preserve and strengthen the social licence for public funding of research?

The ARC does not currently do a very good job of promoting outcomes of research which it has supported. A better media & comms strategy (including social media) more suited to the 21st centuary is required. The wider public need to associate major advances and breakthroughs with support from the ARC.

Q6. What elements of ARC processes or practices create administrative burdens and/or duplication of effort for researchers, research offices and research partners?

I would consider revising the way that TR information is presented. We do not need to see 100 publications listed and I doubt many (even the detailed assessors) read this. The effort to reward from the ARC is worse then any other funding system I have engaged with (including internationally). The project proposal and impact should be the focus, the TR section and even the budget could be reduced and simplified. Every ARC application seems to be 100+ pages when we are lucky if the assessors even spend a few mins looking through it (!) Shorter simpler applications, with greater emphasis on the project compared to the TR/budget is required.

Applications for industry funding or support within industry are typically much simpler and to the point. Way to much time (particularly in the DP) is wasted on TR not enough on what the actual project is and how it will lead to positive impact.

Q7. What improvements could be made:

(a) to ARC processes to promote excellence, improve agility, and better facilitate globally collaborative research and partnerships while maintaining rigour, excellence and peer review at an international standard?

(b) to the ARC Act to give effect to these process improvements, or do you suggest other means?

Please include examples of success or best practice from other countries or communities if you have direct experience of these.

ARC (or at least assessors) seem to treat PIs as being in a second tier to CIs. I would treat PIs and CIs the same, this would encourage more international PIs. Reduce the emphasis on TR in the assessment - we should be encouraging well thought out, quality project proposals. Current system just encourages manipulation of e.g. h-index, publication numbers etc. (often at the expense of ECRs/student careers).

I would specify that *no* ARC application should be longer than 30 pages. A 1-2 page CV for each CI/PI is fine. 8 pages for the project and then a 1-2 page budget justification. The non-ARC contributions section could be cut down or reduced to a simple line item in the budget. When (at most) even your detailed assessors are only spending 1-2 hours per application it makes no sense to require that each proposal contains so much information which then never gets read.

ARC College members hone in on just a few key pieces of information when reviewing each application - why not structure proposals to just highlight the key points.

Please, simply and significantly reduce the lengths of the applications and reduce the current emphasis on TR except for fellowships. In fact, I would argue that the TR for DP should be largely irrelevant compared to the quality and impact of the project (in line with many international approaches). People can have great ideas at any stage in their career, the current ARC DP system locks people out of funding (particularly those with career breaks or who have previously had a teaching focus).

Q8. With respect to ERA and EI:

(a) Do you believe there is a need for a highly rigorous, retrospective excellence and impact assessment exercise, particularly in the absence of a link to funding?

(b) What other evaluation measures or approaches (e.g. data driven approaches) could be deployed to inform research standards and future academic capability that are relevant to all disciplines, without increasing the administrative burden?

(c) Should the ARC Act be amended to reference a research quality, engagement and impact assessment function, however conducted?

(d) If so, should that reference include the function of developing new methods in research assessment and keeping up with best practice and global insights?

a. I would stop ERA altogether - I don't think anyone really uses this except for promotional materials (certainly it has zero impact on ARC assessments). EI could be useful but needs to be re-thought, it also takes a long time. It has zero profile internationally.

b. why not just use one of the exisiting recognised systems (e.g. THE or ARWU?) why does Australia need to have another evaluation measure? who is it for?

c. perhaps, within the EI framework.

d. no, I am not even sure what is meant by this question..

Q9. With respect to the ARC’s capability to evaluate research excellence and impact:

(a) How can the ARC best use its expertise and capability in evaluating the outcomes and benefits of research to demonstrate the ongoing value and excellence of Australian research in different disciplines and/or in response to perceived problems?

(b) What elements would be important so that such a capability could inform potential collaborators and end-users, share best practice, and identify national gaps and opportunities?

(c) Would a data-driven methodology assist in fulfilling this purpose?

a. Maybe this could be one area that the ARC College could assist with (whilst staying clear of grant assessment).

b. end-user and industry engagement.

c. identifiable external and linkages. Evidence of translation or change in policy or practice.

Q10. Having regard to the Review’s Terms of Reference, the ARC Act itself, the function, structure and operation of the ARC, and the current and potential role of the ARC in fostering excellent Australian research of global significance, do you have any other comments or suggestions?

1. We need a simpler, less administrative heavy funding system that prioritises the best most impactful research whilst not rewarding exploitation and manipulation of track-record metrics. We need a system that does not lock people out of funding for years (if not forever) because they have taken a break from research e.g. for teaching, retraining, or primary carer duties. Much of this could be solved if the ARC Discovery stream evaluated project quality alone and not track-records. Fellowships must take TR into account, but why is so much weight given to TRs in the discovery stream, it is not clear to me.

2. ARC applications are far too long and convoluted, no more value is added by making them so long - particularly given how little time assessors acutally spend reading them.

3. There is too much ARC money being funnelled into equipment - 'empire building' at Universities (often by senior academics) does not benefit the wider community. National infrastructure accessible by everyone (even if for a nominal fee) should be the future (a huge amount of University based equipment is under utilised). Believe the ARC should primarily be investing in people not infrastructure - there are other schemes that can support this from non-ARC sources.

Submission received

14 December 2022

Publishing statement

Yes, I would like my submission to be published but my and/or the organisation's details kept anonymous. Your submission will need to meet government accessibility requirements.