Fact vs. Fiction: Leveraging AI in Research Management (EARMA Odense 2024)

When AI burst into our collective consciousness a couple of years ago (officially around June 2020), it brought a wave of enormous expectations and myths. We heard that AI would take over jobs, rule the world, develop on its own without human knowledge, and eventually function like the human brain.

But as the dust has settled, we have to ask: is this just mambo-jumbo, or is it a real trend transforming our professional lives?

At EARMA Odense in April 2024, my brilliant co-presenter Katrin Reschwamm and I hosted a highly interactive discussion table to separate fact from fiction. We turned the stage over to the RMAs in the room to hear exactly how they are navigating the AI revolution. Here is what we learned from the front lines.

Co-hosting our interactive EARMA 2024 discussion table with Katrin Reschwamm to uncover how RMAs are actually using AI

The Reality of AI: A Sparring Partner, Not a Replacement

When it comes to daily tasks, RMAs are actively putting AI to work. Participants shared that AI is fantastic for overcoming the dreaded "blank page" syndrome by providing first ideas. It acts as a great sparring partner , helping to draft emails, prepare PowerPoints, structure dissemination plans, and summarize lengthy articles. It can even help compile different texts into a coherent document or produce code for data analysis.

However, the consensus is that it is still very hard to use tools like ChatGPT for writing entire proposals. Furthermore, human RMAs are still better than AI at searching for nuanced funding opportunities.

There are also some amusing growing pains: sometimes the results simply don't fit the user's own language. One participant laughed about entering a prompt in Danish and receiving an answer in Norwegian!

The Dark Side: IP, Security, and "Copy-Paste" Proposals

Despite the benefits, the challenges are very real. The room echoed significant worries about intellectual property (IP) and data security. We are also seeing a concerning trend where researchers are approached directly by AI companies asking them to share their grants.

From an evaluation standpoint, there is a major concern that non-scientific sections in proposals—like Data Management Plans (DMPs)—will all start to look exactly the same if everyone uses the same AI prompts.

Implementation across institutions is also incredibly fragmented. According to the NIH, most applications are already written using AI. Yet, while some universities offer closed AI versions or paid ChatGPT access to their staff, others have absolutely no AI policy, or restrict its use solely to teaching and students.

The collective experiences, challenges, and recommendations crowdsourced from our Odense discussion

How RMAs Should Move Forward

So, how do we adapt? Our discussion table yielded several strong recommendations:

• Control the Tool: Tailor your own internal tools rather than relying strictly on commercial ones, allowing your institution to control the AI and protect its data.

• Mind the Budget: While building an internal tool is safer, institutions must carefully consider the high costs of developing their own Large Language Models (LLMs).

• Adapt Evaluation: In the realm of teaching and academic integrity, many suggest moving toward oral exams to counter AI-generated essays.

• Practice Makes Perfect: Ultimately, the best advice is simply to use AI more often to gain a better understanding of its true capabilities and limits.

  
  

AI won't magically write a winning Horizon Europe proposal for you, but if you learn how to spar with it, it will undoubtedly make your job easier.

Fact vs. Fiction: Leveraging AI in Research Management (EARMA Odense 2024)

Fact vs. Fiction: Leveraging AI in Research Management (EARMA Odense 2024)