AI Evaluations - Library

Elicit - Library AI Evaluation

This evaluation is part of Deakin Library’s AI Evaluation series, providing structured, practice-informed insights into emerging AI technologies. Evaluations are designed to support critical decision-making and responsible engagement with AI, guided by Deakin’s Generative AI Framework and Principles.

Our findings and assessment are shared to inform your judgement. Evaluations are not an endorsement.

Evaluations like this one are not about deciding whether the AI “works,” but about understanding what kind of thinking and behaviours it encourages and whether that aligns with our pedagogical, scholarly, and professional values.

Elicit overview

Summary findings

Evaluation snapshot

• Elicit is a semantic search tool that leverages the Semantic Scholar database. It enables literature discovery, literature summarisation, and has literature/systematic/rapid review and research report automation.  
• Freemium product that includes Enterprise options. 
• An AI-enabled systematic review feature was added February 2025 to aid in searching, screening, data extraction, and report generation, promoting Living Reviews. It integrates with Zotero but limits screening to 500 papers. Searches are not directly reproducible and currently lack transparency required for systematic review reporting. 
• Recently introduced Elicit Reports with deep research functionality allows rapid reviews. Free-version users can screen up to 50 papers and extract data from 8 papers, with reports following the PRISMA flowchart structure. However, Elicit Reports require a clearly defined research question and don’t support overview-style questions.  

Benefits

• Good level of transparency in relation to privacy and data collection, usage, storage (including international servers). Provision of a named privacy contact for enquiries around Elicit’s management of personal information.  
• Clear information on data sources (Semantic Scholar) but provenance of individual records is unclear (e.g. partner attribution not specified). 
• Elicit workflows enhance critical thinking and study evaluation but lack formal critical appraisal functionalities, requiring researchers to assess study quality and relevance.  
• Transparency in research is promoted, aligning with ethical standards from the Australian Code for the Responsible Conduct of Research. It offers a transparent review screening process using Large Language Models (LLM) for screening, allowing human decision-making and efficient data verification. Academic integrity is also somewhat encouraged with citation practice recommendation. 
• A key advantage is integration with tools like Zotero, facilitating effective referencing and source management. Elicit also supports open practices by allowing users to export results into a CSV for further analysis and share projects internally and externally. 

Limitations or risks

• Elicit limitations (accuracy, potential misinterpretation of underlying data, risk of English language bias) are acknowledged and communicated.
• Currently, searches lack reproducibility and transparency for systematic review reporting. 
• There is an increased likelihood of copyright violation by users of Elicit in the upload functionality, due to nuanced rights and permissions for different articles/journals. 

Notable points

• Developers claim Elicit outputs are about 80–90% accurate, urging users to critically evaluate results.
• Elicit reports follow the PRISMA flowchart structure (see Documenting your process and PRISMA advice).
• Searches are not directly reproducible and currently lack the transparency required for systematic review reporting.
• Elicit workflows enhance critical thinking and study evaluation but lack formal critical appraisal functionalities, requiring researchers to assess study quality and relevance. 
• Despite being user-friendly with training resources, users should be aware of copyright and licensing issues, especially when uploading PDFs, and may encounter bugs or incomplete features as Elicit evolves. 

Considerations and implications

This section surfaces key reflections from the evaluation, moving beyond tool description to sense-making. It considers how AI may influence library practice, sector responsibilities, and the broader information landscape. As a boundary-spanning part of the university, the Library draws on its expertise in information practice and knowledge management to surface impacts across cohorts.

Our insights are provisional and reflective, emphasising conditions and contexts rather than certainties or prescriptions.

Our role is not to endorse tools like Elicit but to make meaning. To examine how different forms of AI may reshape how information is produced, accessed, and understood. These considerations are part of the academic library’s role in supporting the organisation, interrogation, and circulation of knowledges within our academic contexts.

Click on the plus (+) icons below to explore considerations related to digital literacies, user behaviours and needs, educator capability, open practices, and library practices: