Seminar: "Towards AI Readiness in Auditing: Benchmarks, Models, and Domain-Structured Reasoning"
Abstract
As financial reporting grows in scale and complexity, the auditing domain demands AI systems capable of structured reasoning, taxonomy alignment, and cross-document consistency. This talk presents a comprehensive effort toward AI readiness in auditing through new benchmarks and domain-specific models. I introduce FinTagging, an LLM-oriented benchmark for end-to-end XBRL tagging that evaluates numerical entity extraction (FinNI) and fine-grained US-GAAP concept linking (FinCL). I then present FinAuditing, the first benchmark built from real XBRL filings to assess semantic matching, hierarchical relationship detection, and multi-step numerical reasoning across structured multi-document data. Zero-shot evaluation of 13 LLMs reveals persistent challenges in schema-aware reasoning and consistency checking. Finally, I introduce AuditWen, an open-source audit-focused LLM trained on a 30K-sample instruction dataset spanning entity recognition, legal reasoning, and audit report generation. Together, these contributions provide the first unified ecosystem for advancing trustworthy AI in auditing.
Invited Speaker: Jimin Huang
Short Bio: Jimin Huang is the founder of The Fin AI community, an initiative dedicated to advancing open science, tooling, and model development for the financial services industry with a focus on responsible innovation. The Fin AI is now an associated member of FINOS and a collaborator with the NVIDIA AI Technology Center (NVAITC). Jimin is also an associated member of The National Centre for Text Mining (NaCTeM). His research spans natural language processing and computational finance, with a particular emphasis on financial large language models (LLMs) and open-source contributions. He is the organizer of the FinLLM Challenge at FinNLP-AgentScen @ IJCAI-2024 and serves as the general chair for The Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FNP), and the 1st Workshop on Large Language Models for Finance and Legal (LLMFinLegal).
