AI-based Question Answering Assistance for Analyzing Natural-language Requirements

by   Saad Ezzini, et al.

By virtue of being prevalently written in natural language (NL), requirements are prone to various defects, e.g., inconsistency and incompleteness. As such, requirements are frequently subject to quality assurance processes. These processes, when carried out entirely manually, are tedious and may further overlook important quality issues due to time and budget pressures. In this paper, we propose QAssist – a question-answering (QA) approach that provides automated assistance to stakeholders, including requirements engineers, during the analysis of NL requirements. Posing a question and getting an instant answer is beneficial in various quality-assurance scenarios, e.g., incompleteness detection. Answering requirements-related questions automatically is challenging since the scope of the search for answers can go beyond the given requirements specification. To that end, QAssist provides support for mining external domain-knowledge resources. Our work is one of the first initiatives to bring together QA and external domain knowledge for addressing requirements engineering challenges. We evaluate QAssist on a dataset covering three application domains and containing a total of 387 question-answer pairs. We experiment with state-of-the-art QA methods, based primarily on recent large-scale language models. In our empirical study, QAssist localizes the answer to a question to three passages within the requirements specification and within the external domain-knowledge resource with an average recall of 90.1 actual answer to the posed question with an average accuracy of 84.2 Keywords: Natural-language Requirements, Question Answering (QA), Language Models, Natural Language Processing (NLP), Natural Language Generation (NLG), BERT, T5.


Modern Question Answering Datasets and Benchmarks: A Survey

Question Answering (QA) is one of the most important natural language pr...

COREQQA – A COmpliance REQuirements Understanding using Question Answering Tool

We introduce COREQQA, a tool for assisting requirements engineers in acq...

Lexical Disambiguation in Natural Language Questions (NLQs)

Question processing is a fundamental step in a question answering (QA) a...

Using Language Models for Enhancing the Completeness of Natural-language Requirements

[Context and motivation] Incompleteness in natural-language requirements...

Incidental Supervision from Question-Answering Signals

Human annotations are costly for many natural language processing (NLP) ...

Improving Requirements Completeness: Automated Assistance through Large Language Models

Natural language (NL) is arguably the most prevalent medium for expressi...

Generative Stock Question Answering

We study the problem of stock related question answering (StockQA): auto...

Please sign up or login with your details

Forgot password? Click here to reset