About NUS Open QA

Overview

NUS Open QA is an abbreviation for NUS Open Question-Answering, NUS being the abbreviation of the National University of Singapore. NUS Open QA is an open-sourced, information-retrieval (IR) based question-answering (QA) system.

This project was formerly named QANUS (for Question Answering NUS).

There are 2 key motivating factors behind the development of NUS Open QA.

  1.  Serve as a framework from which QA systems can be quickly developed.
  2. Act as a baseline system against which QA performance can be easily and reproducibly benchmarked.

Extensible Framework

NUS Open QA is a pipelined QA system. It is designed from the ground-up to be easily extendable. This allows NUS Open QA to serve as a good starting point from which ideas and technologies can be quickly tested and validated.

NUS Open QA is shipped with many of the common techniques used for state-of-the-art QA, including modules for named entity recognition, part-of-speech tagging and question classification. As new techniques mature, these can be easily incorporated into NUS Open QA.

It is easy to customise NUS Open QA to different datasets and techniques by adding/removing input/output modules, or text processing modules as needed. More information can be obtained from the documentation for NUS Open QA.

Reproducible Benchmark

The open-source nature of the system will allow researchers to reliably reproduce experimental results which can serve as a baseline for more advanced and complex QA systems.

It is usually hard to validate the performance of various QA systems and technologies as many systems are proprietary and not freely available to the community. NUS Open QA gives the community access to a common system against which to benchmark new QA systems.

Currently performance is typically measured relative to results from part TREC QA tracks. However these previous results are static, and do not reflect the general performance of the state-of-the-art as the years roll by.

As the field advances, NUS Open QA will be kept updated with the latest technological advances and thus can be kept relevant. This will allow new systems to reproducibly validate their performance against an up-to-date QA system.