Question Answering

  • Special field in AI/NLP
  • Goal: Answer questions in natural language
  • Classification of QA systems:
    • Answer Domain: Open/Closed
    • Supported Question Types: Factoids/Definitions/Yes-No/…
    • Technologies: Shallow QA/Deep QA
  • Dissociation to IR:
    • IR return relevant documents for keyword queries
    • QA should answer complicated questions

Answer Domain

  • Closed-Domain
    • QA within a single domain
    • e.g., medicine, animals, etc.
    • Opportunities:
      • Exploit domain-specific language
      • Use domain-specific ontologies (e.g., MeSH)
  • Open-Domain
    • QA answering any topic
    • Any type of question may be asked (unknown topic and question type)
    • Requires different data sources
      • Aggregation and selection could become hard

Question Types

  • Question Types:
    • Factoid
    • List
    • Definition
    • Relationship
    • Superlative
    • Yes-No
    • Opinion
    • Cause & Effect
  • Answer types:
    • Person
    • Place
    • Date
    • Number
    • Explanation
    • Method
  • Question focus:
    • Entity being asked by the question
  • Question topic:
    • Object or event the question is about
  • e.g. What is the height of Mt. Everest?
    • Focus: Height
    • Topic: Mt. Everest

Technology

Shallow QA

  • Uses simple IR methods to find relevant answer documents
  • Find a passage directly answering the question
  • Keyword-based search (strip filler words)
  • Answer: Use templates
  • Simple case: Question already answered in the web

../_images/14-shallow.png

Deep QA

  • Goal: Answer more complex questions
  • Used techniques:
    • Part-of-Speech Tagging (POS)
    • Named Entity Recognition (NER)
    • Ontologies (linked data, Folksonomy, …)
    • Reasoning (translate to logical query)

Part-of-Speech Tagging

  • Detect grammatical type of words (lexical classification)
  • Detect relationship to other words
  • Problems:
    • Language is ambiguous
    • Semantics are hidden/implicit
    • Using heuristics
  • Classifications (around 150):
    • Noun
    • Verb
    • Article
    • Adjective
    • Adverb
    • Preposition
    • Pronoun
    • Conjunction
    • Interjection
  • Training:
    • Algorithms are trained on a corpus

Named Entity Recognition

  • Identify words representing named entities (e.g., from a domain)
  • Usually, entities are tagged with their type

Common-sense Knowledge Base / Ontology

  • Related to upper ontologies
  • Usually, automatically generated from encyclopedias
  • e.g., DBpedia, YAGO
  • Contents:
    • Ontology of classes + individuals
    • Parts, materials, properties, functions of objects
    • Locations, duration, preconditions, effects of actions and events
  • Use commonsense ontologies to answer queries directly

Showcase: Semi-Shallow Factoid QA

  • Semi-Shallow Factoid System:
    • Input knowledge: Text documents
    • Factoid questions (e.g., when, who, how big)
    • Use WWW data sources
    • Use IR techniques
    • Use some NLP techniques
  • Steps:
    • Query formulation
    • Query classification
    • Document retrieval
    • Passage retrieval
    • Answer processing

../_images/14-showcase.png

Query Formulation

  • Take the query and create a list of keywords
  • Use the bag of words for a normal IR system
  • Steps:
    • Remove stop words
    • Expand query using a thesaurus/ontology (e.g., WordNet)

Query Classification

  • Find required answer type
  • Use question ontologies to get the questioned property
    • e.g., How big relates to size
  • Use supervised learning to learn question and answer types
  • Detect question topic (e.g., NER)
  • Can be used for query formulation
    • e.g., combine queried property and entity

Document Retrieval

  • Retrieve candidate documents containing the query terms
  • Result: Candidate documents

Passage Retrieval

  • Find relevant passages
    • Remove passages not containing the answer
  • Rank using different characteristics:
    • Number of named entities of the correct answer type (e.g., for size: meter, feet)
    • Number of question keywords/topics
    • Sentence proximity between passage and query

Answer Extraction

  • From the selected passage, extract the actual answer
  • Approaches:
    • Answer pattern extraction (designed patters from the answer ontology)
    • N-gram tiling (statistical approach)
  • Problem: Multiple different answers
    • Which to chose?

Watson (D)

  • Deep QA project
  • Watson beat all in Jeopardy
    • Loose money if wrong answer chosen
    • Open domain
    • Complex and obfuscated questions
  • Architecture:
    • Parallelized probabilistic evidence-based approach
    • Combination of shallow IR and Deep Ontology
    • Generate multiple hypotheses
    • Find evidence
    • Model probabilistic confidences
  • Data:
    • Seed corpus (encyclopedias, dictionaries, …)
    • Create expanded corpus (query internet for related documents)
    • Structured corpus (databases, ontologies, taxonomies)
  • Question classification:
    • Type of question
    • Focus and Lexical Answer Type Detection
    • Relation detection
    • Question decomposition
  • Hypothesis scoring (deep QA):
    • IR-based scorer
    • Ontology-based reasoning
    • Geo-spatial reasoning
    • Temporal reasoning