Answering List-Type Questions in Health Domain with Pretrained Large Language Model: A Case for COVID-19 Symptoms

List-type questions, which can have a varying number of answers, are more common in the health domain where people seek for health-related information from a passage or passages. An example of this type of question answering task is to find COVID-19 symptoms from a Twitter post. However, due to the lack of annotated instances for supervised learning, automatic identification of COVID-19 symptoms from Twitter posts is challenging. We investigated detection of symptom mentions in Twitter posts using GPT-3, a pre-trained large language model, along with few-shot learning. Our results of 5-shot and 10-shot learning on a corpus of 655 annotated tweets demonstrate that few-shot learning with pre-trained large language model is a promising approach to answering list-type questions with a minimal amount of effort of annotation.