Bias and Ethics in Datasets

The presentation by Yusuke Hirota discusses gender and racial bias in visual question answering (VQA) datasets. It highlights the challenges of vision language tasks, such as image captioning and VQA, and the societal biases present in these datasets. The analysis reveals gender and racial stereotypes, with men often associated with sports and women with motionless activities. Racial bias is evident in the overrepresentation of white individuals. The presentation suggests solutions like automatic screening and ethical instructions to address these biases.

What is one of the main challenges in vision language tasks?

• Generating high-quality images

• Ensuring data represents the real world

• Improving computational speed

• Reducing the size of datasets

In the analysis of gender bias, what was found about the frequency of questions related to men compared to women?

• Questions about women were non-existent

• Questions about men were about double those about women

• Questions about men and women were equal

• Questions about women were more frequent

What stereotype is often associated with men in visual question answering datasets?

• Gardening

• Sports

• Fashion

• Cooking

How are racial biases identified in datasets?

• By analyzing the color of images

• By counting the number of images

• By checking the size of the dataset

• By examining questions containing race or ethnicity

What is a common issue with racial questions in datasets?

• They are too complex

• They often lack visual grounding

• They are too simple

• They are always accurate

What is one proposed solution to address harmful samples in datasets?

• Use automatic screening to remove ungrounded questions

• Ignore the harmful samples

• Reduce the number of questions

• Increase the dataset size

Why is it important to incorporate ethical instructions in dataset annotation?

• To increase the number of annotators

• To reduce the cost of annotation

• To ensure annotators are aware of biases

• To speed up the annotation process

What is a key takeaway from the study on bias in datasets?

• Biases are not present in datasets

• Biases are only present in small datasets

• Biases are easy to eliminate

• Biases can lead to harmful stereotypes

What is the impact of not addressing bias in training models?

• Models become faster

• Models become more accurate

• Models amplify societal bias

• Models require less data

What is the ultimate goal of raising awareness about bias in datasets?

• To improve computational speed

• To increase dataset size

• To mitigate societal bias

• To reduce the number of questions