Meta Explores Cultural Bias in AI with the Facet Dataset

How Meta explores cultural bias in AI with the facet dataset?

Meta’s FACET is also known as Fairness in Computer Vision Evaluation. It is a dataset that provides a range of images. This images are assessed for various demographic attributes. It includes gender, skin tone, hairstyle, and more.

What Meta’s Facet Provides?

Meta’s FACET  or Fairness in Computer Vision Evaluation dataset provides a range of images. These images are assessed for numerous demographic attributes. It includes gender, skin tone, hairstyle, and more.

The idea is to help more AI developers to factor such elements into their models. It ensures better representation of historically marginalized communities.

What Meta says about FACET?

Meta says that computer vision models allows to accomplish tasks like image classification and semantic segmentation at unprecedented scale.  Meta possess a responsibility to ensure that all the AI systems are fair and equitable.

Benchmarking for fairness in computer vision is notoriously hard task to do. The risk of mislabeling is real, and the people who are using these AI systems will have a better or worse experience.

It is not based on the complexity of the task itself but rather on their demographics. By including a broader set of demographic qualifiers, it can help to address this issue.

In turn, it will make sure about the greater presentation of a wider audience group within the results.

In preliminary studies using FACET, we found that state-of-the-art models are exhibiting performance disparities across demographic groups.

For example, they might struggle to detect people in images whose skin tone is darker, and that challenge can become worse for people with coil rather than straight hair.

What is your goal by releasing FACET?

By releasing FACET, our objective is to enable researchers and practitioners to perform similar benchmarking to better understand the disparities present in their own models.

Moreover, FACET also monitors the impact of mitigation put in place to address fairness concerns. We encourage researchers to use FACET for benchmarking fairness across other vision and multimodal tasks.

It is a valuable dataset that could have a significant impact on AI development. It also ensures better representation and consideration within such tools.

Although Meta notes that FACET is for research evaluation purposes only, and cannot be used for training.

We are releasing the dataset and a dataset explorer with the intention that FACET can become a standard fairness evaluation benchmark for computer vision models.

It helps researchers to evaluate both fairness and robustness across a inclusive set of demographic attributes.

It could end up being a critical update. It maximizes the usage and application of AI tools, and eliminates bias within the existing data collections.

You can read and know more about Meta’s FACET dataset and approach here.

Do you want to know more?

Click here to know about the updates on Meta and other social media platforms.