How to investigate bias while preserving model accuracy?

PMLE By islever Editorial Last updated

Short answer

D Define a fairness metric that is represented by accuracy across the sensitive features. Train a BigQuery ML boosted trees classification model with all features. Use the trained model to make predictions on a test set. Join the data back with the sensitive features, and calculate a fairness metric to investigate whether it meets your requirements.

Question

You work for a telecommunications company. You’re building a model to predict which customers may fail to pay their next phone bill. The purpose of this model is to proactively offer at-risk customers assistance such as service discounts and bill deadline extensions. The data is stored in BigQuery and the predictive features that are available for model training include: - Customer_id - Age - Salary (measured in local currency) - Sex - Average bill value (measured in local currency) - Number of phone calls in the last month (integer) - Average duration of phone calls (measured in minutes) You need to investigate and mitigate potential bias against disadvantaged groups, while preserving model accuracy. What should you do?
  1. Determine whether there is a meaningful correlation between the sensitive features and the other features. Train a BigQuery ML boosted trees classification model and exclude the sensitive features and any meaningfully correlated features.
  2. Train a BigQuery ML boosted trees classification model with all features. Use the ML.GLOBAL_EXPLAIN method to calculate the global attribution values for each feature of the model. If the feature importance value for any of the sensitive features exceeds a threshold, discard the model and tram without this feature.
  3. Train a BigQuery ML boosted trees classification model with all features. Use the ML.EXPLAIN_PREDICT method to calculate the attribution values for each feature for each customer in a test set. If for any individual customer, the importance value for any feature exceeds a predefined threshold, discard the model and train the model again without this feature.
  4. Define a fairness metric that is represented by accuracy across the sensitive features. Train a BigQuery ML boosted trees classification model with all features. Use the trained model to make predictions on a test set. Join the data back with the sensitive features, and calculate a fairness metric to investigate whether it meets your requirements. Correct Answer

Key Takeaways

The correct approach is to measure fairness using a defined metric on a test set containing sensitive attributes, rather than excluding features based on correlation or global importance scores.

To investigate and mitigate bias without sacrificing accuracy, define a fairness metric (like accuracy parity) and evaluate the model on a test set joined with sensitive features. This allows for the quantification of bias across groups before deciding on mitigation strategies.

Common mistake: Selecting Option A is a common error because removing features correlated with sensitive attributes (fairness through unawareness) often reduces model accuracy and fails to address bias effectively.

Expert Analysis

Why the Answer Is Correct

Option D is the best approach because it follows the standard workflow for Responsible AI: define a quantifiable fairness metric relevant to the context, train the model, and then evaluate its performance specifically against that metric using a test set that includes sensitive attributes. This method allows you to directly measure if the model is disadvantaged for specific groups without arbitrarily removing data that might be crucial for maintaining model accuracy.

Why the Other Options Are Wrong

Option A is incorrect because excluding sensitive features and their proxies (correlated features) is a technique known as 'fairness through unawareness,' which often fails to mitigate bias and significantly degrades model accuracy. Option B is incorrect because global feature importance (ML.GLOBAL_EXPLAIN) indicates a feature's overall contribution to the model, not whether it causes disparate performance across demographic groups. Option C is incorrect because local explanations (ML.EXPLAIN_PREDICT) explain individual predictions, and discarding a model based on a single customer's attribution values is not a valid strategy for assessing group-level fairness.

Community Comment Notes

The community consensus strongly supports Option D, emphasizing that it allows for a direct bias assessment and preserves information by avoiding premature feature removal. Several comments referenced Google Cloud documentation on evaluation and fairness, which validates the approach of defining metrics and evaluating on test sets. While one comment suggested Option A, it received no support and contradicts the requirement to preserve accuracy.

Official Reference

https://cloud.google.com/vertex-ai/docs/evaluation/intro-evaluation-fairness

This link opens the official vendor documentation in a new tab.

Exam Strategy

When answering questions about bias, prioritize options that involve defining a specific fairness metric and evaluating the model on a test set with sensitive attributes over options that suggest deleting features or using only global/local explanations.

Community Votes

D
100%

100% of anonymous learners picked answer D. Votes are pick records left by other test-takers — they are not the verified answer.

Discussion 4 contributions

Compiled from public community threads and reader submissions, lightly edited for clarity. Handles and vote tallies are shown as recorded, and comments reflect the views of individual contributors.

From public community threads 4

  1. fitri001 👍 3 Answered D
    Fairness Metric: Defining a metric like parity (equal accuracy) or calibration (similar predicted probabilities) across sensitive features like age, sex, or salary allows you to quantify potential bias. Model Training with All Features (Initially): Training the model with all features provides a baseline performance and allows you to identify potentially biased features later. Test Set Predictions: Making predictions on a held-out test set ensures the evaluation is based on unseen data and avoids overfitting. Joining Back Sensitive Features: Reintroducing sensitive features after prediction allows you to calculate fairness metrics for different customer groups. Iterative Refinement: Based on the fairness metric results, you can determine if further mitigation strategies are needed.
  2. pikachu007 👍 2 Answered D
    Direct Bias Assessment: It directly measures model fairness using a relevant metric, providing clear insights into potential issues. Preserving Information: It avoids prematurely removing features, potentially capturing valuable predictive signals while mitigating bias. Aligning with Goals: It allows tailoring the fairness metric to specific ethical and business objectives.
  3. pinimichele01 👍 1 Answered D
    https://cloud.google.com/vertex-ai/docs/evaluation/intro-evaluation-fairness
  4. prtikare 👍 1
    Answer is A

Reader comments & corrections

No reader corrections yet — spotted an error or have a note? Share it below.

Log in to comment, report an error, or add a note about this question.

How to Minimize Development Time for Weekly TensorFlow Retraining in Vertex AI?

S-grade

How to Handle Categorical Features in BigQuery ML for Fast Deployment?

S-grade

How to preprocess BigQuery data for TensorFlow?

S-grade

How to Track Model and Prediction Lineage in Vertex AI?

S-grade

Practice All PMLE Questions

Access 65 questions with complete answers and detailed explanations.

View Full PMLE Practice Test →

← Back to PMLE Study Guide