Amazon Web Services Updated AIF-C01 Exam Questions and Answers by hubert

The company is developing an ML model to predict customer churn, a binary classification task (churn or no churn). The F1 score is an evaluation metric that balances precision and recall, making it suitable for assessing the performance of binary classification models, especially when dealing with imbalanced datasets, which is common in churn prediction.

Exact Extract from AWS AI Documents:

From the Amazon SageMaker Developer Guide:

"The F1 score is a metric for evaluating binary classification models, combining precision and recall into a single value. It is particularly useful for tasks like churn prediction, where class imbalance may exist, ensuring the model performs well on both positive and negative classes."

(Source: Amazon SageMaker Developer Guide, Model Evaluation Metrics)

Detailed Explanation:

Option A: F1 scoreThis is the correct answer. The F1 score is ideal for binary classification tasks like churn prediction, as it measures the model’s ability to correctly identify both churners and non-churners.

Option B: Mean squared error (MSE)MSE is used for regression tasks to measure the average squared difference between predicted and actual values, not for binary classification.

Option C: R-squaredR-squared is a metric for regression models, indicating how well the model explains the variability of the target variable. It is not applicable to classification tasks.

Option D: Time used to train the modelTraining time is not an evaluation metric for model performance; it measures the duration of training, not the model’s accuracy or effectiveness.

Comprehensive and Detailed Explanation:

SageMaker Clarify provides model explainability, bias detection, and helps visualize how input features affect predictions.

Canvas is for low-code ML building.

Feature Store is for storing and managing ML features.

Ground Truth is for data labeling.

???? Reference:

AWS Documentation – Amazon SageMaker Clarify

The company wants to enhance the response quality of an LLM for complex problem-solving tasks requiring detailed reasoning and step-by-step explanations. Chain-of-thought prompting encourages the LLM to break down the problem into intermediate steps, providing a clear reasoning process before arriving at the final answer, which is ideal for this requirement.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

"Chain-of-thought prompting improves the reasoning capabilities of large language models by encouraging them to break down complex tasks into intermediate steps, providing a step-by-step explanation that leads to the final answer. This technique is particularly effective for problem-solving tasks requiring detailed reasoning."

(Source: AWS Bedrock User Guide, Prompt Engineering Techniques)

Detailed Explanation:

Option A: Few-shot promptingFew-shot prompting provides a few examples to guide the LLM but does not explicitly encourage step-by-step reasoning or detailed explanations.

Option B: Zero-shot promptingZero-shot prompting relies on the LLM’s pre-trained knowledge without examples, making it less effective for complex tasks requiring detailed reasoning.

Option C: Directional stimulus promptingDirectional stimulus prompting is not a standard technique in AWS documentation, likely a distractor, and does not address step-by-step reasoning.

Option D: Chain-of-thought promptingThis is the correct answer. Chain-of-thought prompting enhances response quality for complex tasks by guiding the LLM to reason step-by-step, providing detailed explanations.

The company is building a chatbot using an LLM on Amazon Bedrock, and the chatbot needs to use content from previous customer messages to resolve requests. Adding previous messages to the model prompt (also known as providing conversation history) enables the LLM to maintain context across interactions, allowing it to respond coherently based on the ongoing conversation.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

"To enable a large language model (LLM) to maintain context in a conversation, you can include previous messages in the model prompt. This approach, often referred to as providing conversation history, allows the LLM to generate responses that are contextually relevant toprior interactions."

(Source: AWS Bedrock User Guide, Building Conversational Applications)

Detailed Explanation:

Option A: Turn on model invocation logging to collect messages.Model invocation logging records interactions for auditing or debugging but does not provide the LLM with access to previous messages during inference to maintain conversation context.

Option B: Add messages to the model prompt.This is the correct answer. Including previous messages in the prompt gives the LLM the conversation history it needs to respond appropriately, a common practice for chatbots on Amazon Bedrock.

Option C: Use Amazon Personalize to save conversation history.Amazon Personalize is for building recommendation systems, not for managing conversation history in a chatbot. This option is irrelevant.

Option D: Use Provisioned Throughput for the LLM.Provisioned Throughput in Amazon Bedrock ensures consistent performance for model inference but does not address the need to use previous messages in the conversation.