Amazon Web Services Updated AIF-C01 Exam Questions and Answers by lennie

The correct answer is C because BERTScore is a commonly used metric to evaluate the semantic similarity between generated text (like summaries) and reference text. It uses contextual embeddings from BERT to compare generated and reference sentences, making it highly suitable for evaluating generative tasks like summarization.

From AWS documentation:

"Amazon Bedrock supports BERTScore for evaluating generative text tasks, such as summarization or translation, by comparing the semantic similarity between the output and a reference."

Explanation of other options:

A. AUC is used for binary classification models, not generative text.

B. F1 score is also used for classification problems (precision/recall balance).

D. Real World Knowledge (RWK) score is not a standard or supported evaluation metric in Amazon Bedrock.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Documentation – Model Evaluation Metrics

AWS ML Specialty Guide – Evaluating Generative Models

AWS Generative AI Developer Tools

Few-shot learning involves providing a model with a few examples (shots) to learn from. For improving intent detection accuracy in a chatbot using a large language model (LLM), the data should consist of pairs of user messages and their corresponding correct intents.

Few-shot Learning for Intent Detection:

Few-shot learning aims to enable the model to learn from a small number of examples. For intent detection, the model needs to understand the relationship between user messages and the intended action or meaning.

Providing examples of user messages and the correct user intents allows the model to learn patterns in the phrasing or language that corresponds to each intent.

Why Option C is Correct:

User Messages and Intents: These examples directly teach the model how to map a user’s input to the appropriate intent, which is the goal of intent detection in chatbots.

Improves Accuracy: By using few-shot learning with these examples, the model can generalize better from limited data, improving intent detection.

Why Other Options are Incorrect:

A. Pairs of chatbot responses and correct user intents: Incorrect because it does not focus on user input but rather on outputs.

B. Pairs of user messages and correct chatbot responses: This would be useful for response generation, not intent detection.

D. Pairs of user intents and correct chatbot responses: Again, this is not aligned with detecting intents but with generating responses.

Artificial Intelligence (AI) is the broad field focused on simulating human problem-solving and cognitive abilities, including reasoning, perception, and decision-making.

(Reference: AWS Certified AI Practitioner Official Study Guide)

Machine Learning (ML) is a subset of AI that uses data-driven algorithms to identify patterns and make predictions without explicit programming for each specific task.

(Reference: AWS Machine Learning Overview)

Deep learning is a subset of ML that uses neural networks with many layers (deep neural networks) to process complex data and extract high-level features.

(Reference: AWS Deep Learning on AWS)