Amazon Web Services Updated AIF-C01 Exam Questions and Answers by luisa

The verified answer is C. Faithfulness. The company is evaluating foundation models for a summarization task by using an LLM-as-a-judge evaluation job in Amazon Bedrock. AWS documentation states that with a model evaluation job that uses a judge model, Amazon Bedrock uses one LLM to score another model’s responses and provide an explanation of how it scored each prompt and response pair. AWS also states that Amazon Bedrock provides built-in metrics that can be selected for these judge-based evaluation jobs.

Among the answer choices, Faithfulness is the only valid built-in metric for an Amazon Bedrock LLM-as-a-judge model evaluation job. AWS API documentation lists the valid built-in metric names for LLM-as-a-judge evaluation jobs, including Correctness, Completeness, Faithfulness, Helpfulness, Coherence, Relevance, FollowingInstructions, ProfessionalStyleAndTone, and responsible AI metrics such as Harmfulness, Stereotyping, and Refusal. The same documentation identifies Summarization as a valid task type for model evaluation jobs. Therefore, Faithfulness is a valid built-in metric for evaluating whether generated summaries remain grounded in the source input rather than introducing unsupported claims.

Option A. Context relevance and option B. Context coverage are incorrect for this question because AWS lists those as metrics for knowledge base retrieval-only evaluation jobs, not general LLM-as-a-judge model evaluation of foundation model summarization output. These metrics evaluate retrieved context, not the quality of a summarization model’s generated response. Option D. RMSE is also incorrect because root mean square error is a regression metric. It is not a built-in LLM-as-a-judge metric for summarization in Amazon Bedrock.

Because the task is summarization and the evaluation job is LLM-as-a-judge, the correct built-in evaluation metric from the available choices is Faithfulness.

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Amazon Neptune is a fully managed graph database service optimized for storing and querying highly connected data.

AWS guidance highlights that graph databases are ideal for:

Fraud detection

Relationship analysis

Pattern discovery across entities such as users, transactions, and accounts

Amazon Neptune supports popular graph models and enables ML-driven investigations by efficiently traversing relationships, which is critical for fraud pattern analysis.

Why the other options are incorrect:

OpenSearch (A) is optimized for search and log analytics.

Aurora (B) is a relational database, not graph-based.

MemoryDB (D) is an in-memory data store for low-latency workloads.

AWS AI document references:

Amazon Neptune Overview

Graph-Based Fraud Detection on AWS

Using Graph Databases for Investigations

The correct answer is Text embeddings model, which transforms input text into dense vector representations. These vectors preserve semantic meaning and are ideal for similarity search, retrieval-augmented generation (RAG), and semantic search use cases, especially with Amazon OpenSearch Service. According to AWS Bedrock documentation, embeddings models are specifically designed to map text into a numerical space where related concepts are placed closer together. This allows efficient querying and ranking of documents based on the user’s question. This step is critical in chatbots using private knowledge bases. Text completion and instruction-following models focus on generating or following tasks, not producing embeddings. Image generation models are irrelevant here. Therefore, using a text embeddings model ensures that the chatbot retrieves contextually relevant results from stored vectors.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Developer Guide – Text Embeddings Models

AWS ML Specialty Guide – Semantic Search and Vector Databases

Fine-tuning involves training an already pre-trained FM on a smaller, labeled dataset for task specialization.

Data selection is about curating training data.

Pre-training is the initial training phase on massive datasets.

Evaluation happens after training, not during.

???? Reference:

AWS Documentation – Fine-tuning in Amazon Bedrock