Amazon Web Services Updated AIF-C01 Exam Questions and Answers by lennie

The average response time is the correct metric for measuring the runtime efficiency of operating AI models.

Average Response Time:

Refers to the time taken by the model to generate an output after receiving an input. It is a key metric for evaluating the performance and efficiency of AI models in production.

A lower average response time indicates a more efficient model that can handle queries quickly.

Why Option C is Correct:

Measures Runtime Efficiency: Directly indicates how fast the model processes inputs and delivers outputs, which is critical for real-time applications.

Performance Indicator: Helps identify potential bottlenecks and optimize model performance.

Why Other Options are Incorrect:

A. Customer satisfaction score (CSAT): Measures customer satisfaction, not model runtime efficiency.

B. Training time for each epoch: Measures training efficiency, not runtime efficiency during model operation.

D. Number of training instances: Refers to data used during training, not operational efficiency.

Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the training data. AWS documentation explains that an underfit model performs poorly on both training and test datasets, which directly matches the scenario described.

In this case, the model shows low accuracy during training and evaluation, indicating that it has not learned sufficient relationships from the data. AWS identifies common causes of underfitting as insufficient model complexity, inadequate feature representation, overly aggressive regularization, or insufficient training time.

Underfitting is different from overfitting. Overfitting occurs when a model performs well on training data but poorly on test data, which is not the situation here. Hallucination applies to generative AI outputs, not supervised ML model accuracy. Cross-validation is a model evaluation technique, not a cause of poor performance.

AWS emphasizes the importance of diagnosing underfitting early in the model development lifecycle. Remedies include increasing model complexity, adding relevant features, reducing regularization, or selecting a more expressive algorithm. These steps allow the model to better learn from the data and improve accuracy across both training and test sets.

AWS machine learning best practices clearly associate low performance on both datasets with underfitting, making this the most likely cause of the problem described.

The verified answer is B. Recall. In a Retrieval Augmented Generation system, the retrieval component must find the documents or passages that contain the information needed to answer a user query. The question specifically says the company wants to retrieve all relevant documents and avoid missing important information. That wording points directly to recall. Recall measures how many of the relevant items were successfully retrieved out of all relevant items that should have been retrieved. In practical RAG evaluation, high recall means the retriever is less likely to omit important source documents from the context sent to the language model.

Faithfulness is incorrect because it evaluates whether the generated answer is grounded in, and consistent with, the retrieved context. It is a generation-quality metric, not the primary retrieval-coverage metric. A response can be faithful to retrieved context even if the retriever failed to include some important documents. Mean reciprocal rank is incorrect because it measures how highly the first relevant result appears in the ranked list. It is useful for ranking quality, but it does not directly confirm that all relevant documents were retrieved. Precision is incorrect because precision measures how many retrieved documents are actually relevant. High precision reduces irrelevant results, but it can still miss relevant documents if the retriever returns only a narrow subset.

AWS documentation separates retrieval-focused evaluation metrics from response-generation metrics. For knowledge base retrieval-only evaluations, AWS lists built-in retrieval metrics such as context relevance and context coverage, while response generation evaluations include metrics such as faithfulness, correctness, completeness, and helpfulness. This distinction supports the principle that retrieval coverage and not missing relevant information is a retrieval-side concern. In standard information retrieval terminology, the metric that best captures “do not miss relevant documents” is recall.

The verified answer is D. AWS Artifact. The company needs compliance documentation for submission to regulatory authorities. AWS Artifact is the AWS service designed for this exact requirement. AWS describes AWS Artifact as a self-service portal that provides on-demand access to AWS security and compliance reports and agreements. AWS also states that customers can use AWS Artifact to access AWS and independent software vendor security and compliance reports, download compliance reports, and manage compliance agreements. This directly matches the need to obtain documentation for auditors or regulators.

AWS compliance guidance also explains that AWS supports many security standards and compliance certifications, and AWS reports such as SOC reports help customers and auditors understand AWS controls established to support operations and compliance. AWS Artifact is the mechanism customers use to retrieve many of these compliance documents. Therefore, if the company is using Amazon Bedrock and needs compliance documentation related to AWS services, AWS Artifact is the appropriate service.

AWS Config is incorrect because it records, evaluates, and audits AWS resource configurations. It can help assess whether resources comply with internal rules, but it is not the service for obtaining AWS compliance reports for submission to regulators. Amazon Bedrock is incorrect because Bedrock is the generative AI service being used to process invoices; it does not serve as the compliance report portal. Amazon SageMaker AI is incorrect because SageMaker AI is used to build, train, deploy, and manage machine learning models. It is not the AWS service for downloading AWS compliance documentation.

The question is not asking how to process invoices securely or how to configure Bedrock controls. It is asking where to obtain formal compliance documentation. The correct AWS service is AWS Artifact.