Free iSQI CT-AI Exam Dumps

Factors Affecting ML Functional Performance:The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics.

A/B testing is a statistical testing method that compares two different versions of a system to determine which one performs better. In this scenario, the old NLP algorithm was rated for accuracy, and after the update, the new algorithm was also rated by users. A statistical test was performed to compare the two versions, which is the fundamental approach of A/B testing.

A/B testing is commonly used in:

User experience testing (e.g., comparing different versions of a website).

ML model evaluation (e.g., comparing two AI-based classifiers).

Performance assessment (e.g., determining if a new recommendation algorithm is more effective).

This approach allows for data-driven decisions, ensuring that any changes to the system result in meaningful improvements.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Section 9.4 - A/B Testing states that A/B testing is used to compare updates in AI-based systems to determine if the newer version is better.

The problem describes a classification model that predicts whether a patient is at risk for a stroke. The confusion matrix is provided, and the testers have calculated a performance metric as 2/3 or 0.6667.

From the ISTQB Certified Tester AI Testing (CT-AI) Syllabus, the definitions of functional performance metrics from a confusion matrix include:

Accuracy:

Accuracy=TP+TNTP+TN+FP+FNAccuracy = \frac{TP + TN}{TP + TN + FP + FN}Accuracy=TP+TN+FP+FNTP+TN

Measures the proportion of correctly classified instances (both true positives and true negatives) over the total dataset.

If the value is 0.6667, it suggests that the metric includes both correct positive and negative classifications, aligning with accuracy.

Precision:

Precision=TPTP+FPPrecision = \frac{TP}{TP + FP}Precision=TP+FPTP

Measures how many predicted positive cases were actually positive.

Does not match the given calculation.

Recall (Sensitivity):

Recall=TPTP+FNRecall = \frac{TP}{TP + FN}Recall=TP+FNTP

Measures how many actual positives were correctly identified.

Does not match the 0.6667 value.

F1-Score:

F1=2PrecisionRecallPrecision+RecallF1 = 2 \times \frac{Precision \times Recall}{Precision + Recall}F1=2Precision+RecallPrecisionRecall

A balance between precision and recall.

The formula is different from the provided calculation.

Since the formula for accuracy matches the calculated value of 0.6667, the best answer is D. Accuracy.

Certified Tester AI Testing Study Guide Reference:

ISTQB CT-AI Syllabus v1.0, Section 5.1 (Confusion Matrix and Functional Performance Metrics)

ISTQB CT-AI Syllabus v1.0, Section 5.4 (Selecting ML Functional Performance Metrics)

Threshold coverage is a specific type of coverage measure used in neural network testing. It ensures that each neuron in the network achieves an activation value greater than a specified threshold. This is particularly relevant to the scenario described, where testers verify that neurons activate only when the future value of the commodity exceeds the spot price by at least 0.75%.

Why is Threshold Coverage Correct?

Threshold-based activation: The test case in the question is explicitly verifying whether neurons activate only when a certain threshold (0.75%) is exceeded. This aligns perfectly with the definition of threshold coverage.

Common in Neural Network Testing: Threshold coverage is used to measure whether each neuron in a neural network reaches a specified activation value, ensuring that the neural network behaves as expected when exposed to different test inputs.

Precedent in Research: The DeepXplore framework used a threshold of 0.75% to identify incorrect behaviors in neural networks, making this coverage criterion well-documented in AI testing research.

Why Other Options are Incorrect?

(B) Neuron Coverage

Neuron coverage only checks whether a neuron activates (non-zero value) at some point during testing. It does not consider specific activation thresholds, making it less precise for this scenario.

(C) Sign-Change Coverage

This coverage measures whether each neuron exhibits both positive and negative activation values, which is not relevant to the given scenario (where activation only matters when exceeding a specific threshold).

(D) Value-Change Coverage

This coverage requires each neuron to produce two activation values that differ by a chosen threshold, but the question focuses on whether activation occurs beyond a fixed threshold, not changes in activation values.

Reference from ISTQB Certified Tester AI Testing Study Guide

Threshold coverage ensures that neurons exceed a given activation threshold 'Full threshold coverage requires that each neuron in the neural network achieves an activation value greater than a specified threshold. The researchers who created the DeepXplore framework suggested neuron coverage should be measured based on an activation value exceeding a threshold, changing based on the situation.'

Thus, option A is the correct answer, as threshold coverage ensures the neural network's activation is correctly evaluated based on the required condition (0.75%).