XTC v116

Load Testing

New Feature: AI-Assisted Analysis of Load Test Reports

In addition to standard load test, comparison, and trend reports, XTC now supports AI-assisted analysis reports. These reports leverage Large Language Models (LLMs) to automatically analyze load test results, highlight key performance metrics, summarize test findings, and help identify potential performance bottlenecks.

How It Works

• AI Data File: The XLT load test report generator now creates an additional ai-data.md file containing key load test metrics in a structured format optimized for LLM processing.
• Custom User Prompts: You specify an inquiry prompt for the LLM. The more specific and detailed your prompt, the more tailored and useful the analysis results will be.
• System Guardrails: XTC applies an internal system prompt to guide and guardrail the LLM for consistent and accurate analysis.
• LLM Processing: XTC passes the system prompt, your user prompt, and the ai-data.md file to the LLM (currently supporting Google Gemini).
• Markdown & HTML Output: The LLM analyzes the data and generates a detailed report in Markdown format, which XTC then wraps into a standard, shareable HTML report interface.

Managing User Prompts

User prompts form the foundation of AI-assisted analysis reports. To easily reuse tailored prompts across multiple load tests and specific inquiries (such as error investigations, SLA evaluations, or regression checks), XTC provides a dedicated Prompt Library:

• Easy Reusability: Store and organize prompts for specific analytical inquiries so they can be reused effortlessly across different load test runs and reports.
• Organization Library: Manages global, organization-wide prompts accessible across all projects.
• Project Library: Manages all project-specific prompts while also listing organization-level prompts (inherited in read-only mode) for seamless reuse across projects.
• Prompt Operations: Supports creating new prompts as well as editing, duplicating, and deleting existing ones.
• Retention & Recovery: Deleted prompts are retained for 30 days and can be restored if needed.

Creating AI Analysis Reports

Creating an AI analysis report in XTC is simple:

1. Open the Results tab of the load test you want to analyze.
2. In the Reports table, open the context menu for the target report and select AI Analysis.
3. In the dialog, enter a report name and optional description, select the prompt to use for the analysis, and submit.
4. Once XTC finishes generating the report, click on your new report entry to open and review it.

Report Lifecycle and Sharing

AI analysis reports fully support all standard report operations:

• Pinning: Pin important reports to prevent automatic cleanup.
• Sharing: Share reports with colleagues and stakeholders who do not have an XTC account.
• Downloads: Download reports as standard archive files for offline access.
• Lifecycle Management: Like other reports, AI analysis report HTML files are automatically cleaned up after 180 days (with on-demand restoration available), and deleted reports can be recovered within 30 days.

Steady-State Reports

When evaluating targeted versus achieved load performance, a steady-state report is often required. This is a report that excludes the initial ramp-up period to focus solely on stable test conditions. Because creating steady-state reports is a common task, XTC now provides dedicated shortcuts to generate them quickly.

To create a steady-state report from the latest result set, open the three-dot menu above the Reports table of your load test and click New Steady-State Report. To create one from an earlier result set, open the context menu for that result set in the Results table and select Create Steady-State Report.

Available XLT Execution Environments

• XLT 9.x → 9.2.2 (recommended)
• XLT 10.x → 10.0.0-beta-4

Monitoring

Real-Time Alert Notifications

Previously, while the initial monitoring alert notification was sent immediately, any subsequent follow-up notifications were collected and dispatched in batches at 5-minute intervals. XTC now sends out all alert notifications immediately, providing real-time awareness for every alert.

Deprecation Warning: SMS Notification Channel

Notification List Migration Guide

Refer to the XTC v115 Release Notes for details on how users can set up and configure push notifications for their account.

Follow these steps to migrate your notification lists:

1. Verify user setup: Ensure all target users have enabled push notifications in their XTC user account and verified that notifications are working correctly.
2. Update project notification lists: Review all notification lists in your monitoring projects:
   • For User entries: Disable any active SMS channel and enable push notifications instead.
   • For Custom entries: Disable any active SMS channel. Do not configure a custom push notification topic. Instead, add the relevant XTC users directly to the notification list and enable push notifications for them.