AI in Rapise¶

Overview¶

We have integrated AI capabilities directly into Rapise, allowing users to harness the power of AI without needing to switch between applications. Everything can be managed within Rapise itself. Moreover, the AI in Rapise is well-versed with Rapise's Global Object API, the currently open testing framework, its modules, and object repositories. This knowledge enables the generation of precise and specialized responses. In its initial release, Rapise’s AI features include RVL and JavaScript code generation, as well as data generation functionalities. Rapise 8.2 supports OpenAI and Azure OpenAI models, including multimodal ones that can accept image input. The AI workflow in Rapise also supports incremental improvements in code generation quality by allowing users to save positive and negative examples for use in subsequent prompts.

A key feature of Rapise 8.2 is the AI Command, which translates human-readable test step descriptions into executable code using AI. Once the code is generated for an AI Command, it can be executed independently of AI. If the AI Command text is modified, the code is regenerated accordingly. When generating code for an AI Command, Rapise considers available Global Objects, as well as Modules/Page Objects and objects from the Object Repository associated with the test case containing the AI Command (including objects linked with Repository command).

When using AI Commands, the recommended workflow for test creation is as follows: first, define Page Objects with high-level actions (e.g., Login, Logout, OpenModule, AddRecord, UpdateRecord, DeleteRecord). Then, learn and add test case-specific objects to the Object Repository, and finally, write your test case using AI Commands. The AI will translate these commands into executable JavaScript, which you can review and edit. Once all commands are translated into code, the test case is ready for execution.

Rapise provides two primary interfaces for interacting with AI:

The AI Dashboard allows you to configure AI options, chat with the AI, and view your chat history.
The AI Panel is linked to the active RVL document and is used to translate AI Commands into executable JavaScript code.

AI Dashboard¶

How to Open¶

To open the AI Dashboard, go to the main menu and select View > AI Dashboard.

The dashboard comprises three pages:

Screenshot

Dashboard Tabs

Transcript

AI Dashboard tabs: Settings, Chats, Snapshots.

Settings: Configure your AI API key, select the default model, and set framework-specific options here.
Chats: This page displays the history of AI chats. You can open any chat and continue interacting with the AI.
Snapshots: Starred chats will appear on this page.

Settings¶

In its initial release, Rapise supports both OpenAI and Azure OpenAI models. To use OpenAI models, you only need an OpenAI Api Key. For Azure OpenAI, you also need to specify the endpoint URL.

Screenshot

Dashboard Settings

Transcript

Options

Scope global
- global (selected)
- local [create]

Please select an agent workflow for AI Command:
- AI Command (dropdown option)

Enter parameters for the default model:
- gpt-4o-mini

(Obscured text for input)

Base URL:
- (Empty field)

Test Model (Button)

You can create options that are local to the current testing framework. Simply click the local[create] button. If local options exist, Rapise will prioritize them over global ones. To delete local options, click ... > Delete Local.

Chats¶

On the Chats page, you can select a previous chat and continue interacting with the AI. You can also start a new chat. When creating a new chat, you’ll be prompted to select a workflow. Supported workflows include:

AI Chat¶

This is a general-purpose chat with an AI model. You can ask questions and receive answers. You can also enhance prompts by adding screenshots and files.

Screenshot

AI Chat

Transcript

USER: What are main features of a good UI testing framework?

AI: A good UI testing framework should have several key features to ensure it is effective, efficient, and user-friendly. Here are the main features to consider:

Cross-Browser Compatibility: The framework should support testing across multiple browsers (e.g., Chrome, Firefox, Safari, Edge) to ensure consistent behavior of the application.
Cross-Platform Support: It should be able to test applications on various platforms (e.g., web, mobile, desktop) to accommodate different user environments.
Ease of Use: The framework should have a user-friendly interface and clear documentation, making it easy for testers and developers to write and maintain tests.

AI Command¶

This chat type is typically generated by Rapise when processing an AI Command. The result of prompt processing is JavaScript code. In this chat, you can edit the generated code, copy it to a JavaScript file, or convert it to RVL and apply it to the active RVL sheet.

Screenshot

AI Command

Transcript

USER: Open browser at https://libraryinformationsystem.org

AI:

Navigator.Open("https://libraryinformationsystem.org")

Generate Data List¶

Generates a list of data values that can be directly inserted into an RVL sheet from the chat.

Prompt AI to generate a list of values. E.g.: Generate 10 fake company names.

Screenshot

Generate List

Transcript

USER: Generate 10 fake company names

AI:

Companies
InnovaTech  
EcoSphere  
NexGen Solutions  
Quantum Dynamics  
AeroFusion  
CrestWave  
PixelForge  
VitaNova  
Stratosphere Innovations  
BluePeak Industries

Open the RVL file and sheet where you want to insert the result.
Return to the AI Dashboard and click Apply . You can also edit values before inserting them into the RVL.

The list will be inserted into the RVL.

Screenshot

List Result

Transcribe

Flow	Type	Object
Map	Rows	Companies
	Companies
	Innovatech
	EcoSphere
	NexGen Solutions
	Quantum Dynamics
	AeroFusion
	CrestWave
	PixelForge
	VitaNova
	Stratosphere Innovations
	BluePeak Industries
End

Generate Data Table¶

Generates a table with columns and rows. The result can be directly inserted into an RVL sheet as a Map. The workflow is similar to generating a Data List.

Prompt AI to generate a table of values. E.g.: Generate a table with 10 records and columns: Zip, City, Street Address.

Screenshot

Generate Table

Transcribe

USER: Generate a table with 10 records and columns: Zip, City, Street Address

AI:

{  
"data": [  
    {  
    "Zip": "10001",  
    "City": "New York",  
    "Street Address": "123 Main St"  
    },  
    {  
    "Zip": "20001",  
    "City": "Washington",  
    "Street Address": "456 Elm St"  
    },  
    {  
    "Zip": "30301",  
    "City": "Atlanta",  
    "Street Address": "789 Pine St"  
    },  
    {  
    "Zip": "60601",  
    "City": "Chicago",  
    "Street Address": "101 Maple St"  
    },  
    {  
    "Zip": "70112",  
    "City": "New Orleans",  
    "Street Address": "202 Oak St"  
    },  
    {  
    "Zip": "94105",  
    "City": "San Francisco",  
    "Street Address": "303 Cedar St"  
    },  
    {  
    "Zip": "33101",  
    "City": "Miami",  
    "Street Address": "404 Birch St"  
    },  
    {  
    "Zip": "80202",  
    "City": "Denver",  
    "Street Address": "505 Walnut St"  
    },  
    {  
    "Zip": "98101",  
    "City": "Seattle",  
    "Street Address": "606 Spruce St"  
    },  
    {  
    "Zip": "85001",  
    "City": "Phoenix",  
    "Street Address": "707 Palm St"  
    }  
]  
}

Apply the AI response to get the result.

Screenshot

Table Result

Transcribe

Flow	Type	Object	Action
Map	Rows	TBD
	Zip	City	Street Address
	10001	New York	123 Main St
	20001	Washington	456 Elm St
	30301	Atlanta	789 Pine St
	60601	Chicago	101 Maple St
	70112	New Orleans	202 Oak St
	94105	San Francisco	303 Cedar St
	33101	Miami	404 Birch St
	80202	Denver	505 Walnut St
	98101	Seattle	606 Spruce St
	85001	Phoenix	707 Palm St
End

Generate More Rows¶

Use this to generate additional data for an existing table.

Select a row in the current RVL table.

Screenshot

Select Row

Transcribe

Flow	Type	Object	Action
Map	Rows	TBD
	Zip	City	Street Address
	10001	New York	123 Main St
	20001	Washington	456 Elm St
	30301	Atlanta	789 Pine St
	60601	Chicago	101 Maple St
	70112	New Orleans	202 Oak St
	94105	San Francisco	303 Cedar St
	33101	Miami	404 Birch St
	80202	Denver	505 Walnut St
	98101	Seattle	606 Spruce St
	85001	Phoenix	707 Palm St
End

In the AI Dashboard, create a "Generate More Rows" chat and send a prompt to the AI. For example: Generate 10 more records with cities from countries other than USA.

Screenshot

Generate Rows

Transcribe

USER: Generate 10 more records with cities from countries other than USA

AI:

{  
"data": [  
    {  
    "Zip": "10115",  
    "City": "Berlin",  
    "Street Address": "Friedrichstraße 123"  
    },  
    {  
    "Zip": "M5A 1A1",  
    "City": "Toronto",  
    "Street Address": "Queen St E 456"  
    },  
    {  
    "Zip": "SW1A 1AA",  
    "City": "London",  
    "Street Address": "Buckingham Palace Road 1"  
    },  
    {  
    "Zip": "1000",  
    "City": "Brussels",  
    "Street Address": "Rue de la Loi 10"  
    },  
    {  
    "Zip": "1010",  
    "City": "Vienna",  
    "Street Address": "Karntnerring 5"  
    },  
    {  
    "Zip": "1100",  
    "City": "Athens",  
    "Street Address": "Syntagma Square 2"  
    },  
    {  
    "Zip": "75001",  
    "City": "Paris",  
    "Street Address": "Rue de Rivoli 20"  
    },  
    {  
    "Zip": "2000",  
    "City": "Copenhagen",  
    "Street Address": "Købmagergade 30"  
    },  
    {  
    "Zip": "1010",  
    "City": "Amsterdam",  
    "Street Address": "Damstraat 1"  
    },  
    {  
    "Zip": "1010",  
    "City": "Madrid",  
    "Street Address": "Gran Vía 45"  
    }  
]  
}

Apply the generated values to see the result.

Screenshot

Rows Result

Transcribe

Flow	Type	Object	Action
Map	Rows	TBD
	Zip	City	Street Address
	10001	New York	123 Main St
	20001	Washington	456 Elm St
	30301	Atlanta	789 Pine St
	60601	Chicago	101 Maple St
	70112	New Orleans	202 Oak St
	94105	San Francisco	303 Cedar St
	33101	Miami	404 Birch St
	80202	Denver	505 Walnut St
	98101	Seattle	606 Spruce St
	85001	Phoenix	707 Palm St
	10115	Berlin	Friedrichstraße 123
	M5A 1A1	Toronto	Queen St E 456
	SW1A 1AA	London	Buckingham Palace R
	1000	Brussels	Rue de la Loi 10
	1010	Vienna	Karntnerring 5
	1100	Athens	Syntagma Square 2
	75001	Paris	Rue de Rivoli 20
	2000	Copenhagen	Købmagergade 30
	1010	Amsterdam	Damstraat 1
	1010	Madrid	Gran Vía 45
End

Test Scenario from Text¶

Generates a sequence of AI Commands based on a text prompt.

Provide a high-level description of the scenario you want to implement. For example: Generate an end-to-end scenario to create and delete a book in a web based Library Information System. A step-by-step scenario will be generated.

Screenshot

Scenario from Text

Transcribe

USER: Generate an end-to-end scenario to create and delete a book in a web based Library Information System

AI:

1. Open the web browser and navigate to the Library Information System URL {library_url}.  
2. Click on the "Login" button.  
3. Enter the username {username} in the username field.  
4. Enter the password {password} in the password field.  
5. Click on the "Submit" button to log in.  
6. Once logged in, navigate to the "Books" section of the system.  
7. Click on the "Add New Book" button.  
8. Fill in the book title {book_title} in the title field.  
9. Fill in the author name {author_name} in the author field.  
10. Fill in the ISBN number {isbn_number} in the ISBN field.  
11. Select the genre {genre} from the dropdown menu.  
12. Click on the "Save" button to create the book entry.  
13. Verify that the book {book_title} appears in the list of books.  
14. Locate the newly created book {book_title} in the list.  
15. Click on the "Delete" button next to the book {book_title}.  
16. Confirm the deletion in the pop-up dialog.  
17. Verify that the book {book_title} no longer appears in the list of books.  
18. Log out of the Library Information System.

Select an RVL sheet for insertion, then return to the AI Dashboard and click the Apply icon. A sequence of AI commands will be inserted into the RVL sheet.

Screenshot

Scenario from Text Result

Transcribe

Type	Object	ParamName	ParamType	ParamValue
AI	Open the web browser and navigate to the Library Information System URL
Param		library_url	string	library_url
AI	Click on the "Login" button.
AI	Enter the username {username} in the username field.
Param		username	string	username
AI	Enter the password {password} in the password field.
Param		password	string	password
AI	Click on the "Submit" button to log in.
AI	Once logged in, navigate to the "Books" section of the system.
AI	Click on the "Add New Book" button.
AI	Fill in the book title {book_title} in the title field.
Param		book_title	string	book_title
AI	Fill in the author name {author_name} in the author field.
Param		author_name	string	author_name
AI	Fill in the ISBN number {isbn_number} in the ISBN field.
Param		isbn_number	string	isbn_number
AI	Select the genre {genre} from the dropdown menu.
Param		genre	string	genre
AI	Click on the "Save" button to create the book entry.
AI	Verify that the book {book_title} appears in the list of books.
Param		book_title	string	book_title
AI	Locate the newly created book {book_title} in the list.
Param		book_title	string	book_title
AI	Click on the "Delete" button next to the book {book_title}.
Param		book_title	string	book_title
AI	Confirm the deletion in the pop-up dialog.
AI	Verify that the book {book_title} no longer appears in the list of books.
Param		book_title	string	book_title
AI	Log out of the Library Information System.

Test Scenario from Image¶

Generates a sequence of AI Commands from an image and a text prompt.

In the AUT, navigate to the screen/page you want to use.

Screenshot

Transcribe

Library Information System
Logged in as Librarian, Logout

Create New Book

Name
Enter book name (text input)

Author
A.A. Milne (dropdown selection)

Genre
Adventure (dropdown selection)

Out of Print
[ ] (checkbox)

10/07/2020 (date picker)

[Save Book] (button)

In the AI Dashboard, create a "Test Scenario from Image" chat and provide the image along with a text prompt. For example: Fill the form.

Screenshot

Scenario from Image - Prompt

Transcribe

USER: Fill the form (screenshot of the form is attached)

AI:

1. Enter {book_name} in the "Name" field.  
2. Select {author_name} from the "Author" dropdown.  
3. Select {genre} from the "Genre" dropdown.  
4. Check the "Out of Print" checkbox if {out_of_print} is true.  
5. Enter {date} in the date field.  
6. Click the "Save Book" button.

Select an RVL sheet for insertion, then return to the AI Dashboard and click the Apply icon. A sequence of AI commands will be inserted into the RVL sheet.

Screenshot

Scenario from Image - Result

Transcribe

Type	Object	ParamName	ParamType	ParamValue
AI	Enter {book_name} in the "Name" field.
Param		book_name	string	book_name
AI	Select {author_name} from the "Author" dropdown.
Param		author_name	string	author_name
AI	Select {genre} from the "Genre" dropdown.
Param		genre	string	genre
AI	Check the "Out of Print" checkbox if {out_of_print} is true.
Param		out_of_print	string	out_of_print
AI	Enter {date} in the date field.
Param		date	string	date
AI	Click the "Save Book" button.

Snapshots¶

On this page, find starred chats. To add a chat to Snapshots, locate it in history, and hover your mouse pointer over its top-right corner to reveal the ... icon:

Session Menu

Click the icon to expand the popup menu and select Snapshot.

Make Snapshot

If you no longer need a chat on the Snapshots page, you can remove it.

Delete Snapshot

AI Panel¶

AI panel is used to work with AI Commands.

How to Open¶

To open the panel, double-click the line number of an AI Command or click the Show AI Panel button that appears when you hover your mouse over the corresponding cell.

AI Panel

It’s also possible to work with multiple AI Commands simultaneously. Simply select a range of commands and click Show AI Panel.

AI Panel Multiple Commands

There is also a toolbar button that shows all AI Commands on the active RVL sheet.

AI Panel Show All Commands

You can perform various actions with each command.

Command Menu

- generate or regenerate code for the command.
- run the code for this command.
- copy the command ID to the clipboard.
- open the corresponding chat session in the AI Dashboard.
- like the result of code generation. This will be used as a positive example in subsequent AI Command prompts.
- dislike the result of code generation. This will be used as a negative example in subsequent AI Command prompts.

The panel also includes a top-level menu.

Top Level Menu More Menu Items

- generate or regenerate code for all the commands opened in the AI Panel.
- run all the commands opened in the AI Panel sequentially.
- undock the panel.
- dock the panel to the right.
- dock the panel to the bottom.
- hide the panel.
- open the Workflow selection page.

Framework Folders¶

When you work with AI in Rapise, it creates a set of folders in the root framework directory.

Root Framework Folder
- AI - root folder for AI-related files.
  - commands - cache for AI Commands, containing generated JavaScript code. This folder must be stored along with the framework; do not delete it.
  - messages - archive of AI chat messages.
  - sessions - high-level information about AI chat sessions (name, ID, timestamp).
  - options.json - local options that override the global ones stored in C:\ProgramData\Inflectra\Rapise\AI\options.json.
  - screenshot.base64 - last screenshot taken in Chats.
  - AIExamples.txt - the list of liked/disliked examples of generated code.
  - AIPrompt.txt - text to add to each AI prompt for AI Command processing.

Note

When you put a framework under source control it is important to add

options.json
AIExamples.txt
AIPrompt.txt
commands folder with all the files in it.

The following folders are optional (this is just an archive of AI chats)

messages
sessions

AI in Rapise¶

Overview¶

AI Dashboard¶

How to Open¶

Settings¶

Chats¶

AI Chat¶

AI Command¶

Generate Data List¶

Generate Data Table¶

Generate More Rows¶

Test Scenario from Text¶

Test Scenario from Image¶

Snapshots¶

AI Panel¶

How to Open¶

Command Menu¶

Panel Menu¶

Framework Folders¶