AI/ML FAQ

Smart Query is an AI App that allows the user to ask natural language questions (see Note below) on a dataset that is analyzed, and in return get responses from OpenAI based on interpretation of the questions asked. The AI Smart Query App involves interaction with OpenAI and ChatGPT.

Note: Natural Language Processing (NLP) is what allows ChatGPT (and other AI tools) to understand what you are saying, i.e. question you are asking, and respond like a human would – using real, natural language.

Please see our AI/ML User Guide for more information and guidance.

To access Smart Query, go to the Apps menu, and from the AI-ML menu, select AI01 Smart Query. Please see our AI/ML User Guide for more information and guidance.

While Arbutus provides the Smart Query tool to enable users to interact with their data using natural language, it is important to note that the interpretation and response is handled entirely by the AI engine. 

Our integrated AI capabilities allow you to ask questions in natural language and receive helpful insights about your data. The clarity, detail, and structure of your questions will influence the quality of the responses, and because AI interprets queries in real time, the same question may yield different results. Taking time to learn the basics of prompt engineering—how to frame questions clearly and provide relevant context  will help you get more consistent and useful outcomes.

Smart Query is used for asking questions to query the dataset passed to OpenAI and to then get responses back based on ChatGPT's interpretation of those questions.

For example, here are some examples of questions that you can ask in Smart Query:
•  Which insurance provider has the highest Charge_Amount, and what is it?
•  I have a date of birth field in my data. I would like to know how many patients were born between 1937 and 1941.
•  Show me duplicate insurance provider names.
•  I would like to know the total charge amount (field is Charge_Amount) each for the following Insurance Providers: Aetna, Blue Cross. I would like two separate totals.

Please see our AI/ML User Guide for more information and guidance.

When you run the AI Smart Query App, one of the items you are prompted to select is an Arbutus table. If this happens to be a table that is related to another table, or multiple tables, then only those fields that are included in the current view, including any fields from the related tables, are analyzed.

Therefore, while you can use Smart Query on related tables, only those fields in the current view are analyzed.

Smart Query is an AI App that allows the user to ask natural language questions (see Note below) on a dataset that is analyzed, and in return get responses from OpenAI based on interpretation of the questions asked. The AI Smart Query App involves interaction with OpenAI and ChatGPT.

Note: Natural Language Processing (NLP) is what allows ChatGPT (and other AI tools) to understand what you are saying, i.e. question you are asking, and respond like a human would – using real, natural language.

Please see our AI/ML User Guide for more information and guidance.

While the AI Smart Query App is launched from an Arbutus procedure, the results are ultimately generated through OpenAI's ChatGPT (via a Python procedure) and then returned to Arbutus for display. Therefore, the results and output is not generated by Arbutus.

However, it is important to note that the interpretation and response is handled entirely by the AI engine. Our integrated AI capabilities allow you to ask questions in natural language and receive helpful insights about your data. The clarity, detail, and structure of your questions will influence the quality of the responses, and because AI interprets queries in real time, the same question may yield different results.

As the results are generated by OpenAI, it is strongly recommended that users exercise due diligence by reviewing and validating the output within Analyzer to ensure its accuracy and reliability.

No. Smart Query is used for asking questions to query the dataset passed to OpenAI and to then get responses back based on ChatGPT's interpretation of those questions.

A question entered in Smart Query is sent to OpenAI and processed by ChatGPT. If ChatGPT cannot interpret your question, a generic or error message will be returned to Arbutus for display.
Please review the returned message and try rephrasing your question by providing more details such as actual field names and being more specific about your request.

When you run the AI Smart Query App, all processing takes place in the background, not visible to you. Once processing is done, the results are passed back to Arbutus for display. Very limited actions are recorded in the Command Log, and do not really provide much assistance to new users in terms of learning Arbutus syntax. This is mainly because most of the processing is done outside of Arbutus via a Python procedure, resulting in an interaction with OpenAI and ChatGPT.

Please note that our AI functionality requires internet connectivity and an active ChatGPT account. Otherwise, it is disabled and non-functional by default, to ensures that no data is sent off premise unless explicitly decided upon by the customer’s organization and users. It will only be enabled once configured with your organization’s OpenAI account details.

Please take note that the AI Apps will use any information you share via the input prompts or any data from a table that you specify/select, and this information/data is then passed on to OpenAI (via ChatGPT) for subsequent analysis.

Therefore, it is imperative that you get confirmation on your organization’s policy allowing the use of ChatGPT.

To run Data Categorization, go to the Apps menu, and from the AI-ML menu, select AI02 Data Categorization

Please see our AI/ML User Guide for more information and guidance.

Data categorization is the process of organizing and labelling data into predefined categories based on its content, characteristics, or patterns. Data is organized and labelled using a systematic process where each data point is classified into a predefined category based on its characteristics.

Please see our AI/ML User Guide for more information and guidance.

The categories created are based on your answers to the input prompts, i.e. questions, when you run the Data Categorization App. In addition to Data Categorization, there also is interaction with ChatGPT and consolidation of responses received from ChatGPT. Therefore, by tweaking your questions, it could be possible to control how the categories are created.

The responses received from ChatGPT is essentially a filtered list of records from the current table (that you had selected) and based on the description field from that same table. 
For example, if the task you entered/specified was, “Analyze the comments in the selected field and create an output containing items categorized as 'Food issues' “, then you could expect to see a filtered list of all records from the current table (that you had selected) indicating ‘Food issues’ based on the description field that you had also selected. A new output table is not created.

The Arbutus AI App is not used for querying your data, such as a table in your Arbutus project. It is meant to function just like how ChatGPT or Co-Pilot functions - ask a question and return with the responses.

To run Arbutus Assistant, go to the Apps menu, and from the AI-ML menu, select AI03 Arbutus Assistant. Please see our AI/ML User Guide for more information and guidance.

No, Arbutus Assistant is not able to automate analysis. However, based on the questions asked to ChatGPT when running this App, it may be able to provide some form of guidance in terms of how to use Arbutus, e.g., how can I use the Arbutus Duplicates command?

The main difference between Arbutus Assistant and Smart Query is that the latter uses a dataset, e.g., an Arbutus table containing data, which is selected by the user, in its interaction with ChatGPT. However, both these AI Apps prompt for questions, which are also entered by the user, for ChatGPT to analyze and provide a response. Please see our AI/ML User Guide for more information and guidance.

Clusters and clustering is a process of identifying and grouping data points that are similar to each other. It is a common technique in data analysis, machine learning, and market segmentation. 
Clustering helps in recognizing patterns, organizing data, and making predictions. For example, in real estate, clustering could be used to categorize buyers into groups based on budget, location preference, or property type, allowing for better-targeted marketing strategies. 
Please see our AI/ML User Guide for more information and guidance.

To run Clusters, go to the Apps menu, and from the AI-ML menu, select ML01 - Clusters. Please see our AI/ML User Guide for more information and guidance.

When you run the Clusters ML App, you are prompted for the number of clusters. Choosing/Specifying the right number of clusters could involve some experimentation. The right number of clusters helps ensure that the clustering results are both accurate and useful for your specific application. Choosing the right number is crucial because too few or too many may lead to poor segmentation. Therefore, it is recommended that you first get a good understanding of your data.

Please see our AI/ML User Guide for more information and guidance.

No, you will not be able to see what defines each cluster. As a result of running the Clusters ML App, you will notice a new field/column called kclusters in the resulting output table. The range of values in this table, e.g., 0,1,2, would depend on the number of clusters that was manually entered and prompted at the time of running the Clusters ML App. However, you will not see any descriptions as such to describe what these values mean or represent. At a minimum, you should understand that each of those values indicate a grouping of records with similar characteristics based on the fields clustered on. You can do further analysis on that new field (kclusters), e.g. run the Summarize or Classify command, to verify and understand why the records are grouped as they are.

As a result of running the Clusters ML App, you will notice a new field/column called kclusters in the resulting output table. The range of values in this table, e.g., 0,1,2, would depend on the number of clusters that was manually entered and prompted at the time of running the Clusters ML App. However, you will not see any descriptions as such to describe what these values mean or represent. At a minimum, you should understand that each of those values indicate a grouping of records with similar characteristics based on the fields clustered on. You can do further analysis on that new field (kclusters), e.g. run the Summarize or Classify command, to verify and understand why the records are grouped as they are.

When you run the Clusters ML App, you are prompted for the number of clusters. Choosing/Specifying the right number of clusters could involve some experimentation. The right number of clusters helps ensure that the clustering results are both accurate and useful for your specific application. Choosing the right number is crucial because too few or too many may lead to poor segmentation. Therefore, it is recommended that you first get a good understanding of your data. Please see our AI/ML User Guide for more information and guidance.

As a result of running the Clusters ML App, you will notice a new field/column called kclusters in the resulting output table. The range of values in this table, e.g., 0,1,2, would depend on the number of clusters that was manually entered and prompted at the time of running the Clusters ML App. However, you will not see any descriptions as such to describe what these values mean or represent. At a minimum, you should understand that each of those values indicate a grouping of records with similar characteristics based on the fields clustered on. You can do further analysis on that new field (kclusters), e.g. run the Summarize or Classify command, to verify and understand why the records are grouped as they are.

When you run the Clusters ML App, you are prompted for the number of clusters. You should also be aware that choosing/specifying the right number of clusters could involve some experimentation. The right number of clusters helps ensure that the clustering results are both accurate and useful for your specific application. Choosing the right number is crucial because too few or too many may lead to poor segmentation. Therefore, it is recommended that you first get a good understanding of your data. Please see our AI/ML User Guide for more information and guidance.

When you run the Clusters ML App, you are prompted for the following three fields: Relationship Key Field, Numeric Field 1, Numeric Field 2, Numeric Field 3, and Numeric Field 4. The Relationship Key Field is the key field on which the records are grouped by in your dataset, e.g., Vendor ID. The numeric fields (1 to 4) are the fields on which you would like to run Clusters. In other words, the data points that you are looking to group into their respective clusters. At a minimum, you have to specify at least two numeric fields. Selecting just one numeric field would result in a pop-up message displaying an error and termination of procedure.

Please see our AI/ML User Guide for more information and guidance.

When you run the Clusters ML App, you are prompted for the following three fields: Relationship Key Field, Numeric Field 1, Numeric Field 2, Numeric Field 3, and Numeric Field 4. The Relationship Key Field is the key field on which the records are grouped by in your dataset, e.g., Vendor ID. The numeric fields (1 to 4) are the fields on which you would like to run Clusters. In other words, the data points that you are looking to group into their respective clusters. At a minimum, you have to specify at least two numeric fields. Selecting just one numeric field would result in a pop-up message displaying an error and termination of procedure. 

Please see our AI/ML User Guide for more information and guidance.

When you run the Clusters ML App, one of the items you are prompted to enter is the Result Table Name. The results of your clusters testing will be output to this table.

Outliers, in the context of information evaluation, are records with numeric amounts that deviate or differ significantly from the numeric amounts in the records that they are grouped with. These anomalies can show up as surprisingly or unexpectedly high or low values, thus disrupting the distribution of data. Please see our AI/ML User Guide for more information and guidance.

To run Outliers, go to the Apps menu, and from the AI-ML menu, select ML02 - Outliers. Please see our AI/ML User Guide for more information and guidance.

The Arbutus ML Outliers functionality involves executing an industry-standard Python procedure (iqr.py) designed for identifying outliers using Interquartile Range methodology (IQR), thus calculating the medians and the lower and upper boundaries for identifying outliers in a dataset - any value (from the numeric field being tested for outliers) that falls in between the lower and upper boundaries is considered an outlier. Please see our AI/ML User Guide for more information.

When you run the Clusters ML App, one of the items you are prompted to enter is the Result Table Name. The results of your outliers testing will be output to this table.

When you run the Clusters ML App, one of the items you are prompted to enter is the Result Table Name. The results of your outliers testing will be output to this table.

No, you are not able to customize the sensitivity of Outliers detection in the current release of Arbutus.