Using DATA Step Code to Design Scenarios in SAS Visual Investigator

One of the core functionalities of SAS Visual Investigator is its scenario authoring capability. In other words, the ability to create scenarios based on business rules to capture anomalies in the data. There are three types of Primary Scenarios in SAS Visual Investigator:

Record-level Scenario: Evaluates the data by individual transactions
Aggregation Scenario: Evaluates the data at a group level
DATA Step Scenario: Uses SAS DATA Step code and SAS macro language to create a bespoke scenario

During a recent webinar, I was asked about the usage of the DATA Step scenario. That question inspired this post, where I will dive deeper into the DATA Step scenario. I will cover its use cases and technical requirements, followed by providing two step-by-step examples.

Use cases

While record-level scenarios and aggregation scenarios cover a wide range of use cases, the organization may wish to create more complex scenarios to interrogate their data. That is where DATA Step scenarios step in, allowing for the creation of truly bespoke scenarios using a combination of SAS DATA Step code and SAS macro language.

DATA Step scenarios may be beneficial when:

The scenario logic requires the use of SAS functions not supported in a SAS Visual Investigator expression.
The scenario requires more than one data source as input.
The scenario requires multiple passes through the data source.
The scenario message requires conditional logic.

Code editor

DATA Step scenarios are created using the built-in code editor, accessible through the scenario creation screen. The code editor provides syntax highlighting, autocomplete, keyword lookup, code-folding, and code validation functionalities, as well as allowing for keyboard short-cuts.

Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.

Predefined macro variables

There are several macro variables and one macro call that are pre-defined for use in DATA Step scenario. The complete list can be found in the SAS Visual Investigator product documentation. For this post, I want to highlight a few key macro variables as I will be using them in the examples later:

INPUT_DATASET: This macro variable must be specified in the first DATA Step. It defines the data to be used as input for the scenario. This is the assigned data source table of the alert flow.
OUTPUT_DATASET: This macro variable must be used in the last DATA Step to define the output.
MESSAGE: This macro variable stores the scenario message, which could be quoted character strings, scenario parameters, macro variables, or data fields.
SCORE: This macro variable stores the scenario score.

Lastly, I will also be using the EVENT macro call, which enables the assignment of the standard scenario columns, allowing us to write event data as output from the DATA Step.

Code requirements

When it comes to using DATA Step code to create a scenario, there are a few requirements:

Multiple DATA Steps, written sequentially, can be used in the creation of a single scenario. However, the following requirements must be met:
- The first DATA Step must contain a set statement referencing the INPUT_DATASET macro variable.
  - Example: set &INPUT_DATASET;
- The last DATA Step must output to the OUTPUT_DATASET macro variable.
  - Example: data &OUTPUT_DATASET;
- The last DATA Step must invoke the EVENT macro call to enable the assignment of scenario columns.
  - Example: %EVENT;
Options cannot be assigned to the INPUT_DATASET and OUTPUT_DATASET tables.
When referencing a parameter created in the Parameters tab of the scenario builder, prefix the parameter with an ampersand.
- Example: &scenario_parameter.

If the DATA Step—other than the final DATA Step that writes to OUTPUT_DATASET—includes the CAPTURE_CHAR_VALUE, MESSAGE, or TRIG_TXNS macro variables, there must be a length statement specifying the number of bytes to be assigned for the storage of these variables.

Scenario – Unusual incident time

One common usage of SAS Visual Investigator is for the detection of insurance fraud. A behavior that may be indicative of fraud is when an incident occurs at an unusual time of the day. Using mock auto insurance data, I will create a DATA Step scenario to capture this behavior. I will design this scenario in two ways to highlight the capabilities of DATA Step scenarios.

Example 1: Using the varchar flag

The source table I will be working with is Auto Claim, which contains mock auto insurance claim data. This table contains claims information such as incident date, reported date, policy ID, and claimant information. It is unique on the claim level, meaning that there’s one record per individual claim.

The table contains a character column called Unusual Incident Time Flg—a flag variable indicating whether the incident occurred at an unusual time of the day (“yes”) or not (null). I will begin by using this column to build my DATA Step scenario.

Within the SAS Visual Investigator’s Admin UI (Manage Investigate and Search), go to the Alerts tab and the Flows subtab. The next steps assumes that we have already created a flow—a container for all the components necessary for alert generation—and we have loaded a data source. To create a new DATA Step scenario, use the New dropdown menu and select DATA Step scenario.

In the New DATA Step Scenario screen, confirm the data source, actionable entity, and scenario-fired entity. In this example, all three should point to our mock auto claim data and entity. Name the scenario and optionally add a description.

Next, navigate to the Parameters tab and create a parameter to represent the expected flag value when the incident occurred at an unusual time of the day. As seen previously, the Unusual Incident Time Flg column has the value of “Yes” when the incident occurred at an unusual time, therefore, I will set the default value of this parameter to “Yes”.

Next, navigate to the DATA Step Code tab. On this tab, we have the code editor where we can enter our SAS code. I have written two DATA Steps. In the first DATA Step (line 1 to 4), I created a temporary work table called Datastep1 using &INPUT_DATASET as an input. Recall that the INPUT_DATASET macro variable references the data source. In this case, that would be the Auto Claim table. And on line 3, I have a where statement that creates a subset by comparing the Unusual Incident Time Flg source column to the scenario parameter unusual_time_parameter I created on the Parameters tab. In other words, only output when the Unusual Incident Time Flg column has a value of “Yes”. Note that to use the scenario parameter unusual_time_parameter, I had to prefix it with an ampersand (&) similar to when a macro variable is called.

In the second, and final, DATA Step, I output to &OUTPUT_DATASET, as per the code requirements I mentioned previously. In this step, I also write a scenario message “Claim occurred at unusual time of day” and assign a score of 99 to the scenario. Lastly, notice the invocation of the EVENT macro call on line 10 to enable the assignment of scenario columns.

Once I’m satisfied with the code, I need to click the Validate the code button to validate the code before the scenario can be saved. Note that this code validation doesn’t guarantee the integrity of the code, it merely performs basic checks. One such check is to ensure that all referenced scenario parameters have been defined on the Parameters tab. Even if the code passes validation at this stage, it may still run into errors upon execution. This is where testing comes in.

Use the Test the scenario button to test the scenario. If the DATA Step code runs successfully without errors, the test results will be displayed on the Test Results tab. As seen below, out of the 29 available input records, 12 records met the logic of the scenario. The matched records are listed, together with their score and message as defined in the final DATA Step.

There is a limitation to this design: we do not have control over the definition of unusual incident time. In the next example, I will capture this behavior using a different method.

Using the datetime field

Now, I will approach this scenario in another way. In the source table, Auto Claim, there is a datetime column called Incident Dttm. As seen below, this column has the exact time when the incident occurred. Using this column would allow for greater degree of flexibility, as we can define the exact time window that is considered “unusual”.

As demonstrated in the previous example, navigate to the Parameter tab to create the following scenario parameters:

Early_time: The parameter for incident hour (in 24h time format), below which we consider the incident time to be unusual. The default value is defined to be 04:00, meaning that the hours from midnight (00:00) to 4 a.m. (04:00) is what we consider to be unusual.
Late_time: The parameter for incident hour (in 24h time format), above which we consider the incident time to be unusual. The default value is defined to be 23:00, meaning that the hours from 11 p.m. (23:00) and midnight (00:00) is what we consider to be unusual.

Now that the parameters representing the lower and upper boundaries of the unusual time window have been defined, we can use the previous SAS code, with minor tweaks to complete the scenario. Please see below for the updated code. The differences can be spotted on line 3, 4 and 9. Note that both parameters are defined as strings on the Parameters tab, and I use the input function to convert them to a time value in the DATA step.

Following the same procedure to validate the code and test the scenario, we see that there are now 2 more hits displayed in the Test Results tab. This is because the time boundaries I have defined is different than the definition for the Unusual Incident Time Flg column.

This scenario is ready to be saved and the alert flow can then be versioned, published, and executed to generate alerts.

Hopefully, this post has been helpful in giving you more information on how to use DATA Step code to design and build scenarios in SAS Visual Investigator.

Learn more

Aggregation Scenarios and Entity Relationships in SAS Visual Investigator

From Data to Insights: Leveraging SAS Visual Investigator Scenarios For Anomaly Detection

Find more articles from SAS Global Enablement and Learning here.