KB 061 – Custom Extractor

Extractor class definition

Please follow steps below to implement custom extractor.

1. Create a subclass inherited from /BNWVS/CL_EXT_SUPER.
2. Redefine the method PERFORM_METRIC_CALCULATION.
3. Implement custom data extraction logic in the PERFORM_METRIC_CALCULATION method.
   The aim is to select metric data into the internal table and store it into the metric cluster for further distribution to Splunk.
4. Call STORE_METRIC method and pass extracted data to store data it into the metric cluster.
   The call should be made within the redefined PERFORM_METRIC_CALCULATION method.

Extractor class attributes

• IT_NEXTRUN – UTC timestamp, which contain the next run timestamp (normally should not be used);
• IT_LASTRUN – UTC timestamp, which contain the last run timestamp (usually used for data extraction as start of the period together with IT_RUNDATETIME).

Extractor methods

• GET_GROUP_DEF – should be used to retrieve metric group name defined in the Group Definition config (Control Panel->Administrator->Group Definition);
• PROCESS_APP_SERVER_EXCEPTION – used to store a text of raised exception into the application log. All messages could be viewed from Control Panel;
• GET_TESTONLYFLAG – used to determine whether it is a test extractor run. In this case some actions may be skipped;
• GET_TEST_TYPE – used to retrieve the test type, which could be: Display XML or send test data.
• STORE_METRIC – should be called within PERFORM_METRIC_CALCULATION method to store metric data into the cluster to be sent to Splunk;
• GET_PARAMETERS – get extractor parameters defined in the Metric Config (Control Panel->Administrator->Setup metric->Metric configuration);
• SET_PARAMETERS – set extractor parameters. Please note that all of them should be passed otherwise missing will be deleted;
• PERFORM_METRIC_CALCULATION – is used to extract metric data and store it into the metric cluster;

Important notes

• Input parameter IT_RUNDATETIME (UTC timestamp) of PERFORM_METRIC_CALCULATION method should be used together with IT_LASTRUN extractor attribute to determine start and end of extraction period.
• Please use data dictionary table type to store metric data. Please note that currently only standard tables are supported. Apart from the dictionary table types it is also possible to use local or dynamic types, however it will have impact on performance and memory consumption.
• Once the data is extracted, the method STORE_METRIC should be called one or more times to store metric data into the metric cluster. Please note that ‘i_subgroup’ is optional parameter and should be used in case two or more dependant data tables should be extracted at the same time (for instance, document header will be stored with empty subgroup and document items will be stored with subgroup ‘ITEM’). Subgroup field will be also sent to Splunk and could be filtered using EVENT_SUBTYPE field.

Log messages

In order to add own messages into the PowerConnect application log, following methods of /BNWVS/CL_MTR_ENGINE class could be used:

• ADD_APP_LOG – add custom message;
• ADD_APP_LOG_WITH_MSG – add message with message class and number;
• ADD_APP_LOG_WITH_EXC – add message with exception;

Please note that messages will be collected respecting with Application Log Level (Control Panel->Administrator->App Log Level).

Configuration

Once new metric class is created, following config should be made to get it working.

1. Group definition entry (Control Panel->Administrator->Setup Group Def) – should be made to define a group name and run interval. For example, if you set to 3600 seconds (1 hour), when the extractor runs, IT_LASTRUN will be populated with timestamp from 1 hour ago.
   
2. Task group entry (Control Panel->Administrator->Setup Group Def) – should be made to define the extractor class and supported system attributes.
   

Test extractor

The extractor could be tested using the text tool (Control Panel->Goto->Test extractor):

Please note that Send Metric Data checkbox should be set carefully only if test data should be sent to Splunk.

FAQ

• What value is correct for class attribute C_TABNAME?
  It is obsolete attribute. It will be removed in next versions, please remove or ignore it.
• How to know if parameter I_SUBGROUP should be filled in method STORE_METRIC? And if we have to fill it, what value is right?
  The subgroup is used when you need to store some dependent data at the same time. In this case header records are stored with empty I_SUBGROUP and dependent data should be stored with populated I_SUBGROUP parameter. The subgroup itself could have any value (up to you to choose). It will be sent to Splunk as EVENT_SUBTYPE field, which could be used during the search. For instance, subgroup could be used to extract document header and document items. In this case header will not have subgroup, but items could be stored with subgroup ‘ITEM’. Then in Splunk you may distinct headers and items.
• In SAP we’ve created the dictionary objects with the necessary fields but, do we need to export this structure to Splunk platform? Splunk, when it reads the XML file, expects a some type of XML structure but this XML has new fields and structures.
  No, the framework will take care about the data structure and serialize it accordingly so Splunk will be able to understand and parse it.

Download PDF version here