Infosys’ blog on industry solutions, trends, business process transformation and global implementation in Oracle.

« Avoid Validation failures with DRM Validation checks | Main | Data Science Fundamentals »

Automate your File based Data Loads in FDMEE with Open Batches

Automate your File based Data Loads in FDMEE with Open Batches


In one of my recent implementation assignments, we were faced with the requirement of eliminating any manual intervention and automating our data loads in Financial Data Management Enterprise Edition (FDMEE), the target system being Hyperion Financial Management (HFM) and Hyperion Tax Provisioning (HTP). Open Batches in FDMEE is an effective way of automating your file based data loads. Though the standard User Guides contain precious little on creating and running Open Batches and are not entirely comprehensive, Open Batches are easy to use and implement. In this blog, I attempt to explain in detail the process of creating Open Batches. These details have come purely from my personal research and my experience from my past assignments.


 


What is an Open Batch?


Batches play a critical role in automating the Data/Metadata load process. By creating batches, we can easily club different rules together or execute different rules in an automated way by leveraging the scheduling capabilities of FDMEE.



An Open Batch is instrumental in importing data into a POV in FDMEE from a file based data source. Open Batches can be automated to run at scheduled timings. Most importantly, Open Batches pitch in when we have to load more than 1 data file for any particular location. Also, Open Batches provide the flexibility of loading these multiple data files in 'Serial' or 'Parallel' mode.



Configuring and Executing an Open Batch


Let us assume that we have 2 FDMEE Locations and from each of these locations, we have to load 2 data files:


Step 1: Setting up the required FDMEE Locations


We create 2 locations as shown below (without deep-diving into the details of an FDMEE Location)

a. LOC_Sample_Basic1

b. LOC_Sample_Basic2




Step 2: Setting up the Open Batches



A new Open Batch may be created from: Setup -> Batch -> Batch Definition. Click on 'Add' to add a new Batch Definition.



Key in the basic details such as NAME (Name of the Open Batch), TARGET APPLICATION, WAIT FOR COMPLETION and DESCRIPTION. Apart from these, there are a few other details as below:


 


  1. Type: Select 'Open Batch'.

     

  2. Execution Mode: Select 'Serial' if you wish to load all the data files in that location one by one. Select 'Parallel' for loading all the files simultaneously.

     

  3. Open Batch Directory: This is where the User Guides are specifically vague. All we need to do is to specify the name of the folder where the data files will be placed. But, we need to take special care that this folder is created at a level which is lower than the level of the folder 'openbatch'. So, assuming that we need to place the data files in a folder 'Test1', this folder will be created as shown below:


  1. File Name Separator: For File Name Separator, we have to choose from (~ , _ , @ or : ).

     

  2. Naming convention of the Data Files: For a data file to be executed from an Open Batch, it needs to have the following components in it name, separated by the chosen 'File Name Separator':

     


  • File ID: free-form value used to sort the files for a given Location in an order. The values for this can be 1,2,3... or a,b,c... etc, just to specify the order in which the files will be loaded.

  • Data Load Rule Name: The name of the Rule through which the file will be loaded

  • Period: The period for which the file is to be loaded

  • Load Method: 2-character value where first character denotes 'Import Mode' (possible values 'A' and 'R' for 'append' and 'replace' respectively). The second character denotes 'Export Mode' (possible values 'A', 'R', 'M' and 'S' for 'accumulate', 'replace', 'merge' and 'replace by security' respectively).


So, now let us assume that we have to load 2 data files from the location 'LOC_Sample_Basic' using the Data Load Rule 'DLR_Sample_Basic'. Assuming that the 'File Name Separator' chosen is '@', the file names for the 2 files will be:


  • 1@DLR_Sample_Basic@Jan-17@RR

  • 2@DLR_Sample_Basic@Jan-17@RR


  1. The 2 Open Batches for loading the data files will be as below:

OP_Sample_Basic1

OP_Sample_Basic2




 


Step 3: Configuring the Master Batch


After doing all the above configurations, the default question would be that 'Where is the automation in this when we have ended up creating 2 or more Open Batches? How will we execute 2 or more Open Batches without Manual Intervention?'. The answer to this query lies with the creation of the open batch type 'Batch'. We configure an open batch with the type as 'Batch' and name it 'BatchMaster'.



The feature of the Batch is that we can add multiple Open Batches to the Batch Master and then, schedule just this one batch instead of worrying about multiple Open Batches. To add the 2 Open Batches that we had created earlier to this BatchMaster, click the 'Add' button and enter the details. Ensure that you enter the Job Sequence number to tell the system which Open Batch to execute first.



 


Step 4: Executing the Open Batches


The Open batches or the BatchMaster can be executed from: Workflow -> Other -> Batch Execution


You may simply select the BatchMater now and schedule it easily through the 'Schedule' button.



FDMEE Multi-Period Data Load


There may be a requirement to load data files that contain data/amount values for multiple periods. Files for an open batch multiple period load are stored in 'inbox\batches\openbatchml' directory. There are just a few minor variations between setting up a Batch for Multi-Period Load and a Single Period Load.


 


  1. Open Batch File Name: The file name for multi-period files should be as per the format 'FileID_RuleName_StartPeriod_EndPeriod_Loadmethod':


    • FileID: free-form value used to define the load order

    • RuleName: Name of the data load rule which will be used to process the file

    • StartPeriod: First period in the data file

    • EndPeriod: Last period in the data file

    • Load Method: Import and Export Mode

      So, a multi-period file name would look like for a Data Load Rule named 'Actuaload':


    • 1_Actuaload_Jan-17_Dec-17_RR

    • 2_Actuaload_Jan-17_Dec-17_AR

       

       


  1. Open Batch Directory: We specify the name of the folder in the same way as we did for single period batch. But, we need to take special care that this folder is created at a level which is lower than the level of the folder 'openbatchml' instead of 'openbatch'. So, assuming that we need to place the data files in a folder 'Test3', this folder will be created as shown below:



 


  1. Sample Open Batch for Multi-Period:


Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Blogger Profiles