The evolution of scheduling in EPM and why Pipeline’s have all the answers!

In the past, when data load jobs needed to be scheduled in EPM, there were two main options; Batches or EPM Automate scripts. Now however, there is a third option of the new Pipeline functionality which was released as part of the new Data Integration interface in June 2023. Pipelines are set to replace the existing Batch functionality currently available in Data Management, that will soon be decommissioned by Oracle.

While it’s always exciting when Oracle release new functionality, it can be hard to keep up sometimes – this switch however is one that is definitely worth the effort! This blog will tell you all about the differences between the old Batch functionality, existing EPM Automate scripting and the new Pipelines, and why it’s worth making the switch to Pipelines as soon as possible.

1. Batches  

Let’s start with Batches. If you’ve not used them before, Batches are defined and scheduled within data management and are a way to schedule a range of data management load rules, defining parameters for each load.

Batches – Pros:

  • Code-free scheduling
  • Defined and scheduled within EPM (data management) rather than on an external server
  • Able to schedule a series of data load rules in Serial or Parallel

Batches – Cons:

  • Schedules are hidden within Data Management, rather than showing in the Jobs console alongside other scheduled jobs
  • Cannot use substitution variables for the date parameters so they must be manually updated periodically
  • Limited to scheduling data load rules (including any business rules assigned to those data load rules)
  • Oracle plan to decommission the batch functionality and it will not be moving to the Data Integration interface

2. EPM Automate scripts

EPM Automate scripts are coded scripts which are stored and scheduled on an EPM Automate server. They are much more flexible than batches, however they require a basic level of batch scripting to write and maintain.

EPM Automate Scripts – Pros:

  • Can code in usage of substitution variables and other functions like current date
  • Custom email notifications can be coded into the script
  • Able to code more complex and flexible error handling
  • Range of jobs can be incorporated into the script, as long as there is an EPM Automate command for it
  • No plans for Oracle to decommission this functionality

EPM Automate Scripts – Cons:

  • Requires code and a basic understanding of batch scripting
  • Scripts are stored and scheduled on an external server, rather than directly in EPM
  • Requires maintenance of EPM Automate server and scripts

3. Pipelines

Now onto the new Pipeline functionality which, in my humble opinion, is a huge improvement! Pipelines are built and viewed in the Data Integration area (under Data Exchange) and listed alongside all the data load rules in the system. Pipelines are made up of Stages, which in turn are made up of Jobs, such as data load rules, business rules, or any other job type which is available in EPM.

The Pipeline interface has been designed to help admins visualise the tasks and group them into relevant stage groupings, which can have their own individual settings. Email notifications are built in and contain the status of every job, with a link directly to the log. 

 

Pipeline – Pros:

  •  Code-free scheduling
  • Defined and scheduled within EPM rather than on an external server
  • Able to visualise the jobs in relevant stage groupings
  • Stages can be run in parallel or sequence
  • Built-in email notification functionality, which can be sent always, never, or on failure/success only
  • Use of substitution variables to set load period parameters
  • Ability to abort or skip tasks on error
  • Incorporates all types of job (not just data load rules and business rules), including any REST job type
  • Scheduled via the normal Job scheduler, so all scheduled tasks can be viewed in a single place

Pipeline – Cons:

  • No automatic migration tool from batches to pipeline so schedules must be re-build manually initially

In summary, Pipelines provide the flexibility of EPM Automate scripts without the need to maintain scripts on an external server, while also providing a visual interface within EPM Data Integration for admins to view jobs within stages. Pipelines are a great alternative to Batches which, while code-free, were quite restricted in the jobs that could be scheduled and ultimately will be decommissioned by Oracle at some point in the future.

So there you have it, make the switch to Pipelines soon to make use of all the new features available and remove the reliance on either EPM Automate servers or Batch functionality!

Comments

Post a Comment

Popular posts from this blog

Executing Smart View Retrievals using VBA

Image
When working with Hyperion systems, Smart View is a key tool used for reporting, as well as inputting data. It provides a valuable link between Hyperion and Excel, which allows users to retrieve data from the database straight into Excel with ease. Smart View functions are available through the Smart View toolbar and can be used with Essbase, HFM, Planning or Cloud connections. Occasionally, it is necessary to perform Smart View functions behind the scenes using VBA code, rather than manually using the Smart View toolbar. For example, a spreadsheet which will refresh all the Essbase retrieval sheets on opening, so that users can view the most up to date data, without having to remember to refresh each of the retrieval sheets themselves. This blog will be cover how to retrieve data using a Smart View refresh function in VBA. For more information about Smart View Excel VBA functions, go to the Oracle Hyperion Smart View Developers Guide .  System Requirements T
Read more

Loading Actuals from Fusion ERP Cloud to PBCS

Image
If you’re looking to set up a data load of Actuals from ERP to PBCS and have heard this is an “Out of the box” integration, then good news! Not only is the OOTB connector included within PBCS Data Management but the integration is simple to setup and to schedule using EPM Automate. This blog will detail how to setup the ERP to PBCS connector in Data Management and some of the common errors found when running the data load. The first thing to know about this integration is that the data source is actually the ASO Essbase cube which sits underneath the Fusion ERP tables, holding the period Actuals values. This ERP Essbase cube is created using job ‘Create General Ledger Balances Cube’ when configuring the ERP system. This cube must be created before the integration can be set up correctly and can be accessed directly via Smart View for reconciliations. Set up the Source System As this is an Oracle built connector, it’s as simple as providing a name for the source system
Read more

Loading multi-period row data files using Data Management

Image
The ability to load multi-period data with the Period and/or Year in the rows is not particularly new (since release 16.10), however it is something that most people seem to forget is possible in Data Management. This really is something worth remembering as it eliminates the need to either split the data files into one file per month or try to pivot the month into the columns. In order to set this up, let’s firstly look at how to add the Period and Year columns to the import format. Add the Period and Year columns using the Add -> Source Period Row dropdown, and input: The Source Column name – e.g. PERIOD The Field Number (file column) – e.g. 3 Any required expressions for that field  As these are Period/Year dimensions, the mappings are not held in the usual Data Load Mappings area. Instead they are configured in the Period Mappings , under the Source Mapping tab. Select the Source System as File to add a new mapping calendar for loading the data file
Read more