ROBOCORP DATA MANAGEMENT

Today I’m going to show you exactly how to work with Robocorp Data Management Work Items.

In this comprehensive guide, I’ll cover – 

  • What is Work Items
  • Why Work Items & Benefit 
  • Various Keywords required for Work Items management
  • Useful code snippet 
  • Complete working example using Work Items concept
  • Lots more

So if you want to learn and start working with Robocorp Work Items, You’ll love this guide.

Let’s get started.

 

Work Data Management break up a process into separate work items

What is Work Items?

  • Work Item is the smallest chunk of data that can be processed completely end to end.
  • Each step of your BOT execution can receive work items as input (contains data & file) and Create work items as Output for the next step processing
  • You can relate work items as Traditional Loan application file that needs to be moved across different steps such as checking eligibility, chance of default, credit limits, and finally disbursement. Same file work as work items (input and optionally modified some values for next step)
  • A work item contains a payload of JSON-based data and optionally attached files. You can store anything that’s is serializable.
  • Adding and using files with work items requires no additional setup and by default, it’s stored in Robocorp control Room.
Working Example Using Robocorp Work Items | Definitive Guide 3
Source - Example of Work Items Processing by Robocorp

Benefits of work items

  • Splitting large volume of work into individual (small chunks) work items for better handling and parallel processing
  • Provide the ability to build business-critical large workloads process using a queue to complete processing in the parallel environment (Some say multi-Robot environment)
  • Provides ways for analytics and reporting for the individual piece of work
  • Work Data Management break up a process into separate work items so that the overall process can be tracked at a granular level providing a capability to re-process failed work items or handle manually
  • Enables dynamic scaling of runtime environments for parallel execution to significantly improve throughput
  • Enables processing huge volumes of data efficiently to complete the job faster and increase time-to-value
  • Work Data Explorer also provides business users with the tools to manage work items such as running, modifying, retrying, and marking work items as done according to different use cases.

Pre-requisite and dependency

  • Robocorp VsCode extensions has built-in support making the use and testing of work items.
  • Requires the use of rpaframework v11.4.0 or later in your robot.

Various Keywords in RPA.Robocorp.WorkItems

  • Add Work Item File
  • Add Work Item Files
  • Clear Work Item
  • Create Output Work Item
  • Delete Work Item Variables
  • For Each Input Work Item
  • Get Current Work Item
  • Get Input Work Item
  • Get Work Item File
  • Get Work Item Files
  • Get Work Item Payload
  • Get Work Item Variable
  • Get Work Item Variables
  • List Work Item Files
  • Release Input Work Item
  • Remove Work Item File
  • Remove Work Item Files
  • Save Work Item
  • Set Current Work Item
  • Set Task Variables from Work Item
  • Set Work Item Payload
  • Set Work Item Variable
  • Set Work Item Variables

Read more about the keywords here – https://robocorp.com/docs/libraries/rpa-framework/rpa-robocorp-workitems/keywords

How it works

As stated above work items are used for managing the data that passes through the various steps of automation. Where each step takes input from previous steps to create a work item for the next step.

Working Example Using Robocorp Work Items | Definitive Guide 4

Structure of Work items –

  • Work items are JSON means it allows you to store any kind of information that is serializable. In addition, you can also store files with work items.
  • Using the various keywords, you should be able to Create outputs work items, read the content of active work items, manipulate the content of current work items
  • You can create multiple work items as an output from a task, every time you use the keyword ‘Crete output work item’ a new empty item is created as a child for the currently loaded input, which overall makes it easier to track and move information for steps processing.
  • Work items become active when the execution started and the read and write from the work item will work on the active work item only, however, you can set the work item as active using the keyword ‘Set current work item’
  • Changes to work items are modified locally and can’t be reflected unless you use the keyword ‘Save work item’. So, make sure you save work items when you are done with the process of the dataset.

Example Snippet– Creating Work items from Excel file

  • Create Output Work Item creates a new empty work item.
  • Set Work Item Variable sets a new variable name and value into the work item.
  • Save Work Item saves the item and all contained data/files to the next step.

Work item exception handling

Exceptions are everywhere and we can’t assume that all the work items will be processed successfully, some might fail due to application issues when chrome crashes or any other HTTP error, some might fail due to business exceptions and further processing of the work item is not possible.

RPA.Robocorp.WorkItems Keyword libraries provide the keyword ‘Release Input Work Item’ to set the status of the active input work item.

So that status of each work item can be reported and analyzed.

The work items can be released with either “success” or “failure” using the keywords “DONE” and “FAILED”.

Saying “FAILED” is not going to help the operation team or Business Users so the additional exception details must be sent to the control room.

You can set up status as Filed with exception type, exception code if any, and message.

Exception Example snippet –  

Developing with work items locally Using VS Code

When executing a robot in a cloud environment like Control Room, the RPA.Robocorp.WorkItems library will store the work item in the cloud environment, sharing its contents between steps defined in the same process, without any configuration needed.

However, when running locally, the work items can be simulated using folder structure and JSON files.

  • Robocorp VsCode extensions has built-in support making the use and testing of work items
  • As each task in the robot expects different work item input
  • The RPA.Robocorp.WorkItems library can be controlled with specific environment variables to control the input and output sources
  • By default, the env.json is used by Robocorp Lab so the inputs and output paths defined there decide which input is used.
  • In this example under ./devdata you can find different JSON files that demonstrate the selection:
Working Example Using Robocorp Work Items | Definitive Guide 5

To run specific tasks with specific inputs in the command-line or Robocorp Lab Terminal you can run the following commands:

  • Run Process Data:
    • rcc task run -t “Process Data” -e ./devdata/work-items-out/run3/work-items.json

Complete Working Example: Cryptocurrency ticker with multiple work items

Working Example Using Robocorp Work Items | Definitive Guide 6

Robot Overview

This robot tracks the cryptocurrencies portfolio of customer from an incoming Excel file, the individual crypto coins is tracked for price movement is then handled individually in parallel in cloud container.

The robot demonstrates the concept of work items feature of Robocorp-

  • Creating multistep process
  • Passing data between the process steps using work items
  • Parallel execution of steps
  • Use of Custom Python Library (pycoingecko to get Crypto prices using Coin Gecko API)
  • Sending consolidated HTML email using python Jinja2 template engine

Robot Tasks

The robot is split into 3 tasks.

  1. CreateWorkitems – Read Excel Records to Create Robocloud Work Items, Creates a new work item for each Crypto Coin.
  2. ProcessWorkItems – Process Work items Individually to get Latest price from CoinGecko API, can be set to run in parallel
  3. StatusWorkitems – Send Consolidated Report of All coins from Input in template-based HTML email.

 

Robocorp Library Used

  • Robocorp.WorkItems
  • HTTP
  • JSON
  • Excel.Files

Additional Python Library Used

  • Smtplib, jinja2 ,sqllite3
  • Pycoingecko wrapper for CoinGeckoAPI

Robot Input File - List of Crypto Coins

IDCoinSymbol
1bitcoinBTC
2ethereumETH
3tetherUSDT
4solanaSOL
5cardanoADA
6xrpXRP
7avalancheAVAX
8litecoinLTC
9chainlinkLINK

CreateWorkItems.robot

ProcessWorkItems.robot

StatusWorkItems.robot

Simple Python Library for Email and Database

Robot Output Result - Consolidated Email

Working Example Using Robocorp Work Items | Definitive Guide 7
Download Working Example
GitHub

Important!

  • The above example can be improved using Robcorp Vault to store various credentials
  • The database part is optional, it’s for improvement as it can be used to plot graphs and charts from historical price movement to include in the email.

References –

  1. https://robocorp.com/docs/libraries/rpa-framework/rpa-robocorp-workitems/keywords
  2. https://robocorp.com/docs/development-guide/control-room/work-items#the-consumer-reading-work-item-data