ROBOCORP DATA MANAGEMENT

Working Example Using Robocorp Work Items | Definitive Guide 2021

Today I’m going to show you exactly how to work with Robocorp Data Management Work Items.

In this comprehensive guide, I’ll cover – 

  • What is Work Items
  • Why Work Items & Benefit 
  • Various Keywords required for Work Items management
  • Useful code snippet 
  • Complete working example using Work Items concept
  • Lots more

So if you want to learn and start working with Robocorp Work Items, You’ll love this guide.

Let’s get started.

 

Work Data Management break up a process into separate work items

Table of Contents

What is Work Items?

  • Work Item is the smallest chunk of data that can be processed completely end to end.
  • Each step of your BOT execution can receive work items as input (contains data & file) and Create work items as Output for the next step processing
  • You can relate work items as Traditional Loan application file that needs to be moved across different steps such as checking eligibility, chance of default, credit limits, and finally disbursement. Same file work as work items (input and optionally modified some values for next step)
  • A work item contains a payload of JSON-based data and optionally attached files. You can store anything that’s is serializable.
  • Adding and using files with work items requires no additional setup and by default, it’s stored in Robocorp control Room.
Working Example Using Robocorp Work Items | Definitive Guide 2021 1
Source - Example of Work Items Processing by Robocorp

Benefits of work items

  • Splitting large volume of work into individual (small chunks) work items for better handling and parallel processing
  • Provide the ability to build business-critical large workloads process using a queue to complete processing in the parallel environment (Some say multi-Robot environment)
  • Provides ways for analytics and reporting for the individual piece of work
  • Work Data Management break up a process into separate work items so that the overall process can be tracked at a granular level providing a capability to re-process failed work items or handle manually
  • Enables dynamic scaling of runtime environments for parallel execution to significantly improve throughput
  • Enables processing huge volumes of data efficiently to complete the job faster and increase time-to-value
  • Work Data Explorer also provides business users with the tools to manage work items such as running, modifying, retrying, and marking work items as done according to different use cases.

Pre-requisite and dependency

  • Robocorp VsCode extensions has built-in support making the use and testing of work items.
  • Requires the use of rpaframework v11.4.0 or later in your robot.

Various Keywords in RPA.Robocorp.WorkItems

  • Add Work Item File
  • Add Work Item Files
  • Clear Work Item
  • Create Output Work Item
  • Delete Work Item Variables
  • For Each Input Work Item
  • Get Current Work Item
  • Get Input Work Item
  • Get Work Item File
  • Get Work Item Files
  • Get Work Item Payload
  • Get Work Item Variable
  • Get Work Item Variables
  • List Work Item Files
  • Release Input Work Item
  • Remove Work Item File
  • Remove Work Item Files
  • Save Work Item
  • Set Current Work Item
  • Set Task Variables from Work Item
  • Set Work Item Payload
  • Set Work Item Variable
  • Set Work Item Variables

How it works

As stated above work items are used for managing the data that passes through the various steps of automation. Where each step takes input from previous steps to create a work item for the next step.

Working Example Using Robocorp Work Items | Definitive Guide 2021 2

Structure of Work items –

  • Work items are JSON means it allows you to store any kind of information that is serializable. In addition, you can also store files with work items.
  • Using the various keywords, you should be able to Create outputs work items, read the content of active work items, manipulate the content of current work items
  • You can create multiple work items as an output from a task, every time you use the keyword ‘Crete output work item’ a new empty item is created as a child for the currently loaded input, which overall makes it easier to track and move information for steps processing.
  • Work items become active when the execution started and the read and write from the work item will work on the active work item only, however, you can set the work item as active using the keyword ‘Set current work item’
  • Changes to work items are modified locally and can’t be reflected unless you use the keyword ‘Save work item’. So, make sure you save work items when you are done with the process of the dataset.

Example Snippet– Creating Work items from Excel file

  • Create Output Work Item creates a new empty work item.
  • Set Work Item Variable sets a new variable name and value into the work item.
  • Save Work Item saves the item and all contained data/files to the next step.
				
					*** Settings ***
Documentation   Read Excel Records to Create Robocloud WorkItems
Library    RPA.Browser.Selenium
Library    RPA.HTTP
Library    RPA.Robocorp.WorkItems
Library    RPA.Excel.Files

*** Variables ***
${File}=  %{ROBOT_ROOT}/CoinsToWatch.xlsx

*** Tasks ***
Upload Data
    Open Workbook    ${File}
    ${sheetname}=  Get Active Worksheet
    LOG  ${sheetname}
    ${data}=  Read Worksheet As Table  name=${sheetname}  header=True
    FOR    ${row}    IN    @{data}
        Create Output Work Item
        Set Work Item Variable  ID  ${row}[ID]
        Set Work Item Variable  Coin  ${row}[Coin]
        Set Work Item Variable  Symbol  ${row}[Symbol]
        Save Work Item
    END
    Close Workbook

				
			

Work item exception handling

Exceptions are everywhere and we can’t assume that all the work items will be processed successfully, some might fail due to application issues when chrome crashes or any other HTTP error, some might fail due to business exceptions and further processing of the work item is not possible.

RPA.Robocorp.WorkItems Keyword libraries provide the keyword ‘Release Input Work Item’ to set the status of the active input work item.

So that status of each work item can be reported and analyzed.

The work items can be released with either “success” or “failure” using the keywords “DONE” and “FAILED”.

Saying “FAILED” is not going to help the operation team or Business Users so the additional exception details must be sent to the control room.

You can set up status as Filed with exception type, exception code if any, and message.

Exception Example snippet –  

				
					Release Input Work Item    DONE

Release Input Work Item     FAILED      exception_type=APPLICATION   code=LOGIN_PORTAL_DOWN     message=Unable to login into the portal – not proceeding  

Release Input Work Item
        ...    state=FAILED
        ...    exception_type=BUSINESS
        ...    message=Order prosessing failed for: ${Coin}

				
			

Developing with work items locally Using VS Code

When executing a robot in a cloud environment like Control Room, the RPA.Robocorp.WorkItems library will store the work item in the cloud environment, sharing its contents between steps defined in the same process, without any configuration needed.

However, when running locally, the work items can be simulated using folder structure and JSON files.

  • Robocorp VsCode extensions has built-in support making the use and testing of work items
  • As each task in the robot expects different work item input
  • The RPA.Robocorp.WorkItems library can be controlled with specific environment variables to control the input and output sources
  • By default, the env.json is used by Robocorp Lab so the inputs and output paths defined there decide which input is used.
  • In this example under ./devdata you can find different JSON files that demonstrate the selection:
Working Example Using Robocorp Work Items | Definitive Guide 2021 3

To run specific tasks with specific inputs in the command-line or Robocorp Lab Terminal you can run the following commands:

  • Run Process Data:
    • rcc task run -t “Process Data” -e ./devdata/work-items-out/run3/work-items.json

Complete Working Example: Cryptocurrency ticker with multiple work items

Working Example Using Robocorp Work Items | Definitive Guide 2021 4

Robot Overview

This robot tracks the cryptocurrencies portfolio of customer from an incoming Excel file, the individual crypto coins is tracked for price movement is then handled individually in parallel in cloud container.

The robot demonstrates the concept of work items feature of Robocorp-

  • Creating multistep process
  • Passing data between the process steps using work items
  • Parallel execution of steps
  • Use of Custom Python Library (pycoingecko to get Crypto prices using Coin Gecko API)
  • Sending consolidated HTML email using python Jinja2 template engine

Robot Tasks

The robot is split into 3 tasks.

  1. CreateWorkitems – Read Excel Records to Create Robocloud Work Items, Creates a new work item for each Crypto Coin.
  2. ProcessWorkItems – Process Work items Individually to get Latest price from CoinGecko API, can be set to run in parallel
  3. StatusWorkitems – Send Consolidated Report of All coins from Input in template-based HTML email.

 

Robocorp Library Used

  • Robocorp.WorkItems
  • HTTP
  • JSON
  • Excel.Files

Additional Python Library Used

  • Smtplib, jinja2 ,sqllite3
  • Pycoingecko wrapper for CoinGeckoAPI

Robot Input File - List of Crypto Coins

IDCoinSymbol
1bitcoinBTC
2ethereumETH
3tetherUSDT
4solanaSOL
5cardanoADA
6xrpXRP
7avalancheAVAX
8litecoinLTC
9chainlinkLINK

CreateWorkItems.robot

				
					*** Settings ***
Documentation   Read Excel Records to Create Robocloud WorkItems
Library    RPA.Browser.Selenium
Library    RPA.HTTP
Library    RPA.Robocorp.WorkItems
Library    RPA.Excel.Files


*** Variables ***
${File}=  %{ROBOT_ROOT}/CoinsToWatch.xlsx

*** Tasks ***
Upload Data
    Open Workbook    ${File}
    ${sheetname}=  Get Active Worksheet
    LOG  ${sheetname}
    ${data}=  Read Worksheet As Table  name=${sheetname}  header=True
    FOR    ${row}    IN    @{data}
        Create Output Work Item
        Set Work Item Variable  ID  ${row}[ID]
        Set Work Item Variable  Coin  ${row}[Coin]
        Set Work Item Variable  Symbol  ${row}[Symbol]
        Save Work Item
    END
    Close Workbook
				
			

ProcessWorkItems.robot

				
					*** Settings ***
Documentation     Process Workitems Individually to get Latest price from CoinGecko API
Library           RPA.Robocorp.WorkItems
Library           RPA.HTTP
Library           RPA.JSON
Library           CommonLibrary
Library           DateTime
Library           Collections
Library           RPA.Browser.Selenium
Library           RPA.FileSystem

*** Variables ***
${API_URL}=

*** Keywords ***
Load and Process Coin
    [Documentation]    Get Price
    ${ID}=    Get Work Item Variable    ID
    ${Coin}=    Get Work Item Variable    Coin
    ${Symbol}=    Get Work Item Variable    Symbol
    ${passed}=    Run Keyword And Return Status
    ...    Get Coin Data    ${Coin}    ${Symbol}
    IF    ${passed}
        Log    Price prosessing passed for: ${ID} Coin: ${Coin} items: ${Symbol}    level=INFO
        Release Input Work Item    DONE
    ELSE
        Log    Price prosessing failed for: ${ID} Coin: ${Coin} items: ${Symbol}    level=ERROR
        Release Input Work Item
        ...    state=FAILED
        ...    exception_type=BUSINESS
        ...    message=Order prosessing failed for: ${Coin}
    END

*** Keywords ***
Get Coin Data
    [Arguments]    ${Coin}    ${Symbol}
    ${DATE}=    Get Current Date    result_format=timestamp
    ${Price}=    Get Coin Price    ${Coin}    ${Symbol}
    ${MarketPrice}=    Get Coin Data From CoinMarketCap    ${Coin}    ${Symbol}
    Create output work item
    Set work item variables    Coin=${Coin}    CoinGekoPrice=${Price}    CoinMarketPrice=${MarketPrice}    TimeStamp=${DATE}
    Save work item
    Log    Price for ${Coin} : ${Price} at TimeStamp ${DATE} while at Market is ${MarketPrice}

 *** Keyword ***

 Get Coin Data From CoinMarketCap
    [Arguments]    ${Coin}    ${Symbol}
    Open Available Browser    https://coinmarketcap.com/
    Sleep    2
    Maximize Browser Window
    Go To    https://coinmarketcap.com/currencies/${Coin}
    Sleep    1
    Wait Until Element Is Visible    //*[@id="__next"]/div[1]/div/div[2]/div/div[1]/div[2]/div/div[2]/div[1]/div
    ${MARKET_PRICE}=    Get Text    //*[@id="__next"]/div[1]/div/div[2]/div/div[1]/div[2]/div/div[2]/div[1]/div
    Log    ${MARKET_PRICE}
    Close Browser
    [Return]    ${MARKET_PRICE}

*** Tasks ***
Process Data
    For Each Input Work Item    Load and Process Coin

				
			

StatusWorkItems.robot

				
					*** Settings ***
Documentation     Process Workitems
Library           RPA.Robocorp.WorkItems
Library           RPA.HTTP
Library           RPA.JSON
Library           CommonLibrary


*** Variables ***

*** Keywords ***
Consoidated Report
    [Documentation]    Send Report for All Cryptos
    For Each Input Work Item    Build Report

*** Keywords ***
Build Report
    #Coin=${Coin}    CoinGekoPrice=${Price}    CoinMarketPrice=${MarketPrice}    TimeStamp=${DATE}
    ${Coin}=    Get Work Item Variable    Coin
    ${Price}=    Get Work Item Variable    CoinGekoPrice
    ${marketprice}=    Get Work Item Variable    CoinMarketPrice
    ${updatedon}=    Get Work Item Variable    TimeStamp
    Log    ${Coin} ${price}
    Insert Table    ${Coin}    ${price}    ${marketprice}     ${updatedon}

*** Keywords ***
Send Report
    ${results}=    Read Table
    Send Price List    ${results}

*** Tasks ***
Report Workitems
    Clean Table
    Consoidated Report
    Send Report

				
			

Simple Python Library for Email and Database

				
					import os
import sqlite3
import json
from pycoingecko import CoinGeckoAPI
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from smtplib import SMTP

from jinja2 import Environment, FileSystemLoader

env = Environment(
    loader=FileSystemLoader('%s/templates/' % os.path.dirname(__file__)))

# VARIABLES Change Accrodingly
TO_EMAIL = "prasadsatish@outlook.com"
FROM_EMAIL = "rpabotsworld@gmail.com"
SUBJECT = "PRICE MOVEMENT FOR TRACKED COIN"
EMAIL_APP_PASS = "XXXXXXXXXXXX"


def send_mail(to_email ,from_email, subject, bodyContent):
    message = MIMEMultipart()
    message['Subject'] = subject
    message['From'] = from_email
    message['To'] = to_email

    message.attach(MIMEText(bodyContent, "html"))
    msgBody = message.as_string()

    server = SMTP('smtp.gmail.com', 587)
    server.starttls()
    server.login(from_email,EMAIL_APP_PASS )
    server.sendmail(from_email, to_email, msgBody)
    server.quit()


def send_price_list(data):
    template = env.get_template('list.html')
    output = template.render(data=data)
    send_mail(TO_EMAIL,FROM_EMAIL,SUBJECT, output)    
    return "Mail sent successfully."


def get_coin_price(coin,symbol):
    price = None
    cg = CoinGeckoAPI()
    response = cg.get_price(ids=coin, vs_currencies='usd')
    for key, value in response.items():
        price=value['usd']
    return price
    
    
def create_table():
    con =  con = sqlite3.connect('example.db')
    # Create table
    cur = con.cursor()
    cur.execute('''CREATE TABLE crypto
                (coin text, price text, marketprice text,updatedon timestamp)''')
    con.commit()
    con.close()

def insert_table(coin,price,marketprice,updatedon):
    con =  con = sqlite3.connect('example.db')
    # Create table
    cur = con.cursor()
    sql = '''INSERT INTO crypto(coin, price, marketprice, updatedon ) values(?, ?, ?, ?)'''
    val = (coin,price,marketprice,updatedon)
    cur.execute(sql,val)
    print(cur.rowcount, "details inserted")
    con.commit()
    con.close()

def read_table():
    con =  con = sqlite3.connect('example.db')
    cur = con.cursor()
    sql = "Select coin, price, marketprice, updatedon from  crypto"
    cur.execute(sql)
    variable=cur.fetchall()
    con.close()
    return variable

def clean_table():
    con =  con = sqlite3.connect('example.db')
    cur = con.cursor()
    sql = "delete from  crypto"
    cur.execute(sql)
    con.commit()
    cur.close()
    con.close()
				
			

Robot Output Result - Consolidated Email

Working Example Using Robocorp Work Items | Definitive Guide 2021 5

Important!

  • The above example can be improved using Robcorp Vault to store various credentials
  • The database part is optional, it’s for improvement as it can be used to plot graphs and charts from historical price movement to include in the email.
Scroll to Top