Robotic Process Automation and Data Extraction Principles

Posted on Jan 31, 2026 in Computer Engineering

Basic Recording vs. Desktop Recording

Introduction to Recording

• Recording refers to the process of capturing audio, video, or on-screen activities so they can be saved, edited, or shared later. It is commonly used in teaching, presentations, software demonstrations, content creation, and communication.
• Basic Recording: Involves capturing simple audio or video using built-in or external devices. It does not involve screen capturing.

Examples of Basic Recording

• Recording your voice using a microphone.
• Recording a lecture using a mobile phone.
• Recording video using a webcam.
• Capturing sound using tools like Voice Recorder or Audacity.

Desktop Recording (Screen Recording)

Desktop recording means capturing the activities happening on a computer screen. It records the display and may also capture audio and webcam video.

Screen Scraping vs. Data Scraping

Advantages of Screen Scraping

• Works even when database access is restricted.
• Useful for legacy systems.
• Quick to implement.

Limitations of Screen Scraping

• Only visible screen content can be captured.
• Formatting and accuracy issues.
• Breaks if the screen layout changes.
• Slower than direct data access methods.

Short Notes for Examination

Screen Scraping

Screen scraping is the process of extracting displayed data from a screen or graphical interface. It captures visible text and converts it into usable data, often using OCR or automation tools. It is useful for legacy systems but is less accurate and sensitive to layout changes.

Data Scraping

Data scraping is the automated extraction of data from websites or online sources using HTML parsing, APIs, or scripts. It provides structured, accurate, and large-scale data collection, commonly used in research, analytics, and automation.

Advanced Scraping Techniques

Advanced scraping techniques are used to extract data from dynamic, interactive, or complex websites. These techniques go beyond simple HTML parsing.

Handling Dynamic JavaScript-Based Websites

Many modern websites load data using JavaScript. Common tools include:

• Selenium, Playwright, or Puppeteer to simulate a real browser.
• Headless browsers.
• Network intercepts to capture API calls.

API-Based Scraping

Instead of scraping HTML, tools directly collect data from JSON APIs, XML feeds, or GraphQL endpoints. This is faster and more reliable.

AJAX and Lazy-Loading Handling

When content loads only after scrolling or clicking, developers use:

• Automated scrolling scripts.
• Wait functions.
• JavaScript event triggers.

Defining and Assessing Selectors

Defining a selector means choosing a unique and stable pattern to locate an element on a webpage. Assessing a selector means checking whether it is accurate, unique, stable, and efficient.

Criteria for a Good Selector

1. Uniqueness: The selector must point to one specific element to avoid selecting multiple items unintentionally.
2. Stability: The selector should not change frequently; dynamic attributes should be avoided.
3. Accuracy: The selector must correctly target the desired element, such as a product name or price.
4. Performance: Selectors should be simple, avoid long XPath paths, and execute quickly.
5. Reliability: The selector should work even if the page layout shifts slightly or minor UI changes occur.

Selector Examples

• Bad Selector: div:nth-child(7) > span (Breaks if layout changes).
• Good Selector: span.product-price (Stable, meaningful, and unique).

Customization in Automation

Customization refers to modifying a software system, automation workflow, or scraping tool to meet specific needs. It is necessary to clean data, handle structure changes, and improve reliability.

Examples of Customization

• Custom Selectors: Defining stable selectors instead of auto-generated ones.
• Custom Workflow Logic: Adding loops, conditional statements, and error handling.
• Custom Output Formats: Choosing CSV, JSON, XML, or database storage.
• Custom User Inputs: Allowing users to define search keywords or date ranges.
• Custom Filtering: Filtering for high-priced products or specific categories.

Debugging Automation Workflows

Debugging is the process of finding, analyzing, and fixing errors in a scraping workflow or software program. Tools include Browser Developer Tools, Selenium logs, and the UiPath Debug Panel.

Common Debugging Problems

Steps to Debug

1. Identify the Error: Read error messages and use logging.
2. Inspect the Source: Check if classes or IDs changed in the HTML.
3. Test Selectors: Use CSS/XPath checkers or validation tools.
4. Review Logic: Check for incorrect loops or missing wait times.
5. Add Error Handling: Use try-catch blocks and retry loops.

Dynamic vs. Partial Selectors

Dynamic Selectors contain values that change every time a page loads, such as session-based classes. Partial Selectors match only a portion of an attribute, used when the rest of the value is dynamic.

The RPA Challenge and Image Automation

An RPA Challenge occurs when a bot must perform tasks in difficult environments, such as Citrix or remote desktops, where HTML elements are unavailable.

Image-Based Automation

The bot identifies elements based on visual appearance (icons or shapes). It is ideal for remote environments but sensitive to screen resolution and UI changes.

Text Automation

Text automation extracts visible text using OCR (Optical Character Recognition). It is flexible for labeling-based automation but can be slow and depends on screen quality.

Information Retrieval Methods

Information retrieval is the process of extracting data from UI elements, documents, or images. It is categorized into:

• Structured: Databases, Excel, and tables.
• Semi-Structured: Emails, PDFs, and XML files.
• Unstructured: Scanned documents and handwritten text.

RPA History and Definition

• 18th–19th Century: Mechanical machines replaced manual labor during the Industrial Revolution.
• 20th Century: Introduction of assembly lines and industrial robots.
• 21st Century: Shift to software-based knowledge work automation.

Robotic Process Automation (RPA) is the use of software bots to automate repetitive, rules-based digital tasks. It mimics human interactions like clicks and keystrokes across multiple applications.

Suitable Processes for RPA

RPA is best for high-volume, repetitive tasks such as:

• Finance: Invoice processing and payroll.
• HR: Employee onboarding.
• IT: Password resets.
• Data: Extracting data from PDFs.

RPA vs. Traditional Automation

Types of Bots in RPA

• Attended Bots: Work with humans in real-time; triggered manually.
• Unattended Bots: Run without human intervention; best for back-office tasks.
• Hybrid Bots: Combine both attended and unattended modes.

RPA Development Life Cycle (RPADLC)

1. Identification: Choose suitable processes.
2. Assessment: Check feasibility and ROI.
3. Design: Create the PDD and workflow diagrams.
4. Development: Build bots in RPA tools.
5. Testing: Perform unit and UAT testing.
6. Deployment: Move to the production environment.

Documentation: PDD vs. SDD

The Process Design Document (PDD) describes what the process is, while the Solution Design Document (SDD) describes how the bot will technically automate it.

Typical RPA Team Roles

• RPA Sponsor: Senior leader approving the budget.
• Business Analyst: Maps processes and prepares PDDs.
• RPA Developer: Designs and deploys the bots.
• RPA Tester: Ensures process compliance and handles exceptions.

UiPath Control Flow and Data Manipulation

Arguments are used to pass data between workflows (In, Out, In/Out). Loops (While, Do While, For Each) allow repeating actions until conditions are met.

Advanced Control Flow

• Switch: Handles multiple outcomes based on an expression.
• Parallel: Runs multiple activities simultaneously.
• Pick: Triggers the first event that occurs among many.

Text Manipulation Methods

• Concatenation: Joining strings together.
• Substring: Extracting a portion of a string.
• Replace: Swapping old text with new text.
• Split: Dividing text into an array.
• Trim: Removing leading and trailing spaces.
• Regex: Extracting patterns like emails or phone numbers.