Skip to content

LabID Workflow Management Overview

LabID provides workflow organization and metadata management capabilities that help researchers organize, version, and share their computational workflows and associated datasets. This system is designed to support both manual workflow creation and importing workflows from external sources, such as Git repositories, WorkflowHub, and Galaxy. Once created or imported, specific workflow versions can be referenced by workflow runs where a workflow run represents the execution of a specific workflow version with associated input and output datasets, execution parameters, configuration details, reports and logs.

Manual Workflows

Workflows created within LabID by uploading existing files:

  • Upload workflow files from your local computer through the web interface
  • All files are tracked as WorkflowFile objects
  • A Git repository is initialized under the hood and managed by LabID
  • Each Workflow version is immutable and linked to a specific commit hash
  • Users can create multiple versions of a workflow, each with its own set of files and annotations

Imported Workflows

Workflows imported from external Git repositories:

  • Source code is cloned from remote repositories (GitHub, GitLab, etc.)
  • Users select which files to track as WorkflowFiles
  • Maintains connection to the original repository for updates to ease fetching additional versions from the original repository
  • Supports both public and organisational repositories with SSH key authentication

Workflow Management and Sharing

  • Workflow File Classification: Organize workflow components using semantic file types (MAIN, CONFIG, TEST, README, etc.) matching WorkflowHub standards
  • Annotate: Attach licenses, and descriptive metadata
  • Collaboration: Share workflows with team members while preserving version history
  • RO-Crate Export: Generate standards-compliant research object packages
  • WorkflowHub Integration: Import & Publish workflows directly from/to WorkflowHub
  • Galaxy Integration: Import workflows from Galaxy instances
  • Provenance Tracking: Maintain records of workflow origins and modifications

Workflow Execution and Artifacts

  • Workflow Runs: Create and manage workflow runs, linking them to specific workflow versions
  • Link Datasets: Associate workflow runs with input and output datasets, configuration files, logs and reports
  • Galaxy Integration: Import WorkflowInvocations as WorkflowRuns
  • Workflow Execution Metadata: Store user-provided execution metadata and parameters
  • Automate Workflow Run Import: The Command Line Interface (CLI) allows for custom and automated import of workflows executed locally.

Documentation Guide

Features Documentation

  • Workflows and Provenance - Complete step-by-step user guide for creating, importing, and managing workflows; as well as capturing workflow runs, generated data and their provenance.

Technical Documentation

Administration Documentation