Skip to content

Workflow Versioning and Release Management

LabID implements a sophisticated versioning system for workflows that combines Git-based version control with immutable releases. This system ensures reproducibility, traceability, and proper workflow lifecycle management.

Core Versioning Concepts

WorkflowVersion States

Each WorkflowVersion exists in one of two fundamental states:

Development State (Uncommitted)

  • Mutable: Files can be added, modified, or removed
  • commit_hash: Empty/null - no Git commit yet created
  • is_committed: False (computed property)
  • File Operations: Full upload, edit, and delete capabilities
  • Purpose: Active development and iteration

Released State (Committed)

  • Immutable: Files become read-only
  • commit_hash: Contains Git commit SHA hash
  • is_committed: True (computed property)
  • File Operations: Read-only access only
  • Purpose: Stable, reproducible workflow versions

Version Lifecycle

Create Version → Upload Files → Iterate → Release → [Create New Version]
     ↓                ↓           ↓         ↓              ↓
 is_latest=True    Mutable     Mutable    Immutable      is_latest=True
 commit_hash=""     Files       Files     commit_hash    commit_hash=""

Version Management Rules

Latest Version Constraint

Only one version per workflow can be marked as is_latest=True:

class Meta:
    constraints = [
        models.UniqueConstraint(
            fields=["workflow"], 
            condition=Q(is_latest=True), 
            name="unique_latest_workflowversion"
        ),
    ]

Release Prerequisites

Before creating a new version:

  1. Previous version must be committed - Cannot have multiple uncommitted versions
  2. Changes must exist - Git repository must have modifications to commit
  3. Files must be present - At least one WorkflowFile must exist

Automatic Version Numbering

When no name is provided, versions are automatically numbered:

if is_new and not self.name:
    self.name = f"Version {self.index}"

The index field tracks version order within a workflow.

Git Integration

Commit Process

When a WorkflowVersion is released via the /commit/ endpoint:

  1. Validation: Check for uncommitted state and file changes
  2. Git Commit: Create commit with all WorkflowFiles
  3. Hash Storage: Store commit SHA in commit_hash field
  4. Immutability: Version becomes read-only
@action(detail=True, methods=["put"])
def commit(self, request, *args, **kwargs):
    workflowversion = self.get_object()
    if workflowversion.commit_hash:
        raise ValidationError("Already committed")

    commit_hash = workflow.repository.commit(commit_msg)
    workflowversion.commit_hash = commit_hash
    workflowversion.save()

Commit Metadata

Each commit includes:

  • Commit Message: Descriptive message (default: "Commit version '{version_name}'")
  • Timestamp: Automatic Git timestamp
  • Author: LabID system user
  • File Changes: All WorkflowFiles staged and committed

Content Retrieval

File content is retrieved differently based on version state:

@cached_property
def content(self) -> str:
    if self.workflowversion.commit_hash:
        # Released version: use Git commit
        return self._git_repo.git.show(f"{commit_hash}:{file_path}")
    else:
        # Development version: use current file
        return self.get_absolute_file_path().read_text()

Version Creation Patterns

Manual Version Creation

Users create new versions through the UI:

  1. Release Current: Commit the current development version
  2. Create New: System automatically creates new development version
  3. File Inheritance: New version inherits file structure from previous

Automatic Version Creation (Imported Workflows)

For workflows imported from Git repositories:

def sync_remote(self, versions: list[dict[str, str]] = None):
    """Sync workflow with remote repository and create versions for new commits"""
    commit_list = self.repository.get_commits(latest_known_commit_hash)

    for commit_hash, version in versions_dict.items():
        if not self.workflowversions.filter(commit_hash=commit_hash).exists():
            wfv = self.create_new_version(commit_hash, name=version["name"])

Version Naming Strategies

  1. Automatic Numbering: "Version 1", "Version 2", etc.
  2. Git Tags: "Version v1.0.0 (abc1234)" for tagged commits
  3. Commit-based: "Version abc1234" for specific commits
  4. Custom Names: User-defined descriptive names

File Inheritance and Detection

File Inheritance (Manual Workflows)

When creating new versions for manual workflows:

def detect_and_set_workflowfiles(self, previous_wfv: WorkflowVersion = None):
    """Set up files from previous version if available and not yet committed"""
    if not self.workflow.repository_is_remote and not self.commit_hash:
        # Inherit files from previous version
        commit_hash = previous_wfv.commit_hash

File Detection (Imported Workflows)

For imported workflows, files are detected from repository content:

repository_files = self.workflow.repository.list_content(commit_hash, files_only=True)
wf_type, wf_specific_files = get_workflow_type_and_files(self.workflow.repository, commit_hash)

Version Constraints and Validation

Commit Validation

# Cannot commit already committed version
if workflowversion.commit_hash:
    raise ValidationError("This version has already been committed")

# Must have changes to commit
if not workflow.repository.has_changes():
    raise ValidationError("No file changes to commit")

Creation Validation

# Previous version must be committed before creating new one
if previous_version and not previous_version.is_committed:
    raise ValidationError("First commit the previous version before creating a new one")

File Operation Validation

# Cannot modify files in committed versions
if workflowversion.is_committed:
    raise ValidationError("This version has already been committed to the repository")

API Endpoints and Operations

Version Release

PUT /api/v2/workflows/workflowversions/{id}/commit/
Content-Type: application/json

{
  "message": "Release version with new features"
}

Version Creation

New versions are created automatically when:

  1. Previous version is committed
  2. User uploads files to a workflow with only committed versions
  3. Imported workflow is synced with new commits

Version Export

GET /api/v2/workflows/workflowversions/{id}/export/?type=full

Only committed versions can be exported as RO-Crates.

UI Integration

Version Status Indicators

The UI displays version status through computed properties:

const isReleased = computed(
  () => workflowVersion.value?.commit_hash?.value !== ''
)
const isLatest = computed(
  () => workflowVersion.value?.is_latest?.value === true
)

Release Process

async function releaseWorkflowVersion() {
  const releaseUrl = `workflows/workflowversions/${versionId}/commit/`
  const response = await put(releaseUrl)
  // Handle success/error and reload version data
}

Imported Workflow Versioning

Import with Specific Versions

When importing workflows from remote repositories, users can select specific commits/tags:

{
  "versions": [
    {
      "commit_hash": "abc123...",
      "name": "v1.0.0",
      "files": [...]
    },
    {
      "commit_hash": "def456...",
      "name": "Latest",
      "files": [...]
    }
  ]
}

Sync Operations

Existing imported workflows can sync new commits:

PUT /api/v2/workflows/workflows/{id}/sync-from-remote/

This creates new WorkflowVersions for selected remote commits.

Best Practices

Version Management

  1. Regular Releases: Commit versions at logical milestones
  2. Descriptive Names: Use meaningful version names
  3. File Organization: Organize files before committing
  4. Testing: Validate workflows before release

Development Workflow

  1. Create Version: Start with new development version
  2. Upload Files: Add and organize workflow files
  3. Iterate: Modify files as needed
  4. Release: Commit when ready for production use
  5. Repeat: Create new version for next iteration

Imported Workflow Management

  1. Selective Import: Choose specific tags/commits to import
  2. Regular Sync: Keep workflows updated with upstream changes
  3. Version Tracking: Monitor which commits are imported
  4. Conflict Resolution: Handle divergent local/remote changes

Troubleshooting

Common Issues

  1. "Already committed": Attempting to modify released version
  2. "No changes to commit": No file modifications since last commit
  3. "Previous version not committed": Multiple uncommitted versions
  4. "No main file": Missing required MAIN file for publishing

Resolution Strategies

  1. Check Version State: Verify if version is committed
  2. Review File Changes: Ensure modifications exist
  3. Commit Previous: Release previous version first
  4. Add Required Files: Ensure MAIN file is designated

Debugging Version Issues

# Check version state
workflowversion.is_committed  # True if committed
workflowversion.commit_hash   # Git commit hash or empty
workflowversion.is_latest     # True if current development version

# Check repository state
workflow.repository.has_changes()  # True if uncommitted changes exist
workflow.repository.latest_commit  # Latest commit hash

Integration with Publishing

Publishing Requirements

  • Committed Version: Only released versions can be published
  • MAIN File: Must have exactly one MAIN file
  • Complete Metadata: Version must have proper metadata

RO-Crate Generation

Released versions can be exported as Research Object Crates:

  • Full Export: Complete workflow with all files
  • Minimal Export: Essential files only
  • Metadata Preservation: Version information included

WorkflowHub Integration

When publishing to WorkflowHub:

  • Version commit hash ensures reproducibility
  • Git history provides provenance
  • Immutable versions guarantee consistency