Skip to content

25.12.0 - Data Management Enhancements

This release introduces single-folder datafile expansion for simplified building of multidatafile datasets, enhanced S3 storage support with HTTPS URI form and MinIO configuration updates, and the ability to remove annotations from multiple items at once. It also includes bug fixes and interface improvements.

Datasets & Storage

  • Single-folder datafile expansion: Users can now expand folders in the dataset builder to individually register contained files as datafiles, as an alternative to using the folder itself as a single dataset. This is particularly useful to register all image files contained in a folder as individual datafiles of a unique dataset. This feature is powered by recursive directory listing for Dropbox storage volumes.
  • Enhanced S3 support: Added HTTPS URI form support (e.g., https://domain.org/bucketname/path/to/object), MinIO configuration updates, and endpoint_url in storage volume constraints.
  • Batch export: Users can now export Datafile and DatafileCopy records in bulk.
  • New supported formats: Added support for shell scripts (.sh, .bash, .zsh) and BMP images as attachments.
  • Storage volume selection: Fixed identification of non-primary storage volumes during data registration.
  • Broken symlinks: Broken symlinks encountered during data registration now raise clear error messages instead of causing unexpected failures.
  • Permission handling: Chmod errors on directories during datafile protection/un-protection now raise explicit errors.

Assays & Run Directories

  • Safer assay deletion: When deleting an assay with "Remove Datafiles From Disk", the system now checks if other assays share the same run directory to prevent accidental data loss.
  • Run directory validation: Comprehensive validation rules based on assay state ensure run directories exist on the filesystem before state transitions to INCOMING, INITIALIZED, or REGISTERED.
  • State-specific run_dir rules: Template assays cannot have a run directory set; new assays can set one initially but cannot change it once set; initialized and registered assays block run_dir changes except for admins.
  • Run directory conflict detection: The system now validates paths to prevent nested run directories within the same storage volume.
  • AVITI assay support: Paired-end datasets are now correctly marked with the 'PAIRED-END' type upon data-registration.
  • Illumina header parsing: Improved regex to support additional sequencer formats e.g. AVITI.
  • Instrument validation: Fixed validation in Generic Assay forms. Instruments now correctly load when they have a defined technology value (non-NA and non-empty) for generic assays.

Workflow Run Management

  • RO-Crate enhancements: Exported items now include LabID UUIDs, and datasets are represented as collection entities in RO-Crates. c8e260c0
  • Galaxy integration: Normalized Galaxy tags for regex matching, and improved workflow invocation annotation display.
  • Workflow import improvements: Enhanced search to filter by version and tag, improved validation and interactivity of the import page and fixed orphan commit handling when importing workflows from git
  • Search and filtering: Fixed main search bar functionality and filtering by workflow/workflow version in the Workflow Run list page.
  • Ingestion optimization: The system now prevents starting an ingestion task if no new datasets need to be created. fe52a384
  • Data type handling: Fixed an issue where workflow data types were incorrectly set as empty lists.

User Interface Improvements

  • File preview indicator: Files now display a preview icon when previewable (non-directory, valid size, under 50MB).
  • Batch annotation removal: Annotations can now be removed from multiple items at once, across datasets, samples, and other models.
  • UUID fields as hyperlinks: UUIDs throughout the application (notes, string fields, notifications) are now automatically converted to clickable links navigating to detail pages.
  • Unified deleted items display: Improved visual display of deleted items across the application, with trash icon and color state indication.
  • Easier access to location details: Locations are now displayed as clickable links to easily access location details.
  • A new 'Datafile Copies' entry has been added to the 'Dataset' menu. The Datafile Copies entry has been added to the left navigation menu (it is still available in the ellipsis menu of the top navigation bar), making the concept of data file copies more prominent.
  • Freeze button: Resolved display issues in certain contexts where the freeze button was not accessible while it should have (e.g. re-freezing).
  • Revamped Landing page: the landing page has been redesigned and updated to better reflect LabID's capabilities.

CLI Improvements

  • New --dry-run option added to labid register wfrun allowing to simulate the registration without actually performing it.
  • New --type option added to labid get sniffers to filter sniffers by type
  • Added feedback animation to labid register assays notifying users that work is ongoing (hide by passing --no-animation, useful for CI or cron jobs).
  • Added a new sniffer for workflow run crates allowing to register a workflow run provided as a WF Run RO-Crate
  • Fix issues fetching and registering datasets having parent datasets (as part of a run registration)
  • A table mapping input files to existing datasets (mapping file path to dataset_uuid) can now be passed to labid register wfrun
  • New options added to workflow run sniffers allowing to create parent-children relationships between e.g. input and output datasets based on dataset names or file/directory names. For example, when child dataset are named after their parent likw {parent}.suffix.
  • Updated pydantic dependency from 1.10.7 to 2.11.9 f3277688

API & Performance

  • Query optimization: Dataset counting now uses prefetching to reduce N+1 queries.
  • Request timeouts: Added configuration with graceful ScanCode import handling.
  • Experiment polling: Optimized polling in LabNoteDetailView to reduce API load.

Infrastructure & Deployment

  • UI package management: Download prebuilt UI package as part of start.sh with flag to update the package.
  • Docker improvements: Fixed storage volume mounting in docker-compose to use named volumes instead of bind mounts, preventing ownership and permission issues.
  • Dependencies: Fixed scancode-toolkit version.
  • Email notifications: Updated environment settings.

Bug Fixes

  • Experiments can now reference deleted projects in list view
  • Fixed experiment fetching with deleted=false to handle deleted project contexts
  • Fixed duplicate annotation loading after sample cloning
  • Fixed select dropdown option display with inconsistent label properties
  • Fixed string view handling of object values with choice_label property
  • Fixed location breadcrumb type returning 'UNDEFINED'
  • Fixed imaging-channel creation modal validation

Changelog