Skip to content

Add backup file extraction feature - #13935

Open
JoaoJandre wants to merge 3 commits into
apache:mainfrom
scclouds:add-kboss-file-extraction
Open

Add backup file extraction feature#13935
JoaoJandre wants to merge 3 commits into
apache:mainfrom
scclouds:add-kboss-file-extraction

Conversation

@JoaoJandre

Copy link
Copy Markdown
Contributor

Description

This PR adds backup file extraction feature. Currently only supported by KBOSS.

This feature must be enabled in the backup offering through the allowextractfile parameter. If it is set to true, it will be possible to extract files from backups created with that offering.

The listBackupFilesystems API was added to allow listing file systems inside backup volumes:

Parameter Description Required?
backupId ID of the backup for which the file systems should be listed Yes
volumeId ID of the volume belonging to the backup for which the file systems should be listed. If not provided, the file systems of all backup volumes will be listed No
Example execution of the listBackupFilesystems API
(labinterno) 🐱 > list backupfilesystems backupid=ecb15182-d6a8-4044-8251-6d437b27b971 
{
  "count": 1,
  "datastoreobject": [
    {
      "format": "ext4",
      "isdirectory": false,
      "isfilesystem": true,
      "name": "/dev/sda1",
      "size": 0,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    }
  ]
}

The listBackupFiles API was added to allow listing files inside backup volumes:

Parameter Description Required?
backupId ID of the backup for which the files should be listed Yes
volumeId ID of the volume belonging to the backup for which the files should be listed Yes
filesystem Name of the file system that should be mounted in order to list the files Yes
path Path to be listed inside the mounted file system Yes
issymlink If the directory is a symbolic link to another directory. If not informed, ACS will check it No

The filesystem and path parameters expect that the files do not contain special characters such as (, |, or >, for example. Both are expected to receive values in the format /path/to/something.

Example execution of the listBackupFiles API
(labinterno) 🐱 > list backupfiles backupid=ecb15182-d6a8-4044-8251-6d437b27b971 volumeid=4af9cee2-6014-44fa-ac86-09a649b67363 filesystem=/dev/sda1 path=/tmp
{
  "count": 8,
  "datastoreobject": [
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".ICE-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".Test-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".X11-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".XIM-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": ".font-unix",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:45+0000",
      "name": "netplan_a2jbieem",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:42+0000",
      "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    },
    {
      "isdirectory": true,
      "lastupdated": "2026-05-26T13:26:39+0000",
      "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-timesyncd.service-3aOZJF",
      "size": 4096,
      "volumeid": "4af9cee2-6014-44fa-ac86-09a649b67363",
      "volumename": "ROOT-198"
    }
  ]
}

To download files, the downloadBackupFile API was added:

Parameter Description Required?
backupId ID of the backup from which the file should be downloaded Yes
volumeId ID of the volume belonging to the backup from which the file should be downloaded Yes
filesystem Name of the file system that should be mounted in order to download the file Yes
path Path to be downloaded inside the mounted filesystem. If it is a file, it will be compressed using gzip; if it is a directory, it will be added to a tar archive and compressed using gzip Yes

The filesystem and path parameters expect that the files do not contain special characters such as (, |, or >, for example. Both are expected to receive values in the format /path/to/something.

Example execution of the downloadBackupFile API
(labinterno) 🐱 > download backupfile backupid=ecb15182-d6a8-4044-8251-6d437b27b971 volumeid=4af9cee2-6014-44fa-ac86-09a649b67363 filesystem=/dev/sda1 path=/tmp/systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz
{
  "downloadbackupfileresponse": {
    "name": "systemd-private-5f39ee6fdaa3434caa2eb42ad39db34f-systemd-resolved.service-jLhWCz",
    "state": "DOWNLOAD_URL_CREATED",
    "url": "http://192.168.123.142/userdata/215a45db-70e1-4649-93cb-258abe572573/5967473b-29b1-4057-85a3-cd924a458033.gz"
  }
}

The list and download file workflow was also added to the GUI.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

image

How Has This Been Tested?

Test descriptions

Test Result
List filesystems for a volume containing filesystems Filesystems returned
List filesystems for a volume without filesystems Empty list returned
List filesystems for a backup without specifying the volume List of all filesystems across all backup volumes returned
List files in a standard directory List of files returned, with correct file types and sizes
List files in a symlink to a directory, specifying the isSymlink=true flag List of files returned, with correct file types and sizes
List files in a symlink to a directory, without specifying the isSymlink flag List of files returned, with correct file types and sizes
List files in an empty directory Empty list returned
List files in a directory containing symlinks List of files returned, including the symlink file, with its canonical path information
List files in a directory containing a symlink pointing to a non-existent path List of files returned, excluding the symlink file
List files in a non-existent directory Error (expected)
List files in a symlink pointing to another symlink Error (expected)
Download a standard file File downloaded successfully
Download a standard directory File downloaded successfully
Download a symlink pointing to a file File downloaded successfully
Download a symlink pointing to a directory File downloaded successfully
Download a symlink pointing to another symlink Error (expected)

All the tests above were executed for both Linux ROOT volumes (ext4) and Windows ROOT volumes (NTFS).

@weizhouapache weizhouapache added this to the 4.24.0 milestone Aug 20, 2026
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 23.87173% with 641 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.66%. Comparing base (64178ea) to head (4f86194).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
.../apache/cloudstack/backup/KbossBackupProvider.java 0.00% 104 Missing ⚠️
...rg/apache/cloudstack/backup/BackupManagerImpl.java 0.00% 73 Missing and 1 partial ⚠️
.../apache/cloudstack/utils/qemu/GuestfishClient.java 73.45% 52 Missing and 8 partials ⚠️
.../apache/cloudstack/backup/to/BackupFileObject.java 0.00% 50 Missing ⚠️
...he/cloudstack/backup/ExtractBackupFileCommand.java 0.00% 43 Missing ⚠️
...api/command/user/backup/DownloadBackupFileCmd.java 0.00% 34 Missing ⚠️
...org/apache/cloudstack/backup/ListFilesCommand.java 0.00% 34 Missing ⚠️
...ck/api/command/user/backup/ListBackupFilesCmd.java 0.00% 30 Missing ⚠️
...apper/LibvirtDownloadBackupFileCommandWrapper.java 3.70% 26 Missing ⚠️
.../command/user/backup/ListBackupFilesystemsCmd.java 0.00% 21 Missing ⚠️
... and 13 more
Additional details and impacted files
@@            Coverage Diff             @@
##               main   #13935    +/-   ##
==========================================
  Coverage     19.65%   19.66%            
- Complexity    19798    19833    +35     
==========================================
  Files          6368     6383    +15     
  Lines        574943   575857   +914     
  Branches      70355    70444    +89     
==========================================
+ Hits         113015   113230   +215     
- Misses       449658   450336   +678     
- Partials      12270    12291    +21     
Flag Coverage Δ
uitests 3.40% <ø> (-0.01%) ⬇️
unittests 20.94% <23.87%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JoaoJandre

Copy link
Copy Markdown
Contributor Author

It would be nice if someone could test this using RPM, since I have not tested installing the RPM packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants