Revisions of benji

Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 1006021 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 16)
- Update to 0.17.0:
  * Support fsfreeze feature on Kubernetes 1.24+ (requires the updated Helm chart)
  * Support for storage classes in S3
  * Small change to get Benji to work with MySQL/MariaDB
  * k8s-tools: Switch logging to structlog and JSON output
  * k8s-tools: Pass through JSON logging from benji unaltered
  * k8s-tools: Create PVC with default storage class if nothing is specified
  * k8s-tools: Put an upper bound on the time we wait for PV creation
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 971237 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 15)
- Update to 0.16.1:
  * Fix typo in PersistentVolume key name (#140)
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 969341 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 14)
- Update to 0.16.0:
  * ACTION REQUIRED: The Helm chart has seen some changes to default values
    which might break some setups. Especially the default Ceph credential
    volume mounts are no longer there. In addition, the dependant charts
    have been updated and the PostgreSQL chart has changed significantly
    and some keys and values will need adjustment if PostgreSQL is
    deployed via Benji Helm chart.
  * Support for RADOS/RBD namespaces has been added to both the rbd and the
    rbdaio io modules and the Kubernetes integration.
  * The fsfreeze feature works with Ceph CSI provisioned RBD volumes (includes
    volumes provisioned by Rook).
  * There is a new filtering operator like which mimics its SQL counterpart.
  * Pushgateway grouping keys based on the Kubernetes namespace and PVC name
    are used to separate the backup status of different PVCs.
  * Deep-scrubbing with source comparison also checks sparse blocks. This
    will detect blocks that are sparse in the backup and not in the source.
  * A bug preventing restores of large images to succeed due using excessive
    amounts of RAM when using the rbdaio io module has been fixed.
  * Container images are based on AlmaLinux 8 and Ceph Pacific.
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 896938 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 13)
- Update to 0.15.0:
  * Helm chart: Migrate Helm chart to version 2 of the chart
    format (only compatible with Helm 3)
  * Helm chart: Add option to set activeDeadlineSeconds for
    cronjobs (benji.cronJob.activeDeadlineSeconds) (#108)
  * Update container images to fix CVE-2021-20288 in the
    Ceph client code
  * Revalidate invalid blocks during deep-scrubbing (#105)
Richard Brown's avatar Richard Brown (RBrownSUSE) accepted request 879120 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 12)
- Update to 0.14.1:
  * Fixed wrong container image repository reference in Helm chart

- Update to 0.14.0:
  * ACTION REQUIRED: New container images are only published to the
    GitHub container registry. The new image locations are:
    - ghcr.io/elemental-lf/benji
    - ghcr.io/elemental-lf/benji-k8s
    The image tag latest now points to the latest released version
    and not the current master branch. To get the container image
    for the master branch use the tag master as with all other branches.
  * Add options to specify number of retries and timeouts to the S3
    storage module (#100, #102, #103)
  * Switch to improved retry logic of the boto3 library in the S3
    storage module (#103, fixes #101)
  * Fix a bug where Benji would not error out if the base version's
    storage did not match the storage that would've been taken if
    no base version was specified (fixes #105)
  * Use proper fadvise flags on a Ceph RBD read (don't cache us
    but cache other clients)
  * Migrate b2 storage module to b2sdk (closes #46)
  * Kubernetes integration:
    - Add support for Ceph CSI in both backup and restore
      scripts (#106, #104, #107)
    - Add option to restore to a specific storage class (#104, #107)
    - Add option to template Benji restore URLs (#104, #107)
    - Refactor: Move K8s helper functions to benji-k8s-tools
      package to facilitate code sharing
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 851381 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 11)
- Update to 0.13.0:
  * Fix Helm chart repo URLs because of Helm chart repostitory
    deprecation (see https://helm.sh/blog/charts-repo-deprecation)
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 835931 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 10)
- Update to 0.12.0:
  * Fix type conversion bug in benji storage-usage command (issue #96)

- Update to 0.11.0:
  * Fix SQL generation for benji storage-usage command (issue #95)
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 832695 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 9)
- Update to 0.10.0:
  * Helm chart changes:
    - Change chart's requirements to use URL based repository references.
      This should help when deploying Benji via FluxCD's helm-operator (#89)
    - Fix rendering error when specifying a nodeSelector, node
      affinities or tolerations. (#90)
    - Use API group rbac.authorization.k8s.io/v1 for RBAC related resources
  * Add new transform module aes_256_gcm_ecc (#86)
  * Add support for discovering RBD images provisioned by Ceph's CSI
    provisioner to the benji-backup-pvc script (#91)
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 823070 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 8)
- Update to 0.9.0:
  * Sparse blocks are no longer explicitly represented in the database.
    This greatly speeds up the initial step of backup creation if the
    backup is based on an older version, and a lot of the blocks are
    sparse. It also reduces database size and speeds up database
    operations overall due to the reduced number of rows in the blocks
    table. No database migration is necessary, but the changed
    representation will only apply to newly created versions.
  * Implement benji storage-usage command. See the documentation
    on how the usage is calculated and its inherent limitations.
  * For the rbd and rbdaio IO modules add a feature to pass all Ceph
    credentials as part of the URL of the benji backup command. This
    can also be used to pass other Ceph configuration options, and it
    is also possible to override the client_identifier set in Benji's
    configuration file. This allows Benji to connect to any Ceph cluster
    without setting up /etc/ceph/ceph.conf beforehand.
    See the documentation for the full URL syntax.
  * benji enforce now skips removed or otherwise unavailable versions
    and continues with the removal.
  * NBD server: The block cache is split up into a block cache and a
    copy-on-write store for changed blocks. For the block cache a maximum
    size can be specified and least frequently used blocks are evicted if
    the cache gets full. Some adjustments are needed to Benji's
    configuration due to this change. See etc/benji.yaml for an example.
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 749872 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 7)
- Update to 0.8.0:
  * This release contains significant changes related to the naming, format and
    structure of internal and external data representations. They derive from the
    experience of using Benji in the last few months and from the challenges
    uncovered by the ongoing Kubernetes integration efforts. The changes have been
    bundled to avoid multiple metadata version changes and migrations.
  * Old metadata backups and exports and old object metadata can still be read by
    this version of Benji. Existing databases can be migrated to the new database
    structure with benji database-migrate. While this process has been tested with
    both PostgreSQL and SQLite it is strongly recommended to make a consistent
    backup of the database before attempting the migration. The migration process
    requires a significant amount of time and disk space when there are a lot of old
    backups in the database. The versions and blocks tables are completely recreated
    and the old data is moved over. Expect the disk usage to more than double during
    the migration.
  
  Database and metadata changes:
  * The version of metadata exports has changed from 1.1.0 to 2.0.0. Old
    exports (1.0.0 and 1.1.0) can still be imported.
  * snapshot_name in the versions table has been renamed to snapshot in the
    database and in metadata exports. The long version of the corresponding command
    line option has also been renamed from --snapshot-name to --snapshot.
  * name in the versions table has been renamed to volume in the database and in metadata exports.
  * bytes_dedup in the versions table has been renamed to bytes_deduplicated in the database and in metadata exports.
  * id in the blocks table has been renamed to idx in the database and in metadata exports.
  * The type of uid in the versions table has been changed from integer to
    string. This also affects any metadata exports. This removes the inconsistency
    where uid was represented as a string in some places and as an integer in
    others. uids are automatically generated for new versions, but there is also
    the option to set the uid of a version on backup via the new -u/--uid option.
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 727170 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 5)
- Update to 0.7.1:
  * This release pins two package dependencies to older versions
    as newer releases of these dependencies broke Benji.
  * Fixes #49.
  * Fixes #51.
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 719473 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 4)
- Update to 0.7.0:
  * Added a new I/O module rbdaio which uses the asynchronous API of librbd.
    Performance results in relation to rbd have been mixed but performance
	should be at least 10-20% higher on restore. In one case performance has
	been increased tenfold.
  * Almost all Bash helper scripts have been rewritten in Python.
    The new scripts are calling Benji via the command line just like before.
	This is intentional to minimize the interdependence between Benji and these
	helpers. The scripts are examples only and not part of the API.
	There still is one example Bash script at scripts/ceph.sh to show how to
	interact with Benji via Bash. The helpers have additional dependencies
	which can be installed with pip install benji[helpers].
  * The Prometheus metrics exported by benji-k8s have changed:
  * Backup metrics now longer include the auxiliary_data label.
  * Command metrics now longer include the arguments label. The arguments
    have been folded into the command label.
  * benji-k8s: The included scripts have been replaced by Python scripts and
    are using the new helper modules. They should be calling compatible.
  * benji-k8s: All calls to kubectl have been replaced with direct API requests.
    The official Python client for Kubernetes is used. kubectl is still included
	in the image.
  * Helm chart: Volumes and volume mounts are now configurable via values.yaml.
    This is mostly for getting the Ceph credentials into the container but could
	also be used to mount file-based storage.
  * Helm chart: The PostgreSQL chart dependency was updated from 2.7.6 to 4.2.2.
    This is the last chart which uses PostgreSQL 10 and requires no upgrade of the
	database data structures.
  * benji-k8s and Helm chart: The image was simplified to only include the
    Kubernetes specific scripts and kubectl. Instead of running backups or other
	jobs via crond inside the container, the Helm chart now generates separate
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 704864 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 3)
- Update to 0.6.0:
  * URL parsing of I/O resources is now conforming to standards.
    Especially for the RBD I/O module the two slashes directly after
    the colon are no longer valid and have to be removed
    (rbd://pool/image -> rbd:pool/image).
  * Added I/O module for iSCSI. It is based on libiscsi and requires
    no elevated permissions. Please see the documentation as Benji
    requires a special version of the libiscsi Python bindings.
    The module is single-threaded and synchronous, so performance
    will be limited. Contributions are welcome!
  * The algorithm used by benji enforce has seen an overhaul and
    should be more comprehensible as the time categories are based
    on natural time boundaries (start of the hour, day, week,
    month, and year) now.
  * Added a restore helper script (images/benji-k8s/scripts/benji-restore-pvc)
    for Kubernetes. This script is intended to be run on a management
    system with access to the Kubernetes cluster and can restore
    a version into a new or an existing PVC/PV pair.
  * The container images are now based on the Python 3.6 included in EPEL.
    The RBD support has been updated to Ceph Nautilus. Nautilus also
    added RADOS and RBD Python bindings for Python 3.6 which are now
    used instead of building them themselves.
- Add requires on python3-diskcache

- Update to 0.5.0:
  * Added fsfreeze support to the benji-k8s Docker image. Just add
    the benji-backup.me/fsfreeze: yes annotation to the PVC.
    Kubernetes hosts are accessed via pods which are deployed by
    a DaemonSet, see the Helm chart for details.
  * Use bulk inserts to speed up backups of images based on a
Dominique Leuenberger's avatar Dominique Leuenberger (dimstar_suse) accepted request 687136 from Michael Vetter's avatar Michael Vetter (jubalh) (revision 2)
- Update to 0.4.0:
  * Documentation updates
  * Added new CLI command benji storage-stats to get storage usage information
  * Fixed backup progress reporting on console
  * Added fdatasync() calls to the file module to ensure backup integrity
    in case of a system crash or a power failure (will impact performance when using the file I/O module)
  * Speed up command line completion
  * Added benji completion CLI command
Displaying all 16 revisions
openSUSE Build Service is sponsored by