fpart

Fpart is a high-performance filesystem partitioner and its companion tool Fpsync parallelizes data migration tasks using it.

Description

Fpart is a filesystem partitioner written in C and licensed under BSD. It efficiently sorts file trees and packages them into partitions, or "bags." It features a blazing-fast crawling mechanism and allows for partition generation based on various criteria like file number or size. Fpart offers a live mode with hooks for immediate actions on file lists and supports generating partitions from arbitrary input. Its companion tool, Fpsync, leverages Fpart to parallelize rsync, cpio, or tar jobs, enabling efficient and distributed data migration across SSH clusters. Fpsync provides features like transfer status tracking, resume and replay capabilities, and low memory footprint.

Features

Fpart's key features include blazing fast crawling, partition generation based on number, file number, or size, live mode operation with hooks, and support for arbitrary input. Fpsync parallelizes rsync, cpio, or tar jobs using Fpart, supports SSH cluster usage for distributed data migration, starts transfers during the crawl, allows remote target URLs with rsync, parallelizes deletion passes, and provides transfer status, resume, and replay functionality. Both tools boast low memory usage and minimal external dependencies.

Benefits

Fpart and Fpsync offer significant time savings for large file operations. The parallelization capabilities of Fpsync greatly reduce the time required for data migration and backups. Fpart's speed and efficiency in file tree processing make it useful for many system administration tasks. The live mode allows for custom scripting and automation. The support for various input types allows it to work with many other system tools. The low memory footprint of Fpsync is particularly beneficial for systems with limited resources. The project is open source, actively maintained, and benefits from a supportive community.

Links

• Home: https://www.fpart.org/