summaryrefslogtreecommitdiff
path: root/src/chunker.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-10-21chore: fix blemishes found by clippyLars Wirzenius1-2/+1
Signed-off-by: Lars Wirzenius <liw@liw.fi> Sponsored-by: author
2022-04-16feat: use one checksum for all chunks in a backupLars Wirzenius1-3/+13
When making a backup, use the same checksum for any chunks it re-uses or creates. This is for performance: if we allowed two checksums to be used, we would have to compute the checksum for a chunk twice, and potentially look up both on the server. This is just a lot of work. Instead, we use only one. The trade-off here is that when (not if) the user wants to switch to a new checksum type, they'll have to do a full backup, uploading all their data to the server, even when it's already there, just with a different checksum. Hopefully this will be rare. Full backups always use the built-in, hardcoded default checksum, and incremental backups use whatever the previous backup used. The default is still SHA256, but this commit add code to support BLAKE2 if we decide to switch that as a default. It's also easy to add support for others, now. BLAKE2 was added to verify that Obnam can actually handle the checksum changing (manual test: not in the test suite). I don't think users need to be offered even the option of choosing a checksum algorithm to use. When one cares about both security and performance, choosing a checksum requires specialist, expert knowledge. Obnam developers should choose the default. Giving users a knob they can twiddle just makes it that much harder to configure and use Obnam. If the choice Obnam developers have made is shown to be sub-optimal, it seems better to change the default for everyone, rather than hope that every user changes their configuration to gain the benefit. Experience has shown that people mostly don't change the default configuration, and that they are especially bad at choosing well when security is a concern. (Obnam is free software. Expert users can choose their checksum by changing the source code. I'm not fundamentally limiting anyone's freedom or choice here.) Users can switch to a new default algorithm by triggering a full backup with the new "obnam backup --full". Sponsored-by: author
2022-04-16refactor: rename Checksum to LabelLars Wirzenius1-2/+2
Label is a clearer and more accurate name for the type now that it is not just a checksum. Also, serialize a Label in tests, rather than using string literals. This is more correct, and we'll be changing serialization later. Sponsored-by: author
2022-01-16refactor: rename Chunker to FileChunksLars Wirzenius1-3/+3
This should make it a little clearer that it can act as an iterator. Sponsored-by: author
2021-12-31docs: add documentation comments to crateLars Wirzenius1-1/+8
Also, make it an error for a public symbol to not be documented. Sponsored-by: author
2021-09-18refactor: define a Checksum type and use it where appropriateLars Wirzenius1-2/+2
This will make it harder to compare, say, a SHA-256 and a SHA3, later, when we add more checksum types. Sponsored-by: author
2021-07-21Replace ChunkerResult with plain ResultAlexander Batischev1-5/+3
2021-05-29refactor: make metadata be part of datachunkLars Wirzenius1-6/+6
This makes it harder to lose the metadata for a chunk, or to use unrelated metadata and chunk. Also, soon I will refactor things for encrypting chunks, which will need metadata embedded in the encrypted chunk. Sponsored-by: author
2021-04-29feat: improve error messagesLars Wirzenius1-4/+10
All unclear error messages should now be clearer. For example, all the ones related to a file mention the file name and the attempted operation that failed.
2021-02-04refactor: have per-module error enumsLars Wirzenius1-3/+11
This means that a function that parses step bindings can't return an error that the document is missing a title. Such an error return would be nonsensical, and we use the Rust type system to prevent it, at a small cost of being a bit verbose. Additional benefit is that the library portion of Obnam doesn't return anyhow::Result values anymore.
2020-11-09refactor: move sha256 checksumming to its own moduleLars Wirzenius1-6/+2
2020-11-08feat(src/chunker.rs): add abstraction for chunking live dataLars Wirzenius1-0/+60
This is very rudimentary for now