Mixing
ZFS: re-compress existing files after change in compression algorithm
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I have a pool that was created in 2011, using lzjb compression, and it wasn't until a couple of years later that an upgrade allowed me to set the compression to lz4. I estimate that at least 20% of the content (by space) on the array was created prior to 2013, which means it's still compressed using lzjb.
I can think of a couple of options to fix this and regain (some) space:
Back up and restore to a new pool. Not really practical, as I do not have sufficient redundant storage to hold the temporary copy. The restore would also require the pool to be offline for several hours.
Write a script to re-copy any file with a timestamp older than 2013. Potentially risky, especially if it chokes on spaces or other special characters and ends up mangling the original name.
So... is there some way to get ZFS to recompress any legacy blocks using the current compression algorithm? Kind of like a scrub, but healing the compression.
A related question: is there some way to see the usage of each type of compression algorithm? zdb just shows overall compression stats, rather than breaking them down into individual algorithms.
Thanks.
zfs
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |Â
up vote
1
down vote
favorite
I have a pool that was created in 2011, using lzjb compression, and it wasn't until a couple of years later that an upgrade allowed me to set the compression to lz4. I estimate that at least 20% of the content (by space) on the array was created prior to 2013, which means it's still compressed using lzjb.
I can think of a couple of options to fix this and regain (some) space:
Back up and restore to a new pool. Not really practical, as I do not have sufficient redundant storage to hold the temporary copy. The restore would also require the pool to be offline for several hours.
Write a script to re-copy any file with a timestamp older than 2013. Potentially risky, especially if it chokes on spaces or other special characters and ends up mangling the original name.
So... is there some way to get ZFS to recompress any legacy blocks using the current compression algorithm? Kind of like a scrub, but healing the compression.
A related question: is there some way to see the usage of each type of compression algorithm? zdb just shows overall compression stats, rather than breaking them down into individual algorithms.
Thanks.
zfs
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
2
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have a pool that was created in 2011, using lzjb compression, and it wasn't until a couple of years later that an upgrade allowed me to set the compression to lz4. I estimate that at least 20% of the content (by space) on the array was created prior to 2013, which means it's still compressed using lzjb.
I can think of a couple of options to fix this and regain (some) space:
Back up and restore to a new pool. Not really practical, as I do not have sufficient redundant storage to hold the temporary copy. The restore would also require the pool to be offline for several hours.
Write a script to re-copy any file with a timestamp older than 2013. Potentially risky, especially if it chokes on spaces or other special characters and ends up mangling the original name.
So... is there some way to get ZFS to recompress any legacy blocks using the current compression algorithm? Kind of like a scrub, but healing the compression.
A related question: is there some way to see the usage of each type of compression algorithm? zdb just shows overall compression stats, rather than breaking them down into individual algorithms.
Thanks.
zfs
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I have a pool that was created in 2011, using lzjb compression, and it wasn't until a couple of years later that an upgrade allowed me to set the compression to lz4. I estimate that at least 20% of the content (by space) on the array was created prior to 2013, which means it's still compressed using lzjb.
I can think of a couple of options to fix this and regain (some) space:
Back up and restore to a new pool. Not really practical, as I do not have sufficient redundant storage to hold the temporary copy. The restore would also require the pool to be offline for several hours.
Write a script to re-copy any file with a timestamp older than 2013. Potentially risky, especially if it chokes on spaces or other special characters and ends up mangling the original name.
So... is there some way to get ZFS to recompress any legacy blocks using the current compression algorithm? Kind of like a scrub, but healing the compression.
A related question: is there some way to see the usage of each type of compression algorithm? zdb just shows overall compression stats, rather than breaking them down into individual algorithms.
Thanks.
zfs
zfs
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 3 hours ago
asked 3 hours ago
rowan194
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
rowan194
62
asked 3 hours ago
rowan194
62
62
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
rowan194 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
2
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago
add a comment |Â
2
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago
2
2
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
You've have to recopy the data (full or partial) or zfs send/receive the data to a new pool or ZFS filesystem.
There aren't any other options.
answered 3 hours ago
ewwhite
171k73357706
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
You've have to recopy the data (full or partial) or zfs send/receive the data to a new pool or ZFS filesystem.
There aren't any other options.
answered 3 hours ago
ewwhite
171k73357706
add a comment |Â
up vote
2
down vote
You've have to recopy the data (full or partial) or zfs send/receive the data to a new pool or ZFS filesystem.
There aren't any other options.
answered 3 hours ago
ewwhite
171k73357706
add a comment |Â
up vote
2
down vote
up vote
2
down vote
You've have to recopy the data (full or partial) or zfs send/receive the data to a new pool or ZFS filesystem.
There aren't any other options.
answered 3 hours ago
ewwhite
171k73357706
You've have to recopy the data (full or partial) or zfs send/receive the data to a new pool or ZFS filesystem.
There aren't any other options.
answered 3 hours ago
ewwhite
171k73357706
answered 3 hours ago
ewwhite
171k73357706
answered 3 hours ago
ewwhite
171k73357706
answered 3 hours ago
ewwhite
171k73357706
171k73357706
add a comment |Â
add a comment |Â
rowan194 is a new contributor. Be nice, and check out our Code of Conduct.
rowan194 is a new contributor. Be nice, and check out our Code of Conduct.
rowan194 is a new contributor. Be nice, and check out our Code of Conduct.
rowan194 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f933387%2fzfs-re-compress-existing-files-after-change-in-compression-algorithm%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
2
I'm pretty sure you named the only two options. See also the discussion in issue 3013 for why this functionality doesn't exist and you might not want to do this at all.
– Michael Hampton♦
3 hours ago