Is there some tool for finding files in one directory but not in the other?

up vote
1
down vote

favorite

I would like to write a bash script to find those files in a directory but not in another directory.

Does the following script work? When does it not?

for i in "$1"/*; do
 f=$(basename $i);
 if [ ! -e "$2"/"$f" ]
 then
 echo $f
 fi
done

I heard diff can find differences between the files in two directories too. Can it solve my problem or not?

Or some other tool?

Thanks.

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

add a commentÂ |Â

up vote
1
down vote

favorite

I would like to write a bash script to find those files in a directory but not in another directory.

Does the following script work? When does it not?

for i in "$1"/*; do
 f=$(basename $i);
 if [ ! -e "$2"/"$f" ]
 then
 echo $f
 fi
done

I heard diff can find differences between the files in two directories too. Can it solve my problem or not?

Or some other tool?

Thanks.

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

add a commentÂ |Â

up vote
1
down vote

favorite

I would like to write a bash script to find those files in a directory but not in another directory.

Does the following script work? When does it not?

for i in "$1"/*; do
 f=$(basename $i);
 if [ ! -e "$2"/"$f" ]
 then
 echo $f
 fi
done

I heard diff can find differences between the files in two directories too. Can it solve my problem or not?

Or some other tool?

Thanks.

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

I would like to write a bash script to find those files in a directory but not in another directory.

Does the following script work? When does it not?

for i in "$1"/*; do
 f=$(basename $i);
 if [ ! -e "$2"/"$f" ]
 then
 echo $f
 fi
done

I heard diff can find differences between the files in two directories too. Can it solve my problem or not?

Or some other tool?

Thanks.

bash diff

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

edited 1 hour ago

asked 2 hours ago

Tim

24.6k69239427

asked 2 hours ago

Tim

24.6k69239427

asked 2 hours ago

Tim

24.6k69239427

add a commentÂ |Â

2 Answers
2

active

oldest

votes

up vote
3
down vote

If your file names don't contain newline characters, you can do:

(export LC_ALL=C; comm -23 <(ls -A dir1) <(ls -A dir2))

to find out which of the files in dir1 are not found in dir2.

For arbitrary file names, you can use the array disjunction feature of zsh:

dir1_files=(dir1/*(DN:t)) dir2_files=(dir2/*(DN:t))
dir1_and_not_dir2_files=($dir1_files:)

Or with bash4.4+ and recent versions of GNU utilities:

readarray -td '' dir1_and_not_dir2_files < <(
 export LC_ALL=C
 shopt -s nullglob dotglob
 comm -z23 <(cd dir1 && printf '%s' *) 
 <(cd dir2 && printf '%s' *)
)

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

1

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

Â |Â
show 2 more comments

up vote
3
down vote

Yes, you can use diff for that purpose. It's as simple as:

diff -rq dir1 dir2

The -r option tells diff to recurse into sub-directories as well. The -q option tells diff to only report when files differ.

These are the two options I typically use when I want to find out which files are in dir1, but not in dir2, and vice versa. (You can also drop the -r parameter if you do not want to recurse into sub-directories, but only consider the immediate contents of the two directories.)

Note that this will show you both files that exist in dir1 but not in dir2, as well as files that exist in dir2, but not in dir1, e.g.:

$ diff -rq /tmp/dir1/ /tmp/dir2/
Only in /tmp/dir1/: file1
Only in /tmp/dir2/: file2
Only in /tmp/dir2/: file3

If you need only one of the directions (e.g., files that are in dir1 but not in dir2) and get a list of the filenames only (without the "Only in ..." clutter), you could certainly attempt to massage diff's output using grep, sed, awk and the like, but in that case you're better off not using diff in the first place and use StÃƒÂ©phane Chazelas's solution instead.

answered 1 hour ago

Malte Skoruppa

1,1171916

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f479947%2fis-there-some-tool-for-finding-files-in-one-directory-but-not-in-the-other%23new-answer', 'question_page');

);

Post as a guest

Name

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

up vote
3
down vote

If your file names don't contain newline characters, you can do:

(export LC_ALL=C; comm -23 <(ls -A dir1) <(ls -A dir2))

to find out which of the files in dir1 are not found in dir2.

For arbitrary file names, you can use the array disjunction feature of zsh:

dir1_files=(dir1/*(DN:t)) dir2_files=(dir2/*(DN:t))
dir1_and_not_dir2_files=($dir1_files:)

Or with bash4.4+ and recent versions of GNU utilities:

readarray -td '' dir1_and_not_dir2_files < <(
 export LC_ALL=C
 shopt -s nullglob dotglob
 comm -z23 <(cd dir1 && printf '%s' *) 
 <(cd dir2 && printf '%s' *)
)

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

1

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

Â |Â
show 2 more comments

up vote
3
down vote

If your file names don't contain newline characters, you can do:

(export LC_ALL=C; comm -23 <(ls -A dir1) <(ls -A dir2))

to find out which of the files in dir1 are not found in dir2.

For arbitrary file names, you can use the array disjunction feature of zsh:

dir1_files=(dir1/*(DN:t)) dir2_files=(dir2/*(DN:t))
dir1_and_not_dir2_files=($dir1_files:)

Or with bash4.4+ and recent versions of GNU utilities:

readarray -td '' dir1_and_not_dir2_files < <(
 export LC_ALL=C
 shopt -s nullglob dotglob
 comm -z23 <(cd dir1 && printf '%s' *) 
 <(cd dir2 && printf '%s' *)
)

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

1

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

Â |Â
show 2 more comments

up vote
3
down vote

If your file names don't contain newline characters, you can do:

(export LC_ALL=C; comm -23 <(ls -A dir1) <(ls -A dir2))

to find out which of the files in dir1 are not found in dir2.

For arbitrary file names, you can use the array disjunction feature of zsh:

dir1_files=(dir1/*(DN:t)) dir2_files=(dir2/*(DN:t))
dir1_and_not_dir2_files=($dir1_files:)

Or with bash4.4+ and recent versions of GNU utilities:

readarray -td '' dir1_and_not_dir2_files < <(
 export LC_ALL=C
 shopt -s nullglob dotglob
 comm -z23 <(cd dir1 && printf '%s' *) 
 <(cd dir2 && printf '%s' *)
)

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

If your file names don't contain newline characters, you can do:

(export LC_ALL=C; comm -23 <(ls -A dir1) <(ls -A dir2))

to find out which of the files in dir1 are not found in dir2.

For arbitrary file names, you can use the array disjunction feature of zsh:

dir1_files=(dir1/*(DN:t)) dir2_files=(dir2/*(DN:t))
dir1_and_not_dir2_files=($dir1_files:)

Or with bash4.4+ and recent versions of GNU utilities:

readarray -td '' dir1_and_not_dir2_files < <(
 export LC_ALL=C
 shopt -s nullglob dotglob
 comm -z23 <(cd dir1 && printf '%s' *) 
 <(cd dir2 && printf '%s' *)
)

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

edited 2 hours ago

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

answered 2 hours ago

StÃ©phane Chazelas

291k54543882

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

1

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

Â |Â
show 2 more comments

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

1

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

@derobert, yes, you do get that guarantee (unless you enabled some nocaseglob or numericglobsort option). Though as always, you need LC_ALL=C to be guaranteed a total order.
â€“Â StÃ©phane Chazelas
2 hours ago

Indeed, and when I went and checked it seems sorted according to LC_COLLATE is a POSIX requirement.
â€“Â derobert
2 hours ago

Thanks. Will my original script work as well as your scripts?
â€“Â Tim
1 hour ago

Question: In the first solution, wouldn't -A1 be preferable over -A, since otherwise multiple files might get listed in a single line?
â€“Â Malte Skoruppa
1 hour ago

@MalteSkoruppa When piped, results from ls are listed 1 per line by default. man ls doesn't tell you, but info ls somehow does.
â€“Â fra-san
37 mins ago

Â |Â
show 2 more comments

up vote
3
down vote

Yes, you can use diff for that purpose. It's as simple as:

diff -rq dir1 dir2

The -r option tells diff to recurse into sub-directories as well. The -q option tells diff to only report when files differ.

Note that this will show you both files that exist in dir1 but not in dir2, as well as files that exist in dir2, but not in dir1, e.g.:

$ diff -rq /tmp/dir1/ /tmp/dir2/
Only in /tmp/dir1/: file1
Only in /tmp/dir2/: file2
Only in /tmp/dir2/: file3

answered 1 hour ago

Malte Skoruppa

1,1171916

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

add a commentÂ |Â

up vote
3
down vote

Yes, you can use diff for that purpose. It's as simple as:

diff -rq dir1 dir2

The -r option tells diff to recurse into sub-directories as well. The -q option tells diff to only report when files differ.

Note that this will show you both files that exist in dir1 but not in dir2, as well as files that exist in dir2, but not in dir1, e.g.:

$ diff -rq /tmp/dir1/ /tmp/dir2/
Only in /tmp/dir1/: file1
Only in /tmp/dir2/: file2
Only in /tmp/dir2/: file3

answered 1 hour ago

Malte Skoruppa

1,1171916

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

add a commentÂ |Â

up vote
3
down vote

Yes, you can use diff for that purpose. It's as simple as:

diff -rq dir1 dir2

The -r option tells diff to recurse into sub-directories as well. The -q option tells diff to only report when files differ.

Note that this will show you both files that exist in dir1 but not in dir2, as well as files that exist in dir2, but not in dir1, e.g.:

$ diff -rq /tmp/dir1/ /tmp/dir2/
Only in /tmp/dir1/: file1
Only in /tmp/dir2/: file2
Only in /tmp/dir2/: file3

answered 1 hour ago

Malte Skoruppa

1,1171916

Yes, you can use diff for that purpose. It's as simple as:

diff -rq dir1 dir2

The -r option tells diff to recurse into sub-directories as well. The -q option tells diff to only report when files differ.

Note that this will show you both files that exist in dir1 but not in dir2, as well as files that exist in dir2, but not in dir1, e.g.:

$ diff -rq /tmp/dir1/ /tmp/dir2/
Only in /tmp/dir1/: file1
Only in /tmp/dir2/: file2
Only in /tmp/dir2/: file3

answered 1 hour ago

Malte Skoruppa

1,1171916

answered 1 hour ago

Malte Skoruppa

1,1171916

answered 1 hour ago

Malte Skoruppa

1,1171916

answered 1 hour ago

Malte Skoruppa

1,1171916

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

add a commentÂ |Â

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

Thanks. It is more convenient in a single direction. But it is nice to know what diff can do at the best.
â€“Â Tim
1 hour ago

I assumed so, I only wanted to add an answer to the specific question: "Can diff be used for this task?". :-) Do note, however, that StÃ©phane's solution will not work recursively. So if you want to compare directories recursively, you'd need a more involved solution.
â€“Â Malte Skoruppa
1 hour ago

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

Search This Blog

Iyfjky