Mixing
Does `car` on an non-lazy function generated list evals the whole list?
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I want to get in a huge directory the first file which satisfies some condition, something like
(require 'seq)
(require 'f)
(defun lazy-filter-file (pred path)
(car (seq-filter pred (f--files path t))))
(setq fake-filter-file-fn (lambda (x) x))
(lazy-filter-file fake-filter-file-fn path)
And I want to know whether f--files will walk the whole directory, or just find the first file which satisfied the condition and stops. And can you also explain why?
If not lazy, how can I make it lazy?
elisp
asked 4 hours ago
cmal
321110
add a comment |Â
up vote
1
down vote
favorite
I want to get in a huge directory the first file which satisfies some condition, something like
(require 'seq)
(require 'f)
(defun lazy-filter-file (pred path)
(car (seq-filter pred (f--files path t))))
(setq fake-filter-file-fn (lambda (x) x))
(lazy-filter-file fake-filter-file-fn path)
And I want to know whether f--files will walk the whole directory, or just find the first file which satisfied the condition and stops. And can you also explain why?
If not lazy, how can I make it lazy?
elisp
asked 4 hours ago
cmal
321110
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I want to get in a huge directory the first file which satisfies some condition, something like
(require 'seq)
(require 'f)
(defun lazy-filter-file (pred path)
(car (seq-filter pred (f--files path t))))
(setq fake-filter-file-fn (lambda (x) x))
(lazy-filter-file fake-filter-file-fn path)
And I want to know whether f--files will walk the whole directory, or just find the first file which satisfied the condition and stops. And can you also explain why?
If not lazy, how can I make it lazy?
elisp
asked 4 hours ago
cmal
321110
I want to get in a huge directory the first file which satisfies some condition, something like
(require 'seq)
(require 'f)
(defun lazy-filter-file (pred path)
(car (seq-filter pred (f--files path t))))
(setq fake-filter-file-fn (lambda (x) x))
(lazy-filter-file fake-filter-file-fn path)
And I want to know whether f--files will walk the whole directory, or just find the first file which satisfied the condition and stops. And can you also explain why?
If not lazy, how can I make it lazy?
elisp
elisp
asked 4 hours ago
cmal
321110
asked 4 hours ago
cmal
321110
edited 3 hours ago
asked 4 hours ago
cmal
321110
asked 4 hours ago
cmal
321110
asked 4 hours ago
cmal
321110
321110
add a comment |Â
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
2
down vote
The Lisp interpreter always evaluates the arguments before it passes them to the function you're calling, so the answer is that yes, the entire list will be generated first, and then car will select the first element of it.
This may or may not be a performance problem. If the test you're doing on each file name is expensive, then you might want to go to some effort to avoid it. However, in most cases, it's not any more work to get all of the file names from a directory listing than it is to get just the first one.
Most of the time, however, it's not a performance problem. Imagine that the directory list is just a normal text file containing all of the file names. Files on your disk are stored in 4 KB blocks, and any read at the beginning of the file has to request at least one whole block. This will frequently give you the whole list of names, unless that list is pretty long. Additionally, your operating system knows that if you're reading one block of the file then you're very likely to want to read the next few blocks as well. It will generally extend any read that you do to pre-fetch the next few blocks. Finally, there is a certain round-trip time that you will have to wait any time you ask for blocks from a disk. You pay that same cost whether you asked for one block or many sequential blocks. It is therefore much more cost-effective to read many sequential blocks at once than it is to read one block at a time. This is still true even if the disk is an SSD and not a mechanical disk. All of this together means that it is usually fastest to read the whole directory list even if sometimes you only want the first entry.
answered 3 hours ago
db48x
3,529411
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
add a comment |Â
up vote
2
down vote
The question misses the point. Sure, suppose you do (car huge-list), the evaluator will eval huge-list by checking for its type, finding that it's a symbol, looking it up in the environment and returning the bound value (a list object that has been created earlier). However, by that point the huge list has already been generated. Evaluating it over and over again won't change anything about this fact. Even if we were evaluating a list literal, we'd just get a pointer back to the object that was created at read time. (car (huge-list)) isn't much different, here car would get a Lisp object and evaluating it will return it unchanged.
The point of lazy data structures is deferring the creation of the huge list as long as possible. What you really want is a function that when given a directory, produces one directory item at a time, allowing you to stop early. Unfortunately such a thing doesn't exist. The closest thing to this is the readdir function in C which isn't directly exposed to users, directory-files (used by f--files and everything else) is using it to create the full list. The only thing you can do to limit this list is using the match argument to only list files matching the given regular expression. Or write an Emacs module that gives you readdir. If you're absolutely sure it's a performance issue (have you proven it by measuring?), you know what to do...
answered 1 hour ago
wasamasa
14.7k13466
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators
– phils
35 mins ago
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
The Lisp interpreter always evaluates the arguments before it passes them to the function you're calling, so the answer is that yes, the entire list will be generated first, and then car will select the first element of it.
This may or may not be a performance problem. If the test you're doing on each file name is expensive, then you might want to go to some effort to avoid it. However, in most cases, it's not any more work to get all of the file names from a directory listing than it is to get just the first one.
Most of the time, however, it's not a performance problem. Imagine that the directory list is just a normal text file containing all of the file names. Files on your disk are stored in 4 KB blocks, and any read at the beginning of the file has to request at least one whole block. This will frequently give you the whole list of names, unless that list is pretty long. Additionally, your operating system knows that if you're reading one block of the file then you're very likely to want to read the next few blocks as well. It will generally extend any read that you do to pre-fetch the next few blocks. Finally, there is a certain round-trip time that you will have to wait any time you ask for blocks from a disk. You pay that same cost whether you asked for one block or many sequential blocks. It is therefore much more cost-effective to read many sequential blocks at once than it is to read one block at a time. This is still true even if the disk is an SSD and not a mechanical disk. All of this together means that it is usually fastest to read the whole directory list even if sometimes you only want the first entry.
answered 3 hours ago
db48x
3,529411
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
add a comment |Â
up vote
2
down vote
The Lisp interpreter always evaluates the arguments before it passes them to the function you're calling, so the answer is that yes, the entire list will be generated first, and then car will select the first element of it.
This may or may not be a performance problem. If the test you're doing on each file name is expensive, then you might want to go to some effort to avoid it. However, in most cases, it's not any more work to get all of the file names from a directory listing than it is to get just the first one.
Most of the time, however, it's not a performance problem. Imagine that the directory list is just a normal text file containing all of the file names. Files on your disk are stored in 4 KB blocks, and any read at the beginning of the file has to request at least one whole block. This will frequently give you the whole list of names, unless that list is pretty long. Additionally, your operating system knows that if you're reading one block of the file then you're very likely to want to read the next few blocks as well. It will generally extend any read that you do to pre-fetch the next few blocks. Finally, there is a certain round-trip time that you will have to wait any time you ask for blocks from a disk. You pay that same cost whether you asked for one block or many sequential blocks. It is therefore much more cost-effective to read many sequential blocks at once than it is to read one block at a time. This is still true even if the disk is an SSD and not a mechanical disk. All of this together means that it is usually fastest to read the whole directory list even if sometimes you only want the first entry.
answered 3 hours ago
db48x
3,529411
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
add a comment |Â
up vote
2
down vote
up vote
2
down vote
The Lisp interpreter always evaluates the arguments before it passes them to the function you're calling, so the answer is that yes, the entire list will be generated first, and then car will select the first element of it.
This may or may not be a performance problem. If the test you're doing on each file name is expensive, then you might want to go to some effort to avoid it. However, in most cases, it's not any more work to get all of the file names from a directory listing than it is to get just the first one.
Most of the time, however, it's not a performance problem. Imagine that the directory list is just a normal text file containing all of the file names. Files on your disk are stored in 4 KB blocks, and any read at the beginning of the file has to request at least one whole block. This will frequently give you the whole list of names, unless that list is pretty long. Additionally, your operating system knows that if you're reading one block of the file then you're very likely to want to read the next few blocks as well. It will generally extend any read that you do to pre-fetch the next few blocks. Finally, there is a certain round-trip time that you will have to wait any time you ask for blocks from a disk. You pay that same cost whether you asked for one block or many sequential blocks. It is therefore much more cost-effective to read many sequential blocks at once than it is to read one block at a time. This is still true even if the disk is an SSD and not a mechanical disk. All of this together means that it is usually fastest to read the whole directory list even if sometimes you only want the first entry.
answered 3 hours ago
db48x
3,529411
The Lisp interpreter always evaluates the arguments before it passes them to the function you're calling, so the answer is that yes, the entire list will be generated first, and then car will select the first element of it.
This may or may not be a performance problem. If the test you're doing on each file name is expensive, then you might want to go to some effort to avoid it. However, in most cases, it's not any more work to get all of the file names from a directory listing than it is to get just the first one.
Most of the time, however, it's not a performance problem. Imagine that the directory list is just a normal text file containing all of the file names. Files on your disk are stored in 4 KB blocks, and any read at the beginning of the file has to request at least one whole block. This will frequently give you the whole list of names, unless that list is pretty long. Additionally, your operating system knows that if you're reading one block of the file then you're very likely to want to read the next few blocks as well. It will generally extend any read that you do to pre-fetch the next few blocks. Finally, there is a certain round-trip time that you will have to wait any time you ask for blocks from a disk. You pay that same cost whether you asked for one block or many sequential blocks. It is therefore much more cost-effective to read many sequential blocks at once than it is to read one block at a time. This is still true even if the disk is an SSD and not a mechanical disk. All of this together means that it is usually fastest to read the whole directory list even if sometimes you only want the first entry.
answered 3 hours ago
db48x
3,529411
answered 3 hours ago
db48x
3,529411
answered 3 hours ago
db48x
3,529411
answered 3 hours ago
db48x
3,529411
3,529411
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
add a comment |Â
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
This is one of the reasons why lazy sequences in languages like Clojure use chunking internally. The core machinery fetches a chunk of values if needed, the higher-level functions fetch the next unread item from that chunk.
– wasamasa
1 hour ago
add a comment |Â
up vote
2
down vote
The question misses the point. Sure, suppose you do (car huge-list), the evaluator will eval huge-list by checking for its type, finding that it's a symbol, looking it up in the environment and returning the bound value (a list object that has been created earlier). However, by that point the huge list has already been generated. Evaluating it over and over again won't change anything about this fact. Even if we were evaluating a list literal, we'd just get a pointer back to the object that was created at read time. (car (huge-list)) isn't much different, here car would get a Lisp object and evaluating it will return it unchanged.
The point of lazy data structures is deferring the creation of the huge list as long as possible. What you really want is a function that when given a directory, produces one directory item at a time, allowing you to stop early. Unfortunately such a thing doesn't exist. The closest thing to this is the readdir function in C which isn't directly exposed to users, directory-files (used by f--files and everything else) is using it to create the full list. The only thing you can do to limit this list is using the match argument to only list files matching the given regular expression. Or write an Emacs module that gives you readdir. If you're absolutely sure it's a performance issue (have you proven it by measuring?), you know what to do...
answered 1 hour ago
wasamasa
14.7k13466
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators
– phils
35 mins ago
add a comment |Â
up vote
2
down vote
The question misses the point. Sure, suppose you do (car huge-list), the evaluator will eval huge-list by checking for its type, finding that it's a symbol, looking it up in the environment and returning the bound value (a list object that has been created earlier). However, by that point the huge list has already been generated. Evaluating it over and over again won't change anything about this fact. Even if we were evaluating a list literal, we'd just get a pointer back to the object that was created at read time. (car (huge-list)) isn't much different, here car would get a Lisp object and evaluating it will return it unchanged.
The point of lazy data structures is deferring the creation of the huge list as long as possible. What you really want is a function that when given a directory, produces one directory item at a time, allowing you to stop early. Unfortunately such a thing doesn't exist. The closest thing to this is the readdir function in C which isn't directly exposed to users, directory-files (used by f--files and everything else) is using it to create the full list. The only thing you can do to limit this list is using the match argument to only list files matching the given regular expression. Or write an Emacs module that gives you readdir. If you're absolutely sure it's a performance issue (have you proven it by measuring?), you know what to do...
answered 1 hour ago
wasamasa
14.7k13466
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators
– phils
35 mins ago
add a comment |Â
up vote
2
down vote
up vote
2
down vote
The question misses the point. Sure, suppose you do (car huge-list), the evaluator will eval huge-list by checking for its type, finding that it's a symbol, looking it up in the environment and returning the bound value (a list object that has been created earlier). However, by that point the huge list has already been generated. Evaluating it over and over again won't change anything about this fact. Even if we were evaluating a list literal, we'd just get a pointer back to the object that was created at read time. (car (huge-list)) isn't much different, here car would get a Lisp object and evaluating it will return it unchanged.
The point of lazy data structures is deferring the creation of the huge list as long as possible. What you really want is a function that when given a directory, produces one directory item at a time, allowing you to stop early. Unfortunately such a thing doesn't exist. The closest thing to this is the readdir function in C which isn't directly exposed to users, directory-files (used by f--files and everything else) is using it to create the full list. The only thing you can do to limit this list is using the match argument to only list files matching the given regular expression. Or write an Emacs module that gives you readdir. If you're absolutely sure it's a performance issue (have you proven it by measuring?), you know what to do...
answered 1 hour ago
wasamasa
14.7k13466
The question misses the point. Sure, suppose you do (car huge-list), the evaluator will eval huge-list by checking for its type, finding that it's a symbol, looking it up in the environment and returning the bound value (a list object that has been created earlier). However, by that point the huge list has already been generated. Evaluating it over and over again won't change anything about this fact. Even if we were evaluating a list literal, we'd just get a pointer back to the object that was created at read time. (car (huge-list)) isn't much different, here car would get a Lisp object and evaluating it will return it unchanged.
The point of lazy data structures is deferring the creation of the huge list as long as possible. What you really want is a function that when given a directory, produces one directory item at a time, allowing you to stop early. Unfortunately such a thing doesn't exist. The closest thing to this is the readdir function in C which isn't directly exposed to users, directory-files (used by f--files and everything else) is using it to create the full list. The only thing you can do to limit this list is using the match argument to only list files matching the given regular expression. Or write an Emacs module that gives you readdir. If you're absolutely sure it's a performance issue (have you proven it by measuring?), you know what to do...
answered 1 hour ago
wasamasa
14.7k13466
edited 3 mins ago
answered 1 hour ago
wasamasa
14.7k13466
answered 1 hour ago
wasamasa
14.7k13466
answered 1 hour ago
wasamasa
14.7k13466
14.7k13466
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators
– phils
35 mins ago
add a comment |Â
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators
– phils
35 mins ago
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
Thank you very much, for explaining so carefully. Can you give me some further advice for how to create lazy list in Emacs Lisp?
– cmal
1 hour ago
C-h i g (elisp)Generators– phils
35 mins ago
C-h i g (elisp)Generators– phils
35 mins ago
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2femacs.stackexchange.com%2fquestions%2f45047%2fdoes-car-on-an-non-lazy-function-generated-list-evals-the-whole-list%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
