Mixing
C program outputs text in wrong order when writing to a file
Clash Royale CLAN TAG#URR8PPP
up vote
44
down vote
favorite
I have a C program that compiles to an executable called myprogram. This is its main function:
int main(int argc, char ** argv)
printf("this is a test message.n");
system("ls");
return 0;
When I run myprogram > output.txt in a Linux shell and then examine output.txt, I see the output of ls listed above "this is a test message."
I feel like it should be the other way around. Why is this happening, and what can I do so that "this is a test message" appears at the top of output.txt?
If it matters, I'm new to both C and working in a command line.
c linux io-redirection
edited 11 mins ago
Jake Reece
789117
asked yesterday
Archr
35637
add a comment |Â
up vote
44
down vote
favorite
I have a C program that compiles to an executable called myprogram. This is its main function:
int main(int argc, char ** argv)
printf("this is a test message.n");
system("ls");
return 0;
When I run myprogram > output.txt in a Linux shell and then examine output.txt, I see the output of ls listed above "this is a test message."
I feel like it should be the other way around. Why is this happening, and what can I do so that "this is a test message" appears at the top of output.txt?
If it matters, I'm new to both C and working in a command line.
c linux io-redirection
edited 11 mins ago
Jake Reece
789117
asked yesterday
Archr
35637
4
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Yourprintfoutput could get written out before, after, or even in the middle of thelsoutput! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.
– J-L
20 hours ago
1
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago
add a comment |Â
up vote
44
down vote
favorite
up vote
44
down vote
favorite
I have a C program that compiles to an executable called myprogram. This is its main function:
int main(int argc, char ** argv)
printf("this is a test message.n");
system("ls");
return 0;
When I run myprogram > output.txt in a Linux shell and then examine output.txt, I see the output of ls listed above "this is a test message."
I feel like it should be the other way around. Why is this happening, and what can I do so that "this is a test message" appears at the top of output.txt?
If it matters, I'm new to both C and working in a command line.
c linux io-redirection
edited 11 mins ago
Jake Reece
789117
asked yesterday
Archr
35637
I have a C program that compiles to an executable called myprogram. This is its main function:
int main(int argc, char ** argv)
printf("this is a test message.n");
system("ls");
return 0;
When I run myprogram > output.txt in a Linux shell and then examine output.txt, I see the output of ls listed above "this is a test message."
I feel like it should be the other way around. Why is this happening, and what can I do so that "this is a test message" appears at the top of output.txt?
If it matters, I'm new to both C and working in a command line.
c linux io-redirection
c linux io-redirection
edited 11 mins ago
Jake Reece
789117
asked yesterday
Archr
35637
edited 11 mins ago
Jake Reece
789117
asked yesterday
Archr
35637
edited 11 mins ago
Jake Reece
789117
edited 11 mins ago
Jake Reece
789117
edited 11 mins ago
Jake Reece
789117
789117
asked yesterday
Archr
35637
asked yesterday
Archr
35637
asked yesterday
Archr
35637
35637
4
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Yourprintfoutput could get written out before, after, or even in the middle of thelsoutput! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.
– J-L
20 hours ago
1
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago
add a comment |Â
4
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Yourprintfoutput could get written out before, after, or even in the middle of thelsoutput! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.
– J-L
20 hours ago
1
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago
4
4
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Your
printf output could get written out before, after, or even in the middle of the ls output! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.– J-L
20 hours ago
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Your
printf output could get written out before, after, or even in the middle of the ls output! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.– J-L
20 hours ago
1
1
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago
add a comment |Â
3 Answers
3
active
oldest
votes
up vote
73
down vote
accepted
By default output to stdout is line-buffered when connected to a terminal. That is, the buffer is flushed when it's full or when you add a newline.
However, if stdout is not connected to a terminal, like what happens when you redirect the output from your program to a file, then stdout becomes fully buffered. That means the buffer will be flushed and actually written either when it's full or when explicitly flushed (which happens when the program exits).
This means that the output of a separate process started from your code (like what happens when you call system) will most likely be written first, since the buffer of that process will be flushed when that process ends, which is before your own process.
What happens when using redirection (or pipes for that matter):
- Your
printfcall writes to thestdoutbuffer. - The
systemfunction starts a new process, which writes to its own buffer. - When the external process (started by your
systemcall) exits, its buffer is flushed and written. Your own buffer in your own process, isn't touched. - Your own process ends, and your
stdoutbuffer is flushed and written.
To get the output in the "correct" (or at least expected) order, call fflush before calling system, to explicitly flush stdout.
Out of curiosity: Does the program output correctly even iffflushis called aftersystem, or thesystemdoes everything ( including its ownfflush) in that call?
– auhmaan
23 hours ago
Another curious question - doesn't the output ofsystem()also end up in the samestdoutbuffer thatprintf()used before? After all, it didn't bypass it and write directly to console. Thestdouts of both programs were chained - as is evident by the fact that both ended up inoutput.txt.
– Vilx-
20 hours ago
5
@Vilx-, no,stdiois a library, the buffers it uses exist in memory of the process.system()starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and thewrite()system call; and the C libraryFILEstreams andprintf()(etc.) is important here.
– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
Your answer glosses over one detail that maybe the OP should be aware of: Thestdoutvariable in the C program holds a pointer to a libcFILEobject, but what is shared by the two processes is an open file descriptor.
– besmirched
19 hours ago
add a comment |Â
up vote
11
down vote
It is related to output buffering. I managed to reproduce the same behaviour. Forcing the flush did it for me.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char ** argv)
printf("this is a test message.n");
fflush(stdout);
system("ls");
return 0;
Before adding the fflush:
$ ./main > foo
$ cat foo
main
main.c
this is a test message.
and after:
$ ./main > foo
$ cat foo
this is a test message.
foo
main
main.c
answered yesterday
Aif
8,82912336
add a comment |Â
up vote
5
down vote
I suspect it's because of the order in which the stdout buffer gets flushed, which is not necessarily deterministic. It's possible that the parent spawns the ls process and doesn't flush its own stdout until after that returns. It may not actually flush stdout until the process exits.
Try adding fflush (stdout) after the printf statement and see if that forces the output to appear first.
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
add a comment |Â
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
73
down vote
accepted
By default output to stdout is line-buffered when connected to a terminal. That is, the buffer is flushed when it's full or when you add a newline.
However, if stdout is not connected to a terminal, like what happens when you redirect the output from your program to a file, then stdout becomes fully buffered. That means the buffer will be flushed and actually written either when it's full or when explicitly flushed (which happens when the program exits).
This means that the output of a separate process started from your code (like what happens when you call system) will most likely be written first, since the buffer of that process will be flushed when that process ends, which is before your own process.
What happens when using redirection (or pipes for that matter):
- Your
printfcall writes to thestdoutbuffer. - The
systemfunction starts a new process, which writes to its own buffer. - When the external process (started by your
systemcall) exits, its buffer is flushed and written. Your own buffer in your own process, isn't touched. - Your own process ends, and your
stdoutbuffer is flushed and written.
To get the output in the "correct" (or at least expected) order, call fflush before calling system, to explicitly flush stdout.
Out of curiosity: Does the program output correctly even iffflushis called aftersystem, or thesystemdoes everything ( including its ownfflush) in that call?
– auhmaan
23 hours ago
Another curious question - doesn't the output ofsystem()also end up in the samestdoutbuffer thatprintf()used before? After all, it didn't bypass it and write directly to console. Thestdouts of both programs were chained - as is evident by the fact that both ended up inoutput.txt.
– Vilx-
20 hours ago
5
@Vilx-, no,stdiois a library, the buffers it uses exist in memory of the process.system()starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and thewrite()system call; and the C libraryFILEstreams andprintf()(etc.) is important here.
– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
Your answer glosses over one detail that maybe the OP should be aware of: Thestdoutvariable in the C program holds a pointer to a libcFILEobject, but what is shared by the two processes is an open file descriptor.
– besmirched
19 hours ago
add a comment |Â
up vote
73
down vote
accepted
By default output to stdout is line-buffered when connected to a terminal. That is, the buffer is flushed when it's full or when you add a newline.
However, if stdout is not connected to a terminal, like what happens when you redirect the output from your program to a file, then stdout becomes fully buffered. That means the buffer will be flushed and actually written either when it's full or when explicitly flushed (which happens when the program exits).
This means that the output of a separate process started from your code (like what happens when you call system) will most likely be written first, since the buffer of that process will be flushed when that process ends, which is before your own process.
What happens when using redirection (or pipes for that matter):
- Your
printfcall writes to thestdoutbuffer. - The
systemfunction starts a new process, which writes to its own buffer. - When the external process (started by your
systemcall) exits, its buffer is flushed and written. Your own buffer in your own process, isn't touched. - Your own process ends, and your
stdoutbuffer is flushed and written.
To get the output in the "correct" (or at least expected) order, call fflush before calling system, to explicitly flush stdout.
Out of curiosity: Does the program output correctly even iffflushis called aftersystem, or thesystemdoes everything ( including its ownfflush) in that call?
– auhmaan
23 hours ago
Another curious question - doesn't the output ofsystem()also end up in the samestdoutbuffer thatprintf()used before? After all, it didn't bypass it and write directly to console. Thestdouts of both programs were chained - as is evident by the fact that both ended up inoutput.txt.
– Vilx-
20 hours ago
5
@Vilx-, no,stdiois a library, the buffers it uses exist in memory of the process.system()starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and thewrite()system call; and the C libraryFILEstreams andprintf()(etc.) is important here.
– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
Your answer glosses over one detail that maybe the OP should be aware of: Thestdoutvariable in the C program holds a pointer to a libcFILEobject, but what is shared by the two processes is an open file descriptor.
– besmirched
19 hours ago
add a comment |Â
up vote
73
down vote
accepted
up vote
73
down vote
accepted
By default output to stdout is line-buffered when connected to a terminal. That is, the buffer is flushed when it's full or when you add a newline.
However, if stdout is not connected to a terminal, like what happens when you redirect the output from your program to a file, then stdout becomes fully buffered. That means the buffer will be flushed and actually written either when it's full or when explicitly flushed (which happens when the program exits).
This means that the output of a separate process started from your code (like what happens when you call system) will most likely be written first, since the buffer of that process will be flushed when that process ends, which is before your own process.
What happens when using redirection (or pipes for that matter):
- Your
printfcall writes to thestdoutbuffer. - The
systemfunction starts a new process, which writes to its own buffer. - When the external process (started by your
systemcall) exits, its buffer is flushed and written. Your own buffer in your own process, isn't touched. - Your own process ends, and your
stdoutbuffer is flushed and written.
To get the output in the "correct" (or at least expected) order, call fflush before calling system, to explicitly flush stdout.
By default output to stdout is line-buffered when connected to a terminal. That is, the buffer is flushed when it's full or when you add a newline.
However, if stdout is not connected to a terminal, like what happens when you redirect the output from your program to a file, then stdout becomes fully buffered. That means the buffer will be flushed and actually written either when it's full or when explicitly flushed (which happens when the program exits).
This means that the output of a separate process started from your code (like what happens when you call system) will most likely be written first, since the buffer of that process will be flushed when that process ends, which is before your own process.
What happens when using redirection (or pipes for that matter):
- Your
printfcall writes to thestdoutbuffer. - The
systemfunction starts a new process, which writes to its own buffer. - When the external process (started by your
systemcall) exits, its buffer is flushed and written. Your own buffer in your own process, isn't touched. - Your own process ends, and your
stdoutbuffer is flushed and written.
To get the output in the "correct" (or at least expected) order, call fflush before calling system, to explicitly flush stdout.
edited 20 hours ago
community wiki
3 revs, 2 users 92%
Some programmer dude
Out of curiosity: Does the program output correctly even iffflushis called aftersystem, or thesystemdoes everything ( including its ownfflush) in that call?
– auhmaan
23 hours ago
Another curious question - doesn't the output ofsystem()also end up in the samestdoutbuffer thatprintf()used before? After all, it didn't bypass it and write directly to console. Thestdouts of both programs were chained - as is evident by the fact that both ended up inoutput.txt.
– Vilx-
20 hours ago
5
@Vilx-, no,stdiois a library, the buffers it uses exist in memory of the process.system()starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and thewrite()system call; and the C libraryFILEstreams andprintf()(etc.) is important here.
– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
Your answer glosses over one detail that maybe the OP should be aware of: Thestdoutvariable in the C program holds a pointer to a libcFILEobject, but what is shared by the two processes is an open file descriptor.
– besmirched
19 hours ago
add a comment |Â
Out of curiosity: Does the program output correctly even iffflushis called aftersystem, or thesystemdoes everything ( including its ownfflush) in that call?
– auhmaan
23 hours ago
Another curious question - doesn't the output ofsystem()also end up in the samestdoutbuffer thatprintf()used before? After all, it didn't bypass it and write directly to console. Thestdouts of both programs were chained - as is evident by the fact that both ended up inoutput.txt.
– Vilx-
20 hours ago
5
@Vilx-, no,stdiois a library, the buffers it uses exist in memory of the process.system()starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and thewrite()system call; and the C libraryFILEstreams andprintf()(etc.) is important here.
– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
Your answer glosses over one detail that maybe the OP should be aware of: Thestdoutvariable in the C program holds a pointer to a libcFILEobject, but what is shared by the two processes is an open file descriptor.
– besmirched
19 hours ago
Out of curiosity: Does the program output correctly even if
fflush is called after system, or the system does everything ( including its own fflush ) in that call?– auhmaan
23 hours ago
Out of curiosity: Does the program output correctly even if
fflush is called after system, or the system does everything ( including its own fflush ) in that call?– auhmaan
23 hours ago
Another curious question - doesn't the output of
system() also end up in the same stdout buffer that printf() used before? After all, it didn't bypass it and write directly to console. The stdouts of both programs were chained - as is evident by the fact that both ended up in output.txt.– Vilx-
20 hours ago
Another curious question - doesn't the output of
system() also end up in the same stdout buffer that printf() used before? After all, it didn't bypass it and write directly to console. The stdouts of both programs were chained - as is evident by the fact that both ended up in output.txt.– Vilx-
20 hours ago
5
5
@Vilx-, no,
stdio is a library, the buffers it uses exist in memory of the process. system() starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and the write() system call; and the C library FILE streams and printf() (etc.) is important here.– ilkkachu
20 hours ago
@Vilx-, no,
stdio is a library, the buffers it uses exist in memory of the process. system() starts a new process with a separate memory space, and hence separate buffers. The child process does inherit the file descriptor from the parent, though, which is why the output goes to the same file. The difference between the OS-level file descriptors and the write() system call; and the C library FILE streams and printf() (etc.) is important here.– ilkkachu
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
@ilkkachu - Ahh, got it.
– Vilx-
20 hours ago
1
1
Your answer glosses over one detail that maybe the OP should be aware of: The
stdout variable in the C program holds a pointer to a libc FILE object, but what is shared by the two processes is an open file descriptor.– besmirched
19 hours ago
Your answer glosses over one detail that maybe the OP should be aware of: The
stdout variable in the C program holds a pointer to a libc FILE object, but what is shared by the two processes is an open file descriptor.– besmirched
19 hours ago
add a comment |Â
up vote
11
down vote
It is related to output buffering. I managed to reproduce the same behaviour. Forcing the flush did it for me.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char ** argv)
printf("this is a test message.n");
fflush(stdout);
system("ls");
return 0;
Before adding the fflush:
$ ./main > foo
$ cat foo
main
main.c
this is a test message.
and after:
$ ./main > foo
$ cat foo
this is a test message.
foo
main
main.c
answered yesterday
Aif
8,82912336
add a comment |Â
up vote
11
down vote
It is related to output buffering. I managed to reproduce the same behaviour. Forcing the flush did it for me.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char ** argv)
printf("this is a test message.n");
fflush(stdout);
system("ls");
return 0;
Before adding the fflush:
$ ./main > foo
$ cat foo
main
main.c
this is a test message.
and after:
$ ./main > foo
$ cat foo
this is a test message.
foo
main
main.c
answered yesterday
Aif
8,82912336
add a comment |Â
up vote
11
down vote
up vote
11
down vote
It is related to output buffering. I managed to reproduce the same behaviour. Forcing the flush did it for me.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char ** argv)
printf("this is a test message.n");
fflush(stdout);
system("ls");
return 0;
Before adding the fflush:
$ ./main > foo
$ cat foo
main
main.c
this is a test message.
and after:
$ ./main > foo
$ cat foo
this is a test message.
foo
main
main.c
answered yesterday
Aif
8,82912336
It is related to output buffering. I managed to reproduce the same behaviour. Forcing the flush did it for me.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char ** argv)
printf("this is a test message.n");
fflush(stdout);
system("ls");
return 0;
Before adding the fflush:
$ ./main > foo
$ cat foo
main
main.c
this is a test message.
and after:
$ ./main > foo
$ cat foo
this is a test message.
foo
main
main.c
answered yesterday
Aif
8,82912336
answered yesterday
Aif
8,82912336
answered yesterday
Aif
8,82912336
answered yesterday
Aif
8,82912336
8,82912336
add a comment |Â
add a comment |Â
up vote
5
down vote
I suspect it's because of the order in which the stdout buffer gets flushed, which is not necessarily deterministic. It's possible that the parent spawns the ls process and doesn't flush its own stdout until after that returns. It may not actually flush stdout until the process exits.
Try adding fflush (stdout) after the printf statement and see if that forces the output to appear first.
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
add a comment |Â
up vote
5
down vote
I suspect it's because of the order in which the stdout buffer gets flushed, which is not necessarily deterministic. It's possible that the parent spawns the ls process and doesn't flush its own stdout until after that returns. It may not actually flush stdout until the process exits.
Try adding fflush (stdout) after the printf statement and see if that forces the output to appear first.
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
add a comment |Â
up vote
5
down vote
up vote
5
down vote
I suspect it's because of the order in which the stdout buffer gets flushed, which is not necessarily deterministic. It's possible that the parent spawns the ls process and doesn't flush its own stdout until after that returns. It may not actually flush stdout until the process exits.
Try adding fflush (stdout) after the printf statement and see if that forces the output to appear first.
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
I suspect it's because of the order in which the stdout buffer gets flushed, which is not necessarily deterministic. It's possible that the parent spawns the ls process and doesn't flush its own stdout until after that returns. It may not actually flush stdout until the process exits.
Try adding fflush (stdout) after the printf statement and see if that forces the output to appear first.
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
edited 5 hours ago
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
answered yesterday
ConcernedOfTunbridgeWells
50.8k13123185
50.8k13123185
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52534629%2fc-program-outputs-text-in-wrong-order-when-writing-to-a-file%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
4
When you mix different output and input buffers but don't bother to flush your output, there's no guarantee what order they'll get written out. Your
printfoutput could get written out before, after, or even in the middle of thelsoutput! Granted, having output written out in the middle of other output is rare, but I've seen it happen. Intuitively you think "There's no way one line of output could get written out in the middle of another one," but when you actually see it happen, your perception of reality is challenged.– J-L
20 hours ago
1
Possible duplicate of Why does printf not flush after the call unless a newline is in the format string?
– vaxquis
4 hours ago