if-else vs “or†operation for None-check
Clash Royale CLAN TAG#URR8PPP
up vote
17
down vote
favorite
Let's say we have a dict that will always have keys first_name and last_name but they may be equal to None.
'first_name': None,
'last_name': 'Bloggs'
We want to save the first name if it is passed in or save it as an empty string if None is passed in.
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
Both of these work, however, what is the difference behind the scenes? Is one more efficient than the other?
python
add a comment |Â
up vote
17
down vote
favorite
Let's say we have a dict that will always have keys first_name and last_name but they may be equal to None.
'first_name': None,
'last_name': 'Bloggs'
We want to save the first name if it is passed in or save it as an empty string if None is passed in.
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
Both of these work, however, what is the difference behind the scenes? Is one more efficient than the other?
python
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06
add a comment |Â
up vote
17
down vote
favorite
up vote
17
down vote
favorite
Let's say we have a dict that will always have keys first_name and last_name but they may be equal to None.
'first_name': None,
'last_name': 'Bloggs'
We want to save the first name if it is passed in or save it as an empty string if None is passed in.
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
Both of these work, however, what is the difference behind the scenes? Is one more efficient than the other?
python
Let's say we have a dict that will always have keys first_name and last_name but they may be equal to None.
'first_name': None,
'last_name': 'Bloggs'
We want to save the first name if it is passed in or save it as an empty string if None is passed in.
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
Both of these work, however, what is the difference behind the scenes? Is one more efficient than the other?
python
edited Aug 11 at 22:38


Andrew T.
4,14952649
4,14952649
asked Aug 11 at 19:20
user7692855
222212
222212
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06
add a comment |Â
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06
add a comment |Â
6 Answers
6
active
oldest
votes
up vote
5
down vote
Owing to its greater flexibility, there is more going on behind the scenes in the first version. After all, a if b else c
is an expression with 3 possibly distinct input variables/expressions, while a or b
is binary. You can disassemble the expressions to get a better idea:
def a(x):
return x if x else ''
def b(x):
return x or ''
>>> import dis
>>> dis.dis(a)
2 0 LOAD_FAST 0 (x)
2 POP_JUMP_IF_FALSE 8
4 LOAD_FAST 0 (x)
6 RETURN_VALUE
>> 8 LOAD_CONST 1 ('')
10 RETURN_VALUE
>>> dis.dis(b)
2 0 LOAD_FAST 0 (x)
2 JUMP_IF_TRUE_OR_POP 6
4 LOAD_CONST 1 ('')
>> 6 RETURN_VALUE
add a comment |Â
up vote
2
down vote
TLDR: It does not matter. If you care about correctness, you should instead compare against None
.
account['first_name'] if account['first_name'] is not None else ""
There is a notable impact from whether account['first_name']
is mostly None
or an actual value - however, this is at the nanosecond scale. It is negligible unless run in a very tight loop.
If you seriously require better performance, you should consider using a JIT or static compiler such as PyPy, Cython, or similar.
A look at what happens
Python makes many guarantees that what you write is what is executed. That means the a if a else b
case evaluates a
at most twice. In contrast, a or b
evaluates a
exactly once.
In their disassembly, you can see that the LOAD_NAME
, LOAD_CONST
and BINARY_SUBSCR
happen twice for the first case - but only if the value is true-ish. If it is false-ish, the number of lookups is the same!
dis.dis('''account['first_name'] if account['first_name'] else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (account)
10 LOAD_CONST 0 ('first_name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
dis.dis('''account['first_name'] or ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Technically, the statements also perform a different check: boolean false-ness (POP_JUMP_IF_FALSE
) versus boolean truth-ness (JUMP_IF_TRUE_OR_POP
). Since this is a single operation, it is optimised inside the interpreter and the difference is negligible.
For builtin types, you can generally assume that operations are "fast" - meaning that any non-trivial control flow takes significantly more time. Unless you run this in a tight loop over thousands of accounts, it will not have a notable impact.
While in your case it does not make an observable difference, it is usually better to explicitly test is not None
. This lets you distinguish between None
and other false-ish values, such as False
, or
""
, that may be valid.
account['first_name'] if account['first_name'] is not None else ""
Strictly speaking, it is the least efficient. On top of the added lookup, there is an additional lookup for None
and comparison for is not
.
dis.dis('''account['first_name'] if account['first_name'] is not None else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 LOAD_CONST 1 (None)
8 COMPARE_OP 9 (is not)
10 POP_JUMP_IF_FALSE 20
12 LOAD_NAME 0 (account)
14 LOAD_CONST 0 ('first_name')
16 BINARY_SUBSCR
18 RETURN_VALUE
>> 20 LOAD_CONST 2 ('')
22 RETURN_VALUE
Note that this test can actually be faster. An is not None
test compares for identity - that is builtin pointer comparison. Especially for custom types, this is faster than looking up and evaluating a custom __bool__
or even __len__
method.
In practice, the added lookup will not have a noticeable performance difference. It is up to you whether you prefer the shorter a or b
or the more robust a if a is not None else b
. Using a if a else b
gets you neither terseness nor correctness, so it should be avoided.
Here are the numbers from Python 3.6.4, perf timeit
:
# a is None
a or b | 41.4 ns +- 2.1 ns
a if a else b | 41.4 ns +- 2.4 ns
a if a is not None else b | 50.5 ns +- 4.4 ns
# a is not None
a or b | 41.0 ns +- 2.1 ns
a if a else b | 69.9 ns +- 5.0 ns
a if a is not None else b | 70.2 ns +- 5.4 ns
As you can see, there is an impact from the value of a
- if you care about tens of nanoseconds. The terser statement with fewer underlying instructions is faster, and more importantly stable. There is no significant penalty for the added is not None
check.
Either way, if you care about performance - do not optimise for CPython! If you need speed, taking a JIT/static compiler gives significantly more gain. However, their optimisations make instruction counts as performance metrics misleading.
For pure-Python code, as in your case, the PyPy interpreter is an obvious choice. On top of being faster in general, it seems to optimise the is not None
test. Here are the numbers from from PyPy 5.8.0-beta0, perf timeit
:
# a is None
a or b | 10.5 ns +- 0.7 ns
a if a else b | 10.7 ns +- 0.8 ns
a if a is not None else b | 10.1 ns +- 0.8 ns
# a is not None
a or b | 11.2 ns +- 1.0 ns
a if a else b | 11.3 ns +- 1.0 ns
a if a is not None else b | 10.2 ns +- 0.6 ns
Bottom line, do not try to gain performance by optimising for byte code instructions. Even if you are sure that this is a bottleneck (by profiling your application), such optimisations are generally not worth it. A faster runtime gives significantly more gain, and may not even have the same penalties for byte code instructions.
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check againstNone
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.
– poke
Aug 11 at 20:18
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
 |Â
show 1 more comment
up vote
1
down vote
I know this doesn't answer your question about efficiency or the difference behind the scenes, but I'd like to point out that I think the following code is preferable:
first_name = account.get('first_name') or ''
This way you don't have to access the account['first_name']
twice.
Another side effect of this solution (obviously it depends if you want this behavior or not) is you'll never get a KeyError
, even if first_name
is not in the account
dict. Obviously if you prefer to see the KeyError that’s fine too.
The documentation for dict
's get
is here: https://docs.python.org/3/library/stdtypes.html#dict.get
1
'first_name': None .get('first_name', '')
isNone
. So no, that won’t work.
– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returnsNone
if the key is in the dict but with a value ofNone
. It should be""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
add a comment |Â
up vote
1
down vote
Conditional operator
result = value if value else ""
This is the ternary conditional operator and basically equivalent to the following if statement:
if value:
result = value
else:
result = ""
It is very explicit and allows you to describe exactly what condition you require. In this case, it just looks at the truthy value of value
, but you could easily expand this to make a strict test against None
for example:
result = value if value is not None else ""
This would for example retain falsy values like False
or 0
.
or operator
value or ""
This uses the boolean or
operator:
The expression
x or y
first evaluatesx
; ifx
is true, its value is returned; otherwise,y
is evaluated and the resulting value is returned.
So this is basically a way to get the first truthy value (defaulting to the right operand). So this does the same as the value if value else ""
. Unless the conditional operator, it does not support other checks though, so you can only check for truthiness here.
Comparison
In your case, where you want to just check against None
and fall back to an empty string, there’s no difference at all. Just choose what is most understandable to you. From a “pythonic†point of view, one would probably prefer the or
operator, as that’s also a bit shorter.
From a performance standpoint, the conditional operator is slightly more expensive in this case, as this needs to evaluate the dictionary access twice. In practice that won’t be noticeable though, especially not for a dictionary access.
If you do believe that this could have an impact on your application performance, then you should not believe in numbers you get from isolated benchmarks of a single statement; instead, you should profile your application and then try to identify bottlenecks which you can then improve on. I’ll guarantee you that it will be a long way before a second dictionary access will have any impact.
So yes, you can totally ignore the performance arguments for this. Just choose whatever you prefer, what makes the most sense to you. Also consider whether you just want a truthiness check, or whether a strict check against None
would be better.
add a comment |Â
up vote
0
down vote
For your specific case, the boolean or
operator looks more pythonic, and also a very simple benchmark shows that it is slightly more efficient:
import timeit
setup = "account = 'first_name': None, 'last_name': 'Bloggs'"
statements =
'ternary conditional operator': "first_name = account['first_name'] if account['first_name'] else ''",
'boolean or operator': "first_name = account['first_name'] or ''",
for label, statement in statements.items():
elapsed_best = min(timeit.repeat(statement, setup, number=1000000, repeat=10))
print(': :.3 s'.format(label, elapsed_best))
Output:
ternary conditional operator: 0.0303 s
boolean or operator: 0.0275 s
Taking into account that the numbers above are the total execution times in seconds (1000000 evaluations per each statement), in practice, there is no significant difference in efficiency at all.
add a comment |Â
up vote
0
down vote
What is the difference between the two following expressions?
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
The primary difference is that the first, in Python, is the conditional expression,
The expression
x if C else y
first evaluates the condition,C
rather
thanx
. IfC
is true,x
is evaluated and its value is returned;
otherwise,y
is evaluated and its value is returned.
while the second uses the boolean operation:
The expression
x or y
first evaluatesx
; ifx
is true, its value is
returned; otherwise,y
is evaluated and the resulting value is
returned.
Note that the first may require two key lookups versus the second, which only requires one key lookup.
This lookup is called subscript notation:
name[subscript_argument]
Subscript notation exercises the __getitem__
method of the object referenced by name
.
It requires both the name and the subscript argument to be loaded.
Now, in the context of the question, if it tests as True
in a boolean context (which a non-empty string does, but None
does not) it will require a second (redundant) loading of both the dictionary and the key for the conditional expression, while simply returning the first lookup for the boolean or
operation.
Therefore I would expect the second, the boolean operation, to be slightly more efficient in cases where the value is not None
.
Abstract Syntax Tree (AST) breakdown
Others have compared the bytecode generated by both expressions.
However, the AST represents the first breakdown of the language as parsed by the interpreter.
The following AST demonstrates that the second lookup likely involves more work (note I have formatted the output for easier parsing):
>>> print(ast.dump(ast.parse("account['first_name'] if account['first_name'] else ''").body[0]))
Expr(
value=IfExp(
test=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
body=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
orelse=Str(s='')
))
versus
>>> print(ast.dump(ast.parse("account['first_name'] or ''").body[0]))
Expr(
value=BoolOp(
op=Or(),
values=[
Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
Str(s='')]
)
)
Bytecode analysis
Here we see that the bytecode for the conditional expression is much longer. This usually bodes poorly for relative performance in my experience.
>>> import dis
>>> dis.dis("d['name'] if d['name'] else ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (d)
10 LOAD_CONST 0 ('name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
For the boolean operation, it's almost half as long:
>>> dis.dis("d['name'] or ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Here I would expect the performance to be much quicker relative to the other.
Therefore, let's see if there's much difference in performance then.
Performance
Performance is not very important here, but sometimes I have to see for myself:
def cond(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] if d['name'] else ''
def bool_op(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] or ''
We see that when the name is in the dictionary, the boolean operation is about 10% faster than the conditional.
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
However, when the name is not in the dictionary, we see that there is almost no difference:
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
A note on correctness
In general, I would prefer the or
boolean operation to the conditional expression - with the following caveats:
- The dictionary is guaranteed to only have non-empty strings or
None
. - Performance here is critical.
In the case where either the above is not true, I would prefer the following for correctness:
first_name = account['first_name']
if first_name is None:
first_name = ''
The upsides are that
- the lookup is done one time,
- the check for
is None
is quite fast, - the code is explicitly clear, and
- the code is easily maintainable by any Python programmer.
This should also not be any less performant:
def correct(name=False):
d = 'name': 'thename' if name else None
def _correct():
first_name = d['name']
if first_name is None:
first_name = ''
return _correct
We see that we get quite competitive performance when the key is there:
>>> min(timeit.repeat(correct(name=True), repeat=10))
0.10948465298861265
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
when the key is not in the dictionary, it is not quite as good though:
>>> min(timeit.repeat(correct(name=False), repeat=10))
0.11776355793699622
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
Conclusion
The difference between the conditional expression and the boolean operation is two versus one lookups respectively on a True
condition, making the boolean operation more performant.
For correctness's sake, however, do the lookup one time, check for identity to None
with is None
, and then reassign to the empty string in that case.
add a comment |Â
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
5
down vote
Owing to its greater flexibility, there is more going on behind the scenes in the first version. After all, a if b else c
is an expression with 3 possibly distinct input variables/expressions, while a or b
is binary. You can disassemble the expressions to get a better idea:
def a(x):
return x if x else ''
def b(x):
return x or ''
>>> import dis
>>> dis.dis(a)
2 0 LOAD_FAST 0 (x)
2 POP_JUMP_IF_FALSE 8
4 LOAD_FAST 0 (x)
6 RETURN_VALUE
>> 8 LOAD_CONST 1 ('')
10 RETURN_VALUE
>>> dis.dis(b)
2 0 LOAD_FAST 0 (x)
2 JUMP_IF_TRUE_OR_POP 6
4 LOAD_CONST 1 ('')
>> 6 RETURN_VALUE
add a comment |Â
up vote
5
down vote
Owing to its greater flexibility, there is more going on behind the scenes in the first version. After all, a if b else c
is an expression with 3 possibly distinct input variables/expressions, while a or b
is binary. You can disassemble the expressions to get a better idea:
def a(x):
return x if x else ''
def b(x):
return x or ''
>>> import dis
>>> dis.dis(a)
2 0 LOAD_FAST 0 (x)
2 POP_JUMP_IF_FALSE 8
4 LOAD_FAST 0 (x)
6 RETURN_VALUE
>> 8 LOAD_CONST 1 ('')
10 RETURN_VALUE
>>> dis.dis(b)
2 0 LOAD_FAST 0 (x)
2 JUMP_IF_TRUE_OR_POP 6
4 LOAD_CONST 1 ('')
>> 6 RETURN_VALUE
add a comment |Â
up vote
5
down vote
up vote
5
down vote
Owing to its greater flexibility, there is more going on behind the scenes in the first version. After all, a if b else c
is an expression with 3 possibly distinct input variables/expressions, while a or b
is binary. You can disassemble the expressions to get a better idea:
def a(x):
return x if x else ''
def b(x):
return x or ''
>>> import dis
>>> dis.dis(a)
2 0 LOAD_FAST 0 (x)
2 POP_JUMP_IF_FALSE 8
4 LOAD_FAST 0 (x)
6 RETURN_VALUE
>> 8 LOAD_CONST 1 ('')
10 RETURN_VALUE
>>> dis.dis(b)
2 0 LOAD_FAST 0 (x)
2 JUMP_IF_TRUE_OR_POP 6
4 LOAD_CONST 1 ('')
>> 6 RETURN_VALUE
Owing to its greater flexibility, there is more going on behind the scenes in the first version. After all, a if b else c
is an expression with 3 possibly distinct input variables/expressions, while a or b
is binary. You can disassemble the expressions to get a better idea:
def a(x):
return x if x else ''
def b(x):
return x or ''
>>> import dis
>>> dis.dis(a)
2 0 LOAD_FAST 0 (x)
2 POP_JUMP_IF_FALSE 8
4 LOAD_FAST 0 (x)
6 RETURN_VALUE
>> 8 LOAD_CONST 1 ('')
10 RETURN_VALUE
>>> dis.dis(b)
2 0 LOAD_FAST 0 (x)
2 JUMP_IF_TRUE_OR_POP 6
4 LOAD_CONST 1 ('')
>> 6 RETURN_VALUE
edited Aug 23 at 8:27
answered Aug 11 at 19:36


schwobaseggl
29.3k31732
29.3k31732
add a comment |Â
add a comment |Â
up vote
2
down vote
TLDR: It does not matter. If you care about correctness, you should instead compare against None
.
account['first_name'] if account['first_name'] is not None else ""
There is a notable impact from whether account['first_name']
is mostly None
or an actual value - however, this is at the nanosecond scale. It is negligible unless run in a very tight loop.
If you seriously require better performance, you should consider using a JIT or static compiler such as PyPy, Cython, or similar.
A look at what happens
Python makes many guarantees that what you write is what is executed. That means the a if a else b
case evaluates a
at most twice. In contrast, a or b
evaluates a
exactly once.
In their disassembly, you can see that the LOAD_NAME
, LOAD_CONST
and BINARY_SUBSCR
happen twice for the first case - but only if the value is true-ish. If it is false-ish, the number of lookups is the same!
dis.dis('''account['first_name'] if account['first_name'] else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (account)
10 LOAD_CONST 0 ('first_name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
dis.dis('''account['first_name'] or ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Technically, the statements also perform a different check: boolean false-ness (POP_JUMP_IF_FALSE
) versus boolean truth-ness (JUMP_IF_TRUE_OR_POP
). Since this is a single operation, it is optimised inside the interpreter and the difference is negligible.
For builtin types, you can generally assume that operations are "fast" - meaning that any non-trivial control flow takes significantly more time. Unless you run this in a tight loop over thousands of accounts, it will not have a notable impact.
While in your case it does not make an observable difference, it is usually better to explicitly test is not None
. This lets you distinguish between None
and other false-ish values, such as False
, or
""
, that may be valid.
account['first_name'] if account['first_name'] is not None else ""
Strictly speaking, it is the least efficient. On top of the added lookup, there is an additional lookup for None
and comparison for is not
.
dis.dis('''account['first_name'] if account['first_name'] is not None else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 LOAD_CONST 1 (None)
8 COMPARE_OP 9 (is not)
10 POP_JUMP_IF_FALSE 20
12 LOAD_NAME 0 (account)
14 LOAD_CONST 0 ('first_name')
16 BINARY_SUBSCR
18 RETURN_VALUE
>> 20 LOAD_CONST 2 ('')
22 RETURN_VALUE
Note that this test can actually be faster. An is not None
test compares for identity - that is builtin pointer comparison. Especially for custom types, this is faster than looking up and evaluating a custom __bool__
or even __len__
method.
In practice, the added lookup will not have a noticeable performance difference. It is up to you whether you prefer the shorter a or b
or the more robust a if a is not None else b
. Using a if a else b
gets you neither terseness nor correctness, so it should be avoided.
Here are the numbers from Python 3.6.4, perf timeit
:
# a is None
a or b | 41.4 ns +- 2.1 ns
a if a else b | 41.4 ns +- 2.4 ns
a if a is not None else b | 50.5 ns +- 4.4 ns
# a is not None
a or b | 41.0 ns +- 2.1 ns
a if a else b | 69.9 ns +- 5.0 ns
a if a is not None else b | 70.2 ns +- 5.4 ns
As you can see, there is an impact from the value of a
- if you care about tens of nanoseconds. The terser statement with fewer underlying instructions is faster, and more importantly stable. There is no significant penalty for the added is not None
check.
Either way, if you care about performance - do not optimise for CPython! If you need speed, taking a JIT/static compiler gives significantly more gain. However, their optimisations make instruction counts as performance metrics misleading.
For pure-Python code, as in your case, the PyPy interpreter is an obvious choice. On top of being faster in general, it seems to optimise the is not None
test. Here are the numbers from from PyPy 5.8.0-beta0, perf timeit
:
# a is None
a or b | 10.5 ns +- 0.7 ns
a if a else b | 10.7 ns +- 0.8 ns
a if a is not None else b | 10.1 ns +- 0.8 ns
# a is not None
a or b | 11.2 ns +- 1.0 ns
a if a else b | 11.3 ns +- 1.0 ns
a if a is not None else b | 10.2 ns +- 0.6 ns
Bottom line, do not try to gain performance by optimising for byte code instructions. Even if you are sure that this is a bottleneck (by profiling your application), such optimisations are generally not worth it. A faster runtime gives significantly more gain, and may not even have the same penalties for byte code instructions.
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check againstNone
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.
– poke
Aug 11 at 20:18
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
 |Â
show 1 more comment
up vote
2
down vote
TLDR: It does not matter. If you care about correctness, you should instead compare against None
.
account['first_name'] if account['first_name'] is not None else ""
There is a notable impact from whether account['first_name']
is mostly None
or an actual value - however, this is at the nanosecond scale. It is negligible unless run in a very tight loop.
If you seriously require better performance, you should consider using a JIT or static compiler such as PyPy, Cython, or similar.
A look at what happens
Python makes many guarantees that what you write is what is executed. That means the a if a else b
case evaluates a
at most twice. In contrast, a or b
evaluates a
exactly once.
In their disassembly, you can see that the LOAD_NAME
, LOAD_CONST
and BINARY_SUBSCR
happen twice for the first case - but only if the value is true-ish. If it is false-ish, the number of lookups is the same!
dis.dis('''account['first_name'] if account['first_name'] else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (account)
10 LOAD_CONST 0 ('first_name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
dis.dis('''account['first_name'] or ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Technically, the statements also perform a different check: boolean false-ness (POP_JUMP_IF_FALSE
) versus boolean truth-ness (JUMP_IF_TRUE_OR_POP
). Since this is a single operation, it is optimised inside the interpreter and the difference is negligible.
For builtin types, you can generally assume that operations are "fast" - meaning that any non-trivial control flow takes significantly more time. Unless you run this in a tight loop over thousands of accounts, it will not have a notable impact.
While in your case it does not make an observable difference, it is usually better to explicitly test is not None
. This lets you distinguish between None
and other false-ish values, such as False
, or
""
, that may be valid.
account['first_name'] if account['first_name'] is not None else ""
Strictly speaking, it is the least efficient. On top of the added lookup, there is an additional lookup for None
and comparison for is not
.
dis.dis('''account['first_name'] if account['first_name'] is not None else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 LOAD_CONST 1 (None)
8 COMPARE_OP 9 (is not)
10 POP_JUMP_IF_FALSE 20
12 LOAD_NAME 0 (account)
14 LOAD_CONST 0 ('first_name')
16 BINARY_SUBSCR
18 RETURN_VALUE
>> 20 LOAD_CONST 2 ('')
22 RETURN_VALUE
Note that this test can actually be faster. An is not None
test compares for identity - that is builtin pointer comparison. Especially for custom types, this is faster than looking up and evaluating a custom __bool__
or even __len__
method.
In practice, the added lookup will not have a noticeable performance difference. It is up to you whether you prefer the shorter a or b
or the more robust a if a is not None else b
. Using a if a else b
gets you neither terseness nor correctness, so it should be avoided.
Here are the numbers from Python 3.6.4, perf timeit
:
# a is None
a or b | 41.4 ns +- 2.1 ns
a if a else b | 41.4 ns +- 2.4 ns
a if a is not None else b | 50.5 ns +- 4.4 ns
# a is not None
a or b | 41.0 ns +- 2.1 ns
a if a else b | 69.9 ns +- 5.0 ns
a if a is not None else b | 70.2 ns +- 5.4 ns
As you can see, there is an impact from the value of a
- if you care about tens of nanoseconds. The terser statement with fewer underlying instructions is faster, and more importantly stable. There is no significant penalty for the added is not None
check.
Either way, if you care about performance - do not optimise for CPython! If you need speed, taking a JIT/static compiler gives significantly more gain. However, their optimisations make instruction counts as performance metrics misleading.
For pure-Python code, as in your case, the PyPy interpreter is an obvious choice. On top of being faster in general, it seems to optimise the is not None
test. Here are the numbers from from PyPy 5.8.0-beta0, perf timeit
:
# a is None
a or b | 10.5 ns +- 0.7 ns
a if a else b | 10.7 ns +- 0.8 ns
a if a is not None else b | 10.1 ns +- 0.8 ns
# a is not None
a or b | 11.2 ns +- 1.0 ns
a if a else b | 11.3 ns +- 1.0 ns
a if a is not None else b | 10.2 ns +- 0.6 ns
Bottom line, do not try to gain performance by optimising for byte code instructions. Even if you are sure that this is a bottleneck (by profiling your application), such optimisations are generally not worth it. A faster runtime gives significantly more gain, and may not even have the same penalties for byte code instructions.
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check againstNone
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.
– poke
Aug 11 at 20:18
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
 |Â
show 1 more comment
up vote
2
down vote
up vote
2
down vote
TLDR: It does not matter. If you care about correctness, you should instead compare against None
.
account['first_name'] if account['first_name'] is not None else ""
There is a notable impact from whether account['first_name']
is mostly None
or an actual value - however, this is at the nanosecond scale. It is negligible unless run in a very tight loop.
If you seriously require better performance, you should consider using a JIT or static compiler such as PyPy, Cython, or similar.
A look at what happens
Python makes many guarantees that what you write is what is executed. That means the a if a else b
case evaluates a
at most twice. In contrast, a or b
evaluates a
exactly once.
In their disassembly, you can see that the LOAD_NAME
, LOAD_CONST
and BINARY_SUBSCR
happen twice for the first case - but only if the value is true-ish. If it is false-ish, the number of lookups is the same!
dis.dis('''account['first_name'] if account['first_name'] else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (account)
10 LOAD_CONST 0 ('first_name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
dis.dis('''account['first_name'] or ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Technically, the statements also perform a different check: boolean false-ness (POP_JUMP_IF_FALSE
) versus boolean truth-ness (JUMP_IF_TRUE_OR_POP
). Since this is a single operation, it is optimised inside the interpreter and the difference is negligible.
For builtin types, you can generally assume that operations are "fast" - meaning that any non-trivial control flow takes significantly more time. Unless you run this in a tight loop over thousands of accounts, it will not have a notable impact.
While in your case it does not make an observable difference, it is usually better to explicitly test is not None
. This lets you distinguish between None
and other false-ish values, such as False
, or
""
, that may be valid.
account['first_name'] if account['first_name'] is not None else ""
Strictly speaking, it is the least efficient. On top of the added lookup, there is an additional lookup for None
and comparison for is not
.
dis.dis('''account['first_name'] if account['first_name'] is not None else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 LOAD_CONST 1 (None)
8 COMPARE_OP 9 (is not)
10 POP_JUMP_IF_FALSE 20
12 LOAD_NAME 0 (account)
14 LOAD_CONST 0 ('first_name')
16 BINARY_SUBSCR
18 RETURN_VALUE
>> 20 LOAD_CONST 2 ('')
22 RETURN_VALUE
Note that this test can actually be faster. An is not None
test compares for identity - that is builtin pointer comparison. Especially for custom types, this is faster than looking up and evaluating a custom __bool__
or even __len__
method.
In practice, the added lookup will not have a noticeable performance difference. It is up to you whether you prefer the shorter a or b
or the more robust a if a is not None else b
. Using a if a else b
gets you neither terseness nor correctness, so it should be avoided.
Here are the numbers from Python 3.6.4, perf timeit
:
# a is None
a or b | 41.4 ns +- 2.1 ns
a if a else b | 41.4 ns +- 2.4 ns
a if a is not None else b | 50.5 ns +- 4.4 ns
# a is not None
a or b | 41.0 ns +- 2.1 ns
a if a else b | 69.9 ns +- 5.0 ns
a if a is not None else b | 70.2 ns +- 5.4 ns
As you can see, there is an impact from the value of a
- if you care about tens of nanoseconds. The terser statement with fewer underlying instructions is faster, and more importantly stable. There is no significant penalty for the added is not None
check.
Either way, if you care about performance - do not optimise for CPython! If you need speed, taking a JIT/static compiler gives significantly more gain. However, their optimisations make instruction counts as performance metrics misleading.
For pure-Python code, as in your case, the PyPy interpreter is an obvious choice. On top of being faster in general, it seems to optimise the is not None
test. Here are the numbers from from PyPy 5.8.0-beta0, perf timeit
:
# a is None
a or b | 10.5 ns +- 0.7 ns
a if a else b | 10.7 ns +- 0.8 ns
a if a is not None else b | 10.1 ns +- 0.8 ns
# a is not None
a or b | 11.2 ns +- 1.0 ns
a if a else b | 11.3 ns +- 1.0 ns
a if a is not None else b | 10.2 ns +- 0.6 ns
Bottom line, do not try to gain performance by optimising for byte code instructions. Even if you are sure that this is a bottleneck (by profiling your application), such optimisations are generally not worth it. A faster runtime gives significantly more gain, and may not even have the same penalties for byte code instructions.
TLDR: It does not matter. If you care about correctness, you should instead compare against None
.
account['first_name'] if account['first_name'] is not None else ""
There is a notable impact from whether account['first_name']
is mostly None
or an actual value - however, this is at the nanosecond scale. It is negligible unless run in a very tight loop.
If you seriously require better performance, you should consider using a JIT or static compiler such as PyPy, Cython, or similar.
A look at what happens
Python makes many guarantees that what you write is what is executed. That means the a if a else b
case evaluates a
at most twice. In contrast, a or b
evaluates a
exactly once.
In their disassembly, you can see that the LOAD_NAME
, LOAD_CONST
and BINARY_SUBSCR
happen twice for the first case - but only if the value is true-ish. If it is false-ish, the number of lookups is the same!
dis.dis('''account['first_name'] if account['first_name'] else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (account)
10 LOAD_CONST 0 ('first_name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
dis.dis('''account['first_name'] or ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Technically, the statements also perform a different check: boolean false-ness (POP_JUMP_IF_FALSE
) versus boolean truth-ness (JUMP_IF_TRUE_OR_POP
). Since this is a single operation, it is optimised inside the interpreter and the difference is negligible.
For builtin types, you can generally assume that operations are "fast" - meaning that any non-trivial control flow takes significantly more time. Unless you run this in a tight loop over thousands of accounts, it will not have a notable impact.
While in your case it does not make an observable difference, it is usually better to explicitly test is not None
. This lets you distinguish between None
and other false-ish values, such as False
, or
""
, that may be valid.
account['first_name'] if account['first_name'] is not None else ""
Strictly speaking, it is the least efficient. On top of the added lookup, there is an additional lookup for None
and comparison for is not
.
dis.dis('''account['first_name'] if account['first_name'] is not None else ""''')
1 0 LOAD_NAME 0 (account)
2 LOAD_CONST 0 ('first_name')
4 BINARY_SUBSCR
6 LOAD_CONST 1 (None)
8 COMPARE_OP 9 (is not)
10 POP_JUMP_IF_FALSE 20
12 LOAD_NAME 0 (account)
14 LOAD_CONST 0 ('first_name')
16 BINARY_SUBSCR
18 RETURN_VALUE
>> 20 LOAD_CONST 2 ('')
22 RETURN_VALUE
Note that this test can actually be faster. An is not None
test compares for identity - that is builtin pointer comparison. Especially for custom types, this is faster than looking up and evaluating a custom __bool__
or even __len__
method.
In practice, the added lookup will not have a noticeable performance difference. It is up to you whether you prefer the shorter a or b
or the more robust a if a is not None else b
. Using a if a else b
gets you neither terseness nor correctness, so it should be avoided.
Here are the numbers from Python 3.6.4, perf timeit
:
# a is None
a or b | 41.4 ns +- 2.1 ns
a if a else b | 41.4 ns +- 2.4 ns
a if a is not None else b | 50.5 ns +- 4.4 ns
# a is not None
a or b | 41.0 ns +- 2.1 ns
a if a else b | 69.9 ns +- 5.0 ns
a if a is not None else b | 70.2 ns +- 5.4 ns
As you can see, there is an impact from the value of a
- if you care about tens of nanoseconds. The terser statement with fewer underlying instructions is faster, and more importantly stable. There is no significant penalty for the added is not None
check.
Either way, if you care about performance - do not optimise for CPython! If you need speed, taking a JIT/static compiler gives significantly more gain. However, their optimisations make instruction counts as performance metrics misleading.
For pure-Python code, as in your case, the PyPy interpreter is an obvious choice. On top of being faster in general, it seems to optimise the is not None
test. Here are the numbers from from PyPy 5.8.0-beta0, perf timeit
:
# a is None
a or b | 10.5 ns +- 0.7 ns
a if a else b | 10.7 ns +- 0.8 ns
a if a is not None else b | 10.1 ns +- 0.8 ns
# a is not None
a or b | 11.2 ns +- 1.0 ns
a if a else b | 11.3 ns +- 1.0 ns
a if a is not None else b | 10.2 ns +- 0.6 ns
Bottom line, do not try to gain performance by optimising for byte code instructions. Even if you are sure that this is a bottleneck (by profiling your application), such optimisations are generally not worth it. A faster runtime gives significantly more gain, and may not even have the same penalties for byte code instructions.
edited Aug 13 at 11:58
answered Aug 11 at 19:35
MisterMiyagi
5,8101637
5,8101637
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check againstNone
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.
– poke
Aug 11 at 20:18
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
 |Â
show 1 more comment
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check againstNone
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.
– poke
Aug 11 at 20:18
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
1
1
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
“If you care about performance, you should instead compare against None and use PyPy†– That’s a very confusing statement. You are seriously recommending switching to PyPy on the basis of a tiny and insignificant instruction which has a 40ns difference in a benchmark?
– poke
Aug 11 at 20:06
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
@poke Of course not. There is no performance difference between statements - especially with PyPy, the stdev makes the difference of means insignificant. That is why the more correct test against None should be taken. There is a difference between implementations, though, and that matches the factor of commonly used benchmarks. That is why the objectively faster PyPy should be taken if one cares about performance.
– MisterMiyagi
Aug 11 at 20:14
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check against
None
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.– poke
Aug 11 at 20:18
When there’s no difference, then why are you recommending anything based on performance? If there is no performance difference, you shouldn’t even show meaningless numbers. That just gives off a wrong impression. Also, “more correct test†is fairly subjective when you don’t actually know the context in which OP is executing this. Yes, it is a good idea to mention that a strict check against
None
might be better for a various number of reasons but not because of performance, no. So leave it up to OP to make an educated decision based on that.– poke
Aug 11 at 20:18
1
1
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
@MisterMiyagi FWIW I think you did a decent job of editing the answer to be more balanced. Clearing up the comment trail, it's no-longer relevant :)
– roganjosh
Aug 11 at 20:41
1
1
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
Note that in OP's version, the first example does two dictionary lookups while the second example only does one. If you're using some weird thing that looks like a dictionary but isn't, that could have side effects.
– Kevin
Aug 11 at 22:05
 |Â
show 1 more comment
up vote
1
down vote
I know this doesn't answer your question about efficiency or the difference behind the scenes, but I'd like to point out that I think the following code is preferable:
first_name = account.get('first_name') or ''
This way you don't have to access the account['first_name']
twice.
Another side effect of this solution (obviously it depends if you want this behavior or not) is you'll never get a KeyError
, even if first_name
is not in the account
dict. Obviously if you prefer to see the KeyError that’s fine too.
The documentation for dict
's get
is here: https://docs.python.org/3/library/stdtypes.html#dict.get
1
'first_name': None .get('first_name', '')
isNone
. So no, that won’t work.
– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returnsNone
if the key is in the dict but with a value ofNone
. It should be""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
add a comment |Â
up vote
1
down vote
I know this doesn't answer your question about efficiency or the difference behind the scenes, but I'd like to point out that I think the following code is preferable:
first_name = account.get('first_name') or ''
This way you don't have to access the account['first_name']
twice.
Another side effect of this solution (obviously it depends if you want this behavior or not) is you'll never get a KeyError
, even if first_name
is not in the account
dict. Obviously if you prefer to see the KeyError that’s fine too.
The documentation for dict
's get
is here: https://docs.python.org/3/library/stdtypes.html#dict.get
1
'first_name': None .get('first_name', '')
isNone
. So no, that won’t work.
– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returnsNone
if the key is in the dict but with a value ofNone
. It should be""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
add a comment |Â
up vote
1
down vote
up vote
1
down vote
I know this doesn't answer your question about efficiency or the difference behind the scenes, but I'd like to point out that I think the following code is preferable:
first_name = account.get('first_name') or ''
This way you don't have to access the account['first_name']
twice.
Another side effect of this solution (obviously it depends if you want this behavior or not) is you'll never get a KeyError
, even if first_name
is not in the account
dict. Obviously if you prefer to see the KeyError that’s fine too.
The documentation for dict
's get
is here: https://docs.python.org/3/library/stdtypes.html#dict.get
I know this doesn't answer your question about efficiency or the difference behind the scenes, but I'd like to point out that I think the following code is preferable:
first_name = account.get('first_name') or ''
This way you don't have to access the account['first_name']
twice.
Another side effect of this solution (obviously it depends if you want this behavior or not) is you'll never get a KeyError
, even if first_name
is not in the account
dict. Obviously if you prefer to see the KeyError that’s fine too.
The documentation for dict
's get
is here: https://docs.python.org/3/library/stdtypes.html#dict.get
edited Aug 11 at 23:00
answered Aug 11 at 20:06


Matthew Horst
1,7411316
1,7411316
1
'first_name': None .get('first_name', '')
isNone
. So no, that won’t work.
– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returnsNone
if the key is in the dict but with a value ofNone
. It should be""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
add a comment |Â
1
'first_name': None .get('first_name', '')
isNone
. So no, that won’t work.
– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returnsNone
if the key is in the dict but with a value ofNone
. It should be""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
1
1
'first_name': None .get('first_name', '')
is None
. So no, that won’t work.– poke
Aug 11 at 20:07
'first_name': None .get('first_name', '')
is None
. So no, that won’t work.– poke
Aug 11 at 20:07
I don't think this will have the desired output. As @poke said it returns
None
if the key is in the dict but with a value of None
. It should be ""
– user7692855
Aug 11 at 20:14
I don't think this will have the desired output. As @poke said it returns
None
if the key is in the dict but with a value of None
. It should be ""
– user7692855
Aug 11 at 20:14
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
You're right, sorry, clearly I need coffee even in the afternoons. I still prefer .get to handle the case of the key not being present, in most cases.
– Matthew Horst
Aug 11 at 20:22
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
This approach indeed is preferable - precisely because it means the input has to be better formed.
– MisterMiyagi
Aug 11 at 20:45
add a comment |Â
up vote
1
down vote
Conditional operator
result = value if value else ""
This is the ternary conditional operator and basically equivalent to the following if statement:
if value:
result = value
else:
result = ""
It is very explicit and allows you to describe exactly what condition you require. In this case, it just looks at the truthy value of value
, but you could easily expand this to make a strict test against None
for example:
result = value if value is not None else ""
This would for example retain falsy values like False
or 0
.
or operator
value or ""
This uses the boolean or
operator:
The expression
x or y
first evaluatesx
; ifx
is true, its value is returned; otherwise,y
is evaluated and the resulting value is returned.
So this is basically a way to get the first truthy value (defaulting to the right operand). So this does the same as the value if value else ""
. Unless the conditional operator, it does not support other checks though, so you can only check for truthiness here.
Comparison
In your case, where you want to just check against None
and fall back to an empty string, there’s no difference at all. Just choose what is most understandable to you. From a “pythonic†point of view, one would probably prefer the or
operator, as that’s also a bit shorter.
From a performance standpoint, the conditional operator is slightly more expensive in this case, as this needs to evaluate the dictionary access twice. In practice that won’t be noticeable though, especially not for a dictionary access.
If you do believe that this could have an impact on your application performance, then you should not believe in numbers you get from isolated benchmarks of a single statement; instead, you should profile your application and then try to identify bottlenecks which you can then improve on. I’ll guarantee you that it will be a long way before a second dictionary access will have any impact.
So yes, you can totally ignore the performance arguments for this. Just choose whatever you prefer, what makes the most sense to you. Also consider whether you just want a truthiness check, or whether a strict check against None
would be better.
add a comment |Â
up vote
1
down vote
Conditional operator
result = value if value else ""
This is the ternary conditional operator and basically equivalent to the following if statement:
if value:
result = value
else:
result = ""
It is very explicit and allows you to describe exactly what condition you require. In this case, it just looks at the truthy value of value
, but you could easily expand this to make a strict test against None
for example:
result = value if value is not None else ""
This would for example retain falsy values like False
or 0
.
or operator
value or ""
This uses the boolean or
operator:
The expression
x or y
first evaluatesx
; ifx
is true, its value is returned; otherwise,y
is evaluated and the resulting value is returned.
So this is basically a way to get the first truthy value (defaulting to the right operand). So this does the same as the value if value else ""
. Unless the conditional operator, it does not support other checks though, so you can only check for truthiness here.
Comparison
In your case, where you want to just check against None
and fall back to an empty string, there’s no difference at all. Just choose what is most understandable to you. From a “pythonic†point of view, one would probably prefer the or
operator, as that’s also a bit shorter.
From a performance standpoint, the conditional operator is slightly more expensive in this case, as this needs to evaluate the dictionary access twice. In practice that won’t be noticeable though, especially not for a dictionary access.
If you do believe that this could have an impact on your application performance, then you should not believe in numbers you get from isolated benchmarks of a single statement; instead, you should profile your application and then try to identify bottlenecks which you can then improve on. I’ll guarantee you that it will be a long way before a second dictionary access will have any impact.
So yes, you can totally ignore the performance arguments for this. Just choose whatever you prefer, what makes the most sense to you. Also consider whether you just want a truthiness check, or whether a strict check against None
would be better.
add a comment |Â
up vote
1
down vote
up vote
1
down vote
Conditional operator
result = value if value else ""
This is the ternary conditional operator and basically equivalent to the following if statement:
if value:
result = value
else:
result = ""
It is very explicit and allows you to describe exactly what condition you require. In this case, it just looks at the truthy value of value
, but you could easily expand this to make a strict test against None
for example:
result = value if value is not None else ""
This would for example retain falsy values like False
or 0
.
or operator
value or ""
This uses the boolean or
operator:
The expression
x or y
first evaluatesx
; ifx
is true, its value is returned; otherwise,y
is evaluated and the resulting value is returned.
So this is basically a way to get the first truthy value (defaulting to the right operand). So this does the same as the value if value else ""
. Unless the conditional operator, it does not support other checks though, so you can only check for truthiness here.
Comparison
In your case, where you want to just check against None
and fall back to an empty string, there’s no difference at all. Just choose what is most understandable to you. From a “pythonic†point of view, one would probably prefer the or
operator, as that’s also a bit shorter.
From a performance standpoint, the conditional operator is slightly more expensive in this case, as this needs to evaluate the dictionary access twice. In practice that won’t be noticeable though, especially not for a dictionary access.
If you do believe that this could have an impact on your application performance, then you should not believe in numbers you get from isolated benchmarks of a single statement; instead, you should profile your application and then try to identify bottlenecks which you can then improve on. I’ll guarantee you that it will be a long way before a second dictionary access will have any impact.
So yes, you can totally ignore the performance arguments for this. Just choose whatever you prefer, what makes the most sense to you. Also consider whether you just want a truthiness check, or whether a strict check against None
would be better.
Conditional operator
result = value if value else ""
This is the ternary conditional operator and basically equivalent to the following if statement:
if value:
result = value
else:
result = ""
It is very explicit and allows you to describe exactly what condition you require. In this case, it just looks at the truthy value of value
, but you could easily expand this to make a strict test against None
for example:
result = value if value is not None else ""
This would for example retain falsy values like False
or 0
.
or operator
value or ""
This uses the boolean or
operator:
The expression
x or y
first evaluatesx
; ifx
is true, its value is returned; otherwise,y
is evaluated and the resulting value is returned.
So this is basically a way to get the first truthy value (defaulting to the right operand). So this does the same as the value if value else ""
. Unless the conditional operator, it does not support other checks though, so you can only check for truthiness here.
Comparison
In your case, where you want to just check against None
and fall back to an empty string, there’s no difference at all. Just choose what is most understandable to you. From a “pythonic†point of view, one would probably prefer the or
operator, as that’s also a bit shorter.
From a performance standpoint, the conditional operator is slightly more expensive in this case, as this needs to evaluate the dictionary access twice. In practice that won’t be noticeable though, especially not for a dictionary access.
If you do believe that this could have an impact on your application performance, then you should not believe in numbers you get from isolated benchmarks of a single statement; instead, you should profile your application and then try to identify bottlenecks which you can then improve on. I’ll guarantee you that it will be a long way before a second dictionary access will have any impact.
So yes, you can totally ignore the performance arguments for this. Just choose whatever you prefer, what makes the most sense to you. Also consider whether you just want a truthiness check, or whether a strict check against None
would be better.
edited Aug 12 at 10:49
answered Aug 11 at 19:31
poke
197k42310365
197k42310365
add a comment |Â
add a comment |Â
up vote
0
down vote
For your specific case, the boolean or
operator looks more pythonic, and also a very simple benchmark shows that it is slightly more efficient:
import timeit
setup = "account = 'first_name': None, 'last_name': 'Bloggs'"
statements =
'ternary conditional operator': "first_name = account['first_name'] if account['first_name'] else ''",
'boolean or operator': "first_name = account['first_name'] or ''",
for label, statement in statements.items():
elapsed_best = min(timeit.repeat(statement, setup, number=1000000, repeat=10))
print(': :.3 s'.format(label, elapsed_best))
Output:
ternary conditional operator: 0.0303 s
boolean or operator: 0.0275 s
Taking into account that the numbers above are the total execution times in seconds (1000000 evaluations per each statement), in practice, there is no significant difference in efficiency at all.
add a comment |Â
up vote
0
down vote
For your specific case, the boolean or
operator looks more pythonic, and also a very simple benchmark shows that it is slightly more efficient:
import timeit
setup = "account = 'first_name': None, 'last_name': 'Bloggs'"
statements =
'ternary conditional operator': "first_name = account['first_name'] if account['first_name'] else ''",
'boolean or operator': "first_name = account['first_name'] or ''",
for label, statement in statements.items():
elapsed_best = min(timeit.repeat(statement, setup, number=1000000, repeat=10))
print(': :.3 s'.format(label, elapsed_best))
Output:
ternary conditional operator: 0.0303 s
boolean or operator: 0.0275 s
Taking into account that the numbers above are the total execution times in seconds (1000000 evaluations per each statement), in practice, there is no significant difference in efficiency at all.
add a comment |Â
up vote
0
down vote
up vote
0
down vote
For your specific case, the boolean or
operator looks more pythonic, and also a very simple benchmark shows that it is slightly more efficient:
import timeit
setup = "account = 'first_name': None, 'last_name': 'Bloggs'"
statements =
'ternary conditional operator': "first_name = account['first_name'] if account['first_name'] else ''",
'boolean or operator': "first_name = account['first_name'] or ''",
for label, statement in statements.items():
elapsed_best = min(timeit.repeat(statement, setup, number=1000000, repeat=10))
print(': :.3 s'.format(label, elapsed_best))
Output:
ternary conditional operator: 0.0303 s
boolean or operator: 0.0275 s
Taking into account that the numbers above are the total execution times in seconds (1000000 evaluations per each statement), in practice, there is no significant difference in efficiency at all.
For your specific case, the boolean or
operator looks more pythonic, and also a very simple benchmark shows that it is slightly more efficient:
import timeit
setup = "account = 'first_name': None, 'last_name': 'Bloggs'"
statements =
'ternary conditional operator': "first_name = account['first_name'] if account['first_name'] else ''",
'boolean or operator': "first_name = account['first_name'] or ''",
for label, statement in statements.items():
elapsed_best = min(timeit.repeat(statement, setup, number=1000000, repeat=10))
print(': :.3 s'.format(label, elapsed_best))
Output:
ternary conditional operator: 0.0303 s
boolean or operator: 0.0275 s
Taking into account that the numbers above are the total execution times in seconds (1000000 evaluations per each statement), in practice, there is no significant difference in efficiency at all.
answered Aug 11 at 20:00
Gevorg Davoian
9617
9617
add a comment |Â
add a comment |Â
up vote
0
down vote
What is the difference between the two following expressions?
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
The primary difference is that the first, in Python, is the conditional expression,
The expression
x if C else y
first evaluates the condition,C
rather
thanx
. IfC
is true,x
is evaluated and its value is returned;
otherwise,y
is evaluated and its value is returned.
while the second uses the boolean operation:
The expression
x or y
first evaluatesx
; ifx
is true, its value is
returned; otherwise,y
is evaluated and the resulting value is
returned.
Note that the first may require two key lookups versus the second, which only requires one key lookup.
This lookup is called subscript notation:
name[subscript_argument]
Subscript notation exercises the __getitem__
method of the object referenced by name
.
It requires both the name and the subscript argument to be loaded.
Now, in the context of the question, if it tests as True
in a boolean context (which a non-empty string does, but None
does not) it will require a second (redundant) loading of both the dictionary and the key for the conditional expression, while simply returning the first lookup for the boolean or
operation.
Therefore I would expect the second, the boolean operation, to be slightly more efficient in cases where the value is not None
.
Abstract Syntax Tree (AST) breakdown
Others have compared the bytecode generated by both expressions.
However, the AST represents the first breakdown of the language as parsed by the interpreter.
The following AST demonstrates that the second lookup likely involves more work (note I have formatted the output for easier parsing):
>>> print(ast.dump(ast.parse("account['first_name'] if account['first_name'] else ''").body[0]))
Expr(
value=IfExp(
test=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
body=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
orelse=Str(s='')
))
versus
>>> print(ast.dump(ast.parse("account['first_name'] or ''").body[0]))
Expr(
value=BoolOp(
op=Or(),
values=[
Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
Str(s='')]
)
)
Bytecode analysis
Here we see that the bytecode for the conditional expression is much longer. This usually bodes poorly for relative performance in my experience.
>>> import dis
>>> dis.dis("d['name'] if d['name'] else ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (d)
10 LOAD_CONST 0 ('name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
For the boolean operation, it's almost half as long:
>>> dis.dis("d['name'] or ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Here I would expect the performance to be much quicker relative to the other.
Therefore, let's see if there's much difference in performance then.
Performance
Performance is not very important here, but sometimes I have to see for myself:
def cond(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] if d['name'] else ''
def bool_op(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] or ''
We see that when the name is in the dictionary, the boolean operation is about 10% faster than the conditional.
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
However, when the name is not in the dictionary, we see that there is almost no difference:
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
A note on correctness
In general, I would prefer the or
boolean operation to the conditional expression - with the following caveats:
- The dictionary is guaranteed to only have non-empty strings or
None
. - Performance here is critical.
In the case where either the above is not true, I would prefer the following for correctness:
first_name = account['first_name']
if first_name is None:
first_name = ''
The upsides are that
- the lookup is done one time,
- the check for
is None
is quite fast, - the code is explicitly clear, and
- the code is easily maintainable by any Python programmer.
This should also not be any less performant:
def correct(name=False):
d = 'name': 'thename' if name else None
def _correct():
first_name = d['name']
if first_name is None:
first_name = ''
return _correct
We see that we get quite competitive performance when the key is there:
>>> min(timeit.repeat(correct(name=True), repeat=10))
0.10948465298861265
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
when the key is not in the dictionary, it is not quite as good though:
>>> min(timeit.repeat(correct(name=False), repeat=10))
0.11776355793699622
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
Conclusion
The difference between the conditional expression and the boolean operation is two versus one lookups respectively on a True
condition, making the boolean operation more performant.
For correctness's sake, however, do the lookup one time, check for identity to None
with is None
, and then reassign to the empty string in that case.
add a comment |Â
up vote
0
down vote
What is the difference between the two following expressions?
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
The primary difference is that the first, in Python, is the conditional expression,
The expression
x if C else y
first evaluates the condition,C
rather
thanx
. IfC
is true,x
is evaluated and its value is returned;
otherwise,y
is evaluated and its value is returned.
while the second uses the boolean operation:
The expression
x or y
first evaluatesx
; ifx
is true, its value is
returned; otherwise,y
is evaluated and the resulting value is
returned.
Note that the first may require two key lookups versus the second, which only requires one key lookup.
This lookup is called subscript notation:
name[subscript_argument]
Subscript notation exercises the __getitem__
method of the object referenced by name
.
It requires both the name and the subscript argument to be loaded.
Now, in the context of the question, if it tests as True
in a boolean context (which a non-empty string does, but None
does not) it will require a second (redundant) loading of both the dictionary and the key for the conditional expression, while simply returning the first lookup for the boolean or
operation.
Therefore I would expect the second, the boolean operation, to be slightly more efficient in cases where the value is not None
.
Abstract Syntax Tree (AST) breakdown
Others have compared the bytecode generated by both expressions.
However, the AST represents the first breakdown of the language as parsed by the interpreter.
The following AST demonstrates that the second lookup likely involves more work (note I have formatted the output for easier parsing):
>>> print(ast.dump(ast.parse("account['first_name'] if account['first_name'] else ''").body[0]))
Expr(
value=IfExp(
test=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
body=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
orelse=Str(s='')
))
versus
>>> print(ast.dump(ast.parse("account['first_name'] or ''").body[0]))
Expr(
value=BoolOp(
op=Or(),
values=[
Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
Str(s='')]
)
)
Bytecode analysis
Here we see that the bytecode for the conditional expression is much longer. This usually bodes poorly for relative performance in my experience.
>>> import dis
>>> dis.dis("d['name'] if d['name'] else ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (d)
10 LOAD_CONST 0 ('name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
For the boolean operation, it's almost half as long:
>>> dis.dis("d['name'] or ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Here I would expect the performance to be much quicker relative to the other.
Therefore, let's see if there's much difference in performance then.
Performance
Performance is not very important here, but sometimes I have to see for myself:
def cond(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] if d['name'] else ''
def bool_op(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] or ''
We see that when the name is in the dictionary, the boolean operation is about 10% faster than the conditional.
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
However, when the name is not in the dictionary, we see that there is almost no difference:
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
A note on correctness
In general, I would prefer the or
boolean operation to the conditional expression - with the following caveats:
- The dictionary is guaranteed to only have non-empty strings or
None
. - Performance here is critical.
In the case where either the above is not true, I would prefer the following for correctness:
first_name = account['first_name']
if first_name is None:
first_name = ''
The upsides are that
- the lookup is done one time,
- the check for
is None
is quite fast, - the code is explicitly clear, and
- the code is easily maintainable by any Python programmer.
This should also not be any less performant:
def correct(name=False):
d = 'name': 'thename' if name else None
def _correct():
first_name = d['name']
if first_name is None:
first_name = ''
return _correct
We see that we get quite competitive performance when the key is there:
>>> min(timeit.repeat(correct(name=True), repeat=10))
0.10948465298861265
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
when the key is not in the dictionary, it is not quite as good though:
>>> min(timeit.repeat(correct(name=False), repeat=10))
0.11776355793699622
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
Conclusion
The difference between the conditional expression and the boolean operation is two versus one lookups respectively on a True
condition, making the boolean operation more performant.
For correctness's sake, however, do the lookup one time, check for identity to None
with is None
, and then reassign to the empty string in that case.
add a comment |Â
up vote
0
down vote
up vote
0
down vote
What is the difference between the two following expressions?
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
The primary difference is that the first, in Python, is the conditional expression,
The expression
x if C else y
first evaluates the condition,C
rather
thanx
. IfC
is true,x
is evaluated and its value is returned;
otherwise,y
is evaluated and its value is returned.
while the second uses the boolean operation:
The expression
x or y
first evaluatesx
; ifx
is true, its value is
returned; otherwise,y
is evaluated and the resulting value is
returned.
Note that the first may require two key lookups versus the second, which only requires one key lookup.
This lookup is called subscript notation:
name[subscript_argument]
Subscript notation exercises the __getitem__
method of the object referenced by name
.
It requires both the name and the subscript argument to be loaded.
Now, in the context of the question, if it tests as True
in a boolean context (which a non-empty string does, but None
does not) it will require a second (redundant) loading of both the dictionary and the key for the conditional expression, while simply returning the first lookup for the boolean or
operation.
Therefore I would expect the second, the boolean operation, to be slightly more efficient in cases where the value is not None
.
Abstract Syntax Tree (AST) breakdown
Others have compared the bytecode generated by both expressions.
However, the AST represents the first breakdown of the language as parsed by the interpreter.
The following AST demonstrates that the second lookup likely involves more work (note I have formatted the output for easier parsing):
>>> print(ast.dump(ast.parse("account['first_name'] if account['first_name'] else ''").body[0]))
Expr(
value=IfExp(
test=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
body=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
orelse=Str(s='')
))
versus
>>> print(ast.dump(ast.parse("account['first_name'] or ''").body[0]))
Expr(
value=BoolOp(
op=Or(),
values=[
Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
Str(s='')]
)
)
Bytecode analysis
Here we see that the bytecode for the conditional expression is much longer. This usually bodes poorly for relative performance in my experience.
>>> import dis
>>> dis.dis("d['name'] if d['name'] else ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (d)
10 LOAD_CONST 0 ('name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
For the boolean operation, it's almost half as long:
>>> dis.dis("d['name'] or ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Here I would expect the performance to be much quicker relative to the other.
Therefore, let's see if there's much difference in performance then.
Performance
Performance is not very important here, but sometimes I have to see for myself:
def cond(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] if d['name'] else ''
def bool_op(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] or ''
We see that when the name is in the dictionary, the boolean operation is about 10% faster than the conditional.
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
However, when the name is not in the dictionary, we see that there is almost no difference:
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
A note on correctness
In general, I would prefer the or
boolean operation to the conditional expression - with the following caveats:
- The dictionary is guaranteed to only have non-empty strings or
None
. - Performance here is critical.
In the case where either the above is not true, I would prefer the following for correctness:
first_name = account['first_name']
if first_name is None:
first_name = ''
The upsides are that
- the lookup is done one time,
- the check for
is None
is quite fast, - the code is explicitly clear, and
- the code is easily maintainable by any Python programmer.
This should also not be any less performant:
def correct(name=False):
d = 'name': 'thename' if name else None
def _correct():
first_name = d['name']
if first_name is None:
first_name = ''
return _correct
We see that we get quite competitive performance when the key is there:
>>> min(timeit.repeat(correct(name=True), repeat=10))
0.10948465298861265
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
when the key is not in the dictionary, it is not quite as good though:
>>> min(timeit.repeat(correct(name=False), repeat=10))
0.11776355793699622
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
Conclusion
The difference between the conditional expression and the boolean operation is two versus one lookups respectively on a True
condition, making the boolean operation more performant.
For correctness's sake, however, do the lookup one time, check for identity to None
with is None
, and then reassign to the empty string in that case.
What is the difference between the two following expressions?
first_name = account['first_name'] if account['first_name'] else ""
vs
first_name = account['first_name'] or ""
The primary difference is that the first, in Python, is the conditional expression,
The expression
x if C else y
first evaluates the condition,C
rather
thanx
. IfC
is true,x
is evaluated and its value is returned;
otherwise,y
is evaluated and its value is returned.
while the second uses the boolean operation:
The expression
x or y
first evaluatesx
; ifx
is true, its value is
returned; otherwise,y
is evaluated and the resulting value is
returned.
Note that the first may require two key lookups versus the second, which only requires one key lookup.
This lookup is called subscript notation:
name[subscript_argument]
Subscript notation exercises the __getitem__
method of the object referenced by name
.
It requires both the name and the subscript argument to be loaded.
Now, in the context of the question, if it tests as True
in a boolean context (which a non-empty string does, but None
does not) it will require a second (redundant) loading of both the dictionary and the key for the conditional expression, while simply returning the first lookup for the boolean or
operation.
Therefore I would expect the second, the boolean operation, to be slightly more efficient in cases where the value is not None
.
Abstract Syntax Tree (AST) breakdown
Others have compared the bytecode generated by both expressions.
However, the AST represents the first breakdown of the language as parsed by the interpreter.
The following AST demonstrates that the second lookup likely involves more work (note I have formatted the output for easier parsing):
>>> print(ast.dump(ast.parse("account['first_name'] if account['first_name'] else ''").body[0]))
Expr(
value=IfExp(
test=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
body=Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
orelse=Str(s='')
))
versus
>>> print(ast.dump(ast.parse("account['first_name'] or ''").body[0]))
Expr(
value=BoolOp(
op=Or(),
values=[
Subscript(value=Name(id='account', ctx=Load()),
slice=Index(value=Str(s='first_name')), ctx=Load()),
Str(s='')]
)
)
Bytecode analysis
Here we see that the bytecode for the conditional expression is much longer. This usually bodes poorly for relative performance in my experience.
>>> import dis
>>> dis.dis("d['name'] if d['name'] else ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 POP_JUMP_IF_FALSE 16
8 LOAD_NAME 0 (d)
10 LOAD_CONST 0 ('name')
12 BINARY_SUBSCR
14 RETURN_VALUE
>> 16 LOAD_CONST 1 ('')
18 RETURN_VALUE
For the boolean operation, it's almost half as long:
>>> dis.dis("d['name'] or ''")
1 0 LOAD_NAME 0 (d)
2 LOAD_CONST 0 ('name')
4 BINARY_SUBSCR
6 JUMP_IF_TRUE_OR_POP 10
8 LOAD_CONST 1 ('')
>> 10 RETURN_VALUE
Here I would expect the performance to be much quicker relative to the other.
Therefore, let's see if there's much difference in performance then.
Performance
Performance is not very important here, but sometimes I have to see for myself:
def cond(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] if d['name'] else ''
def bool_op(name=False):
d = 'name': 'thename' if name else None
return lambda: d['name'] or ''
We see that when the name is in the dictionary, the boolean operation is about 10% faster than the conditional.
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
However, when the name is not in the dictionary, we see that there is almost no difference:
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
A note on correctness
In general, I would prefer the or
boolean operation to the conditional expression - with the following caveats:
- The dictionary is guaranteed to only have non-empty strings or
None
. - Performance here is critical.
In the case where either the above is not true, I would prefer the following for correctness:
first_name = account['first_name']
if first_name is None:
first_name = ''
The upsides are that
- the lookup is done one time,
- the check for
is None
is quite fast, - the code is explicitly clear, and
- the code is easily maintainable by any Python programmer.
This should also not be any less performant:
def correct(name=False):
d = 'name': 'thename' if name else None
def _correct():
first_name = d['name']
if first_name is None:
first_name = ''
return _correct
We see that we get quite competitive performance when the key is there:
>>> min(timeit.repeat(correct(name=True), repeat=10))
0.10948465298861265
>>> min(timeit.repeat(cond(name=True), repeat=10))
0.11814919696189463
>>> min(timeit.repeat(bool_op(name=True), repeat=10))
0.10678509017452598
when the key is not in the dictionary, it is not quite as good though:
>>> min(timeit.repeat(correct(name=False), repeat=10))
0.11776355793699622
>>> min(timeit.repeat(cond(name=False), repeat=10))
0.10031125508248806
>>> min(timeit.repeat(bool_op(name=False), repeat=10))
0.10030031995847821
Conclusion
The difference between the conditional expression and the boolean operation is two versus one lookups respectively on a True
condition, making the boolean operation more performant.
For correctness's sake, however, do the lookup one time, check for identity to None
with is None
, and then reassign to the empty string in that case.
edited Aug 12 at 0:34
answered Aug 11 at 23:54


Aaron Hall♦
150k44274241
150k44274241
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51802974%2fif-else-vs-or-operation-for-none-check%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Are the first names predominantly set or not?
– MisterMiyagi
Aug 13 at 12:06