Find positions of certain value in a big boolean list

up vote
8
down vote

favorite

I would like to know is it possible to find positions of False in t:

t = Table[RandomChoice[True, False], 6000];
Flatten[Position[t, False]] // AbsoluteTiming

faster than the above method.
I will appreciate any help.

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

asked Aug 9 at 17:12

Maria Sargsyan

984

add a commentÂ |Â

up vote
8
down vote

favorite

I would like to know is it possible to find positions of False in t:

t = Table[RandomChoice[True, False], 6000];
Flatten[Position[t, False]] // AbsoluteTiming

faster than the above method.
I will appreciate any help.

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

asked Aug 9 at 17:12

Maria Sargsyan

984

add a commentÂ |Â

up vote
8
down vote

favorite

I would like to know is it possible to find positions of False in t:

t = Table[RandomChoice[True, False], 6000];
Flatten[Position[t, False]] // AbsoluteTiming

faster than the above method.
I will appreciate any help.

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

asked Aug 9 at 17:12

Maria Sargsyan

984

I would like to know is it possible to find positions of False in t:

t = Table[RandomChoice[True, False], 6000];
Flatten[Position[t, False]] // AbsoluteTiming

faster than the above method.
I will appreciate any help.

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

asked Aug 9 at 17:12

Maria Sargsyan

984

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

edited Aug 10 at 5:06

Mr.Wizardâ™¦

227k284631014

asked Aug 9 at 17:12

Maria Sargsyan

984

asked Aug 9 at 17:12

Maria Sargsyan

984

asked Aug 9 at 17:12

Maria Sargsyan

984

add a commentÂ |Â

3 Answers
3

active

oldest

votes

up vote
5
down vote

accepted

Speed here is hindered by the fact that True/False is not a packable type in Mathematica, although I personally think it should be.

If it is possible to reformulate your problem to use 1/0 instead, which can be packed, the methods already provided (Pick and SparseArray) each become much faster.

You can convert your data using With faster than using Boole, but the overhead is still significant:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

b = Developer`ToPackedArray @ 
 With[True = 1, False = 0, Evaluate @ t]; // RepeatedTiming

0.000151, Null

Now observe how fast Pick becomes compared to its direct application on t:

r1 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming

r2 = Pick[Range @ Length @ b, b, 0]; // RepeatedTiming

r1 === r2

0.000343, Null

0.0000509, Null

True

Combined with the overhead of the conversion this is a little slower than kglr's method (0.000176 second on my machine), but it gives an idea of the performance that is possible if you can avoid True/False and use packed integers instead.

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

1

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

1

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

add a commentÂ |Â

up vote
6
down vote

You can use Pick:

t = Table[RandomChoice[True,False],6000];
r1 = Flatten[Position[t,False]]; //RepeatedTiming
r2 = Pick[Range@Length@t,t,False]; //RepeatedTiming

r1 === r2

0.0023, Null

0.00036, Null

True

Another useful alternative is PositionIndex, especially when you want to know the positions of different values:

r3 = PositionIndex[t]; //RepeatedTiming

r1 === r2 === r3[False]

0.00070, Null

True

answered Aug 9 at 17:20

Carl Woll

55.4k271144

add a commentÂ |Â

up vote
5
down vote

Using SparseArray with "AdjacencyLists" or with "NonzeroPositions" is faster than alternatives posted so far:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

r4 =SparseArray[t, Automatic, True]["AdjacencyLists"]; //RepeatedTiming // First

0.00021

r5 = Flatten@SparseArray[t, Automatic, True]["NonzeroPositions"]; //RepeatedTiming// First

0.00021

versus

r1 = Flatten[Position[t, False]]; // RepeatedTiming // First

0.0027

r2 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming // First

0.000414

r3 = PositionIndex[t][False]; // RepeatedTiming // First

0.000830

(b = Developer`ToPackedArray @ With[True =1, False =0, Evaluate @ t]; 
 r6 = Pick[Range @ Length @ b, b, 0];) //RepeatedTiming // First

0.00028

SameQ[r1, r2, r3, r4, r5, r6]

True

where r1 is from OP, r2 and r3 are from Carl Woll's and r6 is from Mr.Wizard's answer.

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

add a commentÂ |Â

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\$","\$"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "387"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f179770%2ffind-positions-of-certain-value-in-a-big-boolean-list%23new-answer', 'question_page');

);

Post as a guest

Name

3 Answers
3

active

oldest

votes

3 Answers
3

active

oldest

votes

up vote
5
down vote

accepted

Speed here is hindered by the fact that True/False is not a packable type in Mathematica, although I personally think it should be.

If it is possible to reformulate your problem to use 1/0 instead, which can be packed, the methods already provided (Pick and SparseArray) each become much faster.

You can convert your data using With faster than using Boole, but the overhead is still significant:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

b = Developer`ToPackedArray @ 
 With[True = 1, False = 0, Evaluate @ t]; // RepeatedTiming

0.000151, Null

Now observe how fast Pick becomes compared to its direct application on t:

r1 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming

r2 = Pick[Range @ Length @ b, b, 0]; // RepeatedTiming

r1 === r2

0.000343, Null

0.0000509, Null

True

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

1

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

1

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

add a commentÂ |Â

up vote
5
down vote

accepted

Speed here is hindered by the fact that True/False is not a packable type in Mathematica, although I personally think it should be.

If it is possible to reformulate your problem to use 1/0 instead, which can be packed, the methods already provided (Pick and SparseArray) each become much faster.

You can convert your data using With faster than using Boole, but the overhead is still significant:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

b = Developer`ToPackedArray @ 
 With[True = 1, False = 0, Evaluate @ t]; // RepeatedTiming

0.000151, Null

Now observe how fast Pick becomes compared to its direct application on t:

r1 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming

r2 = Pick[Range @ Length @ b, b, 0]; // RepeatedTiming

r1 === r2

0.000343, Null

0.0000509, Null

True

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

1

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

1

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

add a commentÂ |Â

up vote
5
down vote

accepted

Speed here is hindered by the fact that True/False is not a packable type in Mathematica, although I personally think it should be.

If it is possible to reformulate your problem to use 1/0 instead, which can be packed, the methods already provided (Pick and SparseArray) each become much faster.

You can convert your data using With faster than using Boole, but the overhead is still significant:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

b = Developer`ToPackedArray @ 
 With[True = 1, False = 0, Evaluate @ t]; // RepeatedTiming

0.000151, Null

Now observe how fast Pick becomes compared to its direct application on t:

r1 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming

r2 = Pick[Range @ Length @ b, b, 0]; // RepeatedTiming

r1 === r2

0.000343, Null

0.0000509, Null

True

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

Speed here is hindered by the fact that True/False is not a packable type in Mathematica, although I personally think it should be.

If it is possible to reformulate your problem to use 1/0 instead, which can be packed, the methods already provided (Pick and SparseArray) each become much faster.

You can convert your data using With faster than using Boole, but the overhead is still significant:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

b = Developer`ToPackedArray @ 
 With[True = 1, False = 0, Evaluate @ t]; // RepeatedTiming

0.000151, Null

Now observe how fast Pick becomes compared to its direct application on t:

r1 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming

r2 = Pick[Range @ Length @ b, b, 0]; // RepeatedTiming

r1 === r2

0.000343, Null

0.0000509, Null

True

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

answered Aug 10 at 4:46

Mr.Wizardâ™¦

227k284631014

1

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

1

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

add a commentÂ |Â

1

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

1

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

Wow, I am really surprised that With[True = 1, False = 0, Evaluate @ t] is faster than Boole.
â€“Â Henrik Schumacher
Aug 10 at 6:54

@Henrik It's kind of disappointing isn't it?
â€“Â Mr.Wizardâ™¦
Aug 10 at 14:58

Yes, it is... As well as the general slowness of Booleans in Mathematica.
â€“Â Henrik Schumacher
Aug 10 at 20:06

@Henrik Yet another case where there is room for improvement in the core language that is neglected, all the while new functions applicable only for some are added.
â€“Â Mr.Wizardâ™¦
Aug 10 at 20:20

sigh I am totally with you in that respect.
â€“Â Henrik Schumacher
Aug 10 at 20:21

add a commentÂ |Â

up vote
6
down vote

You can use Pick:

t = Table[RandomChoice[True,False],6000];
r1 = Flatten[Position[t,False]]; //RepeatedTiming
r2 = Pick[Range@Length@t,t,False]; //RepeatedTiming

r1 === r2

0.0023, Null

0.00036, Null

True

Another useful alternative is PositionIndex, especially when you want to know the positions of different values:

r3 = PositionIndex[t]; //RepeatedTiming

r1 === r2 === r3[False]

0.00070, Null

True

answered Aug 9 at 17:20

Carl Woll

55.4k271144

add a commentÂ |Â

up vote
6
down vote

You can use Pick:

t = Table[RandomChoice[True,False],6000];
r1 = Flatten[Position[t,False]]; //RepeatedTiming
r2 = Pick[Range@Length@t,t,False]; //RepeatedTiming

r1 === r2

0.0023, Null

0.00036, Null

True

Another useful alternative is PositionIndex, especially when you want to know the positions of different values:

r3 = PositionIndex[t]; //RepeatedTiming

r1 === r2 === r3[False]

0.00070, Null

True

answered Aug 9 at 17:20

Carl Woll

55.4k271144

add a commentÂ |Â

up vote
6
down vote

You can use Pick:

t = Table[RandomChoice[True,False],6000];
r1 = Flatten[Position[t,False]]; //RepeatedTiming
r2 = Pick[Range@Length@t,t,False]; //RepeatedTiming

r1 === r2

0.0023, Null

0.00036, Null

True

Another useful alternative is PositionIndex, especially when you want to know the positions of different values:

r3 = PositionIndex[t]; //RepeatedTiming

r1 === r2 === r3[False]

0.00070, Null

True

answered Aug 9 at 17:20

Carl Woll

55.4k271144

You can use Pick:

t = Table[RandomChoice[True,False],6000];
r1 = Flatten[Position[t,False]]; //RepeatedTiming
r2 = Pick[Range@Length@t,t,False]; //RepeatedTiming

r1 === r2

0.0023, Null

0.00036, Null

True

Another useful alternative is PositionIndex, especially when you want to know the positions of different values:

r3 = PositionIndex[t]; //RepeatedTiming

r1 === r2 === r3[False]

0.00070, Null

True

answered Aug 9 at 17:20

Carl Woll

55.4k271144

answered Aug 9 at 17:20

Carl Woll

55.4k271144

answered Aug 9 at 17:20

Carl Woll

55.4k271144

answered Aug 9 at 17:20

Carl Woll

55.4k271144

add a commentÂ |Â

up vote
5
down vote

Using SparseArray with "AdjacencyLists" or with "NonzeroPositions" is faster than alternatives posted so far:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

r4 =SparseArray[t, Automatic, True]["AdjacencyLists"]; //RepeatedTiming // First

0.00021

r5 = Flatten@SparseArray[t, Automatic, True]["NonzeroPositions"]; //RepeatedTiming// First

0.00021

versus

r1 = Flatten[Position[t, False]]; // RepeatedTiming // First

0.0027

r2 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming // First

0.000414

r3 = PositionIndex[t][False]; // RepeatedTiming // First

0.000830

(b = Developer`ToPackedArray @ With[True =1, False =0, Evaluate @ t]; 
 r6 = Pick[Range @ Length @ b, b, 0];) //RepeatedTiming // First

0.00028

SameQ[r1, r2, r3, r4, r5, r6]

True

where r1 is from OP, r2 and r3 are from Carl Woll's and r6 is from Mr.Wizard's answer.

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

add a commentÂ |Â

up vote
5
down vote

Using SparseArray with "AdjacencyLists" or with "NonzeroPositions" is faster than alternatives posted so far:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

r4 =SparseArray[t, Automatic, True]["AdjacencyLists"]; //RepeatedTiming // First

0.00021

r5 = Flatten@SparseArray[t, Automatic, True]["NonzeroPositions"]; //RepeatedTiming// First

0.00021

versus

r1 = Flatten[Position[t, False]]; // RepeatedTiming // First

0.0027

r2 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming // First

0.000414

r3 = PositionIndex[t][False]; // RepeatedTiming // First

0.000830

(b = Developer`ToPackedArray @ With[True =1, False =0, Evaluate @ t]; 
 r6 = Pick[Range @ Length @ b, b, 0];) //RepeatedTiming // First

0.00028

SameQ[r1, r2, r3, r4, r5, r6]

True

where r1 is from OP, r2 and r3 are from Carl Woll's and r6 is from Mr.Wizard's answer.

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

add a commentÂ |Â

up vote
5
down vote

Using SparseArray with "AdjacencyLists" or with "NonzeroPositions" is faster than alternatives posted so far:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

r4 =SparseArray[t, Automatic, True]["AdjacencyLists"]; //RepeatedTiming // First

0.00021

r5 = Flatten@SparseArray[t, Automatic, True]["NonzeroPositions"]; //RepeatedTiming// First

0.00021

versus

r1 = Flatten[Position[t, False]]; // RepeatedTiming // First

0.0027

r2 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming // First

0.000414

r3 = PositionIndex[t][False]; // RepeatedTiming // First

0.000830

(b = Developer`ToPackedArray @ With[True =1, False =0, Evaluate @ t]; 
 r6 = Pick[Range @ Length @ b, b, 0];) //RepeatedTiming // First

0.00028

SameQ[r1, r2, r3, r4, r5, r6]

True

where r1 is from OP, r2 and r3 are from Carl Woll's and r6 is from Mr.Wizard's answer.

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

Using SparseArray with "AdjacencyLists" or with "NonzeroPositions" is faster than alternatives posted so far:

SeedRandom[1]
t = Table[RandomChoice[True, False], 6000];

r4 =SparseArray[t, Automatic, True]["AdjacencyLists"]; //RepeatedTiming // First

0.00021

r5 = Flatten@SparseArray[t, Automatic, True]["NonzeroPositions"]; //RepeatedTiming// First

0.00021

versus

r1 = Flatten[Position[t, False]]; // RepeatedTiming // First

0.0027

r2 = Pick[Range @ Length @ t, t, False]; // RepeatedTiming // First

0.000414

r3 = PositionIndex[t][False]; // RepeatedTiming // First

0.000830

(b = Developer`ToPackedArray @ With[True =1, False =0, Evaluate @ t]; 
 r6 = Pick[Range @ Length @ b, b, 0];) //RepeatedTiming // First

0.00028

SameQ[r1, r2, r3, r4, r5, r6]

True

where r1 is from OP, r2 and r3 are from Carl Woll's and r6 is from Mr.Wizard's answer.

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

edited Aug 10 at 11:06

answered Aug 10 at 3:49

kglr

157k8182379

answered Aug 10 at 3:49

kglr

157k8182379

answered Aug 10 at 3:49

kglr

157k8182379

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

Search This Blog

Iyfjky