Mixing
Removing dupes in list of lists in Python
Clash Royale CLAN TAG#URR8PPP
up vote
9
down vote
favorite
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
asked Aug 26 at 19:18
Jamie Lunn
514
add a comment |Â
up vote
9
down vote
favorite
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
asked Aug 26 at 19:18
Jamie Lunn
514
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22
add a comment |Â
up vote
9
down vote
favorite
up vote
9
down vote
favorite
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
asked Aug 26 at 19:18
Jamie Lunn
514
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
asked Aug 26 at 19:18
Jamie Lunn
514
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
edited Aug 27 at 2:16
Peter Mortensen
12.9k1983111
12.9k1983111
asked Aug 26 at 19:18
Jamie Lunn
514
asked Aug 26 at 19:18
Jamie Lunn
514
asked Aug 26 at 19:18
Jamie Lunn
514
514
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22
add a comment |Â
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22
2
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22
add a comment |Â
4 Answers
4
active
oldest
votes
up vote
5
down vote
accepted
You can use collections.Counter with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
answered Aug 26 at 19:27
blhsing
13k2628
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
add a comment |Â
up vote
6
down vote
An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
answered Aug 26 at 19:30
Joe Iddon
13k31338
2
filtered = [l for l in a if counts[l[0]] == 1]is probably clearer but yes.
– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
add a comment |Â
up vote
6
down vote
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.
answered Aug 26 at 19:41
jpp
63.4k173680
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist+dictare the only way to structure data!
– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandaspart obviously :)
– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkoutnumpy- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
add a comment |Â
up vote
0
down vote
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
answered Aug 26 at 20:35
vash_the_stampede
1567
add a comment |Â
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
5
down vote
accepted
You can use collections.Counter with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
answered Aug 26 at 19:27
blhsing
13k2628
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
add a comment |Â
up vote
5
down vote
accepted
You can use collections.Counter with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
answered Aug 26 at 19:27
blhsing
13k2628
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
add a comment |Â
up vote
5
down vote
accepted
up vote
5
down vote
accepted
You can use collections.Counter with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
answered Aug 26 at 19:27
blhsing
13k2628
You can use collections.Counter with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
answered Aug 26 at 19:27
blhsing
13k2628
edited Aug 26 at 19:52
answered Aug 26 at 19:27
blhsing
13k2628
answered Aug 26 at 19:27
blhsing
13k2628
answered Aug 26 at 19:27
blhsing
13k2628
13k2628
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
add a comment |Â
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
that sounds overcomplicated
– Jean-François Fabre
Aug 26 at 19:39
@Jean-FrançoisFabre its basically
[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]– Fabian N.
Aug 26 at 19:42
@Jean-FrançoisFabre its basically
[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]– Fabian N.
Aug 26 at 19:42
1
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 at 19:44
1
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 at 19:54
add a comment |Â
up vote
6
down vote
An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
answered Aug 26 at 19:30
Joe Iddon
13k31338
2
filtered = [l for l in a if counts[l[0]] == 1]is probably clearer but yes.
– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
add a comment |Â
up vote
6
down vote
An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
answered Aug 26 at 19:30
Joe Iddon
13k31338
2
filtered = [l for l in a if counts[l[0]] == 1]is probably clearer but yes.
– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
add a comment |Â
up vote
6
down vote
up vote
6
down vote
An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
answered Aug 26 at 19:30
Joe Iddon
13k31338
An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
answered Aug 26 at 19:30
Joe Iddon
13k31338
edited Aug 26 at 19:39
answered Aug 26 at 19:30
Joe Iddon
13k31338
answered Aug 26 at 19:30
Joe Iddon
13k31338
answered Aug 26 at 19:30
Joe Iddon
13k31338
13k31338
2
filtered = [l for l in a if counts[l[0]] == 1]is probably clearer but yes.
– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
add a comment |Â
2
filtered = [l for l in a if counts[l[0]] == 1]is probably clearer but yes.
– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
2
2
filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.– Jean-François Fabre
Aug 26 at 19:37
filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.– Jean-François Fabre
Aug 26 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 at 19:38
3
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 at 19:38
add a comment |Â
up vote
6
down vote
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.
answered Aug 26 at 19:41
jpp
63.4k173680
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist+dictare the only way to structure data!
– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandaspart obviously :)
– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkoutnumpy- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
add a comment |Â
up vote
6
down vote
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.
answered Aug 26 at 19:41
jpp
63.4k173680
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist+dictare the only way to structure data!
– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandaspart obviously :)
– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkoutnumpy- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
add a comment |Â
up vote
6
down vote
up vote
6
down vote
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.
answered Aug 26 at 19:41
jpp
63.4k173680
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.
answered Aug 26 at 19:41
jpp
63.4k173680
answered Aug 26 at 19:41
jpp
63.4k173680
answered Aug 26 at 19:41
jpp
63.4k173680
answered Aug 26 at 19:41
jpp
63.4k173680
63.4k173680
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist+dictare the only way to structure data!
– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandaspart obviously :)
– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkoutnumpy- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
add a comment |Â
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist+dictare the only way to structure data!
– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandaspart obviously :)
– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkoutnumpy- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
3
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 at 19:48
2
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python
list + dict are the only way to structure data!– jpp
Aug 26 at 19:52
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python
list + dict are the only way to structure data!– jpp
Aug 26 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the
import pandas part obviously :)– Jean-François Fabre
Aug 26 at 19:56
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the
import pandas part obviously :)– Jean-François Fabre
Aug 26 at 19:56
@FabianN. You should also checkout
numpy - very fast, especially on large data sets like when image processing.– Joe Iddon
Aug 26 at 20:56
@FabianN. You should also checkout
numpy - very fast, especially on large data sets like when image processing.– Joe Iddon
Aug 26 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 at 21:27
add a comment |Â
up vote
0
down vote
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
answered Aug 26 at 20:35
vash_the_stampede
1567
add a comment |Â
up vote
0
down vote
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
answered Aug 26 at 20:35
vash_the_stampede
1567
add a comment |Â
up vote
0
down vote
up vote
0
down vote
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
answered Aug 26 at 20:35
vash_the_stampede
1567
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
answered Aug 26 at 20:35
vash_the_stampede
1567
edited Aug 26 at 20:46
answered Aug 26 at 20:35
vash_the_stampede
1567
answered Aug 26 at 20:35
vash_the_stampede
1567
answered Aug 26 at 20:35
vash_the_stampede
1567
1567
add a comment |Â
add a comment |Â
Â
draft saved
draft discarded
Â
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52029529%2fremoving-dupes-in-list-of-lists-in-python%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 at 19:22