Merge two DataFrames based on columns and values of a specific column with Pandas in Python 3.x

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
6
down vote

favorite
1












Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question



















  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago















up vote
6
down vote

favorite
1












Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question



















  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago













up vote
6
down vote

favorite
1









up vote
6
down vote

favorite
1






1





Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question















Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge




python pandas csv merge






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 3 hours ago

























asked 3 hours ago









Michael Gann

343




343







  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago













  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago








1




1




The easiest way to do this would be to melt df2 and then do a left join on df1.
– CJ59
3 hours ago




The easiest way to do this would be to melt df2 and then do a left join on df1.
– CJ59
3 hours ago




1




1




Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
– Michael Gann
3 hours ago





Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
– Michael Gann
3 hours ago













2 Answers
2






active

oldest

votes

















up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago

















up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214900%2fmerge-two-dataframes-based-on-columns-and-values-of-a-specific-column-with-panda%23new-answer', 'question_page');

);

Post as a guest

















2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago














up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












up vote
9
down vote










up vote
9
down vote









You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer














You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64






share|improve this answer














share|improve this answer



share|improve this answer








edited 3 hours ago

























answered 3 hours ago









Vaishali

16.2k3927




16.2k3927







  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago







1




1




Thank you i will try that out!
– Michael Gann
3 hours ago




Thank you i will try that out!
– Michael Gann
3 hours ago




1




1




Works fine! Thank you very much!
– Michael Gann
2 hours ago




Works fine! Thank you very much!
– Michael Gann
2 hours ago












up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago














up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












up vote
3
down vote










up vote
3
down vote









Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer












Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64






share|improve this answer












share|improve this answer



share|improve this answer










answered 3 hours ago









W-B

90.5k72754




90.5k72754











  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago
















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago















Works fine! Thank you very much!
– Michael Gann
2 hours ago




Works fine! Thank you very much!
– Michael Gann
2 hours ago

















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214900%2fmerge-two-dataframes-based-on-columns-and-values-of-a-specific-column-with-panda%23new-answer', 'question_page');

);

Post as a guest
































































Comments

Post a Comment

Popular posts from this blog

Long meetings (6-7 hours a day): Being “babysat” by supervisor

Image
Clash Royale CLAN TAG #URR8PPP .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0; up vote 31 down vote favorite 4 I hold a PhD in mechanical engineering with a 7 year background in self-guided, self-paced research including 2 years as a graduate level course instructor and a subject matter expert for various engineering projects in a university setting in the United States. For the last 10-11 months, I am working in a research industry in France where my task is to debug C++ spaghetti code written over a period of several years by my current supervisor. It is a team of 2: just I and my supervisor. Diurnally, I am posed with a "bug of the day" to get out of this C++ code. My supervisor generally gives me about 15-30 minutes to solve it and then accosts me about whether or not the task is completed and then rinses and repeats this until the end of the day. Off-late, my supervisor has been holding 6-7 hour long "meetin
Read more

What does second last employer means? [closed]

Image
Clash Royale CLAN TAG #URR8PPP .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0; up vote -3 down vote favorite Say, for example I have worked in the following companies, Company A => 2010 - 2011 Company B => 2011 - 2012 Company C => 2012 - Till Date Company D => Have offer Now assume Company D is asking to submit documents from my second last employer. So from the list of companies given, which company would be qualified as second last employer? software-industry employer share | improve this question asked Aug 10 '14 at 5:16 Rajaprabhu Aravindasamy 559 1 6 13 closed as off-topic by Jim G., jmort253 ♦ Aug 11 '14 at 1:58 This question appears to be off-topic. The users who voted to close gave this specific reason: "Questions seeking advice on company-specific regulations, agreements, or policies should be directed to your manager or HR department. Questions that address only
Read more

One-line joke

A one-liner is a joke that is delivered in a single line. A good one-liner is said to be pithy - concise and meaningful. [1] Comedians and actors use this comedic method as part of their act, e.g. Jimmy Carr, Tommy Cooper, Rodney Dangerfield, Norm Macdonald, Ken Dodd, Stewart Francis, Zach Galifianakis, Mitch Hedberg, Anthony Jeselnik, Milton Jones, Shappi Khorsandi, Jay London, Mark Linn-Baker, Demetri Martin, Groucho Marx, Dan Mintz, Emo Philips, Tim Vine, Steven Wright and Henny Youngman and Arnold Schwarzenegger. [ citation needed ] Many fictional characters are also known to deliver one-liners, including James Bond, who usually includes pithy and laconic quips after disposing of a villain. [ citation needed ] Examples "A baby seal walks into a club." "A dyslexic man walks into a bra." (Anonymous) "There are three types of people, those who can count and those who can't." "Venison's dear, isn't it?" (Jimmy Carr) "An escala
Read more