Как соединить линии, которые имеют один и тот же ключ, в одну линию?

Eсть DataFrame, и хотел бы сделать еще один столбец, который объединяет столбцы, чье имя начинается с того же самого значения в Answer и QID.

Вот данные, и их пример:

    QID     Category    Text    QType   Question    Answer0     Answer1
0   16  Automotive  Access to car   Single  Do you have access to a car?    I own a car/cars    I own a car/cars
1   16  Automotive  Access to car   Single  Do you have access to a car?    I lease/ have a company car     I lease/have a company car
2   16  Automotive  Access to car   Single  Do you have access to a car?    I have access to a car/cars     I have access to a car/cars
3   16  Automotive  Access to car   Single  Do you have access to a car?    No, I don’t have access to a car/cars   No, I don't have access to a car
4   16  Automotive  Access to car   Single  Do you have access to a car?    Prefer not to say   Prefer not to say
5   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Audi    Audi
6   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Alfa Romeo  Alfa Romeo
7   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    BMW     BMW
8   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Cadillac    Cadillac
9   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chevrolet   Chevrolet
10  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Chrysler    Chrysler
11  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Citroen     Citroen
12  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Daihatsu    Daihatsu
13  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Fiat    Fiat
14  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Ford    Ford
15  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Honda   Honda
16  17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Hyundai     Hyundai
...

Xотел бы получить что-то подобное:

    QID     Category    Text    QType   Question    Answer0     Answer1     Answer3     Answer4     Answer5     Answer6     Answer7     Answer8     Answer9     Answer10    Answer11     Answer12     ...      
4   16  Automotive  Access to car   Single  Do you have access to a car?    I own a car/cars    I lease/ have a company car     I have access to a car/cars     No, I don’t have access to a car/cars   Prefer not to say       
5   17  Automotive  Make of car/cars    Multiple    If you own/lease a car(s), which brand are they?    Audi    Alfa Romeo  BMW     Cadillac    Chevrolet   Chrysler    Citroen     ...

Я могу комбинировать подаренное/статическое число столбцов, чье имя начинается с одного и того же значения в Answer и QID:

df = pd.DataFrame('path/to/file')

# ленивый - в первую очередь нужны атрибуты, кроме столбцов QID и Answer
agg = {col:"first" for col in list(df.columns) if col!="QID" and "Answer" not in col}
# получить список всех ответов в Answer0 для QID
agg = {**agg, **{"Answer0":lambda s: list(s)}}

# вспомогательная функция для вызова ряда. не нужна, но делает более читабельной.
def ans(r, i):
    return "" if i>=len(r["AnswerT"]) else r["AnswerT"][i]

# разделить список от объединения обратно на столбцы с помощью назначения
# переименовать Answer0 в AnserT из агрегирования, чтобы на него можно было ссылаться.  
# AnswerT бросай, когда больше не хочешь.
dfgrouped = df.groupby("QID").agg(agg).reset_index().rename(columns={"Answer0":"AnswerT"}).assign(
    Answer0=lambda dfa: dfa.apply(lambda r: ans(r, 0), axis=1),
    Answer1=lambda dfa: dfa.apply(lambda r: ans(r, 1), axis=1),
    Answer2=lambda dfa: dfa.apply(lambda r: ans(r, 2), axis=1),
    Answer3=lambda dfa: dfa.apply(lambda r: ans(r, 3), axis=1),
    Answer4=lambda dfa: dfa.apply(lambda r: ans(r, 4), axis=1),
    Answer5=lambda dfa: dfa.apply(lambda r: ans(r, 5), axis=1),
    Answer6=lambda dfa: dfa.apply(lambda r: ans(r, 6), axis=1),
).drop("AnswerT", axis=1)

print(dfgrouped.to_string(index=False))

А как можно комбинировать динамическое число столбцов, где эти столбцы имеют имена, которые начинаются с одного и того же значения в Answer и QID?


Ответы (1 шт):

Автор решения: MaxU

Попробуйте так:

res = (df
       .assign(x=df.groupby("Question")["Category"].cumcount())
       .drop(columns=["Unnamed: 6"])
       .pivot_table(index=["Question","Category","Text","Choice","Question:"], 
                    columns="x", 
                    values="Answer Option",
                    aggfunc="first")
       .add_prefix("Answer")
       .reset_index())

результат:

In [10]: res
Out[10]:
x   Question     Category                             Text    Choice                                Question:  ...  \
0          4    Household               Number of children    Single  How many children under the age of 1...  ...
1         12      Finance   Personal Income Classification    Single  What is your personal income, before...  ...
2         14    Household                 Ages of children  Multiple            When were your children born?  ...
3         15    Household                    Accommodation    Single    What is your accommodation situation?  ...
4         16   Automotive                    Access to car    Single             Do you have access to a car?  ...
..       ...          ...                              ...       ...                                      ...  ...
148      683       Travel  Travel destination past 3 years  Multiple  Which of the following countries/reg...  ...
149      689       Gaming      Online video gaming content  Multiple  On which of the following platforms ...  ...
150      693        Media         Music streaming services  Multiple  Which of following music streaming s...  ...
151      696  Electronics         electronic purchases P3Y  Multiple  Which of the following electronic de...  ...
152       Id     Category                             Name    Choice                                     Text  ...

x   Answer134 Answer135 Answer136 Answer137 Answer138
0         NaN       NaN       NaN       NaN       NaN
1         NaN       NaN       NaN       NaN       NaN
2         NaN       NaN       NaN       NaN       NaN
3         NaN       NaN       NaN       NaN       NaN
4         NaN       NaN       NaN       NaN       NaN
..        ...       ...       ...       ...       ...
148       NaN       NaN       NaN       NaN       NaN
149       NaN       NaN       NaN       NaN       NaN
150       NaN       NaN       NaN       NaN       NaN
151       NaN       NaN       NaN       NaN       NaN
152       NaN       NaN       NaN       NaN       NaN

[153 rows x 144 columns]
→ Ссылка