Как усреднить значения по точным группам времени?

Question

Есть массив

data = [
   {"timestamp": 1604029392, "val": 23.88},
   {"timestamp": 1604028792, "val": 28.99},
   {"timestamp": 1604028192, "val": 22.20},
   {"timestamp": 1604027592, "val": 29.02},
   {"timestamp": 1604026992, "val": 28.46},
   {"timestamp": 1604026392, "val": 20.67},
   {"timestamp": 1604025792, "val": 20.80},
   {"timestamp": 1604025192, "val": 22.51},
   {"timestamp": 1604024592, "val": 23.72},
   {"timestamp": 1604023992, "val": 28.37},
   {"timestamp": 1604023392, "val": 22.33},
   {"timestamp": 1604022792, "val": 21.80},
   {"timestamp": 1604022192, "val": 24.11},
   {"timestamp": 1604021592, "val": 26.96},
   {"timestamp": 1604020992, "val": 27.22},
   {"timestamp": 1604020392, "val": 21.24},
   {"timestamp": 1604019792, "val": 20.75},
   {"timestamp": 1604019192, "val": 21.81},
   {"timestamp": 1604018592, "val": 20.08},
   {"timestamp": 1604017992, "val": 24.58},
   {"timestamp": 1604017392, "val": 28.68},
   {"timestamp": 1604016792, "val": 29.03},
   {"timestamp": 1604016192, "val": 29.84},
   {"timestamp": 1604015592, "val": 29.30},
   {"timestamp": 1604014992, "val": 21.05},
   {"timestamp": 1604014392, "val": 21.68},
   {"timestamp": 1604013792, "val": 25.40},
   {"timestamp": 1604013192, "val": 29.71},
   {"timestamp": 1604012592, "val": 24.74},
   {"timestamp": 1604011992, "val": 28.97},
   {"timestamp": 1604011392, "val": 25.45},
   {"timestamp": 1604010792, "val": 22.80},
   {"timestamp": 1604010192, "val": 29.69},
   {"timestamp": 1604009592, "val": 25.21},
   {"timestamp": 1604008992, "val": 29.52},
   {"timestamp": 1604008392, "val": 27.05},
   {"timestamp": 1604007792, "val": 22.23},
   {"timestamp": 1604007192, "val": 26.83},
   {"timestamp": 1604006592, "val": 29.44},
   {"timestamp": 1604005992, "val": 26.33},
   {"timestamp": 1604005392, "val": 20.59},
   {"timestamp": 1604004792, "val": 23.01},
   {"timestamp": 1604004192, "val": 20.93},
   {"timestamp": 1604003592, "val": 26.58},
   {"timestamp": 1604002992, "val": 29.65},
   {"timestamp": 1604002392, "val": 27.04},
   {"timestamp": 1604001792, "val": 27.91},
   {"timestamp": 1604001192, "val": 28.34},
   {"timestamp": 1604000592, "val": 29.63},
   {"timestamp": 1603999992, "val": 20.81},
   {"timestamp": 1603999392, "val": 25.77},
   {"timestamp": 1603998792, "val": 21.31},
   {"timestamp": 1603998192, "val": 21.18},
   {"timestamp": 1603997592, "val": 23.52},
   {"timestamp": 1603996992, "val": 22.35},
   {"timestamp": 1603996392, "val": 20.99},
   {"timestamp": 1603995792, "val": 27.41},
   {"timestamp": 1603995192, "val": 21.08},
   {"timestamp": 1603994592, "val": 26.19},
   {"timestamp": 1603993992, "val": 23.32},
   {"timestamp": 1603993392, "val": 28.46},
   {"timestamp": 1603992792, "val": 24.04},
   {"timestamp": 1603992192, "val": 28.37},
   {"timestamp": 1603991592, "val": 22.59},
   {"timestamp": 1603990992, "val": 27.60},
   {"timestamp": 1603990392, "val": 22.78},
   {"timestamp": 1603989792, "val": 21.05},
   {"timestamp": 1603989192, "val": 28.48},
   {"timestamp": 1603988592, "val": 23.44},
   {"timestamp": 1603987992, "val": 22.00},
   {"timestamp": 1603987392, "val": 22.57},
   {"timestamp": 1603986792, "val": 22.96},
   {"timestamp": 1603986192, "val": 27.96},
   {"timestamp": 1603985592, "val": 26.46},
   {"timestamp": 1603984992, "val": 24.10},
   {"timestamp": 1603984392, "val": 28.32},
   {"timestamp": 1603983792, "val": 21.27},
   {"timestamp": 1603983192, "val": 27.93},
   {"timestamp": 1603982592, "val": 20.47},
   {"timestamp": 1603981992, "val": 21.16},
   {"timestamp": 1603981392, "val": 26.48},
   {"timestamp": 1603980792, "val": 24.69},
   {"timestamp": 1603980192, "val": 24.16},
   {"timestamp": 1603979592, "val": 23.41},
   {"timestamp": 1603978992, "val": 27.45},
   {"timestamp": 1603978392, "val": 25.15},
   {"timestamp": 1603977792, "val": 26.00},
   {"timestamp": 1603977192, "val": 23.48},
   {"timestamp": 1603976592, "val": 29.27},
   {"timestamp": 1603975992, "val": 28.72},
   {"timestamp": 1603975392, "val": 22.39},
   {"timestamp": 1603974792, "val": 24.94},
   {"timestamp": 1603974192, "val": 22.12},
   {"timestamp": 1603973592, "val": 20.75},
   {"timestamp": 1603972992, "val": 28.00},
   {"timestamp": 1603972392, "val": 24.77},
   {"timestamp": 1603971792, "val": 25.98},
   {"timestamp": 1603971192, "val": 24.46},
   {"timestamp": 1603970592, "val": 29.23},
   {"timestamp": 1603969992, "val": 29.09},
   {"timestamp": 1603969392, "val": 21.12},
   {"timestamp": 1603968792, "val": 20.82},
   {"timestamp": 1603968192, "val": 28.81},
   {"timestamp": 1603967592, "val": 20.29},
   {"timestamp": 1603966992, "val": 28.82},
   {"timestamp": 1603966392, "val": 25.81},
   {"timestamp": 1603965792, "val": 28.11},
   {"timestamp": 1603965192, "val": 28.74},
   {"timestamp": 1603964592, "val": 28.73},
   {"timestamp": 1603963992, "val": 26.85},
   {"timestamp": 1603963392, "val": 26.63},
   {"timestamp": 1603962792, "val": 29.83},
   {"timestamp": 1603962192, "val": 25.30},
   {"timestamp": 1603961592, "val": 29.42},
   {"timestamp": 1603960992, "val": 20.78},
   {"timestamp": 1603960392, "val": 28.60},
   {"timestamp": 1603959792, "val": 20.11},
   {"timestamp": 1603959192, "val": 27.56},
   {"timestamp": 1603958592, "val": 28.56},
   {"timestamp": 1603957992, "val": 25.85},
   {"timestamp": 1603957392, "val": 25.07},
   {"timestamp": 1603956792, "val": 28.39},
   {"timestamp": 1603956192, "val": 22.40},
   {"timestamp": 1603955592, "val": 29.27},
   {"timestamp": 1603954992, "val": 23.59},
   {"timestamp": 1603954392, "val": 25.85},
   {"timestamp": 1603953792, "val": 22.03},
   {"timestamp": 1603953192, "val": 28.38},
   {"timestamp": 1603952592, "val": 29.77},
   {"timestamp": 1603951992, "val": 23.33},
   {"timestamp": 1603951392, "val": 23.05},
   {"timestamp": 1603950792, "val": 29.68},
   {"timestamp": 1603950192, "val": 29.34},
   {"timestamp": 1603949592, "val": 28.74},
   {"timestamp": 1603948992, "val": 24.90},
   {"timestamp": 1603948392, "val": 22.56},
   {"timestamp": 1603947792, "val": 27.37},
   {"timestamp": 1603947192, "val": 26.80},
   {"timestamp": 1603946592, "val": 20.18},
   {"timestamp": 1603945992, "val": 28.89},
   {"timestamp": 1603945392, "val": 27.89},
   {"timestamp": 1603944792, "val": 26.54},
   {"timestamp": 1603944192, "val": 27.17},
   {"timestamp": 1603943592, "val": 29.96}
]

Массив берется из БД за последний промежуток, там сохранены результаты измерения температуры с интервалом примерно 30-50 секунд

Надо усреднить значения с шагом кратным заданному времени.

Например сейчас [5 марта 17:12]

В массиве data окажется выборка от [5 марта 17:12] до [4 марта 17:12]

Нужно будет создать массив где время будет с точным шагом, 17:00, 16:00, 15:00,... и до 17:00 прошлого дня

т.е. первое значение будет усреднено всего за 12 последних минут, но будет показано как за 17:00

А у последнего лишние 12 минут будут выкинуты

Answer 1

если аппроксимировать не надо, то можно сделать так:

data = [{'time': 10, 'val': 15}, {'time': 18, 'val': 15}, {'time': 13, 'val': 18}]

data_new = []

for time in range(8, 20, 2):
    count = 0
    sum = 0

    for obj in data:
        if time <= obj['time'] < time + 2:
            count += 1
            sum += obj['val']

    data_new.append((time, 0 if count == 0 else sum / count))

print(data_new)

ну или так:

data_new = []

for time in range(8, 20, 2):
    local = [obj['val'] for obj in data if time <= obj['time'] < time + 2]
    data_new.append((time, 0 if len(local) == 0 else sum(local) / len(local)))

Answer 2

Попробуйте сделать с помощью модуля pandas:

import pandas as pd
import datetime

df = pd.DataFrame(sorted(data, key=lambda x: x["timestamp"])) # data-ваш список словарей
df["timestamp"] = pd.to_datetime(df["timestamp"], unit="s")

df_24h = df[df["timestamp"]>=(datetime.datetime.now()-datetime.timedelta(hours=24))]
res = df_24h.groupby(df["timestamp"].dt.hour).mean()

тогда res будет:

                 val
timestamp           
0          25.670000
1          23.680000
2          23.066667
3          26.510000
8          25.100000
9          27.680000
10         25.443333
11         25.775000
12         24.486667
13         25.793333
14         24.148333
15         25.178333
16         23.386667
17         25.730000
18         23.590000
19         24.506667
20         25.853333
21         25.411667
22         26.940000
23         25.313333

(здесь 20 значений, а не 24, потому что я делал выборку от моего текущего времени, и данных в data просто не хватило.)

либо, если вам нужен не датафрейм, а именно список, то так можно добавить конвертацию:

res.values.tolist()

что даст, соответственно:

[[25.67],
 [23.679999999999996],
 [23.066666666666666],
 [26.51],
 [25.1],
 [27.679999999999996],
 [25.44333333333333],
 [25.775000000000002],
 [24.486666666666668],
 [25.793333333333333],
 [24.14833333333333],
 [25.178333333333338],
 [23.386666666666667],
 [25.73],
 [23.59],
 [24.506666666666664],
 [25.853333333333328],
 [25.411666666666665],
 [26.939999999999998],
 [25.313333333333333]]

БЛОГ НА HUSL

Как усреднить значения по точным группам времени?

Ответы (2 шт):