The high pass filter is not completely removing the DC component. See the picture. Blue is the signal before filtering, green is after.
Here is some background on the subject matter. I'm studying the beats of the heart in order to get something called HRV. The important aspects of HRV are between about 0.07 and 0.15 Hz. The standard practice in the literature is to pass the data through a band pass before analysis. I'm passing it through a low and high pass filter because I wanted to inspect the affects of both operations on the data. It is a bit of an art, because I want to smooth the data, but not distort the important information.
As requested here is the code. The data points are not evenly spaced in time, so the first step is to use interpolation to make evenly spaced points. Also, below that is 500 points from the data set.
RRData = RRDataCorrected;
(*Below is plot of raw data before interpolation or filtering.*)
(*For this plot I subtract out the DC signal.*)
dumplot =
ListLinePlot[
MapAt[# - Mean[RRDataCorrected[[All, 2]]] &, #, 2] & /@
RRDataCorrected, PlotStyle -> {Blue},
PlotMarkers -> Graphics@{Disk[{0, 0}, [email protected]]},
AspectRatio -> 1/10, ImageSize -> Full];
myInterpolation = Interpolation[RRData, InterpolationOrder -> 3];
filteredUniformRRData =
Table[{i*beatinterval, myInterpolation[i*beatinterval]}, {i, 0,
Length[RRData] - 1}];
filteredUniformRRData =
Transpose[{Table[i*beatinterval, {i, 0, Length[RRData] - 1}],
LowpassFilter[filteredUniformRRData[[All, 2]], 1.5000000]}];
filteredUniformRRData =
Transpose[{Table[i*beatinterval, {i, 0, Length[RRData] - 1}],
HighpassFilter[RRData[[All, 2]], 00.1]}];
Show[ListLinePlot[filteredUniformRRData, PlotStyle -> Green,
PlotRange -> {All, All}], dumplot, AspectRatio -> 1/10,
ImageSize -> Full, PlotRange -> {All, All}]
RRDataCorrected = {{1058.403`, 1.025`}, {1059.4905`,
1.1500000000000001`}, {1060.6830000000002`,
1.235`}, {1061.9095000000002`, 1.218`}, {1063.1030000000003`,
1.169`}, {1064.2330000000004`, 1.091`}, {1065.2795000000003`,
1.002`}, {1066.2455000000002`, 0.93`}, {1067.1585000000002`,
0.896`}, {1068.0600000000002`, 0.907`}, {1069.0095000000001`,
0.992`}, {1070.0900000000001`, 1.169`}, {1071.303`,
1.2570000000000001`}, {1072.545`, 1.227`}, {1073.731`,
1.145`}, {1074.8295`, 1.052`}, {1075.844`, 0.977`}, {1076.796`,
0.927`}, {1077.7195000000002`, 0.92`}, {1078.6645`,
0.97`}, {1079.6915000000001`, 1.084`}, {1080.832`,
1.197`}, {1082.0485`, 1.236`}, {1083.2810000000002`,
1.229`}, {1084.4685000000002`, 1.1460000000000001`}, {1085.5585`,
1.034`}, {1086.5575000000001`, 0.964`}, {1087.5020000000002`,
0.925`}, {1088.428`, 0.927`}, {1089.3880000000001`,
0.993`}, {1090.4410000000003`, 1.113`}, {1091.6165000000003`,
1.238`}, {1092.8745000000004`, 1.278`}, {1094.1370000000004`,
1.247`}, {1095.3565000000003`, 1.192`}, {1096.4975000000004`,
1.09`}, {1097.5295000000003`, 0.974`}, {1098.4695000000004`,
0.906`}, {1099.3610000000003`, 0.877`}, {1100.2520000000004`,
0.905`}, {1101.2055000000005`, 1.002`}, {1102.3195000000005`,
1.226`}, {1103.6075000000005`, 1.35`}, {1104.9475000000004`,
1.33`}, {1106.2390000000005`,
1.2530000000000001`}, {1107.4570000000006`,
1.183`}, {1108.5915000000005`, 1.086`}, {1109.6305000000004`,
0.992`}, {1110.5895000000005`, 0.926`}, {1111.4985000000006`,
0.892`}, {1112.3960000000006`, 0.903`}, {1113.3565000000006`,
1.018`}, {1114.4830000000006`, 1.235`}, {1115.7570000000005`,
1.313`}, {1117.0475000000006`, 1.268`}, {1118.2775000000006`,
1.192`}, {1119.4360000000006`, 1.125`}, {1120.5250000000005`,
1.053`}, {1121.5330000000006`, 0.963`}, {1122.4665000000007`,
0.904`}, {1123.3610000000008`, 0.885`}, {1124.2635000000007`,
0.92`}, {1125.2390000000007`, 1.031`}, {1126.3540000000007`,
1.199`}, {1127.5935000000006`, 1.28`}, {1128.8580000000006`,
1.249`}, {1130.0695000000005`, 1.174`}, {1131.1985000000004`,
1.084`}, {1132.2360000000003`, 0.991`}, {1133.1925000000003`,
0.922`}, {1134.1020000000003`, 0.897`}, {1135.0065000000004`,
0.912`}, {1135.9595000000004`, 0.994`}, {1137.0465000000004`,
1.18`}, {1138.2480000000003`, 1.223`}, {1139.4695000000004`,
1.22`}, {1140.6605000000004`, 1.162`}, {1141.7880000000005`,
1.093`}, {1142.8550000000005`, 1.041`}, {1143.8660000000004`,
0.981`}, {1144.8235000000004`, 0.934`}, {1145.7395000000004`,
0.898`}, {1146.6385000000002`, 0.9`}, {1147.5685000000003`,
0.96`}, {1148.5915000000002`, 1.086`}, {1149.7515000000003`,
1.234`}, {1151.0115000000003`, 1.286`}, {1152.2950000000003`,
1.281`}, {1153.5345000000002`, 1.198`}, {1154.6995000000002`,
1.1320000000000001`}, {1155.8005000000003`,
1.07`}, {1156.8405000000002`, 1.01`}, {1157.8195000000003`,
0.9480000000000001`}, {1158.7465000000002`,
0.906`}, {1159.6475000000003`, 0.896`}, {1160.5590000000002`,
0.927`}, {1161.5225000000003`, 1.`}, {1162.5915000000002`,
1.1380000000000001`}, {1163.7825000000003`,
1.244`}, {1165.0295000000003`, 1.25`}, {1166.2570000000003`,
1.205`}, {1167.4325000000003`,
1.1460000000000001`}, {1168.5515000000003`,
1.092`}, {1169.6120000000003`, 1.029`}, {1170.6180000000004`,
0.983`}, {1171.5740000000003`, 0.929`}, {1172.4765000000002`,
0.876`}, {1173.3390000000002`, 0.849`}, {1174.1865000000003`,
0.846`}, {1175.0540000000003`, 0.889`}, {1176.0430000000003`,
1.089`}, {1177.2250000000004`,
1.2750000000000001`}, {1178.5140000000004`,
1.303`}, {1179.8060000000003`, 1.281`}, {1181.0405000000003`,
1.188`}, {1182.1965000000002`, 1.124`}, {1183.2825000000003`,
1.048`}, {1184.2870000000003`, 0.961`}, {1185.2220000000002`,
0.909`}, {1186.1135000000002`, 0.874`}, {1186.9895000000001`,
0.878`}, {1187.9005000000002`,
0.9440000000000001`}, {1188.9050000000002`,
1.065`}, {1190.0370000000003`, 1.199`}, {1191.2500000000002`,
1.227`}, {1192.4645000000003`, 1.202`}, {1193.6490000000003`,
1.167`}, {1194.7950000000003`, 1.125`}, {1195.8840000000002`,
1.053`}, {1196.8990000000003`, 0.977`}, {1197.8460000000002`,
0.917`}, {1198.7505000000003`, 0.892`}, {1199.6490000000003`,
0.905`}, {1200.6035000000004`, 1.004`}, {1201.6920000000005`,
1.173`}, {1202.9030000000005`, 1.249`}, {1204.1580000000006`,
1.2610000000000001`}, {1205.3860000000006`,
1.195`}, {1206.5545000000006`,
1.1420000000000001`}, {1207.6795000000006`,
1.108`}, {1208.7565000000006`, 1.046`}, {1209.7820000000006`,
1.0050000000000001`}, {1210.7595000000006`,
0.9500000000000001`}, {1211.6845000000005`,
0.9`}, {1212.5805000000005`, 0.892`}, {1213.4845000000005`,
0.916`}, {1214.4660000000006`, 1.047`}, {1215.6065000000006`,
1.234`}, {1216.8455000000006`, 1.244`}, {1218.0780000000007`,
1.221`}, {1219.2735000000007`, 1.17`}, {1220.4235000000008`,
1.1300000000000001`}, {1221.5260000000007`,
1.075`}, {1222.5700000000008`,
1.0130000000000001`}, {1223.551500000001`,
0.9500000000000001`}, {1224.4730000000009`,
0.893`}, {1225.354000000001`, 0.869`}, {1226.225500000001`,
0.874`}, {1227.1325000000008`,
0.9400000000000001`}, {1228.1650000000009`,
1.125`}, {1229.339500000001`, 1.224`}, {1230.489000000001`,
1.075`}, {1231.5545000000009`, 1.056`}, {1232.6345000000008`,
1.104`}, {1233.736500000001`, 1.1`}, {1234.819500000001`,
1.066`}, {1235.854000000001`,
1.0030000000000001`}, {1236.827000000001`,
0.9430000000000001`}, {1237.7465000000009`,
0.896`}, {1238.6355000000008`, 0.882`}, {1239.5380000000007`,
0.923`}, {1240.5105000000008`, 1.022`}, {1241.5740000000008`,
1.105`}, {1242.7010000000007`, 1.149`}, {1243.8560000000007`,
1.161`}, {1245.0045000000007`,
1.1360000000000001`}, {1246.1165000000008`,
1.088`}, {1247.1730000000007`, 1.025`}, {1248.1610000000007`,
0.9510000000000001`}, {1249.0835000000006`,
0.894`}, {1249.9675000000007`, 0.874`}, {1250.8520000000005`,
0.895`}, {1251.7920000000006`, 0.985`}, {1252.8460000000007`,
1.123`}, {1254.0085000000006`, 1.202`}, {1255.2235000000005`,
1.228`}, {1256.4200000000005`, 1.165`}, {1257.5600000000006`,
1.115`}, {1258.6380000000006`, 1.041`}, {1259.6340000000007`,
0.9510000000000001`}, {1260.5600000000006`,
0.901`}, {1261.4515000000006`, 0.882`}, {1262.3495000000005`,
0.914`}, {1263.3165000000006`, 1.02`}, {1264.3975000000005`,
1.1420000000000001`}, {1265.5830000000005`,
1.229`}, {1266.8205000000005`, 1.246`}, {1268.0245000000004`,
1.162`}, {1269.1580000000004`, 1.105`}, {1270.2215000000003`,
1.022`}, {1271.2035000000003`,
0.9420000000000001`}, {1272.1190000000004`,
0.889`}, {1272.9995000000004`, 0.872`}, {1273.8895000000005`,
0.908`}, {1274.8595000000005`, 1.032`}, {1275.9880000000005`,
1.225`}, {1277.2640000000006`, 1.327`}, {1278.5780000000007`,
1.301`}, {1279.8385000000007`, 1.22`}, {1281.0215000000007`,
1.1460000000000001`}, {1282.1345000000008`,
1.08`}, {1283.173000000001`, 0.997`}, {1284.135500000001`,
0.928`}, {1285.044000000001`, 0.889`}, {1285.933000000001`,
0.889`}, {1286.856500000001`,
0.9580000000000001`}, {1287.892000000001`,
1.113`}, {1289.059500000001`, 1.222`}, {1290.303500000001`,
1.266`}, {1291.5590000000009`, 1.245`}, {1292.7710000000009`,
1.179`}, {1293.9125000000008`, 1.104`}, {1294.991000000001`,
1.053`}, {1296.0075000000008`, 0.98`}, {1296.9580000000008`,
0.921`}, {1297.8640000000007`, 0.891`}, {1298.7570000000007`,
0.895`}, {1299.6950000000008`, 0.981`}, {1300.7665000000009`,
1.162`}, {1301.9740000000008`,
1.2530000000000001`}, {1303.236500000001`,
1.272`}, {1304.4840000000008`, 1.223`}, {1305.677500000001`,
1.164`}, {1306.8100000000009`, 1.101`}, {1307.883000000001`,
1.045`}, {1308.895500000001`, 0.98`}, {1309.8425000000009`,
0.914`}, {1310.736500000001`, 0.874`}, {1311.6125000000009`,
0.878`}, {1312.5340000000008`, 0.965`}, {1313.6045000000008`,
1.176`}, {1314.836000000001`, 1.287`}, {1316.1360000000009`,
1.313`}, {1317.416500000001`, 1.248`}, {1318.617500000001`,
1.154`}, {1319.743000000001`, 1.097`}, {1320.805000000001`,
1.027`}, {1321.793500000001`,
0.9500000000000001`}, {1322.716500000001`,
0.896`}, {1323.6000000000008`, 0.871`}, {1324.4795000000008`,
0.888`}, {1325.4160000000008`, 0.985`}, {1326.5230000000008`,
1.229`}, {1327.7840000000008`, 1.293`}, {1329.0775000000008`,
1.294`}, {1330.3460000000007`, 1.243`}, {1331.5505000000007`,
1.166`}, {1332.6940000000006`, 1.121`}, {1333.7890000000007`,
1.069`}, {1334.8365000000006`, 1.026`}, {1335.8265000000006`,
0.9540000000000001`}, {1336.7540000000006`,
0.901`}, {1337.6465000000005`, 0.884`}, {1338.5420000000006`,
0.907`}, {1339.5115000000005`, 1.032`}, {1340.6540000000005`,
1.2530000000000001`}, {1341.9415000000004`,
1.322`}, {1343.2380000000003`,
1.2710000000000001`}, {1344.4770000000003`,
1.207`}, {1345.6560000000004`, 1.151`}, {1346.7665000000004`,
1.07`}, {1347.7905000000003`, 0.978`}, {1348.7335000000003`,
0.908`}, {1349.6255000000003`, 0.876`}, {1350.5165000000004`,
0.906`}, {1351.4815000000003`, 1.024`}, {1352.5830000000003`,
1.179`}, {1353.7905000000003`, 1.236`}, {1355.0145000000002`,
1.212`}, {1356.2060000000001`, 1.171`}, {1357.3485`,
1.114`}, {1358.4370000000001`, 1.063`}, {1359.4605000000001`,
0.984`}, {1360.4115000000002`, 0.918`}, {1361.3095`,
0.878`}, {1362.1865`, 0.876`}, {1363.0875`,
0.926`}, {1364.0945000000002`, 1.088`}, {1365.2415`,
1.206`}, {1366.4640000000002`, 1.239`}, {1367.6855000000003`,
1.204`}, {1368.8615000000002`,
1.1480000000000001`}, {1369.9835000000003`,
1.096`}, {1371.0400000000002`, 1.0170000000000001`}, {1372.0205`,
0.9440000000000001`}, {1372.94`, 0.895`}, {1373.823`,
0.871`}, {1374.7060000000001`, 0.895`}, {1375.66`,
1.0130000000000001`}, {1376.7575000000002`,
1.182`}, {1377.9830000000002`,
1.2690000000000001`}, {1379.2485000000001`,
1.262`}, {1380.4720000000002`, 1.185`}, {1381.6355000000003`,
1.1420000000000001`}, {1382.7420000000004`,
1.071`}, {1383.7715000000005`, 0.988`}, {1384.7265000000004`,
0.922`}, {1385.6350000000004`, 0.895`}, {1386.5440000000006`,
0.923`}, {1387.5095000000006`, 1.008`}, {1388.5805000000005`,
1.1340000000000001`}, {1389.7535000000005`,
1.212`}, {1390.9650000000004`, 1.211`}, {1392.1605000000004`,
1.18`}, {1393.3145000000004`,
1.1280000000000001`}, {1394.4140000000004`,
1.071`}, {1395.4440000000004`, 0.989`}, {1396.4030000000005`,
0.929`}, {1397.3145000000004`, 0.894`}, {1398.2075000000004`,
0.892`}, {1399.1295000000005`,
0.9520000000000001`}, {1400.1610000000005`,
1.111`}, {1401.3430000000005`,
1.2530000000000001`}, {1402.6100000000006`,
1.281`}, {1403.8695000000005`, 1.238`}, {1405.0695000000005`,
1.162`}, {1406.1760000000006`, 1.051`}, {1407.1880000000006`,
0.973`}, {1408.1360000000006`, 0.923`}, {1409.0565000000006`,
0.918`}, {1409.9935000000005`,
0.9560000000000001`}, {1410.9915000000005`,
1.04`}, {1412.0745000000006`,
1.1260000000000001`}, {1413.2300000000007`,
1.185`}, {1414.4055000000008`, 1.166`}, {1415.5480000000007`,
1.119`}, {1416.6475000000007`, 1.08`}, {1417.6935000000008`,
1.012`}, {1418.6680000000008`, 0.937`}, {1419.5830000000008`,
0.893`}, {1420.4710000000007`, 0.883`}, {1421.3815000000006`,
0.9380000000000001`}, {1422.3980000000006`,
1.095`}, {1423.5490000000007`, 1.207`}, {1424.7725000000007`,
1.24`}, {1425.9980000000007`, 1.211`}, {1427.1875000000007`,
1.168`}, {1428.3220000000006`, 1.101`}, {1429.3825000000006`,
1.02`}, {1430.3660000000007`,
0.9470000000000001`}, {1431.2850000000008`,
0.891`}, {1432.1660000000008`, 0.871`}, {1433.057000000001`,
0.911`}, {1434.0445000000009`, 1.064`}, {1435.2160000000008`,
1.2790000000000001`}, {1436.4930000000008`,
1.2750000000000001`}, {1437.740000000001`,
1.219`}, {1438.928000000001`, 1.157`}, {1440.031000000001`,
1.049`}, {1441.038500000001`, 0.966`}, {1441.979000000001`,
0.915`}, {1442.885500000001`, 0.898`}, {1443.798500000001`,
0.928`}, {1444.776500000001`, 1.028`}, {1445.8595000000012`,
1.1380000000000001`}, {1447.0170000000012`,
1.177`}, {1448.1925000000012`, 1.174`}, {1449.3520000000012`,
1.145`}, {1450.4670000000012`, 1.085`}, {1451.5025000000012`,
0.986`}, {1452.4580000000012`, 0.925`}, {1453.3645000000013`,
0.888`}, {1454.2505000000012`, 0.884`}, {1455.1710000000012`,
0.9570000000000001`}, {1456.2055000000012`,
1.112`}, {1457.3850000000011`, 1.247`}, {1458.649000000001`,
1.281`}, {1459.905000000001`, 1.231`}, {1461.1015000000011`,
1.162`}, {1462.242000000001`, 1.119`}, {1463.3270000000011`,
1.051`}, {1464.3585000000012`, 1.012`}, {1465.3400000000013`,
0.9510000000000001`}, {1466.2620000000013`,
0.893`}, {1467.1465000000012`, 0.876`}, {1468.0340000000012`,
0.899`}, {1468.9810000000011`, 0.995`}, {1470.0825000000011`,
1.208`}, {1471.3285000000012`, 1.284`}, {1472.6095000000012`,
1.278`}, {1473.8475000000012`, 1.198`}, {1475.0220000000013`,
1.151`}, {1476.1395000000014`, 1.084`}, {1477.2010000000014`,
1.039`}, {1478.2110000000014`, 0.981`}, {1479.1595000000013`,
0.916`}, {1480.0555000000013`, 0.876`}, {1480.9285000000013`,
0.87`}, {1481.8270000000014`, 0.927`}, {1482.8455000000013`,
1.11`}, {1484.0285000000013`, 1.256`}, {1485.3055000000013`,
1.298`}, {1486.5900000000013`,
1.2710000000000001`}, {1487.8175000000012`,
1.184`}, {1488.9820000000013`, 1.145`}, {1490.0920000000012`,
1.075`}, {1491.1255000000012`, 0.992`}, {1492.0885000000012`,
0.934`}, {1493.0050000000012`, 0.899`}, {1493.9120000000012`,
0.915`}, {1494.872500000001`, 1.006`}, {1495.9575000000011`,
1.164`}, {1497.161500000001`, 1.244`}, {1498.408000000001`,
1.249`}, {1499.626000000001`, 1.187`}, {1500.7890000000011`,
1.139`}, {1501.877000000001`, 1.037`}, {1502.874500000001`,
0.9580000000000001`}, {1503.802000000001`,
0.897`}, {1504.693500000001`, 0.886`}, {1505.605500000001`,
0.9380000000000001`}, {1506.617000000001`,
1.085`}, {1507.7780000000012`, 1.237`}, {1509.0430000000013`,
1.293`}, {1510.3170000000011`,
1.2550000000000001`}, {1511.554500000001`,
1.22`}, {1512.7470000000012`, 1.165`}, {1513.8595000000012`,
1.06`}, {1514.8750000000011`, 0.971`}, {1515.8225000000011`,
0.924`}, {1516.7515000000012`, 0.934`}, {1517.7230000000013`,
1.0090000000000001`}, {1518.8060000000014`,
1.157`}, {1520.0205000000014`, 1.272`}, {1521.2845000000013`,
1.256`}, {1522.5085000000013`, 1.192`}, {1523.6675000000014`,
1.1260000000000001`}, {1524.7680000000014`,
1.075`}, {1525.7985000000015`, 0.986`}, {1526.7500000000014`,
0.917`}, {1527.6545000000015`, 0.892`}, {1528.5515000000014`,
0.902`}, {1529.5075000000013`, 1.01`}, {1530.6035000000013`,
1.182`}, {1531.8180000000013`, 1.247`}, {1533.0550000000014`,
1.227`}, {1534.2630000000015`, 1.189`}, {1535.4190000000015`,
1.123`}, {1536.5095000000015`, 1.058`}, {1537.5155000000016`,
0.9540000000000001`}, {1538.4345000000017`,
0.884`}, {1539.3005000000016`, 0.848`}, {1540.1540000000016`,
0.859`}, {1541.0470000000016`, 0.927`}, {1542.0655000000015`,
1.11`}, {1543.2390000000016`, 1.237`}, {1544.4935000000016`,
1.272`}, {1545.7405000000017`, 1.222`}, {1546.9235000000017`,
1.1440000000000001`}, {1548.0340000000017`,
1.077`}, {1549.0835000000018`, 1.022`}, {1550.0765000000017`,
0.964`}, {1551.0070000000017`, 0.897`}, {1551.8825000000018`,
0.854`}, {1552.7375000000018`, 0.856`}, {1553.6325000000018`,
0.934`}, {1554.6665000000019`,
1.1340000000000001`}, {1555.8950000000018`,
1.323`}, {1557.203500000002`, 1.294`}, {1558.4675000000018`,
1.234`}, {1559.6660000000018`, 1.163`}, {1560.7935000000018`,
1.092`}, {1561.8555000000017`, 1.032`}, {1562.8585000000016`,
0.974`}, {1563.7990000000016`, 0.907`}, {1564.6860000000015`,
0.867`}, {1565.5495000000014`, 0.86`}, {1566.4415000000015`,
0.924`}, {1567.4485000000016`, 1.09`}, {1568.5875000000015`,
1.188`}, {1569.8060000000014`, 1.249`}, {1571.0450000000014`,
1.229`}, {1572.2425000000014`, 1.166`}, {1573.3705000000014`,
1.09`}, {1574.4315000000013`, 1.032`}, {1575.4315000000013`,
0.968`}, {1576.3670000000013`, 0.903`}, {1577.2500000000014`,
0.863`}, {1578.1110000000015`, 0.859`}, {1578.9955000000014`,
0.91`}, {1579.9910000000013`, 1.081`}, {1581.1335000000013`,
1.204`}, {1582.3710000000012`,
1.2710000000000001`}, {1583.6280000000013`,
1.243`}, {1584.8395000000012`, 1.18`}, {1585.988500000001`,
1.118`}, {1587.0805000000012`, 1.066`}, {1588.118000000001`,
1.0090000000000001`}, {1589.100000000001`,
0.9550000000000001`}, {1590.019500000001`,
0.884`}, {1590.888000000001`, 0.853`}}
beatinterval
is missing, 2nd argument ofTable
is not properly modified, there may be more. ) Please double check it. $\endgroup$