5
$\begingroup$

The high pass filter is not completely removing the DC component. See the picture. Blue is the signal before filtering, green is after.

Here is some background on the subject matter. I'm studying the beats of the heart in order to get something called HRV. The important aspects of HRV are between about 0.07 and 0.15 Hz. The standard practice in the literature is to pass the data through a band pass before analysis. I'm passing it through a low and high pass filter because I wanted to inspect the affects of both operations on the data. It is a bit of an art, because I want to smooth the data, but not distort the important information.

As requested here is the code. The data points are not evenly spaced in time, so the first step is to use interpolation to make evenly spaced points. Also, below that is 500 points from the data set.

RRData = RRDataCorrected;
 (*Below is plot of raw data before interpolation or filtering.*)
 (*For this plot I subtract out the DC signal.*)
 dumplot = 
 ListLinePlot[
  MapAt[# - Mean[RRDataCorrected[[All, 2]]] &, #, 2] & /@ 
   RRDataCorrected, PlotStyle -> {Blue}, 
  PlotMarkers -> Graphics@{Disk[{0, 0}, [email protected]]}, 
  AspectRatio -> 1/10, ImageSize -> Full];

myInterpolation = Interpolation[RRData, InterpolationOrder -> 3];
filteredUniformRRData = 
  Table[{i*beatinterval, myInterpolation[i*beatinterval]}, {i, 0, 
    Length[RRData] - 1}];
filteredUniformRRData = 
  Transpose[{Table[i*beatinterval, {i, 0, Length[RRData] - 1}], 
    LowpassFilter[filteredUniformRRData[[All, 2]], 1.5000000]}];
filteredUniformRRData = 
  Transpose[{Table[i*beatinterval, {i, 0, Length[RRData] - 1}], 
    HighpassFilter[RRData[[All, 2]], 00.1]}];

Show[ListLinePlot[filteredUniformRRData, PlotStyle -> Green, 
  PlotRange -> {All, All}], dumplot, AspectRatio -> 1/10, 
 ImageSize -> Full, PlotRange -> {All, All}]




RRDataCorrected = {{1058.403`, 1.025`}, {1059.4905`, 
   1.1500000000000001`}, {1060.6830000000002`, 
   1.235`}, {1061.9095000000002`, 1.218`}, {1063.1030000000003`, 
   1.169`}, {1064.2330000000004`, 1.091`}, {1065.2795000000003`, 
   1.002`}, {1066.2455000000002`, 0.93`}, {1067.1585000000002`, 
   0.896`}, {1068.0600000000002`, 0.907`}, {1069.0095000000001`, 
   0.992`}, {1070.0900000000001`, 1.169`}, {1071.303`, 
   1.2570000000000001`}, {1072.545`, 1.227`}, {1073.731`, 
   1.145`}, {1074.8295`, 1.052`}, {1075.844`, 0.977`}, {1076.796`, 
   0.927`}, {1077.7195000000002`, 0.92`}, {1078.6645`, 
   0.97`}, {1079.6915000000001`, 1.084`}, {1080.832`, 
   1.197`}, {1082.0485`, 1.236`}, {1083.2810000000002`, 
   1.229`}, {1084.4685000000002`, 1.1460000000000001`}, {1085.5585`, 
   1.034`}, {1086.5575000000001`, 0.964`}, {1087.5020000000002`, 
   0.925`}, {1088.428`, 0.927`}, {1089.3880000000001`, 
   0.993`}, {1090.4410000000003`, 1.113`}, {1091.6165000000003`, 
   1.238`}, {1092.8745000000004`, 1.278`}, {1094.1370000000004`, 
   1.247`}, {1095.3565000000003`, 1.192`}, {1096.4975000000004`, 
   1.09`}, {1097.5295000000003`, 0.974`}, {1098.4695000000004`, 
   0.906`}, {1099.3610000000003`, 0.877`}, {1100.2520000000004`, 
   0.905`}, {1101.2055000000005`, 1.002`}, {1102.3195000000005`, 
   1.226`}, {1103.6075000000005`, 1.35`}, {1104.9475000000004`, 
   1.33`}, {1106.2390000000005`, 
   1.2530000000000001`}, {1107.4570000000006`, 
   1.183`}, {1108.5915000000005`, 1.086`}, {1109.6305000000004`, 
   0.992`}, {1110.5895000000005`, 0.926`}, {1111.4985000000006`, 
   0.892`}, {1112.3960000000006`, 0.903`}, {1113.3565000000006`, 
   1.018`}, {1114.4830000000006`, 1.235`}, {1115.7570000000005`, 
   1.313`}, {1117.0475000000006`, 1.268`}, {1118.2775000000006`, 
   1.192`}, {1119.4360000000006`, 1.125`}, {1120.5250000000005`, 
   1.053`}, {1121.5330000000006`, 0.963`}, {1122.4665000000007`, 
   0.904`}, {1123.3610000000008`, 0.885`}, {1124.2635000000007`, 
   0.92`}, {1125.2390000000007`, 1.031`}, {1126.3540000000007`, 
   1.199`}, {1127.5935000000006`, 1.28`}, {1128.8580000000006`, 
   1.249`}, {1130.0695000000005`, 1.174`}, {1131.1985000000004`, 
   1.084`}, {1132.2360000000003`, 0.991`}, {1133.1925000000003`, 
   0.922`}, {1134.1020000000003`, 0.897`}, {1135.0065000000004`, 
   0.912`}, {1135.9595000000004`, 0.994`}, {1137.0465000000004`, 
   1.18`}, {1138.2480000000003`, 1.223`}, {1139.4695000000004`, 
   1.22`}, {1140.6605000000004`, 1.162`}, {1141.7880000000005`, 
   1.093`}, {1142.8550000000005`, 1.041`}, {1143.8660000000004`, 
   0.981`}, {1144.8235000000004`, 0.934`}, {1145.7395000000004`, 
   0.898`}, {1146.6385000000002`, 0.9`}, {1147.5685000000003`, 
   0.96`}, {1148.5915000000002`, 1.086`}, {1149.7515000000003`, 
   1.234`}, {1151.0115000000003`, 1.286`}, {1152.2950000000003`, 
   1.281`}, {1153.5345000000002`, 1.198`}, {1154.6995000000002`, 
   1.1320000000000001`}, {1155.8005000000003`, 
   1.07`}, {1156.8405000000002`, 1.01`}, {1157.8195000000003`, 
   0.9480000000000001`}, {1158.7465000000002`, 
   0.906`}, {1159.6475000000003`, 0.896`}, {1160.5590000000002`, 
   0.927`}, {1161.5225000000003`, 1.`}, {1162.5915000000002`, 
   1.1380000000000001`}, {1163.7825000000003`, 
   1.244`}, {1165.0295000000003`, 1.25`}, {1166.2570000000003`, 
   1.205`}, {1167.4325000000003`, 
   1.1460000000000001`}, {1168.5515000000003`, 
   1.092`}, {1169.6120000000003`, 1.029`}, {1170.6180000000004`, 
   0.983`}, {1171.5740000000003`, 0.929`}, {1172.4765000000002`, 
   0.876`}, {1173.3390000000002`, 0.849`}, {1174.1865000000003`, 
   0.846`}, {1175.0540000000003`, 0.889`}, {1176.0430000000003`, 
   1.089`}, {1177.2250000000004`, 
   1.2750000000000001`}, {1178.5140000000004`, 
   1.303`}, {1179.8060000000003`, 1.281`}, {1181.0405000000003`, 
   1.188`}, {1182.1965000000002`, 1.124`}, {1183.2825000000003`, 
   1.048`}, {1184.2870000000003`, 0.961`}, {1185.2220000000002`, 
   0.909`}, {1186.1135000000002`, 0.874`}, {1186.9895000000001`, 
   0.878`}, {1187.9005000000002`, 
   0.9440000000000001`}, {1188.9050000000002`, 
   1.065`}, {1190.0370000000003`, 1.199`}, {1191.2500000000002`, 
   1.227`}, {1192.4645000000003`, 1.202`}, {1193.6490000000003`, 
   1.167`}, {1194.7950000000003`, 1.125`}, {1195.8840000000002`, 
   1.053`}, {1196.8990000000003`, 0.977`}, {1197.8460000000002`, 
   0.917`}, {1198.7505000000003`, 0.892`}, {1199.6490000000003`, 
   0.905`}, {1200.6035000000004`, 1.004`}, {1201.6920000000005`, 
   1.173`}, {1202.9030000000005`, 1.249`}, {1204.1580000000006`, 
   1.2610000000000001`}, {1205.3860000000006`, 
   1.195`}, {1206.5545000000006`, 
   1.1420000000000001`}, {1207.6795000000006`, 
   1.108`}, {1208.7565000000006`, 1.046`}, {1209.7820000000006`, 
   1.0050000000000001`}, {1210.7595000000006`, 
   0.9500000000000001`}, {1211.6845000000005`, 
   0.9`}, {1212.5805000000005`, 0.892`}, {1213.4845000000005`, 
   0.916`}, {1214.4660000000006`, 1.047`}, {1215.6065000000006`, 
   1.234`}, {1216.8455000000006`, 1.244`}, {1218.0780000000007`, 
   1.221`}, {1219.2735000000007`, 1.17`}, {1220.4235000000008`, 
   1.1300000000000001`}, {1221.5260000000007`, 
   1.075`}, {1222.5700000000008`, 
   1.0130000000000001`}, {1223.551500000001`, 
   0.9500000000000001`}, {1224.4730000000009`, 
   0.893`}, {1225.354000000001`, 0.869`}, {1226.225500000001`, 
   0.874`}, {1227.1325000000008`, 
   0.9400000000000001`}, {1228.1650000000009`, 
   1.125`}, {1229.339500000001`, 1.224`}, {1230.489000000001`, 
   1.075`}, {1231.5545000000009`, 1.056`}, {1232.6345000000008`, 
   1.104`}, {1233.736500000001`, 1.1`}, {1234.819500000001`, 
   1.066`}, {1235.854000000001`, 
   1.0030000000000001`}, {1236.827000000001`, 
   0.9430000000000001`}, {1237.7465000000009`, 
   0.896`}, {1238.6355000000008`, 0.882`}, {1239.5380000000007`, 
   0.923`}, {1240.5105000000008`, 1.022`}, {1241.5740000000008`, 
   1.105`}, {1242.7010000000007`, 1.149`}, {1243.8560000000007`, 
   1.161`}, {1245.0045000000007`, 
   1.1360000000000001`}, {1246.1165000000008`, 
   1.088`}, {1247.1730000000007`, 1.025`}, {1248.1610000000007`, 
   0.9510000000000001`}, {1249.0835000000006`, 
   0.894`}, {1249.9675000000007`, 0.874`}, {1250.8520000000005`, 
   0.895`}, {1251.7920000000006`, 0.985`}, {1252.8460000000007`, 
   1.123`}, {1254.0085000000006`, 1.202`}, {1255.2235000000005`, 
   1.228`}, {1256.4200000000005`, 1.165`}, {1257.5600000000006`, 
   1.115`}, {1258.6380000000006`, 1.041`}, {1259.6340000000007`, 
   0.9510000000000001`}, {1260.5600000000006`, 
   0.901`}, {1261.4515000000006`, 0.882`}, {1262.3495000000005`, 
   0.914`}, {1263.3165000000006`, 1.02`}, {1264.3975000000005`, 
   1.1420000000000001`}, {1265.5830000000005`, 
   1.229`}, {1266.8205000000005`, 1.246`}, {1268.0245000000004`, 
   1.162`}, {1269.1580000000004`, 1.105`}, {1270.2215000000003`, 
   1.022`}, {1271.2035000000003`, 
   0.9420000000000001`}, {1272.1190000000004`, 
   0.889`}, {1272.9995000000004`, 0.872`}, {1273.8895000000005`, 
   0.908`}, {1274.8595000000005`, 1.032`}, {1275.9880000000005`, 
   1.225`}, {1277.2640000000006`, 1.327`}, {1278.5780000000007`, 
   1.301`}, {1279.8385000000007`, 1.22`}, {1281.0215000000007`, 
   1.1460000000000001`}, {1282.1345000000008`, 
   1.08`}, {1283.173000000001`, 0.997`}, {1284.135500000001`, 
   0.928`}, {1285.044000000001`, 0.889`}, {1285.933000000001`, 
   0.889`}, {1286.856500000001`, 
   0.9580000000000001`}, {1287.892000000001`, 
   1.113`}, {1289.059500000001`, 1.222`}, {1290.303500000001`, 
   1.266`}, {1291.5590000000009`, 1.245`}, {1292.7710000000009`, 
   1.179`}, {1293.9125000000008`, 1.104`}, {1294.991000000001`, 
   1.053`}, {1296.0075000000008`, 0.98`}, {1296.9580000000008`, 
   0.921`}, {1297.8640000000007`, 0.891`}, {1298.7570000000007`, 
   0.895`}, {1299.6950000000008`, 0.981`}, {1300.7665000000009`, 
   1.162`}, {1301.9740000000008`, 
   1.2530000000000001`}, {1303.236500000001`, 
   1.272`}, {1304.4840000000008`, 1.223`}, {1305.677500000001`, 
   1.164`}, {1306.8100000000009`, 1.101`}, {1307.883000000001`, 
   1.045`}, {1308.895500000001`, 0.98`}, {1309.8425000000009`, 
   0.914`}, {1310.736500000001`, 0.874`}, {1311.6125000000009`, 
   0.878`}, {1312.5340000000008`, 0.965`}, {1313.6045000000008`, 
   1.176`}, {1314.836000000001`, 1.287`}, {1316.1360000000009`, 
   1.313`}, {1317.416500000001`, 1.248`}, {1318.617500000001`, 
   1.154`}, {1319.743000000001`, 1.097`}, {1320.805000000001`, 
   1.027`}, {1321.793500000001`, 
   0.9500000000000001`}, {1322.716500000001`, 
   0.896`}, {1323.6000000000008`, 0.871`}, {1324.4795000000008`, 
   0.888`}, {1325.4160000000008`, 0.985`}, {1326.5230000000008`, 
   1.229`}, {1327.7840000000008`, 1.293`}, {1329.0775000000008`, 
   1.294`}, {1330.3460000000007`, 1.243`}, {1331.5505000000007`, 
   1.166`}, {1332.6940000000006`, 1.121`}, {1333.7890000000007`, 
   1.069`}, {1334.8365000000006`, 1.026`}, {1335.8265000000006`, 
   0.9540000000000001`}, {1336.7540000000006`, 
   0.901`}, {1337.6465000000005`, 0.884`}, {1338.5420000000006`, 
   0.907`}, {1339.5115000000005`, 1.032`}, {1340.6540000000005`, 
   1.2530000000000001`}, {1341.9415000000004`, 
   1.322`}, {1343.2380000000003`, 
   1.2710000000000001`}, {1344.4770000000003`, 
   1.207`}, {1345.6560000000004`, 1.151`}, {1346.7665000000004`, 
   1.07`}, {1347.7905000000003`, 0.978`}, {1348.7335000000003`, 
   0.908`}, {1349.6255000000003`, 0.876`}, {1350.5165000000004`, 
   0.906`}, {1351.4815000000003`, 1.024`}, {1352.5830000000003`, 
   1.179`}, {1353.7905000000003`, 1.236`}, {1355.0145000000002`, 
   1.212`}, {1356.2060000000001`, 1.171`}, {1357.3485`, 
   1.114`}, {1358.4370000000001`, 1.063`}, {1359.4605000000001`, 
   0.984`}, {1360.4115000000002`, 0.918`}, {1361.3095`, 
   0.878`}, {1362.1865`, 0.876`}, {1363.0875`, 
   0.926`}, {1364.0945000000002`, 1.088`}, {1365.2415`, 
   1.206`}, {1366.4640000000002`, 1.239`}, {1367.6855000000003`, 
   1.204`}, {1368.8615000000002`, 
   1.1480000000000001`}, {1369.9835000000003`, 
   1.096`}, {1371.0400000000002`, 1.0170000000000001`}, {1372.0205`, 
   0.9440000000000001`}, {1372.94`, 0.895`}, {1373.823`, 
   0.871`}, {1374.7060000000001`, 0.895`}, {1375.66`, 
   1.0130000000000001`}, {1376.7575000000002`, 
   1.182`}, {1377.9830000000002`, 
   1.2690000000000001`}, {1379.2485000000001`, 
   1.262`}, {1380.4720000000002`, 1.185`}, {1381.6355000000003`, 
   1.1420000000000001`}, {1382.7420000000004`, 
   1.071`}, {1383.7715000000005`, 0.988`}, {1384.7265000000004`, 
   0.922`}, {1385.6350000000004`, 0.895`}, {1386.5440000000006`, 
   0.923`}, {1387.5095000000006`, 1.008`}, {1388.5805000000005`, 
   1.1340000000000001`}, {1389.7535000000005`, 
   1.212`}, {1390.9650000000004`, 1.211`}, {1392.1605000000004`, 
   1.18`}, {1393.3145000000004`, 
   1.1280000000000001`}, {1394.4140000000004`, 
   1.071`}, {1395.4440000000004`, 0.989`}, {1396.4030000000005`, 
   0.929`}, {1397.3145000000004`, 0.894`}, {1398.2075000000004`, 
   0.892`}, {1399.1295000000005`, 
   0.9520000000000001`}, {1400.1610000000005`, 
   1.111`}, {1401.3430000000005`, 
   1.2530000000000001`}, {1402.6100000000006`, 
   1.281`}, {1403.8695000000005`, 1.238`}, {1405.0695000000005`, 
   1.162`}, {1406.1760000000006`, 1.051`}, {1407.1880000000006`, 
   0.973`}, {1408.1360000000006`, 0.923`}, {1409.0565000000006`, 
   0.918`}, {1409.9935000000005`, 
   0.9560000000000001`}, {1410.9915000000005`, 
   1.04`}, {1412.0745000000006`, 
   1.1260000000000001`}, {1413.2300000000007`, 
   1.185`}, {1414.4055000000008`, 1.166`}, {1415.5480000000007`, 
   1.119`}, {1416.6475000000007`, 1.08`}, {1417.6935000000008`, 
   1.012`}, {1418.6680000000008`, 0.937`}, {1419.5830000000008`, 
   0.893`}, {1420.4710000000007`, 0.883`}, {1421.3815000000006`, 
   0.9380000000000001`}, {1422.3980000000006`, 
   1.095`}, {1423.5490000000007`, 1.207`}, {1424.7725000000007`, 
   1.24`}, {1425.9980000000007`, 1.211`}, {1427.1875000000007`, 
   1.168`}, {1428.3220000000006`, 1.101`}, {1429.3825000000006`, 
   1.02`}, {1430.3660000000007`, 
   0.9470000000000001`}, {1431.2850000000008`, 
   0.891`}, {1432.1660000000008`, 0.871`}, {1433.057000000001`, 
   0.911`}, {1434.0445000000009`, 1.064`}, {1435.2160000000008`, 
   1.2790000000000001`}, {1436.4930000000008`, 
   1.2750000000000001`}, {1437.740000000001`, 
   1.219`}, {1438.928000000001`, 1.157`}, {1440.031000000001`, 
   1.049`}, {1441.038500000001`, 0.966`}, {1441.979000000001`, 
   0.915`}, {1442.885500000001`, 0.898`}, {1443.798500000001`, 
   0.928`}, {1444.776500000001`, 1.028`}, {1445.8595000000012`, 
   1.1380000000000001`}, {1447.0170000000012`, 
   1.177`}, {1448.1925000000012`, 1.174`}, {1449.3520000000012`, 
   1.145`}, {1450.4670000000012`, 1.085`}, {1451.5025000000012`, 
   0.986`}, {1452.4580000000012`, 0.925`}, {1453.3645000000013`, 
   0.888`}, {1454.2505000000012`, 0.884`}, {1455.1710000000012`, 
   0.9570000000000001`}, {1456.2055000000012`, 
   1.112`}, {1457.3850000000011`, 1.247`}, {1458.649000000001`, 
   1.281`}, {1459.905000000001`, 1.231`}, {1461.1015000000011`, 
   1.162`}, {1462.242000000001`, 1.119`}, {1463.3270000000011`, 
   1.051`}, {1464.3585000000012`, 1.012`}, {1465.3400000000013`, 
   0.9510000000000001`}, {1466.2620000000013`, 
   0.893`}, {1467.1465000000012`, 0.876`}, {1468.0340000000012`, 
   0.899`}, {1468.9810000000011`, 0.995`}, {1470.0825000000011`, 
   1.208`}, {1471.3285000000012`, 1.284`}, {1472.6095000000012`, 
   1.278`}, {1473.8475000000012`, 1.198`}, {1475.0220000000013`, 
   1.151`}, {1476.1395000000014`, 1.084`}, {1477.2010000000014`, 
   1.039`}, {1478.2110000000014`, 0.981`}, {1479.1595000000013`, 
   0.916`}, {1480.0555000000013`, 0.876`}, {1480.9285000000013`, 
   0.87`}, {1481.8270000000014`, 0.927`}, {1482.8455000000013`, 
   1.11`}, {1484.0285000000013`, 1.256`}, {1485.3055000000013`, 
   1.298`}, {1486.5900000000013`, 
   1.2710000000000001`}, {1487.8175000000012`, 
   1.184`}, {1488.9820000000013`, 1.145`}, {1490.0920000000012`, 
   1.075`}, {1491.1255000000012`, 0.992`}, {1492.0885000000012`, 
   0.934`}, {1493.0050000000012`, 0.899`}, {1493.9120000000012`, 
   0.915`}, {1494.872500000001`, 1.006`}, {1495.9575000000011`, 
   1.164`}, {1497.161500000001`, 1.244`}, {1498.408000000001`, 
   1.249`}, {1499.626000000001`, 1.187`}, {1500.7890000000011`, 
   1.139`}, {1501.877000000001`, 1.037`}, {1502.874500000001`, 
   0.9580000000000001`}, {1503.802000000001`, 
   0.897`}, {1504.693500000001`, 0.886`}, {1505.605500000001`, 
   0.9380000000000001`}, {1506.617000000001`, 
   1.085`}, {1507.7780000000012`, 1.237`}, {1509.0430000000013`, 
   1.293`}, {1510.3170000000011`, 
   1.2550000000000001`}, {1511.554500000001`, 
   1.22`}, {1512.7470000000012`, 1.165`}, {1513.8595000000012`, 
   1.06`}, {1514.8750000000011`, 0.971`}, {1515.8225000000011`, 
   0.924`}, {1516.7515000000012`, 0.934`}, {1517.7230000000013`, 
   1.0090000000000001`}, {1518.8060000000014`, 
   1.157`}, {1520.0205000000014`, 1.272`}, {1521.2845000000013`, 
   1.256`}, {1522.5085000000013`, 1.192`}, {1523.6675000000014`, 
   1.1260000000000001`}, {1524.7680000000014`, 
   1.075`}, {1525.7985000000015`, 0.986`}, {1526.7500000000014`, 
   0.917`}, {1527.6545000000015`, 0.892`}, {1528.5515000000014`, 
   0.902`}, {1529.5075000000013`, 1.01`}, {1530.6035000000013`, 
   1.182`}, {1531.8180000000013`, 1.247`}, {1533.0550000000014`, 
   1.227`}, {1534.2630000000015`, 1.189`}, {1535.4190000000015`, 
   1.123`}, {1536.5095000000015`, 1.058`}, {1537.5155000000016`, 
   0.9540000000000001`}, {1538.4345000000017`, 
   0.884`}, {1539.3005000000016`, 0.848`}, {1540.1540000000016`, 
   0.859`}, {1541.0470000000016`, 0.927`}, {1542.0655000000015`, 
   1.11`}, {1543.2390000000016`, 1.237`}, {1544.4935000000016`, 
   1.272`}, {1545.7405000000017`, 1.222`}, {1546.9235000000017`, 
   1.1440000000000001`}, {1548.0340000000017`, 
   1.077`}, {1549.0835000000018`, 1.022`}, {1550.0765000000017`, 
   0.964`}, {1551.0070000000017`, 0.897`}, {1551.8825000000018`, 
   0.854`}, {1552.7375000000018`, 0.856`}, {1553.6325000000018`, 
   0.934`}, {1554.6665000000019`, 
   1.1340000000000001`}, {1555.8950000000018`, 
   1.323`}, {1557.203500000002`, 1.294`}, {1558.4675000000018`, 
   1.234`}, {1559.6660000000018`, 1.163`}, {1560.7935000000018`, 
   1.092`}, {1561.8555000000017`, 1.032`}, {1562.8585000000016`, 
   0.974`}, {1563.7990000000016`, 0.907`}, {1564.6860000000015`, 
   0.867`}, {1565.5495000000014`, 0.86`}, {1566.4415000000015`, 
   0.924`}, {1567.4485000000016`, 1.09`}, {1568.5875000000015`, 
   1.188`}, {1569.8060000000014`, 1.249`}, {1571.0450000000014`, 
   1.229`}, {1572.2425000000014`, 1.166`}, {1573.3705000000014`, 
   1.09`}, {1574.4315000000013`, 1.032`}, {1575.4315000000013`, 
   0.968`}, {1576.3670000000013`, 0.903`}, {1577.2500000000014`, 
   0.863`}, {1578.1110000000015`, 0.859`}, {1578.9955000000014`, 
   0.91`}, {1579.9910000000013`, 1.081`}, {1581.1335000000013`, 
   1.204`}, {1582.3710000000012`, 
   1.2710000000000001`}, {1583.6280000000013`, 
   1.243`}, {1584.8395000000012`, 1.18`}, {1585.988500000001`, 
   1.118`}, {1587.0805000000012`, 1.066`}, {1588.118000000001`, 
   1.0090000000000001`}, {1589.100000000001`, 
   0.9550000000000001`}, {1590.019500000001`, 
   0.884`}, {1590.888000000001`, 0.853`}}
$\endgroup$
9
  • $\begingroup$ Please add your code so that we can copy into a notebook. I'm not going to attempt to type out your code for myself. $\endgroup$
    – Hugh
    Commented Dec 31, 2020 at 22:12
  • $\begingroup$ @Hugh I have added to post. Thanks for your interest. $\endgroup$
    – Chris
    Commented Dec 31, 2020 at 22:27
  • $\begingroup$ @Chris Thank you for adding the code. Can you add the data, or a snippet thereof, or a way to generate a similar dataset? We want to be able to copy / paste and run your code, and to do that we need code and data. $\endgroup$
    – MarcoB
    Commented Dec 31, 2020 at 22:28
  • $\begingroup$ @MarcoB 500 points fit into my post' $\endgroup$
    – Chris
    Commented Dec 31, 2020 at 22:37
  • 2
    $\begingroup$ BTW, the code sample given in your question doesn't work. (Definition of beatinterval is missing, 2nd argument of Table is not properly modified, there may be more. ) Please double check it. $\endgroup$
    – xzczd
    Commented Jan 1, 2021 at 3:44

1 Answer 1

8
$\begingroup$

I don't like LowpassFilter and HighpassFilter. Somewhere there is a discussion on how to make them work, but I can't find it now. (Relevant posts are here and here.) I prefer Butterworth filters. These are standard filters.

The only problem is that they take a bit of work to use. First you make the filter, then you have to convert it to an infinite impulse response filter and then apply it.

To get started I have called your data data and as you had spotted it is not equally spaced sampled data so I interpolate and resample. I work out the average increment from your original data.

 {t1, t2} = {data[[1, 1]], data[[-1, 1]]};
    mean = Mean[data[[All, 2]]];
    inc = Mean[Differences[data[[All, 1]]]];
    int = Interpolation[data];
    d1 = Table[{t, int[t]}, {t, t1, t2 - inc, inc}];
    plotd1 = ListLinePlot[d1, PlotStyle -> {Blue},
      PlotMarkers -> Graphics@{Disk[{0, 0}, [email protected]]},
      AspectRatio -> 1/10, ImageSize -> Full]

Original data

The next step is to make the filter and then apply it to the data. I start by defining the sample rate of the data (equals 1/increment). Then I note the start and stop frequencies you require. These have to be expressed as a ratio of the sample rate and we need to express this as a fraction of 2 Pi. I have only done the high-pass filter here, but the low-pass filter is a similar procedure.

sr = 1/inc;
{f1, f2} = {0.07, 0.15};
filtHP = ToDiscreteTimeModel[
   ButterworthFilterModel[{"Highpass", 2, f1/sr 2 \[Pi]}],
   sr, Method -> { "BilinearTransform", "CriticalFrequency" -> f1/sr}];
d1HP = RecurrenceFilter[filtHP, d1[[All, 2]]];
Show[
 ListLinePlot[Transpose[{d1[[All, 1]], d1HP}], PlotStyle -> Green,
 PlotRange -> {All, All}],
 plotd1, AspectRatio -> 1/10, ImageSize -> Full,
 PlotRange -> {All, All}]

Filtered data

Note that the start falls from a larger value. It takes a few samples to take out the mean.

The Butterworth filter has variants of lowpass, highpass and bandpass, etc. I suggest you experiment with those. I think you are taking the correct approach to check as you progress. There is another step which involves pre-warping frequencies link here (it is only necessary if you are dealing with filter values that approach half the sample rate).

Happy new year!

Hope that helps.

$\endgroup$
4
  • $\begingroup$ Thanks. I noticed the butterworth filter, but it looked complicated. A high band pass filter should be child's play; just throw out the low frequences if the fourier transform. I don't understand why Mathematica has decided to make it complicated. They seem to be using a smoothing window, even if you don't select the option. The default high pass filter should not require a smoothing window. Mathematica has screwed up. $\endgroup$
    – Chris
    Commented Jan 1, 2021 at 0:47
  • $\begingroup$ "Somewhere there is a discussion on how to make them work but I can't find it now. " I guess you mean the one now linked under the question? :) $\endgroup$
    – xzczd
    Commented Jan 1, 2021 at 2:29
  • $\begingroup$ @xzczd Well done! How do you remember the links? Have you just got a good memory? Perhaps this is a question for meta. $\endgroup$
    – Hugh
    Commented Jan 1, 2021 at 10:14
  • $\begingroup$ Oh I happen to remember this just because I seldom answer questions about signal processing. (To be precise, I have only 4 answers under signal-processing. ) As to finding duplicates, there does exist a related post in meta: mathematica.meta.stackexchange.com/q/2402/1871 $\endgroup$
    – xzczd
    Commented Jan 1, 2021 at 10:25

Not the answer you're looking for? Browse other questions tagged or ask your own question.