# Replacing and restructuring list elements in place

I have 200k pieces of data like this (only two elements shown), imported from a CSV file

{{2015, 11, 12, "17:04", 1.0811}, {2015, 11, 12, "17:05", 1.0811}}


and want to turn it into a list suitable for DateListPlot (I will only take small chunks once I have a plottable list], i.e. like

{{{2015, 11, 12, 17,04}, 1.0811}, {{2015, 11, 12, 17:05}, 1.0811}}


EDIT The first two lines of the CSV file are, verbatim,

Date_Time,Day,Month,Year,Date,Time,Open,Close,High,Low
12/11/2015 17:04,12,11,2015,12/11/2015,17:04,1.08110,1.08109,1.08110,1.08109


END EDIT

Having looked at Replace, and general List manipulation instructions I realise I don't know where to begin as I am still new to Mathematica. I have looked at related questions but none has seemed appropriate since I wish to replace and restructure at the same time.

I suppose I could simply build a new table and use DateValue to get the Hour and Minute from Part 4 of each sub list, but this seems heavy handed and I would also like to take the opportunity to learn more elegant/efficient methods of performing such list manipulations.

What would you recommend - and why?

[And - is there a way of forcing a list structure during the import so that the CVS fields for dates/time values are automatically combined into such a sublist?]

• "is there a way of forcing a list structure" ... not sure I understand, but see second argument of SemanticImport (not Import). Is that what you want? – Szabolcs Oct 19 '16 at 15:50
• Indeed, that might deliver the end result I am looking for the overall import operation, but having come across the problem of changing a list as described I would like to know that answer too. Was going AFK when your comment came in so will have to check later. Thx. – Julian Moore Oct 19 '16 at 15:54
• You can get your desired list if you do: {#[[1 ;; 4]], Last[#]} & /@ list – mgamer Oct 19 '16 at 16:34
• If you can upload a small sample of your CSV file, I can look into how to semantically import it properly. – dan7geo Oct 19 '16 at 17:43
• @dan7geo I have added header and 1st line; now that my attention has been drawn to SemanticImport I could do the simple part of making one list per CSV row, it's structuring it as required for DateListPlot that would be most illuminating – Julian Moore Oct 19 '16 at 19:41

To create a list of DateObjects and values you can first do this:

list = {DateObject@Join[#[[;; 3]],
ToExpression /@ StringSplit[#[[4]], ":"]], #[[5]]} & /@ data


Replace data with your data. ( /@ stands for Map and # & notation creates a pure function)

Then you can run the following to plot your data:

DateListPlot@list


EDIT:

To create a function that processes the list:

makePlottable := {DateObject@Join[#[[;; 3]],
ToExpression /@ StringSplit[#[[4]], ":"]], #[[5]]} & /@ # &


(This feature might be useful for further processing: https://reference.wolfram.com/language/ref/TimeSeries.html )

• I expect that works as promised, but having said I am still new to MMA in the question I'm afraid it's too terse for me, but my attempt to grok it is: DateObject (ah! {y, m, d..} or dateobj for DateListPlot) takes ymdhm(s) #[[;; 3]] picks Y M D, StringSplit must be getting h &m; all datetime stuff is then joined and becomes the DateObject and item 5 (the value) appended. # must be a placeholder for the current list item but how the #, & and /@ work together escapes me for now :( – Julian Moore Oct 20 '16 at 8:12
• Could you expand that expression as function calls so it looks like e.g. makePlottable[aList_List] := Map[... I tried and failed and this would help me understand & learn. – Julian Moore Oct 20 '16 at 10:01
• OK, I've got as far as makePlottable[aList_List] := {DateObject[Join[#[[;; 3]]], ToExpression [StringSplit[#[[4]], ":"]]], #[[5]]} & /@ aList but how do I replace the & /@ at the end? – Julian Moore Oct 20 '16 at 10:29
• See update to the answer with links explaining those things. – dan7geo Oct 21 '16 at 4:07
• :) Love the extra "# &" after the "& /@" I was already unclear about :) Where might I find such non-trivial examples explained step by step (or operator by operator) so I can decode for my still rather procedural brain? I've upvoted your answer but, since I did ask for the "why?" (meaning: how it works) I haven't accepted it (yet) – Julian Moore Oct 21 '16 at 13:47