Steven

164 reputation

bio	website
	location
	age
visits	member for	5 months
	seen	Feb 5 at 17:30
stats	profile views	22

	164 reputation	bio	website		visits	member for	5 months
5 badges		location			seen	Feb 5 at 17:30

summary answers questions tags badges favorites bounties reputation activity

48 Actions

suggestions reviews revisions comments badges posts accepts all

Feb 4	accepted	How do you force a decimal output?
Dec 12	revised	How do you force a decimal output? added 151 characters in body
Dec 12	comment	How do you force a decimal output? NumberForm[2.60151925345869310^8, 30] Still results in scientific notation. Annoyingly. AccountingForm seems to be the answer to my question. AccountingForm[2.60151925345869310^-8, 30] gives a nice decimal format I can use.
Dec 12	asked	How do you force a decimal output?
Dec 12	accepted	How do you deal with very large datasets in Mathematica?
Dec 11	comment	Data wrangling messy data Leonid, One of the answers in that question points to hashtables. This is closer to what I'd like. In Clojure I can do (map :bankbalance my-list-of-accounts) It returns a list of bank balances- very elegant, as it frees me from caring about the layout of my data.
Dec 11	revised	Data wrangling messy data added 450 characters in body
Dec 11	asked	Data wrangling messy data
Dec 11	revised	How do you deal with very large datasets in Mathematica? added 1221 characters in body
Dec 11	revised	How do you deal with very large datasets in Mathematica? added 467 characters in body
Dec 11	comment	How do you deal with very large datasets in Mathematica? Well, I know which columns are needed or not needed ahead of time. I get those when I'm training my classifier. But you still have to load a file or a row before discarding it in mathematica. So I could write some code to go through, read a column I want, skip the next few values, and read the next column I want. But that seems like the wrong way to do it. Particularly next to just taking the pain of some wasted memory, and doing it in chunks of 5K rows.
Dec 10	comment	How do you deal with very large datasets in Mathematica? This stream approach looks very promising. Is there any way to read a line of a CSV file in as a list? If so, I can hack up something that'll work quite easily. I see RecordSeparators, but apparently those are three fixed CR/LF Types. It also gets confused by CSV field quoting.
Dec 10	comment	How do you deal with very large datasets in Mathematica? mark: Sorry, finance style. ->2mm=2 million. Joel. You're probably right, I'll think about the phrasing and post a separate question.
Dec 10	comment	How do you deal with very large datasets in Mathematica? I can throw out unused columns, but have to load up the entire row first. Which is where my trouble lies. I think my best approach is just to do predictions in chunks of say 5000 rows at a time. Which will easily fit in memory.
Dec 10	comment	How do you deal with very large datasets in Mathematica? After looking at some of the work involved, there seems to be two options that don't radically increase complexity: 1. Buy more memory. 2. Spit my prediction file into chunks to make do for now, then rewrite in Clojure later. (Clojure is a similar language to mma for my purposes.). I've still no clue how to deal with the varying/shifted fields in otherwise similar datasets.
Dec 10	asked	How do you deal with very large datasets in Mathematica?
Dec 7	revised	How to map over non-Null values in a list added 544 characters in body
Dec 7	comment	How to map over non-Null values in a list Moved comment to question.
Dec 7	asked	How to map over non-Null values in a list
Dec 6	revised	Can I store a DataDistribution in a database? Found a nice, tidy solution.