Clik here to view.
Guess who wins: apply() versus for loops in R
Yesterday I tried to do some data processing on my really big data set in MS Excel. Wow, did it not like handling all those data!! Every time I tried to click on a different ribbon, the screen didn’t...
View ArticleClik here to view.
Memory Management in R, and SOAR
The more I’ve worked with my really large data set, the more cumbersome the work has become to my work computer. Keep in mind I’ve got a quad core with 8 gigs of RAM. With growing irritation at how...
View ArticleClik here to view.
Big data analysis, for free, in R (or “How I learned to load, manipulate, and...
Before choosing to support the purchase of Statistica at my workplace, I came across the ff package as an option for working with really big datasets (with special attention paid to ff dataframes, or...
View ArticleClik here to view.
Know Your Dataset: Specifying colClasses to load up an ffdf
When I finally figured out how to successfully use the ff package to load data into R, I was apparently working with relatively pain free data to load up through read.csv.ffdf (see my previous post)....
View ArticleClik here to view.
save.ffdf and load.ffdf: Save and load your big data – quickly and neatly!
I’m very indebted to the ff and ffbase packages in R. Without them, I probably would have to use some less savoury stats program for my bigger data analysis projects that I do at work. Since I started...
View ArticleClik here to view.
KDD Cup 2015: The story of how I built hundreds of predictive models….And got...
The challenge from the KDD Cup this year was to use their data relating to student enrollment in online MOOCs to predict who would drop out vs who would stay. The short story is that using H2O and a...
View Article