Wed 15 November 2017 | tags: Scientific Computing, R, -- (permalink)

I have recently helped a PhD student read and merge about 150 CSV files. I used R, but the student wanted to use Stata later, so I used the haven package to export to the Stata 14 native file format.

There is nothing to report about the process everything was quick and easy and worked as expected. But I noticed that .dta file (Stata file) was substantially larger than the original data. The original data CSV files were a little above 8GB, the consolidated R file was about 1.3GB (no surprises here as R saves the files in a ...

Mon 25 September 2017 | tags: Scientific Computing, R, -- (permalink)

Scientific computation is nowadays an integral part of most research in many science fields. But the majority of researchers never had any formal training on how to structure, maintain, and collaborate on such projects. I had to learn the hard way, and there are several resources nowadays that I wished I had access to when I was starting.

A good start is to read Good enough practices in scientific computing paper, and Best Practices for Scientific Computing. They provide apt recommendations from data management, collaboration, project organisation, revision control systems to the writing of manuscripts.

I also recommend The Plain ...