Translating between Matlab and R

I mentioned in a previous post that I’d write a guide about translating Matlab code to work in R, so that others can avoid the same mistakes I made. This should also function as an R users guide to learning Matlab syntax and vice versa. I hope some people find it useful!

Full article below.

rmatlab

Continue reading

Translation Issues

Where on earth did the time go? One minute I was looking at ice sculptures and wading through snow drifts and now the snow is gone (mostly) and various wildlife has emerged from hiding.

Look, beavers!

I’ve also managed to get out on the water myself. And eat bacon while doing it. Canadian bacon is Different.

riverbaconRiver bacon!

Perhaps the main reason I’ve lost track of time in the last few weeks is that whether I’ve been awake or asleep, this keeps on flashing in front of my eyes.

codegif.gif

I haven’t been able to escape it. My office desktop has looked like this on and off for the whole month. I hope to later show what this has resulted in, but in the meantime I’m going to moan about the amount of pain it’s caused me.

The main source of trouble was that this code was originally written in Matlab. I decided to save us from having to acquire a Matlab license by translating the many scripts that make up the code into R.

Initially this was tedious. There are enough syntax differences (not to mention different names of functions etc) between R and Matlab that this required me to go through the code line by line. Then, even when I done this a number of errors arose simply due to the differing ways that the two programs handle data. I’ll post a guide based on what I learnt in separate post, featuring less pictures of beavers.

Much hair pulling later I got the code running, fed it my data and got a result. These results were consistent with some previous findings obtained using simpler methods. So far so good.

I then decided that instead of feeding my data to the code all in one go, it would be useful to give it one day at a time and then collate the results. “Fine” I thought. “Just modify my overarching processing code, no trouble”.

I was wrong.

c149182a696e73890de876f3d392e2da(Found via googling evil Matlab)

Once again the way in which the two programs handle data required me to make a lot of modifications to the various scripts. Cue more hair pulling. I should also mention that I’ve written this code to be run in parallel, utilising all of my computers cores to increase speed, which means R’s normal debugging tools don’t work.

Finally I got the code to run again and got a result. However, something had changed. A previously suggested relationship had completely reversed in direction. Was this simply due to the new way of feeding the data in? Or was it due to a bug in my code? Or due to me deleting some faulty data? I ran the code again using the original way of processing the data.

Even using parallel processing, this code can take anything from several hours, to all night to run. This meant that getting results was a slow process. So after waiting several hours for the code to run again using the original data processing, once again I got results.

The relationship had flipped direction in these results too.

Moving-animated-clip-art-dont-panic-picture

This suggested that the changes I’d made to accommodate the new data processing method had resulted in a COMPLETELY DIFFERENT RESULT. On the one hand, this was good. It meant that the biologically unrealistic result was due to my error rather than a fundamental problem with the methods. On the other hand, this is the sort of thing that can wake a scientist up at night screaming. A series of small changes in the way data was analysed leading to completely misleading results. In this case we’d caught it before we went too far, but if we’d approached this naively it might have been very easy to miss.

So, now I needed to work out which of my changes had caused this change. Luckily I save all my working files in Dropbox, which keeps a backup of all previous versions. I found a word document containing graphs I’d made to show my supervisor before I’d made changes and reverted all my code to a date before then. Then one by one I reinstated my changes.

In the end I pinned it down to one file. In that file, one line of code.

as.matrix(Y)

One line of code had resulted in huge, significant changes to my final result. As I said, the stuff of nightmares.

In the end I stripped out all the changes I’d made and carefully rewrote the scripts to deal with the new method of data processing. So my tale of woe has a happy ending, the code now works and perhaps I’ll even have some results soon. For everyone who made it this far, here is a view of Gatineu Park:

IMG_3201

On preparing for fieldwork and preparing for gigapocolypse

So, as the last post said, I’m back on the Scillies. Once again I came over to help out with an undergraduate field-course and was then left behind! Here are the undergrads, a thoroughly good bunch.

Image

As I said in last year’s field diary, it’s a bit surreal being a demonstrator on this field course as it seems like only a couple of years since I myself was on the same field course as an undergrad.

Image

The field course was a nice chance to take stock before starting my own work. While the students wandered the island conducting mini-behavioural experiments on the local wildlife (I am happy to say that one group enthusiastically chose shags!), I could settle back into island life and get into the right frame of mind for fieldwork.

Image

There are shags about (also, Razorbills!), though everything seems to be in lower numbers than last year.

At the end of the field-course, I once again watched the Scillonian disappear over the horizon. Now I was left to my own devices. Rushing headlong into fieldwork would not be a wise decision though. I needed to be sure I knew exactly what I was doing before I started. How I would choose sites, how I would decide where to go each day, what my sampling protocol would be and how I would record data.

Image

This year, I’m going to be videoing rafts of foraging shags around the island, in order to capture all the behaviour that goes on within these large groups. Since we can’t identify individual birds (even the colour ringed ones keep their legs underwater!) I attempt to avoid constantly recording the same birds by filming from several spatially distinct sites.

I ideally want to record data in each of these sites in a variety of tidal conditions. Is the tide ebbing or flooding? Is the tide at its fastest flow (half way between two tidal states) or slowest (right after high or low water)? Or is it in between?

Birds might also forage differently at different times of day, regardless of tide. I want to capture this potential variation with my sampling protocol if I can.

That is an awful lot of combinations of states to think about! Working out where I am going the next day could give me a serious headache. So I got a computer to do it for me. For those interested, here’s how it works:

  • Firstly I got hold of a tide timetable
  • I then wrote a script (using open source statistics package R) to interpolate between tidal states, at every hour. So for every hour I have the direction of the tide, the height of the tide and the stage of flow (according to the rule of twelfths).
  • From this big table of tidal states, I can extract exactly what the tide is doing at each time of day that I am interested in, on every day.
  • Finally I wrote another script, which extracts this information for whatever day it is tomorrow and then randomly generates a site number for me to visit. This script also checks this against what I’ve already done. The output looks like this:

Image

So every night before fieldwork, I run the script and it tells me which sites to go to, and what tidal conditions I’ll be collecting data in. The script makes sure I won’t record a combination of site, time of day and tidal conditions that I’ve already done (for now!).

So soon I’ll begin my fieldwork. This weekend however, is world gig rowing championship here in the Scillies. Hundreds of gig rowers are arriving on the island, possibly causing it to sink slightly.

I had a moment of horror when I went to the shop yesterday to stock up before the crowds arrived and found it utterly bare. Luckily the shop was restocked when the next boat arrives, so I have safely stocked up for gigapocolypse.

For the next few days, the shops will be emptied, the pubs will run dry and the cash machine will run out of money! I’m also going to be sharing my bunker with a whole load of gig rowers. It could be interesting.

(Last year, gig weekend looked like this!:

Image

)