The problem with clouds of data everywhere

Some years ago now, I started a new project. I went into the project with a lot of bold ideas about everything from what to buy, and what side projects I could start – and even one or two ideas about maybe doing the actual project.

But one idea that I felt very strongly about was that, unlike all my previous projects, THIS time I would actually organise all my data nicely, neatly, and not with my normal chaos model.

Obviously that didn’t happen.

I’m only human after all, and the above cartoon is as near as can be an accurate window into exactly what happened. And if that’s where it ended, then I would have at the very least, basically managed to mess up my file organisation at the same level. But during the project, there has very unfortunately, been a technological advance that has somehow made this already chaotic mess even worse.

Cloud storage is amazing.

I feel I need to make that very clear before I immediately start complaining about it. Cloud storage has made sharing files and working on multiple devices an absolute breeze. One central location for all my project data, nice simple interfaces, access rights for project partners – it’s the sort of thing I’ve dreamed of being able to do for years. (I have very strange slightly tech-based dreams…)

But, it turns out the great cloud solution to so many problems has one or two very tiny drawbacks. The first symptom of which is that I am currently signed up to, and actively using, three different clouds.

Why three? Well thanks to the plethora of choices, the people I work with have all chosen different serivces. So in order to work and collaborate with one set of project partners I use google Drive, and with another I use Dropbox.

I’ve tried to persuade them to move to some other shared platform but this resulted in the third partner getting a Box account and the others staying where they were. Nothing like a classic attempt to standardise to spawn yet more problems.

There are also now rumours that the university is starting to catch up to the existence of cloud storage and is potentially going to start offering a fourth. At this rate by 2030 I’ll have more cloud storage services than I do folders on my computer.

But that was just the start, now that the different services have started adding more features, the problems got worse. With one colleague, if we’re grant writing then that means a google Docs. But if we’re sharing data then Box. But if for any reason we need to work in Microsoft then Dropbox integrates that better so we work on there.
There is one person (who will rename nameless) who has managed to send me things on all three platforms over the last 6 months, and last week they shared an Evernote book with me. I have a suspicion they might have been trying to bait me into writing this article.

Navigating the location of certain datasets is starting to feel a bit like plotting a 3D route through an asteroid belt. Only not fun or exciting, and without the joyful possibility of the sweet release of an asteroid in the face.

Cloud storage has led to my project being no longer contained chaos in one small messy corner of my computer, but a vast rainbow of data distributed between different clouds and computers. And I’m pretty sure my project data wouldn’t make a colourful rainbow but one made of shades of brown and grey.

And to make things worse, with the approaching end of the project I need to start sorting the data into data sets and putting them online… does anyone know any data managers willing to do contract work?