The science of data

I am currently between work assignments and, like many people, I have been furloughed until something comes up. I am using the time to educate myself a little more. I am a developer who works with various technologies, but the requirements of any goven job can be quite narrow and may use older technology. I have been working quite a bit with Python recently and wanted to look into the area of data science. This is about extracting knowledge from data. Fortunately Python has some good functionality for this in the Pandas data analysis library.

Image from Open Law Lab with CC licence.

I like some structure to my learning and so looked for some suitable courses. I have used Coursera in the past for various courses on programming as well as music. They have some good teachers who do video lessons. They had a data science course from the University of Michigan.

Previously I was able to do Coursera course without paying. You just did not get an official certificate at the end. Now it seems you need to pay, but at around £1 per day it is not too bad considering you could do a lot of courses over a year. I am doing another on networks concurrently.

My friend @stav is in a similar situation and he is also doing some learning.

Some courses use other students to do the grading, but this one makes use of Jupyter notebooks where you run code on a web page and the results are automatically verified.

I have just completed for the third week of six and I have to say I am finding it challenging. I have done plenty of database programming and this has some similarities with that, but the way things are done is different enough to stretch my brain. Last week I got 100% right, but I got a couple wrong this week. That was still enough to pass. They rely on you doing some research of your own to find appropriate functions to achieve the goals. That can be hard when you are not sure what to search for. This week I had to import some Excel data and massage it into the appropriate form. I expect this is routine for data sciences, but you need experience to know which tools to reach for, much as a mechanic needs to know which tools are needed to strip an engine down.

I am finding this course interesting despite some frustration with the difficulty. I may need to do more reading of other material.

I have hopes that I can apply the knowledge I gain to some projects around Hive. There is plenty of data here to analyse. I may start with some personal projects to look at my own data. There is lots of fun to be had generating charts from it.

Although I may be in the second half of my working life I am still keen to keep learning. I have worked with computers for decades and they are endlessly fascinating. The technology is much more diverse than when I started and nobody can be an expert in everything these days.

Stay curious.