Vasudhendra Badami

“Today’s world is drowning in data and starving for insights. Our digital lives have created an overwhelming flood of information. In the last 5 years data scientists have come to the rescue by trying to make sense of it all. The sexy job in the next 10 years will be statisticians, and I’m not kidding.” – Hal Varian, chief economist at Google

Until the end of the last decade, the word “data scientist” hardly existed. However, new possibilities have opened up new frontiers owing to the huge volumes of data that keeps piling up. And, irrevocable changes in the way businesses are run have spawned loads of analysts and number crunchers to “manage” data and predict successful future strategies and outcomes.

Organizations are still falling over themselves trying to hire data scientist who can harness the power of data to hasten the data-to-action process. Although not as many companies as should be are relying on data-driven decision making, by the turn of this decade, analytics will have taken over. Just ask early adopters such as Facebook, Amazon, and LinkedIn.

Rest assured, automated programs aren’t going to make data scientists obsolete anytime soon.

In this article, you’ll find the most recommended learning path to become a data scientist. In addition, we’ve added links for best tutorials to get started on your data scientist path.

Who is a Data Scientist?

Here’s an interesting definition of a data scientist on the web: “A data scientist is someone who is better at statistics than any software engineer and better at software engineering than any statistician.”

You can read two conflicting views in Larry Wasserman’s Data Science: The End of Statistics? and Andrew Gelman’s Statistics is the least important part of data science.

It is far “safer” to say a data scientist wears many hats!

Data scientists typically uncover commercially valuable information hidden in tons of unstructured and structured data. They apply a formidable skillset of programming, statistics, math, business acumen, great communication, and some psychology on huge data sets to provide actionable insights.

These big data wranglers need to have contextual understanding and intuition to come up with magic. Identifying whether the data is meaningful requires an excellent blend of technical and business skills. And that’s what aspiring data scientists should build on.

Before you organize, package, and deliver data, you need to know how.

What skills do you need to become a consummate data scientist?

Looking at social scientist Drew Conway’s famous Data Science Venn diagram, hacking skills, math and stats knowledge, and substantive expertise (commonly assumed to be domain knowledge) portray the interdisciplinary nature of a data scientist’s strengths.

• A data scientist needs hacking skills to collect and munge e-data, math and stats knowledge to apply the right tools and techniques to glean key insights, and substantive expertise to ask motivating questions and make predictions.
• Modeling follows exploring data. This is where math and stat come into play. The trick lies in finding the most suitable technique to apply on big data to identify the least error-prone predictive model.
• The final step would involve a data scientist knowing how to interpret the results and ask interesting questions.

There is a series of Venn diagrams modeled on Conway’s version.

The 2016 version by Gartner perhaps makes more sense with its specific call-outs.

In the video below, Jeff Hammerbacher, Cloudera Co-founder and a prominent data scientist, calls data scientists “data rats.” Hammerbacher, who coined the term data scientist, says there is no perfect background to becoming a data scientist. In practice, he believes there are five components you need to be trained on to do your job properly:

• Data collection and integration
• Data visualization (dashboard design)
• Large-scale experimentation
• Causal inference and observational studies
• Data products (fitting machine learning models, deploying in production, setting up a regular refresh cycle, and evaluating performance)

Watch video

So what’s in a data scientist’s technical toolkit?

A data scientist has to be more than proficient in the following tools and techniques. We’ve provided a few useful links to help you get an idea about the specific topics.

• Mathematics—linear algebra, calculus
• Statistics—probability, Bayesian stats, inferential and descriptive stats, game theory, optimization
• Programming Languages (R, Python, C++, Java etc)—algorithms and data structures, distributed computing
• Machine learning in R or Python—random forests, neural nets, k-nearest neighbors, decision trees, SVM, etc.
• Databases and querying languages— MySQL, MongoDB, Cassandra
• Big Data technologies—Apache, Hadoop, Pig, Spark, Hive, Flume, Cloudera
• Cloud services like Amazon S3

Check out Top Data Science Skills by Job Role here.

How can you become a data scientist?

Watch this great webinar by Jesse Steinweg-Woods on “How to become a data scientist in 2017” to get answers to some really specific questions.

For the professional:

If you want to switch to a career in data science, then taking free MOOCs (Coursera, Udacity, EdX, Khan Academy) or enrolling for online classes could be your best bet. People from diverse backgrounds can find themselves doing really well in analytical jobs because of their amazing talent for problem-solving, curiosity, and communication skills.

For the student:

Universities world over offer graduate courses in data science, business intelligence, analytics, and big data technologies. For math, statistics, or computer science undergraduates, this could be a fantastic option. If you want to study on your own, that’s fine too because there are lots of free e-books to help you master the skills you need.

Joining competitions, attending data science meet-ups, doing projects for experience, and updating your repertoire of skills will ensure that you are near-perfect for the job. It is really all about practice. And tons of it.

Before you go all out, you can get an internship or join a bootcamp with companies such as Amazon, Zipfian, and Twitter just to be sure that this is the right career choice for you.

Why become a Data Scientist?

You don’t need to be sold the idea. Really?

You love numbers. You love data. But truthfully, aren’t the big bucks and the job security great incentives?

Data scientists make about $130,000 a year on average. Since 2013, job postings for data scientists have grown by 108%. Research says that the career path for a data scientist is expected to touch almost 19% this decade. And, Glassdoor says that data science jobs have great average scores for work-life balance. Data scientists are critical assets for any organization today.

With studies saying that demand is expected to outpace supply and top companies all over looking for the brightest analysts, you can figure out the answer for yourself quite easily.

By the way, did you know there are several job roles in data science?

1. A complete guide to hiring a data scientist
2. The most promising tech jobs for 2018
3. 8 ways to hire a developer [Actionable tips]

• Built-in WiFi
• 4 USB ports
• 1 HDMI 2.0
• 1 MicroSD card slot
• 40 GPIO pins
• A 3.5mm audio jack
• 1 Micro USB for power

In recent times, the Internet of Things is ubiquitous and is now a popular domain in the developer community. According to research by Statista, there are 6.21 million developers working in IoT and 5.36 million developers planning to work in IoT in next 6 months.

If you want to get started in IoT and are wondering which programming language to start with, here is a list of 11 popular programming languages used in IoT.

C

C, the language that was first developed to program telephone switches, is a reliable and reasonable choice for embedded system development. It is impressive because of its proximity to machine language.

It is a procedural language and the code is compiled and not interpreted. The code written in C is more reliable and scalable, and processor independence makes it a strong contender for IoT development. Because C is not platform independent, it enables IoT developers for code reuse, which can run on most of the systems.With the help of pointers, accessing and modifying addresses is easy in C.

C++

C++ is a middle-level programming language with imperative, object-oriented, and generic programming features with low-level memory manipulation.

C++ is designed with a bias toward system programming, embedded programming, resource-constrained devices and large systems. The design highlights of C++ are

• Performance
• Efficiency
• Flexibility of use

C++ is a popular choice among the embedded developers coding for Linux systems. Linux programming expertise, particularly with C++, is crucial for developing efficient, scalable, and secure IoT application. Here are a few features that make C++ a popular choice among IoT developers:

• Data hiding
• Stronger typing/ checking
• Multi-peripheral transparency using classes
• Templates (as always if used carefully)
• Initialisation lists

Go

Go is an open source programming language developed at Google. It combines the benefits of compiled language that is performance and security with that of aspeed of dynamic language.

It supports concurrent input, output, and processing on many different channels. Coordination of an entire fleet of sensors and actuators is possible when used correctly. The biggest risk is that the different channels don’t necessarily know about one another. If a programmer isn’t careful enough, a system could behave unpredictably because of a lack of coordination between channels.

In GO, gathering and sending data to various sensors and actuators is made easy by adding explicit hash table type.

The biggest advantage of GO is its ability to sort an entire network of sensors and making use of related IoT programming related devices. Go is now available on a wide variety of processors and platforms.

JavaScript

JavaScript is a scripting language with syntax similar to C. Initially, it was mainly used to create web pages, but now JavaScript is widely used in web servers, mobile apps, and IoT systems. JavaScript is good at event-driven applications; this allows every device to listen to various other events and respond to the concerned events. It has a garbage collector which eliminates freeing up of memory.

In your project, if you want to use the Apache server on a Raspberry Pi to collect data from a network of Arduino-based sensors, JavaScript would be good to start with.

Here are 10 reasons why JavaScript is used in IoT.

Python

The language, which was developed during a holiday break, went on to become the most preferred language for web development and started gaining popularity in embedded controls and IoT. Python is an interpreted language which can be either submitted for runtime compilation or run through one of theseveral pre-compilers so that compact executable code may be distributed.

The greatest benefit that Python offers to developers is readability with elegant syntax, without compromising on the size. Python’s clean syntax is apt for database arrangement. If your app requires data to be arranged in a database format and use tables for data control, Python is the best choice.

Python supports a huge number of libraries and modules, so you can get more stuff done with less code. It’s handy in more powerful edge devices, gateways, and also the cloud.

Rust

Rust is an open source, general-purpose, multi-paradigm, compiled programming language sponsored by Mozilla. Rust shares many of Go’s qualitiesand solves race condition problems of Go. Rust includes functions that eliminate race conditions for highly concurrent programs, making it a less risky language. Because of its ability to handle concurrent programming, Rust is now popular among IoT developers.

Rust is a safe, concurrent and practical language, supporting functional and imperative-procedural paradigms. It maintains these goals without a garbage collector. This makes Rust a useful language for the following use cases:

• Embedding in other languages
• Programming with specific space and time requirements
• Writing low-level code, like device drivers and operating systems

Rust is an improvement on current languages by having a number of compile-time safety checks which produce no runtime overhead and eliminates all data races.

Java

Java is an object-oriented language, and there are very few hardware dependencies built into the compiler, which makes it incredibly portable.

The biggest concern in IoT is security; with the Generic Connection Framework 8 (GCF 8), the Access Point API in Java provides the latest security standards and the highest levels of networked encryption and authentication which ensure data privacy.

All the object references in Java are implicit pointers which cannot be manipulated by application code. This automatically rules out the potential risk of memory access violations which can inevitably cause an application to stop all of a sudden.

Connectivity at the application level of the IoT system is also easily handled in Java with a comprehensive set of APIs, both standard and freely available through open source projects.

Assembly language

Assembly language is a low-level programming language and is specific to a particular type of computer architecture. In contrast, to other high-level programming languages, Assembly language is not portable across multiple architectures.

Assembly language is also called symbolic machine code. It is converted into executable machine code by a utility program.

Assembly language is best if you want to make your IoT project compact, minimal, and optimal.

ParaSail

ParaSail stands for Parallel Specification and Implementation Language. It is a compiled, object-oriented language with syntax similar to Java, Python, C#, and Ada.

ParaSail is a language that you must consider if you have a requirement for parallel processing in your IoT system.

ParaSail is capable of both specifying and implementing parallel applications. It supports both implicit and explicit parallelism. Every ParaSail expression is defined to have parallel evaluation semantics.

R

R is a programming language and environment for statistical computing and graphics. It is widely used by statisticians and data miners for building statistical software and data analysis.

Here are a few statistical and graphical techniques implemented by R and its libraries:

• Linear and nonlinear modeling
• Time-series analysis
• Classical statistical tests
• Clustering
• Classification and others

R is easily extensible through functions and extensions. It has stronger object-oriented programming facilities than most statistical computing languages. Extending R is also made easy with its lexical scoping rules.

B#

B# was designed as a very small and efficient embedded control language. The embedded virtual machine (EVM) that allows B# to run on a variety of different hardware platforms only takes 24 kb of memory. B# is lean enough for 8-bit MCUs.

The syntax of B# looks a bit like C#, with support for the real-time control functions that are critical to make things happen in the real world. The B# code when coupled with a compact virtual machine could be easily ported and reused across multiple hardware platforms.

B# supports writing portable interrupt handlers and device addressing registers in a uniform way.

It also supports modern object-oriented features such as

• Namespaces
• Abstract and concrete classes
• Interfaces
• Delegates

In addition to these, B# caters to the embedded systems programmer with

• Efficient boxing/unboxing conversions
• Multi-threading statements
• Device addressing registers
• Deterministic memory defragmenter
• Field properties
• Interrupt handlers

Each of these features is directly supported by the constructs of B# and its underlying virtual machine. This helps to create, use, and reuse more portable and decoupled software components across different embedded systems applications.

If your project is going to live on embedded hardware platforms that aren’t as big and complex as a Raspberry Pi, then B# is the best option available.

Forth

Forth has been around since 1970 and is designed and optimized for embedded system programming. It is an imperative stack-based language and environment. It includes features such as

• Structured programming
• Concatenative programming
• Extensibility
• Reflection

Forth features both interactive execution of commands and the ability to compile sequences of commands for later execution.

Forth is used in the Open Firmware boot loader, in space applications such as spacecrafts, and other embedded systems where interaction with hardware is involved.

HiveQL

HiveQL is based on SQL but does not strictly follow the full SQL-92 standard. It runs on the Hadoop map-reduce framework but hides the complexities from the developers.

HiveQL does not offer extensions in SQL, including multi-table inserts and create table as select. It only offers basic support for indexes. Here are few things which HiveQL can do easily:

• Create and manage tables and partitions
• Evaluate functions
• Support various Relational, Arithmetic, and Logical Operators
• Download the contents of a table to a local directory or result of queries to HDFS directory

Pig Latin

Pig Latin is the language of Apache Pig, which is a high-level platform for creating programs that run on Apache Hadoop. Pig Latin can be extended using User Defined Functions (UDFs) which you can write in Java, Python, JavaScript, Ruby, or Groovy and then call directly from the language.

Here are a couple of key properties of Pig Latin:

• Ease of programming. With Pig Latin, it is easy to achieve parallel execution of simple and “embarrassingly parallel” data analysis tasks. Complex tasks with multiple interrelated data transformations are explicitly encoded as data flow sequences; this makes them easy to write, understand, and maintain.
• Optimization opportunities. The way in which tasks are encoded permits the system to optimize their execution automatically, allowing you to focus on semantics rather than efficiency.

Extensibility. Users can create their own functions to do special-purpose processing.

Julia

Julia is a high-level, high-performance dynamic programming language for numerical analysis and computational science. Julia language provides

• A sophisticated compiler
• An extensive mathematical function library
• Numerical accuracy
• Distributed parallel execution

What makes Julia’s design unique is its type system with parametric polymorphism, and it types in a fully dynamic programming language. It also has multiple dispatch as its core programming paradigm. Julia allows concurrent, parallel, and distributed computing and direct calling of C and Fortran libraries without any glue code.

Julia’s LLVM-based JIT compiler combined with the language’s design allows it to often match the performance of C.

It does not impose any particular style of parallelism on the user. Instead, Julia provides flexibility to the user by providing a number of key building blocks for distributed computation through which it supports various styles of parallelism.

In 1995, JavaScript emerged as programming language to create web pages. Brendan Eich developed JavaScript with syntax similar to C, but nobody believed that JavaScript would play a major role in development of commercial softwares. In 1997, JavaScript was made a standard through ECMA international. But what gave the required firepower to JavaScript from breaking out of just being a programming language to creating web pages are the

• The creation of JavaScript Standard Object Notation (JSON)- data interchange format, as a strict subset of JavaScript by Douglas Crockford and
• The introduction of Node.js in 2009 by Ryan Dahl. Node.js has played a crucial role in building web servers in JavaScript by using Google’s super fast V8 JavaScript engine.

Now JavaScript is widely used in web pages, web servers, mobile apps, and IoT systems.

You ask me why JavaScript is used in building IoT systems? Here are my reasons.,

JavaScript is a event-driven language

It is quite good at event-driven applications. In event driven applications, every device listens to various other events and responds to concerned events.

Event loops in JavaScript allow you run numerous tasks without waiting for other tasks to complete. This helps in responding to events in real time, handling multiple tasks parallelly, and allowing multiple devices to respond to the same event. This contributes to a great extent in saving precious battery power.

Memory management

JavaScript has a garbage collector, which eliminates the need of explicitly freeing up the memory. This allows embedded developers to focus on other important aspects of development. The automatic freeing of the unused memory results in a stable product because the garbage collector eliminates memory leaks.

One drawback of garbage collector with constrained devices is trashing – the garbage collector running very often has an adverse impact on performance. This can be avoided with the JavaScript programming style which limits the creation of new objects to major state changes in the embedded device or application. This keeps the memory usage stable without running the garbage collector very often.

Existing JavaScript tools, libraries and plugins

With the increased use of JavaScript in various applications, there are many JavaScript development resources available, such as

• JavaScript libraries like Underscore.js, lodash, traverse, and Async
• Testing tools like Blue Ridge, SugarTest, FireUnit, JSLint etc.
• Client-side development framework
• Server-side JavaScript APIs and others

JavaScript developers in IoT have sophisticated frameworks and engines like CycloneJS, IoT.js, JerryScript, Duktape, etc. specifically designed for constrained devices.

JavaScript compatible hardware solutions

A wide variety of hardware solutions in IoT, such as Raspberry Pi, Espruino, etc.. support Node.js. There are JavaScript-only microcontrollers such as Tessel 2 and Espruino which have proven to be very useful in IoT projects. Thousands of Node Package Manager (NPM) modules for Node.js such as PM2, Socket.io, Mocha etc. have been developed to enhance the power of Node.js in IoT.

End-user scripting

End user scripting can be enabled by embedded devices using JavaScript because JavaScript is a managed execution environment where end-used scriptability is secure by safely sandboxing the scripts. End-used scripting brings new ideas and possibilities because the customization enabled by allows end-users, hobbyists, and professional programmers to script their devices and add new functionalities.

JavaScript is open source

This is another crucial factor that makes JavaScript a suitable programming language for IoT. The open source nature of JavaScript makes it possible for programmers to make useful contributions to various JavaScript projects. This fosters creativity and brings in innovations to IoT enabled hardware, software, and network solutions.

JavaScript is widely used across internet

Already JavaScript is a commonly used language across the Internet, so it makes absolute sense to include the same language in the devices which will be part of the Internet.

JavaScript is easy to learn

JavaScript is very easy to learn, so you can start coding in a short span of time. Consider a situation where a C programmer is asked to do parallel programming.

To do parallel programming in C, he has to be a “stud.” He has to know what he is doing. And here is this JavaScript thing; it is kind of a built-in feature. Now any 12-year-old can do parallel programming!

Familiar Syntax

Currently, a majority of embedded programmers are using C. In the process of developing an IoT system, embedded programmers need not learn different syntax if they are using JavaScript. Syntax of JavaScript is very similar to C, hence an experienced embedded programmer just needs to invest a little time to understand and successfully modify JavaScript code.

Popularity of Node.js !

Node.js has evolved as a robust technology which makes server-side implementation of JavaScript possible. The event-driven nature and asynchronous input-output (IO) model of Node.js makes it a perfect fit to build IoT systems.

A range of tech majors are leveraging the power of Node.js in building a network of devices, sensors, and smartphones; these can be controlled and manipulated remotely. Companies like Microsoft, IBM, and Samsung have already embraced Node.js as the preferred technology for their IoT development projects.

So now,