"Data Scientist: Sexiest Job of the 21st century" – Harvard Business Review, 2012

In more recent times, Glassdoor named it the "best job of the year" for 2016.

Where did the title "Data Scientist" come from?

The title was coined in 2008 by Dr. Dhanurjay Patil, Chief Data Scientist at the White House Office of Science and Technology Policy, and Jeff Hammerbacher, Chief Scientist at Cloudera. Since then, data science has evolved from a niche field to a core business function and one of the most sought-after careers globally.

For those aspiring to become data scientists in India, here's a curated list of top-ranking professionals on Kaggle, highlighting their education, roles, and key achievements.

Bishwarup Bhattacharjee (Current Rank: 9)

Bishwarup holds a bachelor's degree in Statistics from the University of Calcutta and is currently a Senior Business Analyst at VMware. He has previously consulted for multiple companies and co-founded Alphinite Analytics.

Achievements:

1. 1st place – Allstate Claims Severity Challenge
2. 2nd place – BNP Paribas Cardif Claim Challenge
3. 4th place – Santander Customer Satisfaction Challenge

Abhishek Thakur (Current Rank: 19)

Abhishek holds a B.Tech. from NIT Surat and a Master's from the University of Bonn. He has served in several data science roles and is now Principal Data Scientist at ProductsUp.

Achievements:

1. Gold medal – Springleaf Marketing Response Challenge (Rank 2)
2. Rank 3 – How Much Did It Rain? Challenge
3. Rank 3 – Otto Group Product Classification Challenge

Sudalai Rajkumar (SRK) (Current Rank: 26)

SRK, Lead Data Scientist at Freshdesk, is an experienced problem solver with a background in engineering and analytics. He’s also a top solver on CrowdANALYTIX.

Achievements:

1. Gold medal – How Much Did It Rain? Challenge (Rank 2)
2. Rank 5 – EEG Detection Challenge
3. Rank 6 – Taxi Trajectory Prediction Challenge

Thakur Raj Anand (Current Rank: 45)

Thakur is a Data Scientist at DataRobot. He completed his post-graduation in Applied Quantitative Finance from Madras School Of Economics and has held various roles in analytics and consulting.

Achievements:

1. Rank 6 – Bosch Production Line Performance Challenge
2. Gold medal – Homesite Quote Conversion Challenge
3. Rank 9 – Avito Duplicate Ads Detection Challenge

These accomplished professionals serve as an inspiration for aspiring data scientists in India and beyond. Their contributions reflect the power of data-driven problem solving and continuous learning in a competitive global landscape.

In our previous redis blog we gave a brief introduction on how to interface between python and redis. In this post, we will use Redis as a cache, to build the backend of our basic twitter app.

We first start the server, if it’s in a stopped state.

sudo service redis_6379 start
sudo service redis_6379 stop

In case you have not installed the redis server, you can install the server and configure it with python using the previous tutorial.

We will work on creating our own custom Twitter and post tweets to this. Users should be able to post tweets, and there should be a timeline forthe posts. The screenshot of the final product is shown below.

We will use flask and redis for this. Flask is a good python web microframework which lets you focus only on things you need. There is more focus on the modularity of your code base. Redis is a key-value datastore that can be used as a database. Redis is an excellent choice for caching and for constant real-time analysis of data coming in, hence redis is a great tool to build a twitter-like platform.

Let us start building the module. There are some build dependencies; therefore ensure the following dependencies are installed.

sudo apt-get install build-essential
sudo apt-get install python3-dev
sudo apt-get install libncurses5-dev

Once done, fire-up a virtualenv and install the requirements.

virtualenv venv -p python3.5
source venv/bin/activate
wget https://raw.githubusercontent.com/infinite-Joy/retwis-py/master/requirements.txt
pip install -r requirements.txt

Create a folder structure of the following format.

mkdir retwis
cd retwis

Frontend using Jinja templates

Flask lets us create the template files - layout.html, login.html and signup.html. These templates are designed using the Jinja2 templates which Flask uses. We can use template inheritance and login and signup pages will inherit from layout.html.

Check out the three template files shown below.

<!doctype html>
<title>Retwis</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='style.css') }}">
<nav class="navbar navbar-default navbar-fixed-top">
  <div class="container-fluid">
    <div class="navbar-header">
      <h1>Retwis</h1>
    </div>
    <div id="navbar" class="navbar-collapse collapse">
      <ul class="nav navbar-nav navbar-right">
        <li>
        {% if not session.username %}
          <a href="{{ url_for('login') }}">log in</a>
        {% else %}
          <a href="{{ url_for('logout') }}">log out</a>
        {% endif %}
        </li>
      </ul>
    </div>
  </div>
</nav>
<div class="main-body">
  <div class="container">
    {% block body %}{% endblock %}
  </div>
</div>

Note that we have abstracted out the common elements of all the pages. We have defined the header with the title and then in the body; if a session is present, there will be the login link, else there will be the logout link.

Check out the login and the signup html which are almost similar.

{% extends "layout.html" %}
{% block body %}
  <h2>Login</h2>
  {% if error %}<p class="error"><strong>Error:</strong> {{ error }}{% endif %}
  <form action="{{ url_for('login') }}" method="post">
    <div class="form-group">
      <label for="username">Username</label>
      <input class="form-control" type="text" name="username">
    </div>
    <div class="form-group">
      <label for="password">Password</label>
      <input class="form-control" type="password" name="password">
    </div>
    <button class="btn btn-default" type="submit">Login</button>
  </form>
  <a class="btn btn-default" href="{{ url_for('signup') }}">Sign up</a>
{% endblock %}

{% extends "layout.html" %}
{% block body %}
  <h2>Signup</h2>
  {% if error %}<p class="error"><strong>Error:</strong> {{ error }}{% endif %}
  <form action="{{ url_for('signup') }}" method="post">
    <div class="form-group">
      <label for="username">Username</label>
      <input class="form-control" type="text" name="username">
    </div>
    <div class="form-group">
      <label for="password">Password</label>
      <input class="form-control" type="password" name="password">
    </div>
    <button class="btn btn-default" type="submit">Sign up</button>
  </form>
{% endblock %}

As you can see, if there is no error, then we define the username and the password fields that are bound with the “post” method.

We can now create the basic flask app and see if the two templates get rendered correctly. We create two endpoints for the templates and then render them. Check out the code below.

from flask import Flask
from flask import render_template

app = Flask(__name__)
DEBUG = True

@app.route('/signup')
def signup():
    error = None
    return render_template('signup.html', error=error)

@app.route('/')
def login():
    error = None
    return render_template('login.html', error=error)

if __name__ == "__main__":
    app.run()

To run the server use the following command.

python views.py

On your browser, open http://127.0.0.1:5000/signup

And hit http://127.0.0.1:5000/

You should be able to see the two pages above.

We will also need to create the home page which the user will fall back to once he is logged in. Create a home.html in the templates folder and then write the tweets block.

{% extends "layout.html" %}
{% block body %}
  <form action="{{ url_for('home') }}" method="post">
    <div class="form-group">
      <input class="form-control" type="text" name="tweet" placeholder="What are you thinking?">
    </div>
    <button class="btn btn-default" type="submit">Post</button>
  </form>
  {% for post in timeline %}
    <li class="tweet">
      {{ post.username }} at {{ post.ts }}
      {{ post.text }}
    </li>
  {% else %}
    <h2>No posts!</h2>
  {% endfor %}
{% endblock %}

As you see, if there are posts on the timeline, then list the username, time, and the text, else put “No posts” in header format. Let’s build the code for that in view.py and see how it looks.

@app.route('/home')
def home():
    return render_template('home.html', timeline=[{"username": "dummy_username",
                                                   "ts": "today",
                                                   "text": "dummy text"}])

If you check out the url http://localhost:5000/home, you should get the page below.

Now that we have all the pages and have built the frontend, in the next post we will build the redis backend that will handle the user information, the session data, and the posts that the users submit.

Sessions and user information

We will be using redis to get user information. If you don't have redis-py already installed in your virtual environment, install it using pip.

pip install redis

Next, we need to plugin redis to our flask app and see that it gets instantiated before each request.

import redis

from flask import Flask
from flask import render_template

app = Flask(__name__)
DEBUG = True

def init_db():
    db = redis.StrictRedis(
        host=DB_HOST,
        port=DB_PORT,
        db=DB_NO)
    return db

@app.before_request
def before_request():
    g.db = init_db()

# remaining code here.

We will interface the signup page with redis and on signing up, the user information should get populated in the redis datastore.

We change the signup function to the code below.

import redis

from flask import Flask
from flask import render_template
from flask import request
from flask import url_for
from flask import session
from flask import g

app = Flask(__name__)

# other code …

@app.route('/signup', methods=['GET', 'POST'])
def signup():
    error = None
    if request.method == 'GET':
        return render_template('signup.html', error=error)
    username = request.form['username']
    password = request.form['password']
    user_id = str(g.db.incrby('next_user_id', 1000))
    g.db.hmset('user:' + user_id, dict(username=username, password=password))
    g.db.hset('users', username, user_id)
    session['username'] = username
    return redirect(url_for('home'))

Here, we take the username and the password from the form and push them to the redis database. Note that we increment the keys by 1000. This is a standard for redis keys. For more information, consult the official docs.

We will also need to set a secret key to use session information which is used in the code above. You can read about sessions and how to set session keys from the official docs. We will also do a little bit of refactoring and keep the settings information together.

# import statements

app = Flask(__name__)

# settings
DEBUG = True

# I am using a SHA1 hash. Use a more secure algo in your PROD work
SECRET_KEY = '8cb049a2b6160e1838df7cfe896e3ec32da888d7'
app.secret_key = SECRET_KEY

# Redis setup
DB_HOST = 'localhost'
DB_PORT = 6379
DB_NO = 0

# def init_db(): ...
# def before_request(): ...
# def signup(): ...
# def login(): ...
# def home(): ...

if __name__ == "__main__":
    app.run()

Check out the form now and try to submit some user information.

Check on the redis end and check out the values that have been populated.

?  redis-cli
127.0.0.1:6379> HGETALL *
(empty list or set)
127.0.0.1:6379> KEYS *
1) "users"
2) "user:1000"
3) "next_user_id"
127.0.0.1:6379> HGETALL "users"
1) "hackerearth"
2) "1000"
127.0.0.1:6379> HGETALL "user:1000"
1) "username"
2) "hackerearth"
3) "password"
4) "hackerearth"

Once the session and signup functions work fine, we can then focus on the home page where people can login once they have signed up. These two pages should fall back safely to the home page.

@app.route('/', methods=['GET', 'POST'])
def login():
    error = None
    if request.method == 'GET':
        return render_template('login.html', error=error)
    username = request.form['username']
    password = request.form['password']
    user_id = str(g.db.hget('users', username), 'utf-8')
    if not user_id:
        error = 'No such user'
        return render_template('login.html', error=error)
    saved_password = str(g.db.hget('user:' + str(user_id), 'password'), 'utf-8')
    if password != saved_password:
        error = 'Incorrect password'
        return render_template('login.html', error=error)
    session['username'] = username
    return redirect(url_for('home'))

The code tells us if the request method is “GET”, then we render the login page. This is the first page that comes up when we go to the page http://localhost:5000/.

After that, we will fill up the fields with the previous values. The entered username and password is pulled from the form. Using this username, we get the user ID from the redis database and this user ID is used to retrieve the password. This password is then matched with the entered password. If there is a match, then we will be redirected to the “home page.”

We now need to work on the home page. The home page is the biggest of the three modules as these do several things simultaneously. It should handle the session information. If the session information is not there, it should transfer to the login page. It should retrieve the posts of the user and push them to the redis database and get the data in turn. So we will replace the home function in views.py with the code below.

@app.route('/home', methods=['GET', 'POST'])
def home():
    if not session:
        return redirect(url_for('login'))
    user_id = g.db.hget('users', session['username'])
    if request.method == 'GET':
        return render_template('home.html', timeline=_get_timeline(user_id))
    text = request.form['tweet']
    post_id = str(g.db.incr('next_post_id'))
    g.db.hmset('post:' + post_id, dict(user_id=user_id,
                                       ts=datetime.utcnow(), text=text))
    g.db.lpush('posts:' + str(user_id), str(post_id))
    g.db.lpush('timeline:' + str(user_id), str(post_id))
    g.db.ltrim('timeline:' + str(user_id), 0, 100)
    return render_template('home.html', timeline=_get_timeline(user_id))

def _get_timeline(user_id):
    posts = g.db.lrange('timeline:' + str(user_id), 0, -1)
    timeline = []
    for post_id in posts:
        post = g.db.hgetall('post:' + str(post_id, 'utf-8'))
        timeline.append(dict(
            username=g.db.hget('user:' + str(post[b'user_id'], 'utf-8'), 'username'),
            ts=post[b'ts'],
            text=post[b'text']))
    return timeline

Note, the timeline part is handled in the _get_timeline function. We get the timeline from the redis database and then for all the posts we put the username, time and the post text to a timeline list. This list is returned to the home function, which takes the user tweet post and pushes it to redis, after which it renders the current posts in the timeline. We will also need to “import datetime.”

import redis

import datetime

from flask import Flask
from flask import render_template
from flask import request
from flask import url_for
from flask import session
from flask import g
from flask import redirect

# rest of the code

We need to build the url for logout for the template to work correctly.

@app.route('/logout')
def logout():
    session.pop('username', None)
    return redirect(url_for('login'))

Now, check it in the browser. Hit http://localhost:5000; login with your credentials. You should be able to post tweets now to the post.

Please refactor the code to make it more organized. Also, use Test Driven Development and good logging practises when building production-grade apps (although it isn’t in this post). Please find the whole code in this github repo.

Credits

A big shoutout to kushmansingh/retwis-py who inspired me to write the blog.

Last week when I sharedThe top programming languages that will be mostpopular in 2017, the frequent comment was, what does it take to be a better developer?

I’ve met some amazing developers in real life and through React Native Community, and I decided to ask them, “How do I become a better developer?” Thank you to everyone who took the time to answer these questions with passion!

This is a compilation of answers I received from them. Some of these quotes are not limited to answers from that specific question.

Interviewees / Current Position

• Aravind Kumaraguru (Engineering Director @Pioneers in Engineering)
• Brent Vatne (Front-end Developer @Exponent)
• Charlie Cheever (Co-founder @Exponent)
• Christopher Chedeau (Front-end Engineer @Facebook)
• Dan Horrigan (Senior Back-end Developer @Futuri Media)
• Frank W. Zammetti (Lead Architect @BNY Mellon)
• Janic Duplessis (Co-founder @App & Flow)
• Jake Murzy (Co-founder @commitocracy)
• Jun Ho Hwang (Software Engineer @Coupang)
• Keon Kim (Machine Learning Maniac @NYU)
• Munseok Oh (Co-founder and CTO @Sketchware)
• Satyajit Sahoo (UX Lead @ Glucosio & Front-end Engineer @Callstack.io)
• Sonny Lazuardi Hermawan (Engineer @Sale Stock)
• Sunggu Hwang (CTO @ScatterLab)
• Timothy Ko (Software Engineer @Snapchat)

Q&A

Aravind Kumaraguru

Aravind is an undergrad at UC Berkeley pursuing a degree in Electrical Engineering and Computer Science and is Engineering Director for the nonprofit organization Pioneers in Engineering.

Q: How do you think I can become a better developer?

A: Obviously, never stay complacent with what you know – this field changes ridiculously fast, and you need to keep up with it. Follow along with the news in the tech industry, perhaps read up on some source code for a Python module that you recently used.

A friend of mine had some free time over winter break, so he decided to teach himself Django and build a webapp that he could interact with over SMS. It’s sort of a toy project, but he really enjoyed learning the different development paradigms. For context, he specializes in embedded systems and robotics, so this is nowhere near his comfort zone.

But pushing yourself to try different things will make you much stronger as an engineer. I personally wish I had done more web stuff before this year – in my organization (PiE), we’re developing a new iteration of a robotics kit to be used by high school students. While I have a good grasp of the low-level and systems stuff, I’m at a loss when it comes managing our UI design. Never had an interest in doing that type of stuff full-time, but having even a surface-level knowledge can be immensely helpful

Q: Do you have any projects you did to push yourself out of your comfort zone?

A: I built an automated door opener last summer, which operated a mechanical lever to open a door when an RFID card was scanned. The project used a really powerful motor and a mess of sensors to track the state of the arm, which proved to be quite difficult to coordinate. I learned real quick that I would need to do a bunch of offline testing before running my code on the device, which was very different from what I was used to up till then.

In terms of academics, I just finished CS 189, which was a massive crash course in data science, optimization, and probability theory. The programming I did in that class was also very different from what I’m used to, even though it was all in Python.

Brent Vatne

Brent is Front-end web/mobile developer working on Exponent and React Native. He contributes to tons of open-source projects.

Q: I really want to become a better developer; what would you say the first step is?

A: Do stuff you’re excited about and contribute to open source projects:-D

Q: How old are you and how much experience do you have as a programmer?

A: I am 30 years old, and very much 😮

Q: How did you join Exponent? What was the cause?

A: James (ide) and I were the most active contributors to a react-native outside of facebook and so we spoke a lot. He created exponent with Charlie. I ended up doing some consulting work with them and Charlie asked if I’d be interested in working with them full time and year, it was lots of fun so I joined.

Q: I should know objective C and Java thoroughly before I jump into React Native, right?

A: You can learn it as you go if you need to. there’s also tons of pure javascript stuff that need to be done. and documentation. lots of things 🙂

Charlie Cheever

Charlie Cheever is the co-founder of Quora, an online knowledge market. He was formerly an engineer and manager at Facebook, where he oversaw the creation of Facebook Connect and the Facebook Platform. Prior to Facebook, Cheever was employed by Amazon.com in Seattle. He left Facebook to start Quora in June 2009 to work on Exponent.

Q: What’s the motivation of Exponent being free and Open Source?

A: I really want to make something that like a 12-year-old version of me would use. So, someone who doesn’t know tons about programming but can learn new things and doesn’t have a credit card or lots of money, but has time and creativity and a phone and friends. I learned to program making calculator games on TI-85, it’s sad to me that kids can’t make stuff on their phones today.

Q: Why did you leave Quora?

A: I managed the mobile teams there and it was so slow to work on those apps even tho we had good people, I found it so frustrating And after I left I tried to build some mobile stuff and it was so annoying that I decided there needed to be a different way to make stuff. So James and I made something like react Native called Ion. It was strikingly similar actually. But React Native already had android support and 20 people working on it, and we had 2 people. So we decided to make everything else around it that we wanted to make!

Q: What did you do on Facebook?

A: I made the developer platform that all those games like FarmVille were on. Well, not all of it obviously but was one of two main developers. And I worked on the first version of facebook video, then did a lot of random other things. Then was a manager and did log in with Facebook on other sites, and then left to do Quora.

Christopher Chedeau

Christopher has been working at Facebook as a Front-end Engineer for about 5 years. Previously, he worked at Curse Network.

Q: What do you do on Facebook?

A: I was on the photos team when I started, then I discovered React and started adopting and promoting it both internally and externally. I was there at the beginning of reacting native and pushed it through until 3 months ago. I just recently switched to the Nuclide team. I’m still #3 contributor on React Native.😛

Q: Do you have any prior work experience?

A: I was working for Curse (doing website for blizzard games) during my college to pay for it. It was fun to see the company go from 5 people in a guild to a 100 people company.

Q: What’s your day to day like on Facebook? The current project you’re working on?

A: I’m currently working on the Nuclide team, Facebook’s IDE built on top of Atom. I would say my time is spent half coding, half cheerleading all the cool stuff people are doing inside of FB.

Q: How do you think one can become a better developer?

A: I think that there are multiple levels.

The first level is mastering all the concepts. For example yesterday I had to write a function that removes certain keys from a big nested object. Because I’ve done this task so many times in the past, I was able to implement it in one go without even thinking and it worked the first time. For this one, exercises are really good. You want to code the same kind of things many many times to train your muscle memory.

The second level is how do you build things in a way that are not going to break in the future. Ideally, once you build something, you can move to the next thing and it’ll keep working without you there. This is challenging when there’s a ton of developers touching the codebase and product directions changing often.

Finally, the third level is how do you prevent a whole class of problems from even existing in the first place. A good example is with manual dom mutations, it’s very easy to trigger some code that interacts with a dom node that has been removed from the dom. React came in and made this problem go away. You have to go out of your way to do so, and even if you want to do those things, you have the tools to make it work: lifecycle events.

Q: Is there something you wish you’d known or learned earlier as a programmer?

A: Probably the most important thing is: tradeoffs, tradeoffs, tradeoffs. They are everywhere.

If you are working on some random throwaway feature that no one is going to use, who cares if the code is maintainable, you need it to work and now one mistake I see a lot is that people over-engineer the easy things but are not willing to make their architecture less clean from a CS perspective even though it actually provides the user experience you need.

At the end of the day, we write all this code for the users, we should first understand what the user experience should be and then do whatever it takes to get it. If the user just needs to display some content and needs to be able to edit it easily, just install WordPress, pick a good looking theme and call it a day

– Btw, pro-tip, if you want to be successful, always think about the value you are providing. If you are earning $100k a year, this means that the company should be making $200k because you’re here

Dan Horrigan

Dan is a Senior Back-end developer @Futuri Media. He has 20 years of programming experience in many different languages. He’s been contributing to React Native early/mid-2015.

Q: What’s your background as a programmer?

A: I started learning to program (with QBasic) when I was 11 and was hooked. I learned everything I could, as fast as I could. I learned a few languages like Visual Basic and started to dabble with C and C++. Then I found web development and dove in head first. First, learning HTML and CSS, then adding simple CGI scripts written in Perl, and eventually Classic ASP.

My first paying project was when I was 14: A website for the company my dad worked for, with a customer portal to let them see their job progress. This was all in ASP. After that, I started learning PHP, and have been using that as my language of choice ever since. However, I picked up a lot of experience with other languages along the way: JS, Python, Ruby (on Rails), Java, C#, Go, Objective-C.

Q: What are some projects you’re currently working on?

A: I work for Future Media (http://futurimedia.com). We provide SaaS solutions for Broadcast Radio and TV companies. We provide white label mobile applications, social engagement and discovery, audio streaming and podcast solutions, etc. I haven’t had much free time lately to contribute to many OSS projects, but hope to change that soon!

Currently, I am a Senior Back-End Web Developer, but I am transitioning into being the Director of Technical Operations.

Q: Is there something you wish you’d learned or knew earlier as a developer?

A: I wished I would have realized earlier in my career that it is OK to be wrong, and that failure is just a chance to learn.

Q: What’s the first step to becoming a good developer?

A: Come up with a small-ish project that you think would be cool, or would make your life easier, and just jump right in. Too many people try to learn without a goal other than “I want to learn to code.” Without a goal, you are just reading docs or copy/pasting from tutorials…you can’t learn that way.

To become a better developer, you need to do one simple thing: Never. Stop. Learning. Read other people’s code, figure out how that one app does that really cool thing you saw, read blogs, etc. No matter how good you are, or think you are, there is always someone better, and always more to learn.

Q: Is there a certain project you’re currently interested in? Next on your learning list?

A: I have been using, and occasionally contributing to, React Native since early/mid-2015, and continue to be interested in it.

Next, on my learning list is learning Erlang/Elixir. We build heavily distributed systems where I work and think we would really benefit from a language like that.

Frank W. Zammetti

Frank is a lead architect for BNY Mellon by day and the author of eight books on various programming topics for Apress by night

Q: How do I become a better developer?

A: I get asked this question quite a bit both at work from junior developers and from readers of my books. I always give the same answer: make games!

It sounds like a joke answer, but it most definitely is not! Games have a unique ability to touch on so many software engineering topics that you can’t help but learn things from the experience. Whether it’s choosing proper data structures and algorithms, or writing optimized code (without getting lost in micro-optimizations – at least too soon), or various forms of AI, it’s all stuff that is more broadly applicable outside of games. You frequently deal with network coding, obviously audio and visual coding (which tends to open your mind to mathematical concepts you otherwise might not be), efficient I/O and of course overall architecture, which has to be clean and efficient in games (and for many games, extensible). All those topics and more are things that come into play (hehe) when making games.

It also teaches you debugging and defensive programming techniques extremely well because one thing people don’t accept in games is errors. It’s kind of ironic actually: people will deal with some degree of imperfection in their banking website but show a single glitch in a game and they hate it! You have no choice but to write solid code in a game and you figure out what works and what doesn’t, how to recover from unexpected conditions, how to spot edge cases, all of that. It all comes into play and those are skills that developers need generally and which I find are most frequently lacking in many developers.

It doesn’t matter one bit if the game you produce is any good, or whether anyone else ever even plays it. It doesn’t matter if it’s web-based (even if your day job is), or mobile, doesn’t matter what technologies you use. The type of insight and problem-solving skills you build and tune when creating games will serve you well no matter what your day job is, even in ways that are far from obvious.

I’ve been programming games for the better part of 35 years now. No, none of them have been best-sellers or won awards or anything like that. In fact, it’s a safe bet that most people wouldn’t have even heard of my games, even the one’s still available today. None of that matters because the experience of building them is far and away the most rewarding part of it. Perhaps the best thing about programming games is that they are, by their nature, fun! You’re creating something that’s intended to be enjoyable so the process of creating it should absolutely be just as enjoyable. How many things can you do that are really fun while still being challenging and simultaneously help build the skills needed for a long career?

So yeah, make games, that’s my simple two-word answer!

Q: Is there something you wish you’d known or learned earlier as a programmer?

A: Hmm, tough question actually. I guess if there was one thing (and I’ll cheat and combine two things here because they’re related) I would say that early on I didn’t understand two very important phrases: “As simple as possible, but no simpler” and “Don’t let the perfect be the enemy of the good”.

I have a natural perfectionist mentality, so I spend a lot of time pondering architecture, API design, etc. I once spent 33 hours straight working on a Commodore 64 demo because ONE lousy pixel was out of place and my perfectionist brain just couldn’t live with it! Sometimes, I have to force myself to say “okay, it’s good enough, you’ve planned enough, now get to work and actually BUILD stuff and refactor it later if needed”, or I have to force myself to say “okay, it basically does what it’s supposed to, it doesn’t need to be absolutely flawless because nobody but me is even going to notice”. Especially when you’ve got deadlines and people relying on you, you have to make sure you’re working towards concrete goals and not constantly getting stuck trying to achieve perfection because you rarely are going to, at least initially anyway, no matter how hard you plan or try – and the dirty little secret in IT is that perfection rarely matters anyway! Good enough is frequently, err, good enough 🙂

And, your design/development approach should always strive to be as absolutely simple as possible. Of course, what constitutes “simple” is debatable and doesn’t necessarily even always have the same meaning from project to project, but for me some key metrics are how many dependencies I have (web development today is a NIGHTMARE in this regard – less is GENERALLY better) and how many layers of abstraction there are. Developers, especially in the Java world, like to abstract everything and they do so under the assumption that it’s more flexible. But if there’s one thing I’ve learned over the years it’s that the way to write flexible code is to write simple code. It’s better than abstractions and extension points and that sort of stuff because it’s just far easier to understand the consequences of your changes.

As a corollary, a terse code is NOT simpler code! Simple code is code that anyone can quickly understand, even less capable developers, and even yourself years after. Terse and “clever” code tends to be the exact opposite. Often times, the more verbose code is actually simpler because there are fewer assumptions and often less knowledge needed to understand it, less “code hoping” you have to do to follow things. Related to this is that writing less code isn’t AUTOMATICALLY better. No, you shouldn’t re-invent the wheel, but you also shouldn’t be AFRAID to invent a marginally better the wheel when it makes sense. Knowing the difference is hard of course and comes from experience, but if you think it’s ALWAYS better to write less code then you’re going to make your life harder in the long run.

Of course, don’t over-simplify code either. Too simple and suddenly extending it almost MUST mean a refactor. You never want to completely refactor because you HAVE to in order to build an app over time. There’s a balance that’s difficult to strike but it should always be the goal.

Oh yeah, and I wish I knew how to express myself in fewer words… but actually, I’m still obviously working on that one 🙂

Janic Duplessis

Janic is the co-founder of App & Flow, a react-native contributor, and open-source contributor.

Q: Any tips to becoming a better developer?

A: Don’t think there’s anything in particular, you just have keep learning and getting out of your comfort zone. Like trying a new language or framework from time to time. At least that’s what I do but I’m pretty sure there are some other good ways haha 🙂

Q: How can I start contributing to React Native?

A: The best is to start with something small like a bug fix or adding a small feature like an extra prop on a component. Most contributors know either iOS or Android and a bit of JS. There are also some JS devs that work on things like the package and clip. We keep some issues with a Good First Task label that should be a good place to start

Jake Murzy

Jake is an Open-source Archaeologist. He writes buzzword compliant code. Co-founder at @commitocracy.

Q: Hey Jake, any tips to becoming a better programmer? 🙂

A: Number one thing you should do is to learn your tools before you learn the language you work in because it will lead to faster feedback loops and you will get to experience more in less time. So install a linter and it will catch most of your errors as you type. It statically analyzes your code and recommends best practices to follow. You should always follow best practices until you gain enough experience to start questioning them.

Jun Ho Hwang

Jun is a software engineer at Coupang, which is the $5 Billion Startup Filling Amazon’s Void In South Korea. He is a very friendly developer who loves to connect.

Q: How do you become a better developer?

A: The word ‘better’ can be described in various ways–especially in the field of programming. A good developer could be someone who is exceptionally talented in development, someone who is amazing at communicating, or someone who understands Business very well. I personally think a “good” developer is someone who is in the middle–a person who can solve his or her business problem with their development skills, and communicate with others about the issue. Ultimately, to achieve this, it requires a lot of practice, and I recommend you to create your own service. Looking and thinking from the perspective of the user and improving the service to fulfill their needs really helps you grow as a better developer.

Q: Is there something you wish you’d known or learned earlier as a developer?

A: I really wish I started my own service earlier on. The hardest thing to grasp before developing is realizing how you can apply what you learned. Many developers are afraid to start a “service” because it sounds difficult; however, pondering about what to make and where to start, and then connecting those points of thought help you grow as a better developer.

Q: What do you do at Coupang? What are you currently working on?

A: Coupling provides a rocket-delivery-service, and I am working on developing a system called “Coupling Car,” which is related to insurance and monetary management. Furthermore, I’m thinking about adding transportation control system and the ability to analyze data from the log.

Keon Kim

Keon is a student at NYU who is really passionate about Machine Learning. He is a very active GitHub member who tries to contribute to open source projects related to machine learning.

Q: What are your interests? What kind of projects have you worked on?

A: I’ve been working on machine learning projects these days. I am one of the project members of DeepCoding Project, a project with a goal of translating written English to the source code. I’ve been contributing to a C++ machine learning framework called my pack(https://github.com/mlpack/mlpack), which is equivalent to skit-learn in Python.

I’ve also done some fun side projects: DeepStock (https://github.com/keonkim/deepstock) project is an attempt to predict the stock market trends by analyzing daily news headlines. CodeGAN (https://github.com/keonkim/CodeGAN) is a source code generator that uses one of the new deep learning methods called SeqGAN.

Q: How do you become a better developer?

A: I think it is really important to understand the basics. By basics, I mean math, data structures, and algorithms. Deep learning is really hot right now, and I see people jumping into learning it without basic knowledge in computer science and mathematics. And of course, most of them give up as soon as mathematical notations appear in the tutorial. I know this because I was one of them and it took me really long time to understand some concepts that students with a strong fundamentals could understand in a fraction of the time I spent. New languages, libraries, and frameworks are introduced literally every day these days, and you need the fundamentals in order to keep up with them.

Munseok Oh

Munseok is a Full-stack developer and CTO at Sketchware. He previously worked at System Integration for ~7 years.

Q: How do I become a better developer?

A: When I was very young and cocky, I evaluated other developers based on their coding style. There were certain criteria they had to pass in order for me to judge them as a good developer. But now, I really don’t think that way. Now, I believe that every developer is progressive, which means he or she is becoming a better developer every day. It doesn’t really matter if the style is bad or code is good–as long as the program runs, I think it’s great! Whether the program has room for growth or has bugs, I think the motivation to develop is what really matters. Developers usually are never satisfied with their skills. They are always eager to become better–probably why you’re doing this. It’s really hard to justify “good developer”. People like you will become better than me in no time. I still don’t think I am a good developer.

Q: What was the most difficult thing when you were developing Sketchware?

A: Developing Sketchware wasn’t too difficult because we had a good blueprint for the item. The direction was very clear for us to follow, so developing it was a breeze. However, there was a line we had to maintain for Sketchware–this line had two conditions:

1. Sketchware must be an easy tool for anyone to create applications.
2. Whatever the user takes away from Sketchware can be applied in their future career

Since we wanted Sketchware to be an efficient tool that can help users learn programming concepts, I am very considerate and think a lot when it comes to adding new features in the application.

Q: As a developer, is there something you wish you knew or fixed earlier?

A: I really wish I jumped into the Start-up world earlier. When it comes to developing, you need to be passionate and really enjoy what you do. Even if you pull 3 all-nighters, ponder all day long about a new algorithm, or stress about a new bug, everything will be okay if you’re enjoying it. It really goes back to the question #1–I get my energy from the joy I have when I develop, and that joy eventually makes you a better developer. When life hits you, most developers lose the passion for developing if you think of it as work. I used to be like that. But now, I’m really not worried–since developing brings joy to me now. Even if we run out of funds or our company burns down, it’s really okay since I am making the most out of what I am doing.

Satyajit Sahoo

Satyajit is the UX Lead at Glucosio, and Front-end Engineer at Callstack.io. He is an amazing open-source contributor; he is one of the top 5 contributors in React Native

Q: What is your background as a programmer?

A: I don’t really come from a programming background. I did my graduation in Forestry. I left post-graduation after getting a job offer and never looked back.

Q: What’s your day like on day to day basis?

A: It’s pretty boring. I wake up, order some breakfast online or go out, then start office work. In evening I go out to a bar or take a long walk if there’s enough time left. At night I mostly watch TV series or hack on side-projects.

Q: Motivation behind contributing to open source projects?

A: I’ve been involved in Open Source for a long time. When I was doing my graduation I got into Linux and got introduced to the world of Open Source. I loved it how we could learn so much from other projects. It fascinated me that developers were selfless to let us see and use the there code for free (mostly). I did a lot of Open Source projects in form of themes and apps during my college days, and it always made me happy when people forked them and changed to meet their needs, and send pull requests to fix things.

As a developer, I contribute to Open Source projects most of the time because I need a feature, or it improves something on a project I love. I think it’s better if we work together to fix stuff that is important to us rather than just filing issues.

Q: How do I become a better developer?

A: I think it’s important that we are open to new things. There’s a lot to learn, and we cannot learn if we stay in our bubble. Try new things, even if you think you can’t do it, even it looks complex on the surface. I have failed to do things so many times, but eventually succeed. In the process, I understand the problem and the solution, and then it becomes really simple.

Sonny Lazuardi Hermawan

Sonny is a JavaScript Full Stack Engineer, a React & React Native player, and an Open source enthusiast. He currently works as an Engineer at Sale Stock.

Q: How do you become a better developer?

A: I think always eager to learn is the key. Try everything, make mistakes, and learn from that mistakes. I agree that code review from partners and senior engineers will make our code better. Try publishing your own open source projects, meet other great developers and learn from them.

Q: What’s your motivation behind creating open source projects?

A: I just want the people to know about our idea, and try implementing it so that others can use our project. I’m really inspired by people that work on open source projects that used by many devs such as Dan Abramov that created redux.

Sunggu Hwang

Sunggu worked at Daum Communications for 4 years. Then, he left Daum to work at Scatter Lab as the CTO. This is his 5th year at Scatter Lab.

Q: How do you become a better developer?

A: Hmm… Becoming a good developer… Every developer has his or her own personality when it comes to programming. As an analogy, think about blacksmiths! Not all blacksmiths are alike–some enjoy crafting the best sword, while some might enjoy testing out the sword more than crafting it. I am a thinker–who plans and organizes thoughts before I carry out an action. I think a good developer knows how to write concise and clean code; you should practice this habit. Even though the trend for programming is always changing, and many people use different languages, write a piece of code that anyone can understand without comments.

Q: What do you think is the next BIG thing?

A: I’ve observed the evolution of programming languages, and I think it’s becoming more abstract every generation–procedural programming, imperative programming, functional programming… I think in the future, maybe in about 20 to 30 years, we will live in the time where the computer writes the code for us, and we just put them together like legos.

Q: What should I focus on studying?

A: I think deep learning is a must. Try different tutorials and learn it with passion. Math, algorithms–anything will help you in the long run.

Timothy Ko

Timothy is a software engineer at Snapchat. He previously worked at many places such as Riot Games, Square, etc.

Q: What do you do at Snapchat?

A: I’m a software engineer on the monetization team, so I work on anything related to making money. Some example projects are Snapchat Discover, a news platform within the iOS and Android apps; Ad Manager, a control panel used by sales and ad operations to flight ads; Ads API, which allows third-party partners to integrate their own ad platforms into Snapchat. Also, I was a past intern at Snapchat so I occasionally give talks and Q&As to upcoming interns. I’m also heavily invested in hiring and conduct a lot of interviews there.

Q: What do you do on a day-to-day basis?

A: What I’ve mentioned previously. Also, even after I pass on the work to other people, sometimes I have to go back and help support it or be part of the technical discussions on future changes. When new people join the team, usually I’m the one to ramp people up on how the code base looks like the kinds of frameworks we use, how a typical engineer workflow looks like, etc.

Q: What languages/framework do you guys mostly use?

A: For server code, it’s usually Java and for UI we use React Redux. Most teams work in google app engine, which is why we use Java, but some teams switch it up a little bit due to some app engine limitations. And of course, the product teams work in objective C for iOS and Java for Android.

Q: How do you think I can become a better developer?

A: I think the best thing to do is to do as many things as possible. I did seven internships while in school so I already had two years of work experience before I graduated. Work experience is super important because coding in a hackathon, doing personal projects, and doing school assignments are totally different than working with enterprise software and apps with real users. But you have to start somewhere, so that’s where going to school, doing personal projects, and competing in hackathons comes in. And while at work, I think the best way to succeed is to ask lots of questions and learn by doing. You can read and study all you want, but you might not understand what’s going on until you actually do it. Another thing is code reviews — you can do so much knowledge transfer by having a more senior engineer tear your code apart and tell you how to make it better. Also, if you ever come up with a proposal on how to solve a problem, getting a tech lead to bombard you with hard questions forces you to make sure you have every little detail covered.

*The article was originally posted by Sung Park on Github*

Introduction

“There exists a world for Machine Learning beyond R and Python!

Machine Learning is a product of statistics, mathematics, and computer science. As a practice, it has grown phenomenally in the last few years. It has empowered companies to build products like recommendation engines, self driving cars etc. which were beyond imagination until a few years back. In addition, ML algorithms have also given a massive boost to big data analysis.

But, how is ML making all these accomplishments?

After realising the sheer power of machine learning, lots of people and companies have invested their time and resources in creating a supportive ML environment. That's why, we come across several open source projects these days.

You have a great opportunity right now to make most out of machine learning. No longer, you need to write endless codes to implement machine learning algorithms. Some good people have already done the dirty work. Yes, they've made libraries. Your launchpad is set.

In this article, you'll learn about top programming languages which are being used worldwide to create machine learning models/products.

Why are libraries useful?

A library is defined as a collection of non-volatile and pre-compiled codes. Libraries are often used by programs to develop software.

Libraries tend to be relatively stable and free of bugs. If we use appropriate libraries, it reduces the amount of code that is to be written. The fewer the lines of code, the better the functionality. Therefore, in most cases, it is better to use a library than to write our own code.

Libraries can be implemented more efficiently than our own codes in algorithms. So people have to rely on libraries in the field of machine learning.

Correctness is also an important feature like efficiency is in machine learning. We can never be sure if an algorithm is implemented perfectly after reading the original research paper twice. An open source library consists of all the minute details that are dropped out of scientific literature.

7 Programming Languages for Machine Learning

Python

Python is an old and very popular language designed in 1991 by Guido van Rossum. It is open source and is used for web and Internet development (with frameworks such as Django, Flask, etc.), scientific and numeric computing (with the help of libraries such as NumPy, SciPy, etc.), software development, and much more.

Let us now look at a few libraries in Python for machine learning:

1. Scikit-learn

   It was started in 2007 by David Cournapeau as a Google Summer of Code project. Later in 2007, Matthieu Brucher started to work on this project as a part of his thesis. In 2010, Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, and Vincent Michel of INRIA took the leadership of the project. The first edition was released on February 1, 2010. It is built on libraries such as NumPy, SciPy, and Matplotlib.

   Features:

   1. It is open source and commercially usable.
   2. It integrates a wide range of machine learning algorithms for medium-scale supervised and unsupervised problems.
   3. It provides a uniform interface for training and using models.
   4. It also provides a set of tools for chaining, evaluating, and tuning model hyperparameters.
   5. It also supports libraries for data transformation steps such as cleaning data and reducing, expanding, or generating feature representations.
   6. In cases where the number of examples/features or the speed at which it is to be processed is challenging, scikit-learn has a number of options that we can consider when scaling the system.
   7. It has a detailed user guide and documentation.

   A few companies that use scikit-learn are Spotify, Evernote, Inria, and Betaworks.
   Official website: Click here

2. TensorFlow

   It was initially released on November 9, 2015, by the Google Brain Team. It is a machine learning library written in Python and C++.

   Features:

   1. It is an open source software library for machine intelligence.
   2. It is very flexible in that it is not just a rigid neural network library. We can construct graphs and write inner loops that drive computation.
   3. It can run on GPUs, CPUs, desktop, server, or mobile computing platforms.
   4. It connects research and production.
   5. It supports automatic differentiation which is very helpful in gradient-based machine learning algorithms.
   6. It has multiple language options. It comes with an easy to use Python interface and a C++ interface to build and execute computational graphs.
   7. It has detailed tutorials and documentation.

   It is used by companies like Google, DeepMind, Mi, Twitter, Dropbox, eBay, Uber, etc.
   Official Website: Click here

3. Theano

   It is an open source Python library that was built at the Université de Montréal by a machine learning group. Theano is named after the Greek mathematician, who may have been Pythagoras’ wife. It is in tight integration with NumPy.

   Features:

   1. It enables us to define, optimize, and evaluate mathematical expressions including the multi-dimensional arrays which can be difficult in many other libraries.
   2. It combines aspects of an optimizing compiler with aspects of a computer algebra system.
   3. It can optimize execution speeds, that is, it uses g++ or nvcc to compile parts of the expression graph which run faster than pure Python.
   4. It can automatically build symbolic graphs for computing gradients. It also has the ability to recognize some numerically unstable expressions.
   5. It has tons of tutorials and a great documentation.

   A few companies that use Theano are Facebook, Oracle, Google, and Parallel Dots.
   Official Website: Click here

4. Caffe

   Caffe is a framework for machine learning in vision applications. It was created by Yangqing Jia during his PhD at UC Berkeley and was developed by the Berkeley Vision and Learning Center.

   Features:

   1. It is an open source library.
   2. It has got an extensive architecture which encourages innovation and application.
   3. It has extensible code which encourages development.
   4. It is quite fast. It takes 1 ms/image for inference and 4 ms/image for learning. They say "We believe that Caffe is the fastest ConvNet implementation available."
   5. It has a huge community.

   It is used by companies such as Flicker, Yahoo, and Adobe.
   Official Website: Click here

5. GraphLab Create

   The GraphLab Create is a Python package that was started by Prof. Carlos Guestrin of Carnegie Mellon University in 2009. It is now known as Turi and was known as Dato before this. GraphLab Create is a commercial software that comes with a free one year subscription (for academic use only). It allows to perform end-to-end large scale data analysis and data product development.

   Features:

   1. It provides an interactive GUI which allows to explore tabular data, summary plots and statistics.
   2. It includes several toolkits for quick prototyping with fast and scalable algorithms.
   3. It places data and computation using sophisticated new algorithms which makes it scalable.
   4. It has a detailed user guide.

   Official Website: Click here

There are numerous other notable Python libraries for machine learning such as Pattern, NuPIC, PythonXY, Nilearn, Statsmodels, Lasagne, etc.

R

R is a programming language and environment built for statistical computing and graphics. It was designed by Robert Gentleman and Ross Ihaka in August 1993. It provides a wide variety of statistical and graphical techniques such as linear and nonlinear modeling, classical statistical tests, time-series analysis, classification, clustering, etc. It is a free software.

Following are a few packages in R for machine learning:

1. Caret

   The caret package (short for Classification And REgression Training), was written by Max Kuhn. Its development started in 2005. It was later made open source and uploaded to CRAN. It is a set of functions that attempt to unify the process for predictive analysis.

   Features:

   1. It contains tools for data splitting, pre-processing, feature selection, model tuning using resampling, variable importance estimation, etc.
   2. It provides a simple and common interface for many machine learning algorithms such as linear regression, neural networks, and SVMs.
   3. It is easy and simple to learn. Also, there are a lot of useful resources and a good tutorial.

   Official Website: Click here

2. MLR

   It stands for Machine Learning in R. It was written by Bernd Bischl. It is a common interface for machine learning tasks such as classification, regression, cluster analysis, and survival analysis in R.

   Features:

   1. It is possible to fit, predict, evaluate and resample models with only one interface.
   2. It enables easy hyperparameter tuning using different optimization strategies.
   3. It involves built-in parallelization.
   4. It includes filter and wrapper methods for feature selection.

   Official Website: Click here

3. h2o

   It is the R interface for H2O. It was written by Spencer Aiello, Tom Kraljevic and Petr Maj, with the contributions from the H2O.ai team. H2O makes it easy to apply machine learning and predictive analytics to solve the most challenging business problems. h2o is an R scripting functionality for H2O.

   Features:

   1. It is an open source math engine for Big Data.
   2. It computes parallel distributed machine learning algorithms such as generalized linear models, gradient boosting machines, random forests, and neural networks within various cluster environments.
   3. It provides functions for building GLM, K-means, Naive Bayes, Principal Components Analysis, Principal Components Regression, etc.
   4. It can be installed as a standalone or on top of an existing Hadoop installation.

   Official Website: Click here

Other packages in R that are worth considering for machine learning are e1071, rpart, nnet, and randomForest.

Golang

Go language is a programming language which was initially developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson in 2007. It was announced in November 2009 and is used in some of Google's production systems.

It is a statically typed language which has a syntax similar to C. It provides a rich standard library. It is easy to use but the code compiles to a binary that runs almost as fast as C. So it can be considered for tasks dealing with large volumes of data.

Below is a list of libraries in Golang which are useful for data science and related fields:

1. GoLearn

   GoLearn is claimed as a batteries included machine learning library for Go. The aim is simplicity paired with customizability.

   Features:

   1. It implements the scikit-learn interface of Fit/Predict.
   2. It also includes helper functions for data, like cross-validation, and train and test splitting.
   3. It supports performing matrix-like operations on data instances and passing them to estimators.
   4. GoLearn has support for linear and logistic regression, neural networks, K-nearest neighbor, etc.

   Official Website: Click here

2. Gorgonia

   Gorgonia is a library in Go that helps facilitate machine learning. Its idea is quite similar to TensorFlow and Theano. It is low-level but has high goals.

   Features:

   1. It eases the process of writing and evaluating mathematical equations involving multidimensional arrays.
   2. It can perform automatic differentiation, symbolic differentiation, gradient descent optimizations, and numerical stabilization.
   3. It provides many functions which help in creating neural networks conveniently.
   4. It is fast in comparison to TensorFlow and Theano.

   Official website: Click here

3. Goml

   goml is a library for machine learning written entirely in Golang. It lets the developer include machine learning into their applications.

   Features:

   1. It includes comprehensive tests and extensive documentation.
   2. It has clean, expressive, and modular source code.
   3. It currently supports models such as generalized linear models, clustering, text classification, and perceptron (only in online option).

   Official Website: Click here

There are other libraries too that can be considered for machine learning such as gobrain, goglaib, gago, etc.

Java

Java is a general-purpose computer programming language. It was initiated by James Gosling, Mike Sheridan, and Patrick Naughton in June 1991. The first implementation as Java 1.0 was released in 1995 by Sun Microsystems.

Some libraries in Java for machine learning are:

1. WEKA

   It stands for Waikato Environment for Knowledge Analysis. It was created by the machine learning group at the University of Waikato. It is a library with a collection of machine learning algorithms for data mining tasks. These algorithms can either be applied directly to a dataset or we can call it from our own Java code.

   Features:

   1. It is an open source library.
   2. It contains tools for data pre-processing and data visualization.
   3. It also contains tools for classification, regression, clustering, and association rule.
   4. It is also well suited for creating new machine learning schemes.

   Official Website: Click here

2. JDMP

   It stands for Java Data Mining Package. It is a Java library for data analysis and machine learning. Its contributors are Holger Arndt, Markus Bundschus, and Andreas Nägele. It treats every type of data as a matrix.

   Features:

   1. It is an open source Java library.
   2. It facilitates access to data sources and machine learning algorithms and provides visualization modules also.
   3. It provides an easy interface for data sets and algorithms.
   4. It is fast and can handle huge (terabyte-sized) datasets.

   Official Website: Click here

3. MLlib (Spark)

   MLlib is a machine learning library for Apache Spark. It can be used in Java, Python, R, and Scala. It aims at making practical machine learning scalable and easy.

   Features:

   1. It contains many common machine learning algorithms such as classification, regression, clustering, and collaborative filtering.
   2. It contains utilities such as feature transformation and ML pipeline construction.
   3. It includes tools such as model evaluation and hyperparameter tuning.
   4. It also includes utilities such as distributed linear algebra, statistics, data handling, etc.
   5. It has a vast user guide.

   It is used by Oracle.
   Official Website: Click here

Other libraries: Java-ML, JSAT

C++

Bjarne Stroustrup began to work on "C with Classes" which is the predecessor to C++ in 1979. "C with Classes" was renamed to "C++" in 1983. It is a general-purpose programming language. It has imperative, object-oriented, and generic programming features, and it also provides facilities for low-level memory manipulation.

1. mlpack

   mlpack is a machine learning library in C++ which emphasizes scalability, speed, and ease of use. Initially, it was produced by the FASTLab at Georgia Tech. mlpack was presented at the BigLearning workshop of NIPS 2011 and later published in the Journal of Machine Learning Research.

   Features:

   1. An important feature of mlpack is the scalability of the machine learning algorithms that it implements and it is achieved mostly by the use of C++.
   2. It allows kernel functions and arbitrary distance metrics for all its methods.
   3. It has high-quality documentation available.

   Official Website: Click here

2. Shark

   Shark is a C++ machine learning library written by Christian Igel, Verena Heidrich-Meisner, and Tobias Glasmachers. It serves as a powerful toolbox for research as well as real-world applications. It depends on Boost and CMake.

   Features:

   1. It is an open source library.
   2. It provides an accord between flexibility, ease of use, and computational efficiency.
   3. It provides tools for various machine learning techniques such as LDA, linear regression, PCA, clustering, neural networks, etc.

   Official Website: Click here

3. Shogun

   It is a machine learning toolbox developed in 1999 initiated by Soeren Sonnenburg and Gunnar Raetsch.

   Features:

   1. It can be used through a unified interface from multiple languages such as C++, Python, Octave, R, Java, Lua, C#, Ruby, etc.
   2. It enables an easy combination of multiple data representations, algorithm classes, and general purpose tools.
   3. It spans the whole space of machine learning methods including classical (such as regression, dimensionality reduction, clustering) as well as more advanced methods (such as metric, multi-task, structured output, and online learning).

   Official Website: Click here

Other libraries: Dlib-ml, MLC++

Julia

Julia is a high-performance dynamic programming language designed by Jeff Bezanson, Stefan Karpinski, Viral Shah, and Alan Edelman. It first appeared in 2012. The Julia developer community is contributing a number of external packages through Julia's built-in package manager at a rapid pace.

1. ScikitLearn.jl

   The scikit-learn Python library is a very popular library among machine learning researchers and data scientists. ScikitLearn.jl brings the capabilities of scikit-learn to Julia. The primary goal of it is to integrate Julia and Python-defined models together into the scikit-learn framework.

   Features:

   1. It offers around 150 Julia and Python models that can be accessed through a uniform interface.
   2. ScikitLearn.jl provides two types: Pipelines and Feature Unions for data preprocessing and transformation.
   3. It offers a possibility to combine features from DataFrames.
   4. It provides features to find the best set of hyperparameters.
   5. It has a fairly detailed manual and a number of examples.

   Official Website: Click here

2. MachineLearning.jl

   It is a library that aims to be a general-purpose machine learning library for Julia with a number of support tools and algorithms.

   Features:

   1. It includes functionality for splitting datasets into training dataset and test dataset and performing cross-validation.
   2. It also includes a lot of algorithms such as decision tree classifier, random forest classifier, basic neural network, etc.

   Official Website: Click here

3. MLBase.jl

   It is said to be "a swiss knife for machine learning". It is a Julia package which provides useful tools for machine learning applications.

   Features:

   1. It provides many functions for data preprocessing such as data repetition and label processing.
   2. It supports tools such as classification performance, hit rate, etc. for evaluating the performance of a machine learning algorithm.
   3. It implements a variety of cross validation schemes such as k-fold, leave-one-out cross validation, etc.
   4. It has good documentation, and there are a lot of code examples for its tools.

   Official Website: Click here

Scala

Scala is another general-purpose programming language. It was designed by Martin Odersky and first appeared on January 20, 2004. The word Scala is a portmanteau of scalable and language which signifies that it is designed to grow with the demands of its users. It runs on JVM, hence Java and Scala stacks can be mixed. Scala is used in data science.

Here's a list of a few libraries in Scala that can be used for machine learning.

1. ScalaNLP

   ScalaNLP is a suite of machine learning, numerical computing libraries, and natural language processing. It includes libraries like Breeze and Epic.

   • Breeze: It is a set of libraries for machine learning and numerical computing.
   • Epic: It is a natural language processing and prediction library written in Scala.

   Official Website: Click here

This is not an exhaustive list. There are various other languages such as SAS and MATLAB where one can perform machine learning.

Times are changing and have been for a while now. In the world of STEM, women are no longer considered a “bad fit,” which is easily proved by the amazing number of brilliant women in the field today. Women are just as interested in finding out how things work, extracting insight from data, problem-solving, and helping businesses make the right decisions. Staggering amounts of data and knowing that it will only grow has paved the way for boundless opportunities for jobs related to data science, across both genders.

Disappointing Statistics in Favor

Despite being named the sexiest job of the century, data science seems to have few takers among the women folk. Following are some interesting stats about women in the technology domain:

• The American Association of University Women found that the percentage of women in math and computational jobs fell from 35% in 1990 to 26% in 2013.
• BetterBuys’ collated report shows that women make up about 26% of data professionals, with 39% researcher roles, 28% in creative roles, 18% in business management roles, and 13% in developer roles.
• In 2014, women held only 13% of the Chief Information Officer and 25% of Chief Data Officer positions. What is worse is that research found “women were two times more likely than men to quit high-tech positions.”

So, what’s stopping more women from getting into data science and analytics?

We don’t want people talking about gender gap in the world of technology and analytics anymore. Seriously, there is no conspiracy to keep women out of this typically male-dominated sphere. We need diversity in the boardroom, like now.

Why are women not "gung-ho" about such an exciting field?

Women often face challenges in the form of stereotypes and condescension, especially in developing countries like India, when trying to prove their worth. Cultural perception affects their self-confidence and chances of growth. You find women struggling to find work-life balance. Battling these undercurrents and the lack of adequate support and encouragement at home and workplace are sources of stress many talented women are choosing to do without.

Not an ideal situation…

But take heart, aspiring women data scientists. This is what Evenbrite’s Senior Data Scientist, Vesela Gateva, says:

Once you have a very genuine curiosity in a quantitative field or anything science-related, let your curiosity be your main guidance. You shouldn’t think that you’re a woman. I never aspired to be a data scientist. It’s a very recent term. I just ended up being one. All I knew was that I wanted to apply my quantitative skills, solving interesting problems. Women in general tend to give themselves less credit than they deserve. What women should know is that once they have the curiosity, and the basic fundamentals of probability and statistics, computer science, and machine learning, they can figure out the rest on their own.

Gender shouldn’t limit accomplishments, and it certainly shouldn’t define a person’s identity.

14 Women who've hit the stereotype out of the park

What aspiring women data scientists need are to look to bright women who have defied odds to rise to leadership positions in the field of analytics. No point in whining about lack of female representation if you are going to contribute, is there?

Let's appreciate these women for their work and incessant dedication which has helped millions of people around the world to inspire, learn and rise in their respective careers.

Corinna Cortes, Google Research

She needs no introduction to people who in the world of Machine Learning. Corinna Cortes is the head of Google Research (NY), prior to which she was a distinguished researcher for a decade at AT&T Bell Labs. Her development of the algorithm, Support Vector Machines, fetched her the Paris Kanellakis Theory and Practice Award in 2008. She received her PhD in Computer Science in 1993 from the University of Rochester (NY) and has an MS in Physics from the University of Copenhagen. This amazing mother of two is a competitive runner as well. Read her latest tweets here.

Daphne Koller, Co-founder, Coursera

Israeli-American Daphne Koller is a leading expert in the field of machine learning, with special focus on probabilistic graphical models. She is the Chief Computing Officer at Calico Labs. Daphne is also the co-founder of the popular online education platform Coursera. She was a Stanford University professor of Computer Science for nearly two decades. Daphne Koller earned her PhD from Stanford, BS and MS from Hebrew University of Jerusalem, and has done her Post-doctoral research at UCLA. To view her many achievements, go here. When she’s not immersed in her work, you can find her spending time with her daughter or unwinding to music.

Adele Cutler, Random Forest Algorithm Co-Developer

Random Forests (a trademarked statistical classifier) co-developer Adele Cutler has a PhD from University of California, Berkeley, and a math degree from the University of Auckland. She’s been a statistics professor at Utah State University for almost three decades and continues her research in data mining and decision trees. She says, “As statisticians, what we’re really trying to do is think of better ways to get information out of data.” Adele Cutler has varied interests apart from math and stats, including spending time with her family in Taupo and Edinburgh, taking holidays, beading, and knitting. You can find more about her here.

Jenn Wortman Vaughan, Microsoft Research

Jennifer Vaughan is a Senior Researcher at NYC-based Microsoft Research. She is interested in learning models and algorithms related to data aggregation. She received her PhD in 2009 in Computer and Information Science from the University of Pennsylvania, a Masters from Stanford in Computer Science, and a Bachelors in Computer Science from Boston University. She previously worked as an Assistant Professor (CS) in UCLA and was a Harvard University Computing Innovation Fellow. She has a handful of prestigious awards to her name, including a National Science Foundation CAREER award and a Presidential Early Career Award for Scientists and Engineers. In 2006, Jenn co-founded the Annual Workshop for Women in Machine Learning. If you want to know about this rising star, go to here website.

Erin LeDell, Machine Learning Scientist, H2O.ai

California-based H2O.ai Machine Learning scientist, Erin LeDell has a doctorate in “Biostatistics and the Designated Emphasis in Computational Science and Engineering” from the University of California, Berkeley. She has a B.S. and M.A. in Mathematics. Her earlier work history includes working as the Principal Data Scientist at Wise.io and Marvin Mobile Security. Erin is also the founder of DataScientific, Inc. She has co-authored Subsemble: An Ensemble Method for Combining Subset-Specific Algorithm Fits. She is a co-founder of R-Ladies Global, an organization to encourage gender diversity in the R stats community. You can find Erin LeDell here.

Jennifer Bryan, Associate Professor Statistics, UBC

Jennifer Bryan is an Associate Professor, Statistics & Michael Smith Labs, at the University of British Columbia. She's a biostatistician specializing in genomics, and she enjoys statistical computing and data analysis. She has a BA in Economics from Yale and a doctoral degree from the University of California, Berkeley. She takes a popular introductory course in R. Look at her Twitter feed here.

Hilary Mason, Founder, Fast Forward Labs

In her own words, “I love data and cheeseburgers!” Based in New York, Hilary Mason is the founder of Fast Forward Labs, a machine intelligence research company, and the Data Scientist in Residence at Accel. Her magic doesn’t end there. She co-hosts DataGotham, is a member of NYCResistor, and co-founded of HackNY. Apart from being featured in top publications like the Scientific American, she has received the TechFellows Engineering Leadership award and was on the Forbes 40 under 40 Ones to Watch list. She has co-authored Data Driven: Creating a Data Culture. For inspiration, you should look at her LinkedIn profile.

Radhika Kulkarni, Vice President, Advanced Analytics R&D, SAS

Based in Durham, NC, Radhika Kulkarni is the Vice President, Advanced Analytics R&D, at SAS Institute Inc. She has a Masters in Mathematics from IIT-Delhi and a PhD in Operations Research from Cornell University. In her 30-year career with SAS, one of the foremost optimization software vendors, she has received many accolades—she is a SAS CEO Award of Excellence winner and chosen as one of the 100 Diverse Corporate Leaders in STEM by STEMconnector. She loves spending time with her three kids, and is very social. In her own words, “I'm well known to be the party animal.” Check out here tweets here.

Alice Zheng, Senior Manager, Amazon

Alice Zheng is a Senior Manager of Applied Science at Amazon. She heads the optimization team on Amazon's Ad Platform. She was a Microsoft researcher for six years before her stint as the Director of Data Science at Dato. Her focus is on building scalable models in Machine Learning. She has undergraduate degrees in Computer Science and Math and a doctoral degree in electrical engineering from the University of California, Berkeley. Alice Zheng has written two books in the field of data science. She says, “My research focuses on easing the dependence on expertise by making learning algorithms more automated, their outputs more interpretable, and the labeling tasks simpler.” Look at her LinkedIn profile to read more interesting things about her.

Charlotte Wickham, Assistant Professor Statistics, OSU

Charlotte Wickham works as an Assistant Professor of Statistics at the Oregon State University. An R specialist, she creates courseware for Data Camp. She has an Undergraduate degree in Statistics from the University of Auckland and a PhD in Statistics from the University of California, Berkeley. You can visit her website for more information.

Monica Rogati, Former Senior Data Scientist, LinkedIn

Former VP of Data at Jawbone and LinkedIn senior data scientist, Monica Rogati is now an independent data science advisor. Her description on Medium is quite apt: Turning data into products and stories. Based in Sunnyvale, California, she has a PhD in Computer Science from the Carnegie Mellon University and a B.S. in computer science from the University of New Mexico. Her expertise lies in applied machine learning, text mining, and recommender systems. From wearable computing to developing a system to match a job to a candidate, she is an ace at it all. Her LinkedIn profile is chock-full of achievements. You can also follow her at @mrogati.

Alice Daish, Data Scientist, British Museum

Alice Daish is a Data Scientist at the British Museum and a co-Founder of R-Ladies Global. She says, “I love data, R, science and innovation.” Her interests include data analysis, data visualization, predictive modelling, data communication, mentoring, and gender diversity in STEM.S he has a BSc. in Conservation Biology & Ecology from the University of Exeter and an MSc. in Quantitative Biology from Imperial College London. For a more detailed record of her projects and publications, go here. Follow Alice!

Amy O'Connor, Big Data Evangelist, Cloudera

Amy O'Connor is a Big Data evangelist at Cloudera. Prior to this, she was the Senior Director of the Big Data group at Nokia, and prior to that she was Senior Director of Strategy at Sun Microsystems. She describes herself as “a geek in high heels.” Amy O'Connor was on the Information Management’s “10 Big Data Experts to Know” in 2015. She has a BS in Electrical Engineering from the University of Connecticut and an MBA from Northeastern University. Follow her here.

Julia Evans, Machine Learning Engineer, Stripe

Montreal-based Julia Evans says “I love using serious systems in silly ways.” She has undergraduate and graduate degrees in Mathematics and Computer Science from McGill University. She works as a Machine Learning engineer at Stripe. She is passionate about programming and puts events together for women with similar interests. You can read for yourself here. Follow her interesting tweets here.

Women have great communication skills—a necessary skill when you need to tell decision makers what the results of the data analysis are. They are collaborative by nature—a key skill when people from different fields work together. They can think differently and tackle assumptions—vital skills when coupled with business acumen, stats, math, computer science, modeling, and analytical expertise. Admittedly men and women think differently. But that is what analysis is about, isn’t it? Different perspectives?

Like machine learning expert Claudia Perlich, Chief Scientist at Dstillery, said,

“Ultimately, data science is another technical field where women remain statistically a minority, but I do not believe that we need to force the issue or “fight” for a higher female quota. I want to come to work and do what I love and be recognized for what I bring to the table and not waste even one thought on the fact that I am female.”

So there really is no excuse for women to not enter this fascinating world of Data Science is there? Women just need to recognize that they have so much to bring to the table.