Tech Tutorials

All about Machine Learning

Vatsan: Yeah, so these are unprecedented times, unfortunately, and I think this might be the new normal, at least through the end of this year. We'll see. I would say for us, the biggest challenge has been maintaining a healthy balance between work and life. I guess because all of us are working from home these days, we could very easily get up from our bed, log onto a computer, spend hours together in front of a computer, working away, and neglect our health, our family, and so on. So it is important for us to maintain that separation that we used to have and we used to go to an office. The ability to leave work after you leave the office and focus on personal health and family when you're at home. So that's important. Um. I would also say that since we are all remote, it is important for us to over-communicate. You know, we have tons of channels, you know, whether it's Slack or Google Hangouts or WhatsApp or a variety of these tools. It is important that we over-communicate, especially in times such as this, where we are missing those in-person interactions. And often I think, um, you know, when you are at least in an office physically you have your hallway conversations, you have your bankers during lunch, or whether you're grabbing coffee with your team. All of those social events are missing and in the current scenario. So it is important for

Tech Tutorials

Introduction to Object Detection

Humans can easily detect and identify objects present in an image. The human visual system is fast and accurate and can perform complex tasks like identifying multiple objects and detect obstacles with little conscious thought. With the availability of large amounts of data, faster GPUs, and better algorithms, we can now easily train computers to detect and classify multiple objects within an image with high accuracy. In this blog, we will explore terms such as object detection, object localization, loss function for object detection and localization, and finally explore an object detection algorithm known as “You only look once” (YOLO).

Object Localization

An image classification or image recognition model simply detect the probability of an object in an image. In contrast to this, object localization refers to identifying the location of an object in the image. An object localization algorithm will output the coordinates of the location of an object with respect to the image. In computer vision, the most popular way to localize an object in an image is to represent its location with the help of bounding boxes. Fig. 1 shows an example of a bounding box.

bounding box, object detection, localization, self driving cars, computer vision, deep learning, classfication — Fig 1. Bounding box representation used for object localization

A bounding box can be initialized using the following parameters:

bx, by : coordinates of the center of the bounding box
bw : width of the bounding box w.r.t the image width
bh : height of the bounding box w.r.t the image height

Defining the target variable

The target variable for a multi-class image classification problem is defined as:

Loss Function

Since we have defined both the target variable and the loss function, we can now use neural networks to both classify and localize objects.

Machine learning challenge, ML challenge

Object Detection

An approach to building an object detection is to first build a classifier that can classify closely cropped images of an object. Fig 2. shows an example of such a model, where a model is trained on a dataset of closely cropped images of a car and the model predicts the probability of an image being a car.

Now, we can use this model to detect cars using a sliding window mechanism. In a sliding window mechanism, we use a sliding window (similar to the one used in convolutional networks) and crop a part of the image in each slide. The size of the crop is the same as the size of the sliding window. Each cropped image is then passed to a ConvNet model (similar to the one shown in Fig 2.), which in turn predicts the probability of the cropped image is a car.

After running the sliding window through the whole image, we resize the sliding window and run it again over the image again. We repeat this process multiple times. Since we crop through a number of images and pass it through the ConvNet, this approach is both computationally expensive and time-consuming, making the whole process really slow. Convolutional implementation of the sliding window helps resolve this problem.

Convolutional implementation of sliding windows

Before we discuss the implementation of the sliding window using convents, let’s analyze how we can convert the fully connected layers of the network into convolutional layers. Fig. 4 shows a simple convolutional network with two fully connected layers each of shape (400, ).

convolutional sliding window, sliding window, 1d convolution, yolo, object detection — Fig 4. Sliding windows mechanism

A fully connected layer can be converted to a convolutional layer with the help of a 1D convolutional layer. The width and height of this layer are equal to one and the number of filters are equal to the shape of the fully connected layer. An example of this is shown in Fig 5.

full connected layer to 1d convolution, 1 d convolution, full connected layers, dense layers — Fig 5. Converting a fully connected layer into a convolutional layer

We can apply this concept of conversion of a fully connected layer into a convolutional layer to the model by replacing the fully connected layer with a 1-D convolutional layer. The number of the filters of the 1D convolutional layer is equal to the shape of the fully connected layer. This representation is shown in Fig 6. Also, the output softmax layer is also a convolutional layer of shape (1, 1, 4), where 4 is the number of classes to predict.

full convolutional networks , converting dense layers to convolutional layers, computer vision, object detection, object localization — Fig 6. Convolutional representation of fully connected layers.

Now, let’s extend the above approach to implement a convolutional version of sliding window. First, let’s consider the ConvNet that we have trained to be in the following representation (no fully connected layers).

Let’s assume the size of the input image to be 16× 16× 3. If we’re to use a sliding window approach, then we would have passed this image to the above ConvNet four times, where each time the sliding window crops a part of the input image of size 14× 14× 3 and pass it through the ConvNet. But instead of this, we feed the full image (with shape 16× 16 × 3) directly into the trained ConvNet (see Fig. 7). This results in an output matrix of shape 2 × 2 × 4. Each cell in the output matrix represents the result of a possible crop and the classified value of the cropped image. For example, the left cell of the output (the green one) in Fig. 7 represents the result of the first sliding window. The other cells represent the results of the remaining sliding window operations.

Convolutional sliding window, fully convolutional network, sliding window, object detection, object localization, yolo, rcnn, computer vision, perception, self driving cars — Fig 7. Convolutional implementation of the sliding window

Note that the stride of the sliding window is decided by the number of filters used in the Max Pool layer. In the example above, the Max Pool layer has two filters, and as a result, the sliding window moves with a stride of two resulting in four possible outputs. The main advantage of using this technique is that the sliding window runs and computes all values simultaneously. Consequently, this technique is really fast. Although a weakness of this technique is that the position of the bounding boxes is not very accurate.

The YOLO (You Only Look Once) Algorithm

A better algorithm that tackles the issue of predicting accurate bounding boxes while using the convolutional sliding window technique is the YOLO algorithm. YOLO stands for you only look once and was developed in 2015 by Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. It’s popular because it achieves high accuracy while running in real time. This algorithm is called so because it requires only one forward propagation pass through the network to make the predictions.

The algorithm divides the image into grids and runs the image classification and localization algorithm (discussed under object localization) on each of the grid cells. For example, we have an input image of size 256 × 256. We place a 3× 3 grid on the image (see Fig. 8).

YOLO algorithm, you only look once, Joseph Redmon, Computer vision, pattern recognition, Real time object detection — Fig. 8 Grid (3 x 3) representation of the image

Next, we apply the image classification and localization algorithm on each grid cell. For each grid cell, the target variable is defined as

Do everything once with the convolution sliding window. Since the shape of the target variable for each grid cell is 1 × 9 and there are 9 (3 × 3) grid cells, the final output of the model will be:

The advantages of the YOLO algorithm is that it is very fast and predicts much more accurate bounding boxes. Also, in practice to get more accurate predictions, we use a much finer grid, say 19 × 19, in which case the target output is of the shape 19 × 19 × 9.

Conclusion

With this, we come to the end of the introduction to object detection. We now have a better understanding of how we can localize objects while classifying them in an image. We also learned to combine the concept of classification and localization with the convolutional implementation of the sliding window to build an object detection system. In the next blog, we will go deeper into the YOLO algorithm, loss function used, and implement some ideas that make the YOLO algorithm better. Also, we will learn to implement the YOLO algorithm in real time.

Have anything to say? Feel free to comment below for any questions, suggestions, and discussions related to this article. Till then, keep hacking with HackerEarth.

Tech Tutorials

Composing Jazz Music with Deep Learning

Deep Learning is on the rise, extending its application in every field, ranging from computer vision to natural language processing, healthcare, speech recognition, generating art, addition of sound to silent movies, machine translation, advertising, self-driving cars, etc. In this blog, we will extend the power of deep learning to the domain of music production. We will talk about how we can use deep learning to generate new musical beats.

The current technological advancements have transformed the way we produce music, listen, and work with music. With the advent of deep learning, it has now become possible to generate music without the need for working with instruments artists may not have had access to or the skills to use previously. This offers artists more creative freedom and ability to explore different domains of music.

Recurrent Neural Networks

Since music is a sequence of notes and chords, it doesn’t have a fixed dimensionality. Traditional deep neural network techniques cannot be applied to generate music as they assume the inputs and targets/outputs to have fixed dimensionality and outputs to be independent of each other. It is therefore clear that a domain-independent method that learns to map sequences to sequences would be useful.

Recurrent neural networks (RNNs) are a class of artificial neural networks that make use of sequential information present in the data.

recurrent neural network, deep learning, character based learning, — Fig. 1 A basic RNN unit.

A recurrent neural network has looped, or recurrent, connections which allow the network to hold information across inputs. These connections can be thought of as memory cells. In other words, RNNs can make use of information learned in the previous time step. As seen in Fig. 1, the output of the previous hidden/activation layer is fed into the next hidden layer. Such an architecture is efficient in learning sequence-based data.

In this blog, we will be using the Long Short-Term Memory (LSTM) architecture. LSTM is a type of recurrent neural network (proposed by Hochreiter and Schmidhuber, 1997) that can remember a piece of information and keep it saved for many timesteps.

Dataset

Our dataset includes piano tunes stored in the MIDI format. MIDI (Musical Instrument Digital Interface) is a protocol which allows electronic instruments and other digital musical tools to communicate with each other. Since a MIDI file only represents player information, i.e., a series of messages like ‘note on’, ‘note off, it is more compact, easy to modify, and can be adapted to any instrument.

Before we move forward, let us understand some music related terminologies:

Note: A note is either a single sound or its representation in notation. Each note consist of pitch, octave, and an offset.
Pitch: Pitch refers to the frequency of the sound.
Octave: An octave is the interval between one musical pitch and another with half or double its frequency.
Offset: Refers to the location of the note.
Chord: Playing multiple notes at the same time constitutes a chord.

Data Preprocessing

We will use the music21 toolkit (a toolkit for computer-aided musicology, MIT) to extract data from these MIDI files.

Notes Extraction

 def get_notes():  
     notes = []  
     for file in songs:  
       # converting .mid file to stream object  
       midi = converter.parse(file)  
       notes_to_parse = []  
       try:  
         # Given a single stream, partition into a part for each unique instrument  
         parts = instrument.partitionByInstrument(midi)  
       except:  
         pass  
       if parts: # if parts has instrument parts   
         notes_to_parse = parts.parts[0].recurse()  
       else:  
         notes_to_parse = midi.flat.notes  
       for element in notes_to_parse:   
         if isinstance(element, note.Note):  
           # if element is a note, extract pitch   
           notes.append(str(element.pitch))  
         elif(isinstance(element, chord.Chord)):  
           # if element is a chord, append the normal form of the   
           # chord (a list of integers) to the list of notes.   
           notes.append('.'.join(str(n) for n in element.normalOrder))  
     with open('data/notes', 'wb') as filepath:  
       pickle.dump(notes, filepath)  
     return notes

The function get_notes returns a list of notes and chords present in the .mid file. We use the converter.parse function to convert the midi file in a stream object, which in turn is used to extract notes and chords present in the file. The list returned by the function get_notes() looks as follows:

 Out:  
     ['F2', '4.5.7', '9.0', 'C3', '5.7.9', '7.0', 'E4', '4.5.8', '4.8', '4.8', '4', 'G#3',  
     'D4', 'G#3', 'C4', '4', 'B3', 'A2', 'E3', 'A3', '0.4', 'D4', '7.11', 'E3', '0.4.7', 'B4', 'C3', 'G3', 'C4', '4.7', '11.2', 'C3', 'C4', '11.2.4', 'G4', 'F2', 'C3', '0.5', '9.0', '4.7', 'F2', '4.5.7.9.0', '4.8', 'F4', '4', '4.8', '2.4', 'G#3',  
    '8.0', 'E2', 'E3', 'B3', 'A2', '4.9', '0.4', '7.11', 'A2', '9.0.4', ...........]

We can see that the list consists of pitches and chords (represented as a list of integers separated by a dot). We assume each new chord to be a new pitch on the list. As letters are used to generate words in a sentence, similarly the music vocabulary used to generate music is defined by the unique pitches in the notes list.

Generating Input and Output Sequences

A neural network accepts only real values as input and since the pitches in the notes list are in string format, we need to map each pitch in the notes list to an integer. We can do so as follows:

 # Extract the unique pitches in the list of notes.   
   pitchnames = sorted(set(item for item in notes))  
   # create a dictionary to map pitches to integers  
   note_to_int = dict((note, number) for number, note in enumerate(pitchnames))

Next, we will create an array of input and output sequences to train our model. Each input sequence will consist of 100 notes, while the output array stores the 101st note for the corresponding input sequence. So, the objective of the model will be to predict the 101st note of the input sequence of notes.

 # create input sequences and the corresponding outputs  
   for i in range(0, len(notes) - sequence_length, 1):  
     sequence_in = notes[i: i + sequence_length]  
     sequence_out = notes[i + sequence_length]  
     network_input.append([note_to_int[char] for char in sequence_in])  
     network_output.append(note_to_int[sequence_out])

Next, we reshape and normalize the input vector sequence before feeding it to the model. Finally, we one-hot encode our output vector.

 n_patterns = len(network_input)  
   # reshape the input into a format compatible with LSTM layers   
   network_input = np.reshape(network_input, (n_patterns, sequence_length, 1))  
   # normalize input  
   network_input = network_input / float(n_vocab)  
   # One hot encode the output vector  
   network_output = np_utils.to_categorical(network_output)

Model Architecture

We will use keras to build our model architecture. We use a character level-based architecture to train the model. So each input note in the music file is used to predict the next note in the file, i.e., each LSTM cell takes the previous layer activation (a^⟨t−1⟩) and the previous layers actual output (y^⟨t−1⟩) as input at the current time step tt. This is depicted in the following figure (Fig 2.).

LSTM, Long term short architecture, Recurrent neural network, music generation, neural network, — Fig 2. One to Many LSTM architecture

Our model architecture is defined as:

 model = Sequential()  
   model.add(LSTM(128, input_shape=network_in.shape[1:], return_sequences=True))  
   model.add(Dropout(0.2))  
   model.add(LSTM(128, return_sequences=True))  
   model.add(Flatten())  
   model.add(Dense(256))  
   model.add(Dropout(0.3))  
   model.add(Dense(n_vocab))  
   model.add(Activation('softmax'))  
   model.compile(loss='categorical_crossentropy', optimizer='adam')

Our music model consists of two LSTM layers with each layer consisting of 128 hidden layers. We use ‘categorical cross entropy‘ as the loss function and ‘adam‘ as the optimizer. Fig. 3 shows the model summary.

LSTM, Long short term memory, model architecture, music generation, rnn, recurrent neural netowrk — Fig 3. Model summary

Model Training

To train the model, we call the model.fit function with the input and output sequences as the input to the function. We also create a model checkpoint which saves the best model weights.

 from keras.callbacks import ModelCheckpoint  
   def train(model, network_input, network_output, epochs):   
     """  
     Train the neural network  
     """  
     filepath = 'weights.best.music3.hdf5'  
     checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=0, save_best_only=True)  
     model.fit(network_input, network_output, epochs=epochs, batch_size=32, callbacks=[checkpoint])  
   def train_network():  
     epochs = 200  
     notes = get_notes()  
     print('Notes processed')  
     n_vocab = len(set(notes))  
     print('Vocab generated')  
     network_in, network_out = prepare_sequences(notes, n_vocab)  
     print('Input and Output processed')  
     model = create_network(network_in, n_vocab)  
     print('Model created')  
     return model  
     print('Training in progress')  
     train(model, network_in, network_out, epochs)  
     print('Training completed')

The train_network method gets the notes, creates the input and output sequences, creates a model, and trains the model for 200 epochs.

Music Sample Generation

Now that we have trained our model, we can use it to generate some new notes. To generate new notes, we need a starting note. So, we randomly pick an integer and pick a random sequence from the input sequence as a starting point.

 def generate_notes(model, network_input, pitchnames, n_vocab):  
     """ Generate notes from the neural network based on a sequence of notes """  
     # Pick a random integer  
     start = np.random.randint(0, len(network_input)-1)  
     int_to_note = dict((number, note) for number, note in enumerate(pitchnames))  
     # pick a random sequence from the input as a starting point for the prediction  
     pattern = network_input[start]  
     prediction_output = []  
     print('Generating notes........')  
     # generate 500 notes  
     for note_index in range(500):  
       prediction_input = np.reshape(pattern, (1, len(pattern), 1))  
       prediction_input = prediction_input / float(n_vocab)  
       prediction = model.predict(prediction_input, verbose=0)  
       # Predicted output is the argmax(P(h|D))  
       index = np.argmax(prediction)  
       # Mapping the predicted interger back to the corresponding note  
       result = int_to_note[index]  
       # Storing the predicted output  
       prediction_output.append(result)  
       pattern.append(index)  
       # Next input to the model  
       pattern = pattern[1:len(pattern)]  
     print('Notes Generated...')  
     return prediction_output

Next, we use the trained model to predict the next 500 notes. At each time step, the output of the previous layer (ŷ^⟨t−1⟩) is provided as input (x^⟨t⟩) to the LSTM layer at the current time step t. This is depicted in the following figure (see Fig. 4).

sampling, sampling from rnn, LSTM, architecture, music sampling, music generation — Fig 4. Sampling from a trained network.

Since the predicted output is an array of probabilities, we choose the output at the index with the maximum probability. Finally, we map this index to the actual note and add this to the list of predicted output. Since the predicted output is a list of strings of notes and chords, we cannot play it. Hence, we encode the predicted output into the MIDI format using the create_midi method.

 ### Converts the predicted output to midi format  
   create_midi(prediction_output)

To create some new jazz music, you can simply call the generate() method, which calls all the related methods and saves the predicted output as a MIDI file.

 #### Generate a new jazz music   
   generate()  
   Out:   
     Initiating music generation process.......  
     Loading Model weights.....  
     Model Loaded  
     Generating notes........  
     Notes Generated...  
     Saving Output file as midi....

To play the generated MIDI in the Jupyter Notebook you can import the play_midi method from the play.py file or use an external MIDI player or convert the MIDI file to the mp3. Let’s listen to our generated jazz piano music.

 ### Play the Jazz music  
   play.play_midi('test_output3.mid')

“Generated Track 1” Deep Learning Recurrent Neural Network

Audio Player

00:00

Use Up/Down Arrow keys to increase or decrease volume.

1. Generated Sample Music

2:07

Conclusion

Congratulations! You can now generate your own jazz music. You can find the full code in this Github repository. I encourage you to play with the parameters of the model and train the model with input sequences of different sequence lengths. Try to implement the code for some other instrument (such as guitar). Furthermore, such a character-based model can also be applied to a text corpus to generate sample texts, such as a poem.

Also, you can showcase your own personal composer and any similar idea in the World Music Hackathonby HackerEarth.

Have anything to say? Feel free to comment below for any questions, suggestions, and discussions related to this article. Till then, happy coding.

Tech Tutorials

A beginner's tutorial on the apriori algorithm in data mining with R implementation

Now that we have looked at an example of the functionality of Apriori Algorithm, let us formulate the general process.

General Process of the Apriori algorithm

The entire algorithm can be divided into two steps:

Step 1: Apply minimum support to find all the frequent sets with k items in a database.

Step 2: Use the self-join rule to find the frequent sets with k+1 items with the help of frequent k-itemsets. Repeat this process from k=1 to the point when we are unable to apply the self-join rule.

This approach of extending a frequent itemset one at a time is called the “bottom up” approach.

Mining Association Rules

Till now, we have looked at the Apriori algorithm with respect to frequent itemset generation. There is another task for which we can use this algorithm, i.e., finding association rules efficiently.

For finding association rules, we need to find all rules having support greater than the threshold support and confidence greater than the threshold confidence.

But, how do we find these? One possible way is brute force, i.e., to list all the possible association rules and calculate the support and confidence for each rule. Then eliminate the rules that fail the threshold support and confidence. But it is computationally very heavy and prohibitive as the number of all the possible association rules increase exponentially with the number of items.

Given there are n items in the set [latex]I[/latex], the total number of possible association rules is [latex]3^n - 2^{n+1} + 1[/latex].

We can also use another way, which is called the two-step approach, to find the efficient association rules.

The two-step approach is:

Step 1: Frequent itemset generation: Find all itemsets for which the support is greater than the threshold support following the process we have already seen earlier in this article.

Step 2: Rule generation: Create rules from each frequent itemset using the binary partition of frequent itemsets and look for the ones with high confidence. These rules are called candidate rules.

Let us look at our previous example to get an efficient association rule. We found that OPB was the frequent itemset. So for this problem, step 1 is already done. So, let’ see step 2. All the possible rules using OPB are:

OP[latex]\longrightarrow[/latex]B, OB[latex]\longrightarrow[/latex]P, PB[latex]\longrightarrow[/latex]O, B[latex]\longrightarrow[/latex] OP, P[latex]\longrightarrow[/latex]OB, O[latex]\longrightarrow[/latex]PBIf [latex]X[/latex] is a frequent itemset with k elements, then there are [latex]2^k-2[/latex] candidate association rules.We will not go deeper into the theory of the Apriori algorithm for rule generation.

Pros of the Apriori algorithm

It is an easy-to-implement and easy-to-understand algorithm.
It can be used on large itemsets.

Cons of the Apriori Algorithm

Sometimes, it may need to find a large number of candidate rules which can be computationally expensive.
Calculating support is also expensive because it has to go through the entire database.

R implementation

The package which is used to implement the Apriori algorithm in R is called arules. The function that we will demonstrate here which can be used for mining association rules is

apriori(data, parameter = NULL)

The arguments of the function apriori are

data: The data structure which can be coerced into transactions (e.g., a binary matrix or data.frame).

parameter: It is a named list containing the threshold values for support and confidence. The default value of this argument is a list of minimum support of 0.1, minimum confidence of 0.8, maximum of 10 items (maxlen), and a maximal time for subset checking of 5 seconds (maxtime).

https://gist.github.com/HackerEarthBlog/98dca27a7e48694506db6ae413d7570e

Summary

Through this article, we have seen how data mining is helping us make decisions that are advantageous for both customers and industries. We have also seen a simple explanation of the Apriori algorithm, along with its implementation in R. It is not only used by the retail industry to provide us the discount on some bundles of products. The use cases of the Apriori algorithm stretch to Google's auto-completion features and Amazon's recommendation systems.This tutorial aims to make the reader familiar with the fundamentals of the Apriori algorithm and a general process followed to mine frequent itemsets. Hope you are familiar now!

Popular posts like this:

Tech Tutorials

Building your own Lisp Parser Part I

While writing a full-blown compiler for a programming language is a difficult and frustrating task, writing a smaller and more specific parser can be surprisingly easy if you know a small trick.

On the other hand, parsing problems pops up at several places in modern-day programming. So, learning this useful trick can be rewarding.

Source: http://mox.ingenierotraductor.com/2015/12/translation-is-like.html

Prerequisites

You need to know the basics of Python, more specifically, you should know the concepts of recursion and flow of control.

Objectives

After reading and understanding this post, you will be able to create simple calculators, interactive interpreters, parsers, very limited and small programming languages, etc. In general, you should be able to take input, tokenize it, perform whatever actions you want to on the tokens, and output the result of the process.At the end of this post, you will have created a simple, Lisp-like prefix calculator. Following is a demonstration of how it's going to look:> ( + 3 2 )= 5> ( / 2 0 )DivisionByZero> ( - -3 2 )= -5> -2= -2> ( + ( * 3 2 ) 5 )= 11

Step 1: Writing the Grammar

The first step to writing a parser is to write a clear grammar for its syntax. The grammar determines what is and what is not right. Once you have written the grammar, translating it to Python code is a trivial chore. The grammar will serve as our first pseudocode.For our tiny calculator, we know that the input can come in two forms: a Number (-2, .5, +8, 8.5, 9.) or a more complicated Expression begins with a (, followed by an operator, etc.).For writing a grammar, we need to identify different elements of the syntax. So far, we have Expression, Number, and Operator. The next important thing to do is to structure the elements (known as terms) into a hierarchical form. This is shown below:Expression:Number( Operator Expression Expression )Number:a floating-point number ([-+][0-9][*\.][0-9]*)Operators:

You will notice that Operator and Expression have no parent; they are independent terms.A grammar is read from the bottom up and different choices appear on distinct lines. Our grammar says that:

an Operator is one of +, -, *, /.
a Number is a floating-point number which matches the RegEx [-+][0-9]*[\.][0-9]*
an Expression is either a Number or a ( followed by an Operator, followed by two other Expressions, and finally ends in a ). Note that the definition of an Expression is recursive.

Step 2: Translating the Grammar into Pseudocode

Pseudocode is fake code resembling English which is supposed to be an intermediate code that can easily be converted into real code. Although writing pseudocode is optional, it is really helpful.The trick here is to put each term from our grammar into a separate function. Whenever we need to apply the grammar of a certain term, we only have to call the function. Following is the pseudocode implementing the grammar above:https://gist.github.com/HackerEarthBlog/f0a5a4304326936142da39b0d853f944This is our rough pseudo-code that should be good enough for our purpose. In the next step, we will write the real code.

3. Writing the Code

It is said very profoundly about Python that reading and writing Python feels like doing pseudocode. The same applies here, but there is one small caveat— Python doesn't provide any function for “unreading” or putting a character back in the input buffer.For this, I have created a small class which extends the file object to include this feature. To keep things simple, I have avoided inheritance and my class is not compatible with the file object provided by Python. Treat it like a black-box if you don't want to understand it.https://gist.github.com/HackerEarthBlog/6465f93e1ca155ded5e8b0c8294f16baHere is the buffer.py file which handles buffered input:https://gist.github.com/HackerEarthBlog/5330e5f11f96a22608b45affa61fa858

Explanation

expression():

expression() is our top-level function and maps the Expression grammar term. We first ignore all the whitespace. After that, it takes a single non-whitespace character as input and checks it against several possibilities.If the input string starts with +, -, ., or a digit, it is a number. We put the character back and input the entire number.If the input string starts with (, a complete expression is to follow. We input the operator, two more expressions which will serve as the operands, and finally the closing parenthesis. We then calculate the result and return it.

number():

The number function maps the Number grammar term and is very simple—just a wrapper around getword. We input a whole word and if it converts to a float, we return it, otherwise the function returns an error message.

operator():

The operator function inputs a single character and tests it for equality against several known operators. Like the above two functions, it also maps a grammar term, i.e., Operator. In case the given operator is not valid, an error message is returned.

calc():

The calc function is actually not necessary but makes the code substantially better. In an ideal program, each function should do only one logical task. calc removes some burden from expression.

UngetableInput

Although Python 3 supports buffered input through stdin.buffer, Python 2 has no such facility. Plus, Python 3's stdin.buffer would still require us to create some wrapper of our own.The UngetableInput class wraps Python's basic input to go through a buffer. We take input into the buffer and put a character back into the buffer when ungetc is called. Unless the buffer is empty, all input comes from the buffer.

Homework

This code works and leaves a lot of cleaning as homework for the reader. :) Following is a list of things you can do to improve and extend the rudimentary calculator:Improve buffer.py to handle input whitespace more accurately. Hint: You might want to use a string as the buffer.Implement a function to get a single character while skipping all whitespace and replace the whitespace skipping loop with it.Add the ability to create variables. Following the Lisp syntax, it should look something like the following:( define var_name 839457.892 )

What's Coming Next?

One of the most important parts of our program is the input buffer we created. Unfortunately, it's not general purpose and can break when used in something more complicated than our tiny calculator program. In the next article, we will examine a bigger module which does this chore better.

Tech Tutorials

4 performance optimization tips for faster Python code

1. Get the whole setup ready before-hand

This is common sense. Get the whole setup ready. Fire-up your python editor before-hand. If you are writing your files in your local, create a virtual environment and activate it. Along with this, I would advise one other thing which might seem a bit controversial and counter-intuitive, and that is to use TDD. Use your favourite testing tool. I generally use pytest and have that “pip”-d in my virtual environment and start writing small test scripts. I have found that testing helps in clarity of thought, which helps in writing faster programs. Also, this helps in refactoring the code to make it faster. We will get to it later.

2. Get the code working first

People have their own coding styles. Use the coding style that you are most comfortable with. For the first iteration, make the code work, at least and make the submission. See if it passes for all the test cases. If it’s passing then, c'est fait. It's done. And you can move on to the next question.

In case its passing for some of the test cases, while failing for others, citing memory issues, then you know that there is still some work left.

3. Python Coding Tips

Strings:

Do not use the below construct.

s = ""
for x in somelist:
    s += some_function(x)

Instead use this

slist = [some_function(el) for el in somelist]
s = "".join(slist)

This is because in Python, str is immutable, so the left and right strings have to be copied into the new string for every pair of concatenation.

Language Constructs:

Functions: Make functions of your code and although procedural code is supported in Python, it's better to write them in functions.

def main():
    for i in xrange(10**8):
        pass

main()

is better than

for i in xrange(10**8):
    pass

This is because of the underlying CPython implementation. In short, it is faster to store local variables than globals. Read more in this SO post.

I would recommend you keep your procedural code as little as possible. You can use the following standard template.

def solution(args):
    # write the code
    pass

def main():
    # write the input logic to take the input from STDIN
    input_args = ""
    solution(input_args)

if __name__ == "__main__":
    main()

Use the standard library:

Use built-in functions and the standard library as much as possible. Therefore, instead of this:

newlist = []
for item in oldlist:
    newlist.append(myfunc(item))

Use this:

newlist = map(myfunc, oldlist)

There is also the list expressions or the generator expressions.

newlist = [myfunc(item) for item in oldlist]  # list expression
newlist = (myfunc(item) for item in oldlist)  # generator expression

Similarly, use the standard library, like itertools, as they are generally faster and optimized for common operations. So you can have something like permutation for a loop in just three lines of code.

>> import itertools
>>> iter = itertools.permutations([1,2,3])
>>> list(iter)
[(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)]

But if the containers are tiny, then any difference between code using libraries is likely to be minimal, and the cost of creating the containers will outweigh the gains they give.

Generators:

A Python generator is a function which returns a generator iterator (just an object we can iterate over) by calling yield. When a generator function calls yield, the "state" of the generator function is frozen; the values of all variables are saved and the next line of code to be executed is recorded until next() is called again. Generators are excellent constructs to reduce both the average time complexity as well as the memory footprint of the code that you have written. Just look at the following code for prime numbers.

def fib():
    a, b = 0, 1
    while 1:
        yield a
        a, b = b, a + b

So, this will keep on generating Fibonacci numbers infinitely without the need to keep all the numbers in a list or any other construct. Please keep in mind that you should use this construct only when you don't have any absolute need to keep all the generated values because then you will lose the advantage of having a generator construct.

4. Algorithms and Data structures

To make your code run faster, the most important thing that you can do is to take two minutes before writing any code and think about the data-structure that you are going to use. Look at the time complexity for the basic python data-structures and use them based on the operation that is most used in your code. The time complexity of the list taken from python wiki is shown below.

Similarly, keep on reading from all sources about the most efficient data structures and algorithms that you can use. Keep an inventory of the common data structures such as nodes and graphs and remember or keep a handy journal on the situations where they are most appropriate.

Writing fast code is a habit and a skill, which needs to be honed over the years. There are no shortcuts. So do your best and best of luck.

Reference:

stackoverflow.com: optimizing python code

dzone.com: 6 python performance tips

python wiki: Performance Tips

softwareengineering-stackexchange: lkndasldfn

quora: How do I speed up my Python code

python: list to string

monitis: python performance tips part 1

Authors

Meet our Authors

Get to know the experts behind our content. From industry leaders to tech enthusiasts, our authors share valuable insights, trends, and expertise to keep you informed and inspired.

Ruehie Jaiya Karri

Kumari Trishya

Hiring Strategies

7 Tech Recruiting Trends To Watch Out For In 2024

The last couple of years transformed how the world works and the tech industry is no exception. Remote work, a candidate-driven market, and automation are some of the tech recruiting trends born out of the pandemic.

While accepting the new reality and adapting to it is the first step, keeping up with continuously changing hiring trends in technology is the bigger challenge right now.

What does 2024 hold for recruiters across the globe? What hiring practices would work best in this post-pandemic world? How do you stay on top of the changes in this industry?

The answers to these questions will paint a clearer picture of how to set up for success while recruiting tech talent this year.

7 tech recruiting trends for 2024

6 Tech Recruiting Trends To Watch Out For In 2022

Recruiters, we’ve got you covered. Here are the tech recruiting trends that will change the way you build tech teams in 2024.

Trend #1—Leverage data-driven recruiting

Data-driven recruiting strategies are the answer to effective talent sourcing and a streamlined hiring process.

Talent acquisition leaders need to use real-time analytics like pipeline growth metrics, offer acceptance rates, quality and cost of new hires, and candidate feedback scores to reduce manual work, improve processes, and hire the best talent.

The key to capitalizing on talent market trends in 2024 is data. It enables you to analyze what’s working and what needs refinement, leaving room for experimentation.

Trend #2—Have impactful employer branding

98% of recruiters believe promoting company culture helps sourcing efforts as seen in our 2021 State Of Developer Recruitment report.

Having a strong employer brand that supports a clear Employer Value Proposition (EVP) is crucial to influencing a candidate’s decision to work with your company. Perks like upskilling opportunities, remote work, and flexible hours are top EVPs that attract qualified candidates.

A clear EVP builds a culture of balance, mental health awareness, and flexibility—strengthening your employer brand with candidate-first policies.

Trend #3—Focus on candidate-driven market

The pandemic drastically increased the skills gap, making tech recruitment more challenging. With the severe shortage of tech talent, candidates now hold more power and can afford to be selective.

Competitive pay is no longer enough. Use data to understand what candidates want—work-life balance, remote options, learning opportunities—and adapt accordingly.

Recruiters need to think creatively to attract and retain top talent.

Recommended read: What NOT To Do When Recruiting Fresh Talent

Trend #4—Have a diversity and inclusion oriented company culture

Diversity and inclusion have become central to modern recruitment. While urgent hiring can delay D&I efforts, long-term success depends on inclusive teams. Our survey shows that 25.6% of HR professionals believe a diverse leadership team helps build stronger pipelines and reduces bias.

McKinsey’s Diversity Wins report confirms this: top-quartile gender-diverse companies see 25% higher profitability, and ethnically diverse teams show 36% higher returns.

It's refreshing to see the importance of an inclusive culture increasing across all job-seeking communities, especially in tech. This reiterates that D&I is a must-have, not just a good-to-have.

—Swetha Harikrishnan, Sr. HR Director, HackerEarth

Trend #5—Embed automation and AI into your recruitment systems

With the rise of AI tools like ChatGPT, automation is being adopted across every business function—including recruiting.

Manual communication with large candidate pools is inefficient. In 2024, recruitment automation and AI-powered platforms will automate candidate nurturing and communication, providing a more personalized experience while saving time.

Trend #6—Conduct remote interviews

With 32.5% of companies planning to stay remote, remote interviewing is here to stay.

Remote interviews expand access to global talent, reduce overhead costs, and increase flexibility—making the hiring process more efficient for both recruiters and candidates.

Trend #7—Be proactive in candidate engagement

Delayed responses or lack of updates can frustrate candidates and impact your brand. Proactive communication and engagement with both active and passive candidates are key to successful recruiting.

As recruitment evolves, proactive candidate engagement will become central to attracting and retaining talent. In 2023 and beyond, companies must engage both active and passive candidates through innovative strategies and technologies like chatbots and AI-powered systems. Building pipelines and nurturing relationships will enhance employer branding and ensure long-term hiring success.

—Narayani Gurunathan, CEO, PlaceNet Consultants

Recruiting Tech Talent Just Got Easier With HackerEarth

Recruiting qualified tech talent is tough—but we’re here to help. HackerEarth for Enterprises offers an all-in-one suite that simplifies sourcing, assessing, and interviewing developers.

Our tech recruiting platform enables you to:

Tap into a 6 million-strong developer community
Host custom hackathons to engage talent and boost your employer brand
Create online assessments to evaluate 80+ tech skills
Use dev-friendly IDEs and proctoring for reliable evaluations
Benchmark candidates against a global community
Conduct live coding interviews with FaceCode, our collaborative coding interview tool
Guide upskilling journeys via our Learning and Development platform
Integrate seamlessly with all leading ATS systems
Access 24/7 support with a 95% satisfaction score

Recommended read: The A-Zs Of Tech Recruiting - A Guide

Staying ahead of tech recruiting trends, improving hiring processes, and adapting to change is the way forward in 2024. Take note of the tips in this article and use them to build a future-ready hiring strategy.

Ready to streamline your tech recruiting? Try HackerEarth for Enterprises today.

Hiring Strategies

(Part 2) Essential Questions To Ask When Interviewing Developers In 2021

The first part of this blog stresses the importance of asking the right technical interview questions to assess a candidate’s coding skills. But that alone is not enough. If you want to hire the crème de la crème of the developer talent out there, you have to look for a well-rounded candidate.

Honest communication, empathy, and passion for their work are equally important as a candidate’s technical knowledge. Soft skills are like the cherry on top. They set the best of the candidates apart from the rest.

Re-examine how you are vetting your candidates. Identify the gaps in your interviews. Once you start addressing these gaps, you find developers who have the potential to be great. And those are exactly the kind of people that you want to work with!

Let’s get to it, shall we?

What constitutes a good interview question?

An ideal interview should reveal a candidate’s personality along with their technical knowledge. To formulate a comprehensive list of questions, keep in mind three important characteristics.

Questions are open-ended – questions like, “What are some of the programming languages you’re comfortable with,” instead of “Do you know this particular programming language” makes the candidate feel like they’re in control. It is also a chance to let them reply to your question in their own words.
They address the behavioral aspects of a candidate – ensure you have a few questions on your list that allow a candidate to describe a situation. A situation where a client was unhappy or a time when the developer learned a new technology. Such questions help you assess if the candidate is a good fit for the team.
There is no right or wrong answer – it is important to have a structured interview process in place. But this does not mean you have a list of standard answers in mind that you’re looking for. How candidates approach your questions shows you whether they have the makings of a successful candidate. Focus on that rather than on the actual answer itself.

Designing a conversation around these buckets of interview questions brings you to my next question, “What should you look for in each candidate to spot the best ones?”

Hire GREAT developers by asking the right questions

Before we dive deep into the interview questions, we have to think about a few things that have changed. COVID-19 has rendered working from home the new normal for the foreseeable future. As a recruiter, the onus falls upon you to understand whether the developer is comfortable working remotely and has the relevant resources to achieve maximum productivity.

#1 How do you plan your day?

Remote work gives employees the option to be flexible. You don’t have to clock in 9 hours a day as long as you get everything done on time. A developer who hasn’t always been working remotely, but has a routine in place, understands the pitfalls of working from home. It is easy to get distracted and having a schedule to fall back on ensures good productivity.

#2 Do you have experience using tools for collaboration and remote work?

Working from home reduces human interaction heavily. There is no way to just go up to your teammate’s desk and clarify issues. Virtual communication is key to getting work done. Look for what kind of remote working tools your candidate is familiar with and if they know what collaborative tools to use for different tasks.

Value-based interview questions to ask

We went around and spoke to our engineering team, and the recruiting team to see what questions they abide by; what they think makes any candidate tick.

The result? – a motley group of questions that aim to reveal the candidate’s soft skills, in addition to typical technical interview questions and test tasks.

#3 Please describe three recent projects that you worked on. What were the most interesting and challenging parts?

This is an all-encompassing question in that it lets the candidate explain at length about their work ethic—thought process, handling QA, working with a team, and managing user feedback. This also lets you dig enough to assess whether the candidate is taking credit for someone else's work or not.

#4 You’ve worked long and hard to deliver a complex feature for a client and they say it’s not what they asked for. How would you take it?

A good developer will take it in their stride, work closely with the client to find the point of disconnect, and sort out the issue. There are so many things that could go wrong or not be to the client’s liking, and it falls on the developer to remain calm and create solutions.

#5 What new programming languages or technologies have you learned recently?

While being certified in many programming languages doesn't guarantee a great developer, it still is an important technical interview question to ask. It helps highlight a thirst for knowledge and shows that the developer is eager to learn new things.

#6 What does the perfect release look like? Who is involved and what is your role?

Have the developer take you through each phase of a recent software development lifecycle. Ask them to explain their specific role in each phase in this release. This will give you an excellent perspective into a developer’s mind. Do they talk about the before and after of the release? A skilled developer would. The chances of something going wrong in a release are very high. How would the developer react? Will they be able to handle the pressure?

SUBSCRIBE to the HackerEarth blog and enrich your monthly reading with our free e-newsletter – Fresh, insightful and awesome articles straight into your inbox from around the tech recruiting world!

#7 Tell me about a time when you had to convince your lead to try a different approach?

As an example of a behavioral interview question, this is a good one. The way a developer approaches this question speaks volumes about how confident they are expressing their views, and how succinct they are in articulating those views.

#8 What have you done with all the extra hours during the pandemic?

Did you binge-watch your way through the pandemic? I’m sure every one of us has done this. Indulge in a lighthearted conversation with your candidate. This lets them talk about something they are comfortable with. Maybe they learned a new skill or took up a hobby. Get to know a candidate’s interests and little pleasures for a more rounded evaluation.

Over to you! Now that you know what aspects of a candidate to focus on, you are well-equipped to bring out the best in each candidate in their interviews. A mix of strong technical skills and interpersonal qualities is how you spot good developers for your team.

If you have more pressing interview questions to add to this list of ours, please write to us at contact@hackerearth.com.

(Part 1) Essential Questions To Ask When Recruiting Developers In 2021

The minute a developer position opens up, recruiters feel a familiar twinge of fear run down their spines. They recall their previous interview experiences, and how there seems to be a blog post a month that goes viral about bad developer interviews.

While hiring managers, especially the picky ones, would attribute this to a shortage of talented developers, what if the time has come to rethink your interview process? What if recruiters and hiring managers put too much stock into bringing out the technical aspects of each candidate and don’t put enough emphasis on their soft skills?

A report by Robert Half shows that 86% of technology leaders say it’s challenging to find IT talent. Interviewing developers should be a rewarding experience, not a challenging one. If you don’t get caught up in asking specific questions and instead design a simple conversation to gauge a candidate’s way of thinking, it throws up a lot of good insight and makes it fun too.

Developer Hiring Statistics

Asking the right technical interview questions when recruiting developers is important but so is clear communication, good work ethic, and alignment with your organization’s goals.

Let us first see what kind of technical interview questions are well-suited to revealing the coding skills and knowledge of any developer, and then tackle the behavioral aspects of the candidate that sets them apart from the rest.

Recruit GREAT developers by asking the right questions

Here are some technical interview questions that you should ask potential software engineers when interviewing.

#1 Write an algorithm for the following

Minimum Stack - Design a stack that provides 4 functions - push(item), pop, peek, and minimum, all in constant order time complexity. Then move on to coding the actual solution.
Kth Largest Element in an array - This is a standard problem with multiple solutions of best time complexity orders where N log(K) is a common one and O(N) + K log(N) is a lesser-known order. Both solutions are acceptable, not directly comparable to each other, and better than N log(N), which is sorting an array and fetching the Kth element.
Top View of a Binary Tree - Given a root node of the binary tree, return the set of all elements that will get wet if it rains on the tree. Nodes having any nodes directly above them will not get wet.
Internal implementation of a hashtable like a map/dictionary - A candidate needs to specify how key-value pairs are stored, hashing is used and collisions are handled. A good developer not only knows how to use this concept but also how it works. If the developer also knows how the data structure scales when the number of records increases in the hashtable, that is a bonus.

Algorithms demonstrate a candidate’s ability to break down a complex problem into steps. Reasoning and pattern recognition capabilities are some more factors to look for when assessing a candidate. A good candidate can code his thought process of the algorithm finalized during the discussion.

Looking for a great place to hire developers in the US? Try Jooble!

#2 Formulate solutions for the below low-level design (LLD) questions

What is LLD? In your own words, specify the different aspects covered in LLD.
Design a movie ticket booking application like BookMyShow. Ensure that your database schema is tailored for a theatre with multiple screens and takes care of booking, seat availability, seat arrangement, and seat locking. Your solution does not have to extend to the payment option.
Design a basic social media application. Design database schema and APIs for a platform like Twitter with features for following a user, tweeting a post, seeing your tweet, and seeing a user's tweet.

Such questions do not have a right or wrong answer. They primarily serve to reveal a developer’s thought process and the way they approach a problem.

Recommended read: Hardest Tech Roles to Fill (+ solutions!)

#3 Some high-level design (HLD) questions

What do you understand by HLD? Can you specify the difference between LLD and HLD?
Design a social media application. In addition to designing a platform like Twitter with features for following a user, tweeting a post, seeing your tweet, and seeing a user's tweet, design a timeline. After designing a timeline where you can see your followers’ tweets, scale it for a larger audience. If you still have time, try to scale it for a celebrity use case.
Design for a train ticket booking application like IRCTC. Incorporate auth, features to choose start and end stations, view available trains and available seats between two stations, save reservation of seats from start to end stations, and lock them till payment confirmation.
How will you design a basic relational database? The database should support tables, columns, basic field types like integer and text, foreign keys, and indexes. The way a developer approaches this question is important. A good developer designs a solution around storage and memory management.

Here’s a pro-tip for you. LLD questions can be answered by both beginners and experienced developers. Mostly, senior developers can be expected to answer HLD questions. Choose your interview questions set wisely, and ask questions relevant to your candidate’s experience.

#4 Have you ever worked with SQL? Write queries for a specific use case that requires multiple joins.

Example: Create a table with separate columns for student name, subject, and marks scored. Return student names and ranks of each student. The rank of a student depends on the total of marks in all subjects.

Not all developers would have experience working with SQL but some knowledge about how data is stored/structured is useful. Developers should be familiar with simple concepts like joins, retrieval queries, and the basics of DBMS.

#5 What do you think is wrong with this code?

Instead of asking developer candidates to write code on a piece of paper (which is outdated, anyway), ask them to debug existing code. This is another way to assess their technical skills. Place surreptitious errors in the code and evaluate their attention to detail.

Now that you know exactly what technical skills to look for and when questions to ask when interviewing developers, the time has come to assess the soft skills of these candidates. Part 2 of this blog throws light on the how and why of evaluating candidates based on their communication skills, work ethic, and alignment with the company’s goals.

View all

Hiring Strategies

Best Pre-Employment Assessments: Optimizing Your Hiring Process for 2024

In today's competitive talent market, attracting and retaining top performers is crucial for any organization's success. However, traditional hiring methods like relying solely on resumes and interviews may not always provide a comprehensive picture of a candidate's skills and potential. This is where pre-employment assessments come into play.

What is Pre-Employement Assessment?

Pre-employment assessments are standardized tests and evaluations administered to candidates before they are hired. These assessments can help you objectively measure a candidate's knowledge, skills, abilities, and personality traits, allowing you to make data-driven hiring decisions.

By exploring and evaluating the best pre-employment assessment tools and tests available, you can:

Improve the accuracy and efficiency of your hiring process.
Identify top talent with the right skills and cultural fit.
Reduce the risk of bad hires.
Enhance the candidate experience by providing a clear and objective evaluation process.

This guide will provide you with valuable insights into the different types of pre-employment assessments available and highlight some of the best tools, to help you optimize your hiring process for 2024.

Why pre-employment assessments are key in hiring

While resumes and interviews offer valuable insights, they can be subjective and susceptible to bias. Pre-employment assessments provide a standardized and objective way to evaluate candidates, offering several key benefits:

Improved decision-making:

By measuring specific skills and knowledge, assessments help you identify candidates who possess the qualifications necessary for the job.
Reduced bias:

Standardized assessments mitigate the risks of unconscious bias that can creep into traditional interview processes.
Increased efficiency:

Assessments can streamline the initial screening process, allowing you to focus on the most promising candidates.
Enhanced candidate experience:

When used effectively, assessments can provide candidates with a clear understanding of the required skills and a fair chance to showcase their abilities.

Types of pre-employment assessments

There are various types of pre-employment assessments available, each catering to different needs and objectives. Here's an overview of some common types:

1. Skill Assessments:

Technical Skills: These assessments evaluate specific technical skills and knowledge relevant to the job role, such as programming languages, software proficiency, or industry-specific expertise. HackerEarth offers a wide range of validated technical skill assessments covering various programming languages, frameworks, and technologies.
Soft Skills: These employment assessments measure non-technical skills like communication, problem-solving, teamwork, and critical thinking, crucial for success in any role.

2. Personality Assessments:

These employment assessments can provide insights into a candidate's personality traits, work style, and cultural fit within your organization.

3. Cognitive Ability Tests:

These tests measure a candidate's general mental abilities, such as reasoning, problem-solving, and learning potential.

4. Integrity Assessments:

These employment assessments aim to identify potential risks associated with a candidate's honesty, work ethic, and compliance with company policies.

By understanding the different types of assessments and their applications, you can choose the ones that best align with your specific hiring needs and ensure you hire the most qualified and suitable candidates for your organization.

Leading employment assessment tools and tests in 2024

Choosing the right pre-employment assessment tool depends on your specific needs and budget. Here's a curated list of some of the top pre-employment assessment tools and tests available in 2024, with brief overviews:

HackerEarth:

A comprehensive platform offering a wide range of validated skill assessments in various programming languages, frameworks, and technologies. It also allows for the creation of custom assessments and integrates seamlessly with various recruitment platforms.
SHL:

Provides a broad selection of assessments, including skill tests, personality assessments, and cognitive ability tests. They offer customizable solutions and cater to various industries.
Pymetrics:

Utilizes gamified assessments to evaluate cognitive skills, personality traits, and cultural fit. They offer a data-driven approach and emphasize candidate experience.
Wonderlic:

Offers a variety of assessments, including the Wonderlic Personnel Test, which measures general cognitive ability. They also provide aptitude and personality assessments.
Harver:

An assessment platform focusing on candidate experience with video interviews, gamified assessments, and skills tests. They offer pre-built assessments and customization options.

Remember: This list is not exhaustive, and further research is crucial to identify the tool that aligns best with your specific needs and budget. Consider factors like the types of assessments offered, pricing models, integrations with your existing HR systems, and user experience when making your decision.

Choosing the right pre-employment assessment tool

Instead of full individual tool reviews, consider focusing on 2–3 key platforms. For each platform, explore:

Target audience: Who are their assessments best suited for (e.g., technical roles, specific industries)?
Types of assessments offered: Briefly list the available assessment categories (e.g., technical skills, soft skills, personality).
Key features: Highlight unique functionalities like gamification, custom assessment creation, or seamless integrations.
Effectiveness: Briefly mention the platform's approach to assessment validation and reliability.
User experience: Consider including user reviews or ratings where available.

Comparative analysis of assessment options

Instead of a comprehensive comparison, consider focusing on specific use cases:

Technical skills assessment:

Compare HackerEarth and Wonderlic based on their technical skill assessment options, focusing on the variety of languages/technologies covered and assessment formats.
Soft skills and personality assessment:

Compare SHL and Pymetrics based on their approaches to evaluating soft skills and personality traits, highlighting any unique features like gamification or data-driven insights.
Candidate experience:

Compare Harver and Wonderlic based on their focus on candidate experience, mentioning features like video interviews or gamified assessments.

Additional tips:

Encourage readers to visit the platforms' official websites for detailed features and pricing information.
Include links to reputable third-party review sites where users share their experiences with various tools.

Best practices for using pre-employment assessment tools

Integrating pre-employment assessments effectively requires careful planning and execution. Here are some best practices to follow:

Define your assessment goals:

Clearly identify what you aim to achieve with assessments. Are you targeting specific skills, personality traits, or cultural fit?
Choose the right assessments:

Select tools that align with your defined goals and the specific requirements of the open position.
Set clear expectations:

Communicate the purpose and format of the assessments to candidates in advance, ensuring transparency and building trust.
Integrate seamlessly:

Ensure your chosen assessment tool integrates smoothly with your existing HR systems and recruitment workflow.
Train your team:

Equip your hiring managers and HR team with the knowledge and skills to interpret assessment results effectively.

Interpreting assessment results accurately

Assessment results offer valuable data points, but interpreting them accurately is crucial for making informed hiring decisions. Here are some key considerations:

Use results as one data point:

Consider assessment results alongside other information, such as resumes, interviews, and references, for a holistic view of the candidate.
Understand score limitations:

Don't solely rely on raw scores. Understand the assessment's validity and reliability and the potential for cultural bias or individual test anxiety.
Look for patterns and trends:

Analyze results across different assessments and identify consistent patterns that align with your desired candidate profile.
Focus on potential, not guarantees:

Assessments indicate potential, not guarantees of success. Use them alongside other evaluation methods to make well-rounded hiring decisions.

Choosing the right pre-employment assessment tools

Selecting the most suitable pre-employment assessment tool requires careful consideration of your organization's specific needs. Here are some key factors to guide your decision:

Industry and role requirements:

Different industries and roles demand varying skill sets and qualities. Choose assessments that target the specific skills and knowledge relevant to your open positions.
Company culture and values:

Align your assessments with your company culture and values. For example, if collaboration is crucial, look for assessments that evaluate teamwork and communication skills.
Candidate experience:

Prioritize tools that provide a positive and smooth experience for candidates. This can enhance your employer brand and attract top talent.

Budget and accessibility considerations

Budget and accessibility are essential factors when choosing pre-employment assessments:

Budget:

Assessment tools come with varying pricing models (subscriptions, pay-per-use, etc.). Choose a tool that aligns with your budget and offers the functionalities you need.
Accessibility:

Ensure the chosen assessment is accessible to all candidates, considering factors like language options, disability accommodations, and internet access requirements.

Additional Tips:

Free trials and demos: Utilize free trials or demos offered by assessment platforms to experience their functionalities firsthand.
Consult with HR professionals: Seek guidance from HR professionals or recruitment specialists with expertise in pre-employment assessments.
Read user reviews and comparisons: Gain insights from other employers who use various assessment tools.

By carefully considering these factors, you can select the pre-employment assessment tool that best aligns with your organizational needs, budget, and commitment to an inclusive hiring process.

Remember, pre-employment assessments are valuable tools, but they should not be the sole factor in your hiring decisions. Use them alongside other evaluation methods and prioritize building a fair and inclusive hiring process that attracts and retains top talent.

Future trends in pre-employment assessments

The pre-employment assessment landscape is constantly evolving, with innovative technologies and practices emerging. Here are some potential future trends to watch:

Artificial intelligence (AI):

AI-powered assessments can analyze candidate responses, written work, and even resumes, using natural language processing to extract relevant insights and identify potential candidates.
Adaptive testing:

These assessments adjust the difficulty level of questions based on the candidate's performance, providing a more efficient and personalized evaluation.
Micro-assessments:

Short, focused assessments delivered through mobile devices can assess specific skills or knowledge on-the-go, streamlining the screening process.
Gamification:

Engaging and interactive game-based elements can make the assessment experience more engaging and assess skills in a realistic and dynamic way.

Conclusion

Pre-employment assessments, when used thoughtfully and ethically, can be a powerful tool to optimize your hiring process, identify top talent, and build a successful workforce for your organization. By understanding the different types of assessments available, exploring top-rated tools like HackerEarth, and staying informed about emerging trends, you can make informed decisions that enhance your ability to attract, evaluate, and hire the best candidates for the future.

Hiring Strategies

Tech Layoffs: What To Expect In 2024

Layoffs in the IT industry are becoming more widespread as companies fight to remain competitive in a fast-changing market; many turn to layoffs as a cost-cutting measure. Last year, 1,000 companies including big tech giants and startups, laid off over two lakhs of employees. But first, what are layoffs in the tech business, and how do they impact the industry?

Tech layoffs are the termination of employment for some employees by a technology company. It might happen for various reasons, including financial challenges, market conditions, firm reorganization, or the after-effects of a pandemic. While layoffs are not unique to the IT industry, they are becoming more common as companies look for methods to cut costs while remaining competitive.

The consequences of layoffs in technology may be catastrophic for employees who lose their jobs and the firms forced to make these difficult decisions. Layoffs can result in the loss of skill and expertise and a drop in employee morale and productivity. However, they may be required for businesses to stay afloat in a fast-changing market.

This article will examine the reasons for layoffs in the technology industry, their influence on the industry, and what may be done to reduce their negative impacts. We will also look at the various methods for tracking tech layoffs.

What are tech layoffs?

The term "tech layoff" describes the termination of employees by an organization in the technology industry. A company might do this as part of a restructuring during hard economic times.

In recent times, the tech industry has witnessed a wave of significant layoffs, affecting some of the world’s leading technology companies, including Amazon, Microsoft, Meta (formerly Facebook), Apple, Cisco, SAP, and Sony. These layoffs are a reflection of the broader economic challenges and market adjustments facing the sector, including factors like slowing revenue growth, global economic uncertainties, and the need to streamline operations for efficiency.

Each of these tech giants has announced job cuts for various reasons, though common themes include restructuring efforts to stay competitive and agile, responding to over-hiring during the pandemic when demand for tech services surged, and preparing for a potentially tough economic climate ahead. Despite their dominant positions in the market, these companies are not immune to the economic cycles and technological shifts that influence operational and strategic decisions, including workforce adjustments.

This trend of layoffs in the tech industry underscores the volatile nature of the tech sector, which is often at the mercy of rapid changes in technology, consumer preferences, and the global economy. It also highlights the importance of adaptability and resilience for companies and employees alike in navigating the uncertainties of the tech landscape.

Causes for layoffs in the tech industry

Why are tech employees suffering so much?

Yes, the market is always uncertain, but why resort to tech layoffs?

Various factors cause tech layoffs, including company strategy changes, market shifts, or financial difficulties. Companies may lay off employees if they need help to generate revenue, shift their focus to new products or services, or automate certain jobs.

In addition, some common reasons could be:

Financial struggles

Currently, the state of the global market is uncertain due to economic recession, ongoing war, and other related phenomena. If a company is experiencing financial difficulties, only sticking to pay cuts may not be helpful—it may need to reduce its workforce to cut costs.

Also, read: 6 Steps To Create A Detailed Recruiting Budget (Template Included)

Changes in demand

The tech industry is constantly evolving, and companies would have to adjust their workforce to meet changing market conditions. For instance, companies are adopting remote work culture, which surely affects on-premises activity, and companies could do away with some number of tech employees at the backend.

Restructuring

Companies may also lay off employees as part of a greater restructuring effort, such as spinning off a division or consolidating operations.

Automation

With the advancement in technology and automation, some jobs previously done by human labor may be replaced by machines, resulting in layoffs.

Mergers and acquisitions

When two companies merge, there is often overlap in their operations, leading to layoffs as the new company looks to streamline its workforce.

But it's worth noting that layoffs are not exclusive to the tech industry and can happen in any industry due to uncertainty in the market.

Will layoffs increase in 2024?

It is challenging to estimate the rise or fall of layoffs. The overall state of the economy, the health of certain industries, and the performance of individual companies will play a role in deciding the degree of layoffs in any given year.

But it is also seen that, in the first 15 days of this year, 91 organizations laid off over 24,000 tech workers, and over 1,000 corporations cut down more than 150,000 workers in 2022, according to an Economic Times article.

The COVID-19 pandemic caused a huge economic slowdown and forced several businesses to downsize their employees. However, some businesses rehired or expanded their personnel when the world began to recover.

So, given the current level of economic uncertainty, predicting how the situation will unfold is difficult.

Also, read: 4 Images That Show What Developers Think Of Layoffs In Tech

What types of companies are prone to tech layoffs?

Tech layoffs can occur in organizations of all sizes and various areas.

Following are some examples of companies that have experienced tech layoffs in the past:

Large tech firms

Companies such as IBM, Microsoft, Twitter, Better.com, Alibaba, and HP have all experienced layoffs in recent years as part of restructuring initiatives or cost-cutting measures.

Market scenarios are still being determined after Elon Musk's decision to lay off employees. Along with tech giants, some smaller companies and startups have also been affected by layoffs.

Startups

Because they frequently work with limited resources, startups may be forced to lay off staff if they cannot get further funding or need to pivot due to market downfall.

Small and medium-sized businesses

Small and medium-sized businesses face layoffs due to high competition or if the products/services they offer are no longer in demand.

Companies in certain industries

Some sectors of the technological industry, such as the semiconductor industry or automotive industry, may be more prone to layoffs than others.

Companies that lean on government funding

Companies that rely significantly on government contracts may face layoffs if the government cuts technology spending or contracts are not renewed.

How to track tech layoffs?

You can’t stop tech company layoffs, but you should be keeping track of them. We, HR professionals and recruiters, can also lend a helping hand in these tough times by circulating “layoff lists” across social media sites like LinkedIn and Twitter to help people land jobs quicker. Firefish Software put together a master list of sources to find fresh talent during the layoff period.

Because not all layoffs are publicly disclosed, tracking tech industry layoffs can be challenging, and some may go undetected. There are several ways to keep track of tech industry layoffs:

Use tech layoffs tracker

Layoff trackers like thelayoff.com and layoffs.fyi provide up-to-date information on layoffs.

In addition, they aid in identifying trends in layoffs within the tech industry. It can reveal which industries are seeing the most layoffs and which companies are the most affected.

Companies can use layoff trackers as an early warning system and compare their performance to that of other companies in their field.

News articles

Because many news sites cover tech layoffs as they happen, keeping a watch on technology sector stories can provide insight into which organizations are laying off employees and how many individuals have been affected.

Social media

Organizations and employees frequently publish information about layoffs in tech on social media platforms; thus, monitoring companies' social media accounts or following key hashtags can provide real-time updates regarding layoffs.

Online forums and communities

There are online forums and communities dedicated to discussing tech industry news, and they can be an excellent source of layoff information.

Government reports

Government agencies such as the Bureau of Labor Statistics (BLS) publish data on layoffs and unemployment, which can provide a more comprehensive picture of the technology industry's status.

How do companies reduce tech layoffs?

Layoffs in tech are hard – for the employee who is losing their job, the recruiter or HR professional who is tasked with informing them, and the company itself. So, how can we aim to avoid layoffs? Here are some ways to minimize resorting to letting people go:

Salary reductions

Instead of laying off employees, businesses can lower the salaries or wages of all employees. It can be accomplished by instituting compensation cuts or salary freezes.

Implementing a hiring freeze

Businesses can halt employing new personnel to cut costs. It can be a short-term solution until the company's financial situation improves.

Also, read: What Recruiters Can Focus On During A Tech Hiring Freeze

Non-essential expense reduction

Businesses might search for ways to cut or remove non-essential expenses such as travel, training, and office expenses.

Reducing working hours

Companies can reduce employee working hours to save money, such as implementing a four-day workweek or a shorter workday.

These options may not always be viable and may have their problems, but before laying off, a company owes it to its people to consider every other alternative, and formulate the best solution.

Tech layoffs to bleed into this year

While we do not know whether this trend will continue or subside during 2023, we do know one thing. We have to be prepared for a wave of layoffs that is still yet to hit. As of last month, Layoffs.fyi had already tracked 170+ companies conducting 55,970 layoffs in 2023.

So recruiters, let’s join arms, distribute those layoff lists like there’s no tomorrow, and help all those in need of a job! :)

Hiring Tools

What is Headhunting In Recruitment?: Types & How Does It Work?

In today’s fast-paced world, recruiting talent has become increasingly complicated. Technological advancements, high workforce expectations and a highly competitive market have pushed recruitment agencies to adopt innovative strategies for recruiting various types of talent. This article aims to explore one such recruitment strategy – headhunting.

What is Headhunting in recruitment?

In headhunting, companies or recruitment agencies identify, engage and hire highly skilled professionals to fill top positions in the respective companies. It is different from the traditional process in which candidates looking for job opportunities approach companies or recruitment agencies. In headhunting, executive headhunters, as recruiters are referred to, approach prospective candidates with the hiring company’s requirements and wait for them to respond. Executive headhunters generally look for passive candidates, those who work at crucial positions and are not on the lookout for new work opportunities. Besides, executive headhunters focus on filling critical, senior-level positions indispensable to companies. Depending on the nature of the operation, headhunting has three types. They are described later in this article. Before we move on to understand the types of headhunting, here is how the traditional recruitment process and headhunting are different.

How do headhunting and traditional recruitment differ from each other?

Headhunting is a type of recruitment process in which top-level managers and executives in similar positions are hired. Since these professionals are not on the lookout for jobs, headhunters have to thoroughly understand the hiring companies’ requirements and study the work profiles of potential candidates before creating a list.

In the traditional approach, there is a long list of candidates applying for jobs online and offline. Candidates approach recruiters for jobs. Apart from this primary difference, there are other factors that define the difference between these two schools of recruitment.

AspectHeadhuntingTraditional RecruitmentCandidate TypePrimarily passive candidateActive job seekersApproachFocused on specific high-level rolesBroader; includes various levelsScopeproactive outreachReactive: candidates applyCostGenerally more expensive due to expertise requiredTypically lower costsControlManaged by headhuntersManaged internally by HR teams

All the above parameters will help you to understand how headhunting differs from traditional recruitment methods, better.

Types of headhunting in recruitment

Direct headhunting: In direct recruitment, hiring teams reach out to potential candidates through personal communication. Companies conduct direct headhunting in-house, without outsourcing the process to hiring recruitment agencies. Very few businesses conduct this type of recruitment for top jobs as it involves extensive screening across networks outside the company’s expanse.

Indirect headhunting: This method involves recruiters getting in touch with their prospective candidates through indirect modes of communication such as email and phone calls. Indirect headhunting is less intrusive and allows candidates to respond at their convenience.Third-party recruitment: Companies approach external recruitment agencies or executive headhunters to recruit highly skilled professionals for top positions. This method often leverages the company’s extensive contact network and expertise in niche industries.

How does headhunting work?

Finding highly skilled professionals to fill critical positions can be tricky if there is no system for it. Expert executive headhunters employ recruitment software to conduct headhunting efficiently as it facilitates a seamless recruitment process for executive headhunters. Most software is AI-powered and expedites processes like candidate sourcing, interactions with prospective professionals and upkeep of communication history. This makes the process of executive search in recruitment a little bit easier. Apart from using software to recruit executives, here are the various stages of finding high-calibre executives through headhunting.

Identifying the role

Once there is a vacancy for a top job, one of the top executives like a CEO, director or the head of the company, reach out to the concerned personnel with their requirements. Depending on how large a company is, they may choose to headhunt with the help of an external recruiting agency or conduct it in-house. Generally, the task is assigned to external recruitment agencies specializing in headhunting. Executive headhunters possess a database of highly qualified professionals who work in crucial positions in some of the best companies. This makes them the top choice of conglomerates looking to hire some of the best talents in the industry.

Defining the job

Once an executive headhunter or a recruiting agency is finalized, companies conduct meetings to discuss the nature of the role, how the company works, the management hierarchy among other important aspects of the job. Headhunters are expected to understand these points thoroughly and establish a clear understanding of their expectations and goals.

Candidate identification and sourcing

Headhunters analyse and understand the requirements of their clients and begin creating a pool of suitable candidates from their database. The professionals are shortlisted after conducting extensive research of job profiles, number of years of industry experience, professional networks and online platforms.

Approaching candidates

Once the potential candidates have been identified and shortlisted, headhunters move on to get in touch with them discreetly through various communication channels. As such candidates are already working at top level positions at other companies, executive headhunters have to be low-key while doing so.

Assessment and Evaluation

In this next step, extensive screening and evaluation of candidates is conducted to determine their suitability for the advertised position.

Interviews and negotiations

Compensation is a major topic of discussion among recruiters and prospective candidates. A lot of deliberation and negotiation goes on between the hiring organization and the selected executives which is facilitated by the headhunters.

Finalizing the hire

Things come to a close once the suitable candidates accept the job offer. On accepting the offer letter, headhunters help finalize the hiring process to ensure a smooth transition.

The steps listed above form the blueprint for a typical headhunting process. Headhunting has been crucial in helping companies hire the right people for crucial positions that come with great responsibility. However, all systems have a set of challenges no matter how perfect their working algorithm is. Here are a few challenges that talent acquisition agencies face while headhunting.

Common challenges in headhunting

Despite its advantages, headhunting also presents certain challenges:

Cost Implications: Engaging headhunters can be more expensive than traditional recruitment methods due to their specialized skills and services.

Time-Consuming Process: While headhunting can be efficient, finding the right candidate for senior positions may still take time due to thorough evaluation processes.

Market Competition: The competition for top talent is fierce; organizations must present compelling offers to attract passive candidates away from their current roles.

Although the above mentioned factors can pose challenges in the headhunting process, there are more upsides than there are downsides to it. Here is how headhunting has helped revolutionize the recruitment of high-profile candidates.

Advantages of Headhunting

Headhunting offers several advantages over traditional recruitment methods:

Access to Passive Candidates: By targeting individuals who are not actively seeking new employment, organisations can access a broader pool of highly skilled professionals.

Confidentiality: The discreet nature of headhunting protects both candidates’ current employment situations and the hiring organisation’s strategic interests.

Customized Search: Headhunters tailor their search based on the specific needs of the organization, ensuring a better fit between candidates and company culture.

Industry Expertise: Many headhunters specialise in particular sectors, providing valuable insights into market dynamics and candidate qualifications.

Conclusion

Although headhunting can be costly and time-consuming, it is one of the most effective ways of finding good candidates for top jobs. Executive headhunters face several challenges maintaining the g discreetness while getting in touch with prospective clients. As organizations navigate increasingly competitive markets, understanding the nuances of headhunting becomes vital for effective recruitment strategies. To keep up with the technological advancements, it is better to optimise your hiring process by employing online recruitment software like HackerEarth, which enables companies to conduct multiple interviews and evaluation tests online, thus improving candidate experience. By collaborating with skilled headhunters who possess industry expertise and insights into market trends, companies can enhance their chances of securing high-caliber professionals who drive success in their respective fields.

View all

Tech Tutorials

All about Machine Learning

Introduction to Object Detection

Object Localization

Defining the target variable

Loss Function

Object Detection

Convolutional implementation of sliding windows

The YOLO (You Only Look Once) Algorithm

Conclusion

Composing Jazz Music with Deep Learning

Recurrent Neural Networks

Dataset

Data Preprocessing

Notes Extraction

Generating Input and Output Sequences

Model Architecture

Model Training

Music Sample Generation

Conclusion

A beginner's tutorial on the apriori algorithm in data mining with R implementation

General Process of the Apriori algorithm

Mining Association Rules

Pros of the Apriori algorithm

Cons of the Apriori Algorithm

R implementation

Summary

Building your own Lisp Parser Part I

Prerequisites

Objectives

Step 1: Writing the Grammar

Step 2: Translating the Grammar into Pseudocode

3. Writing the Code

Explanation

expression():

number():

operator():

calc():

UngetableInput

Homework

What's Coming Next?

4 performance optimization tips for faster Python code

1. Get the whole setup ready before-hand

2. Get the code working first

3. Python Coding Tips

Strings:

Language Constructs:

Use the standard library:

Generators:

4. Algorithms and Data structures

Reference:

Technical Screening Guide: All You Need To Know

Explore HackerEarth’s top products for Hiring & Innovation

Meet our Authors

7 Tech Recruiting Trends To Watch Out For In 2024

7 tech recruiting trends for 2024

Trend #1—Leverage data-driven recruiting

Trend #2—Have impactful employer branding

Trend #3—Focus on candidate-driven market

Trend #4—Have a diversity and inclusion oriented company culture

Trend #5—Embed automation and AI into your recruitment systems

Trend #6—Conduct remote interviews

Trend #7—Be proactive in candidate engagement

Recruiting Tech Talent Just Got Easier With HackerEarth

(Part 2) Essential Questions To Ask When Interviewing Developers In 2021

What constitutes a good interview question?

Hire GREAT developers by asking the right questions

#1 How do you plan your day?

#2 Do you have experience using tools for collaboration and remote work?

Value-based interview questions to ask

#3 Please describe three recent projects that you worked on. What were the most interesting and challenging parts?

#4 You’ve worked long and hard to deliver a complex feature for a client and they say it’s not what they asked for. How would you take it?

#5 What new programming languages or technologies have you learned recently?

#6 What does the perfect release look like? Who is involved and what is your role?

#7 Tell me about a time when you had to convince your lead to try a different approach?

#8 What have you done with all the extra hours during the pandemic?

(Part 1) Essential Questions To Ask When Recruiting Developers In 2021

Recruit GREAT developers by asking the right questions

#1 Write an algorithm for the following

#2 Formulate solutions for the below low-level design (LLD) questions