What is a Neural Network
Neural Network is a computational algorithm that is used in creating deep learning models for predictions and classifications. It is based on self-learning and training, rather than being explicitly programmed.
Neural Networks are inspired by the central nervous system of human beings; neural networks have connected nodes that are very similar to the neurons in a human body.
In this article, I will build a Neural Network using Deep Learning to make you understand how neural networks work.
Also, read – Dog and Cat Classification with Convolutional neural networks (CNN)
Architecture of a Neural Network
To design the architecture of a neural network, you need to choose how many hidden layers will your neural network will have. I will build a neural network with two hidden layers. Determining how many hidden layers your Neural Networks will have is the part of the architecture design of a neural network. The role of every hidden layer of neural networks is to transform the inputs into something the output layer can work with.
Hidden Layer One
In the process, you also need to determine how many nodes will be present in the hidden layer one. These nodes are also known as neurons or features in simple words. In the figure above each neuron is represented by each circle. In the hidden layer every neuron corresponds to a word in the data set. Each neuron features a weight value, and during the preparing stage the neural alter these values to deliver the most accurate output.
Hidden Layer Two
The hidden layer two does the same as the hidden layer one, but now the input of the hidden layer two is the output of the hidden layer one.
Output Layer
And finally at last we reach the output layer. In the output layer your will use the one-hot encoding to see the results of the layers. In this encoding process only one bit will have the value 1 and all others will have the value 0. For example, if we are encoding three random categories:
+——————-+———–+
| category | value |
+——————-|———–+
| sports | 001 |
| space | 010 |
| computer graphics | 100 |
|——————-|———–|
Classifying Text with Neural Networks
Now I will build a Neural Network, to show you how it works. The dataset I will use here has a lot of texts in English. Now I will manipulate this data to pass it into a neural network.
import numpy as np #numpy is a package for scientific computing
from collections import Counter
vocab = Counter()
text = "Hi from Brazil"
#Get all words
for word in text.split(' '):
vocab[word]+=1
#Convert words to indexes
def get_word_2_index(vocab):
word2index = {}
for i,word in enumerate(vocab):
word2index[word] = i
return word2index
#Now we have an index
word2index = get_word_2_index(vocab)
total_words = len(vocab)
#This is how we create a numpy array (our matrix)
matrix = np.zeros((total_words),dtype=float)
#Now we fill the values
for word in text.split():
matrix[word2index[word]] += 1
print(matrix)Code language: Python (python)[ 1. 1. 1.]
In the above code the text was ‘Hi from Brazil’ and the matrix gave output as [ 1. 1. 1.]. What if the text was only ‘Hi’?
matrix = np.zeros((total_words),dtype=float)
text = "Hi"
for word in text.split():
matrix[word2index[word.lower()]] += 1
print(matrix)Code language: Python (python)[ 1. 0. 0.]
Now I will do the same thing with the labels, here you will see what one-hot encoding means:
y = np.zeros((3),dtype=float)
if category == 0:
y[0] = 1. # [ 1. 0. 0.]
elif category == 1:
y[1] = 1. # [ 0. 1. 0.]
else:
y[2] = 1. # [ 0. 0. 1.]Code language: Python (python)Now lets work on the data set to build the neural networks. Lets start with loading the dataset:
from sklearn.datasets import fetch_20newsgroups
categories = ["comp.graphics","sci.space","rec.sport.baseball"]
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)Code language: Python (python)Training Model to Build Neural Network
In the language of neural networks, one epoch means one forward pass, which means getting the output values and one backward passing of all the training samples.
n_input = total_words # Words in vocab
n_classes = 3 # Categories: graphics, sci.space and baseball
input_tensor = tf.placeholder(tf.float32,[None, n_input],name="input")
output_tensor = tf.placeholder(tf.float32,[None, n_classes],name="output")Code language: Python (python)Now I will separate the training data into batches:
training_epochs = 10
# Launch the graph
with tf.Session() as sess:
sess.run(init) #inits the variables (normal distribution, remember?)
# Training cycle
for epoch in range(training_epochs):
avg_cost = 0.
total_batch = int(len(newsgroups_train.data)/batch_size)
# Loop over all batches
for i in range(total_batch):
batch_x,batch_y = get_batch(newsgroups_train,i,batch_size)
# Run optimization op (backprop) and cost op (to get loss value)
c,_ = sess.run([loss,optimizer], feed_dict={input_tensor: batch_x, output_tensor:batch_y})Code language: Python (python)Now, we have trained our model for Neural Networks, now let’s test our model by looking at the accuracy of the model.
# Test model
index_prediction = tf.argmax(prediction, 1)
index_correct = tf.argmax(output_tensor, 1)
correct_prediction = tf.equal(index_prediction, index_correct)
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
total_test_data = len(newsgroups_test.target)
batch_x_test,batch_y_test = get_batch(newsgroups_test,0,total_test_data)
print("Accuracy:", accuracy.eval({input_tensor: batch_x_test, output_tensor: batch_y_test}))Code language: Python (python)Epoch: 0001 loss= 1133.908114347
Epoch: 0002 loss= 329.093700409
Epoch: 0003 loss= 111.876660109
Epoch: 0004 loss= 72.552971845
Epoch: 0005 loss= 16.673050320
Epoch: 0006 loss= 16.481995190
Epoch: 0007 loss= 4.848220565
Epoch: 0008 loss= 0.759822878
Epoch: 0009 loss= 0.000000000
Epoch: 0010 loss= 0.079848485
Optimization Finished! Accuracy: 0.75
And finally we have created a deep learning model using Neural Networks to classify texts into categories. I hope you likes this article, feel free to ask you questions about neural networks or any other topic in the comments section down below.
Also, read – Artificial Neural Networks with Machine Learning
