Gradient descent

Deep Learning

The gradient descent algorithm performs multidimensional optimization. The objective is to reach the global maximum. Gradient descent is a popular optimization technique used in many machine-learning models. It is used to improve or optimize the model prediction. One...

Backpropagation

Deep Learning

A backpropagation algorithm is commonly used for training artificial neural networks. The weights are updated from backward based on the error calculated as shown in the following image: After calculating the error, gradient descent can be used to calculate the weight...

Training neural networks

Deep Learning

Training ANN is tricky as it contains several parameters to optimize. The procedure of updating the weights is called backpropagation. The procedure to minimize the error is called optimization. 

L1 and L2 regularization

Deep Learning

L1 penalizes the absolute value of the weight and tends to make the weights zero. L2 penalizes the squared value of the weight and tends to make the weight smaller during the training. Both the regularizes assume that models with smaller weights are better.

Batch normalization

Deep Learning

Batch normalization, or batch-norm, increase the stability and performance of neural network training. It normalizes the output from a layer with zero mean and a standard deviation of 1. This reduces overfitting and makes the network train faster. It is very useful in...

Dropout in Neural Network

Deep Learning

Dropout is an effective way of regularizing neural networks to avoid the overfitting of ANN. During training, the dropout layer cripples the neural network by removing hidden units stochastically as shown in the following image:   Note how the neurons are...

Cross-entropy

Deep Learning

Cross-entropy compares the distance between the outputs of softmax and one-hot encoding. Cross-entropy is a loss function for which error has to be minimized. Neural networks estimate the probability of the given data to every class. The probability has to be...

One-hot encoding

data mining, Deep Learning

One-hot encoding is a way to represent the target variables or classes in case of a classification problem. The target variables can be converted from the string labels to one-hot encoded vectors. A one-hot vector is filled with 1 at the index of the target class but...

Perceptron

Deep Learning

An artificial neuron or perceptron takes several inputs and performs a weighted summation to produce an output. The weight of the perceptron is determined during the training process and is based on the training data. The following is a diagram of the perceptron:...

Fire-i and NAVITAR Scope Setup

Computer, image processing

This section details the use of the Fire-i program to control the NAVITAR scope for capturing images and movies to use with the line tracking program or for imaging purposes. 1. Connect the Firewire cable to the NAVITAR scope. 2. Open Fire-i . 3. Ensure the settings...

Paper

Computer

https://scholar.google.com/citations?user=1zIFSngAAAAJ&hl=en   https://matlab1.com/wp-content/uploads/papers/Analog CMOS Implementation of order weight Average operator for fuzzy logic controller chip.pdf  ...