Blanket

Catalan Numbers(2016)

The Catalan numbers are a sequence of natural numbers that occur in various problems in combinatorial mathematics. For example, the number of expressions containing \(n\) valid pairs of parentheses is the \(n\)th Catalan number. Suppose you have a grid of \(n \times n\) squares, the \(n\)th Catalan number represents the number of paths with a length of \(2n\) that lead from the upper left corner to the lower right corner without intersecting the diagonal dotted line running from the upper left to the lower right. Later in the page, we will derive explicit formulas for the Catalan numbers in two different ways.

#Catalan Numbers

July 29, 2023

Layer Normalization (2016)

Layer Normalization, introduced in the 2016 paper Layer Normalization, is a technique used to normalize the summed inputs to the neurons in a layer by using the mean and standard deviation of those inputs within that layer.

Batch Normalization uses the mean and the standard deviation of the summed inputs of a neuron in a mini-batch to normalize the summed inputs to it. While Batch Normalization can accelerate training, it depends on mini-batch size and is challenging to implement in RNNs.

#Layer Normalization

June 17, 2023

Posts

Catalan Numbers(2016)

Layer Normalization (2016)