New Features for Flux.jl

Flux.jl is a high-level neural network library in Julia, which has an intuitive API, easy to debug, and performant. One of the key features of Flux.jl is that it allows user to define a layer in Julia, using most Julia Array operations, while not to worry about size inference, grad calculations, etc. Unlike Knet.jl, a promising competitor of Flux.jl, which do all the tensor calculations from scratch with a lot of magics, Flux.jl simply transcompile the computation graph to mature backends like MXNet and TensorFlow. Thus, it can take the benefits of those advanced technologies and get a cutting-edge performance.

Flux.jl is in alpha and lacks a lot of features before being actually useful. One of the problems is currently it only supports SGD optimizer without decay and momentum. It also only shows the progress during training, while saving checkpoints or doing custom evaluations once a while can help users to build a better model.

After this program finished, Flux.jl will be able to:

• train a model with most popular optimizers like Adagrad, Adam, etc.
• hook functions at specific time point, such as after every 100 batches