End of the first work period

Hi all,

this is the end of this first period of GSoC! It was a challenging but very interesting period and I am very excited about the next two months. It is the first time where I have the opportunity to make something that has a real value, albeit small, on someone else's work. Even when I have to do small tasks, like structure the package or learn Tensorflow APIs, I do it with enthusiasm because I have very clear in mind the ultimate goal and the value that my efforts would bring. It may seem trivial, but for a student this is not the daily bread :) I really have fun coding for this project and I hope this will last until the end!

Speaking about the project, I've spent some time wondering which was the best solution to test the correct installation of the Tensorflow Python APIs on the machine. The last solution was putting the test in a function __nnet_init__ and calling it in the PKG_ADD (code in inst/__nnet_init__ in my repo [1]).

Regarding the code, in this last days I tried to connect the dots, calling a Tensorflow network from Octave in a "Matlab compatible" way. In particular, I use the classes that I made two weeks ago in order to implement a basic version of trainNetwork, that is the core function of this package. As I explained in my post of June 12, trainNetwork takes as input the data and two objects: the layers and the options. I had some some difficulty during the implementation of the Layer class due to the inheritance and the overloading. Eventually, I decided to store the layers in a cell array as attribute of the Layer class. Overloading the subsref, I let the user call a specific layer with the '()' access, like a classic array. With this kind of overloading I managed to solve the main problem of this structure, that is the possibility to get a property of a layer doing for example layers(1).Name

classdef Layer < handle
  properties (Access = private)
    layers = {};
  endproperties

   methods (Hidden, Access = {?Layers})
    function this = Layer (varargin)
        nargin = numel(varargin);
        this.layers = cell(1, nargin);
      for i = 1:nargin
        this.layers{i} = varargin{i};
      end
    endfunction
  endmethods

   methods (Hidden)
    function obj = subsref(this, idx)
      switch idx(1).type
          case '()'
              idx(1).type = '{}';
              obj = builtin('subsref',this.layers,idx);
          case '{}'
              error('{} indexing not supported');
          case '.'
            obj = builtin('subsref',this,idx);
      endswitch
    endfunction

     function obj = numel(this)
      obj = builtin('numel',this.layers);
    endfunction

     function obj = size(this)
      obj = builtin('size',this.layers);
    endfunction
endmethods
endclassdef


Therefore, I implemented the same example of my last post in a proper way. In tests/script you can find the function cnn_linear_model which consists simply in:

1. Loading the datasets
  load('tests/examples/cnn_linear_model/train_70.mat')
  load('tests/examples/cnn_linear_model/test_70.mat')

2. Defining layers and options
  layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(12,25)
    reluLayer
    fullyConnectedLayer(1)
    regressionLayer];
  options = trainingOptions('sgdm', 'MaxEpochs', 1);

3. Training
net = trainNetwork(trainImages, trainAngles, layers, options);
4. Prediction
acc = net.predict(testImages, testAngles) 
TrainNetwork is a draft and I have not yet implemented the classe seriesNetwork, but I think it's a good start :) In next weeks I will focus on the Tensorflow backend of the above mentioned functions, with the goal of having a working version at the end of the second period!

Peace,
Enrico

[1] https://bitbucket.org/cittiberto/octave-nnet/src

Comments

Post a Comment

Popular posts from this blog

Train a Convolutional Neural Network for Regression

Image
Hello, I spent the last period working mostly on Tensorflow, studying the APIs and writing some examples in order to explore the possible implementations of neural networks. For this goal, I chose an interesting example proposed in the Matlab examples at [1]. The dataset is composed by 5000 images, rotated by an angle α, and a corresponding integer label (the rotation angle α). The goal is to make a regression to predict the angle of a rotated image and straighten it up. All files can be found in  tests/examples/cnn_linear_model  in my repo [2]. I have kept the structure as in the Matlab example, but I generated a new dataset starting from LeCun's MNIST digits (datasets at [3]). Each image was rotated by a random angle between 0° and 70°, in order to keep the right orientation of the digits (code in dataset_generation.m ). In Fig. 1 some rotated digits with the corresponding original digits. Figure 1. rotated images in columns 1,3,5 and originals in columns 2,4,6
Read more

Deep learning functions

Image
Hi there, the second part of the project is finishing. This period was quite interesting because I had to dive into the theory behind Neural Networks  In particular [1], [2], [3] and [4] were very useful and I will sum up some concepts here below. On the other hand,  coding became more challenging and t he focus was on the python layer and in particular the way to structure the class in order to make everything scalable and generalizable. Summarizing the situation, in the first period I implemented all the Octave classes for the user interface. Those are Matlab compatible and they call some Python function in a seamless way. On the Python side, the TensorFlow API is used to build the graph of the Neural Network and perform training, evaluation and prediction. I implemented the three core functions: trainNetwork, SeriesNetwork and trainingOptions. To do this, I used a Python class in which I initialize an object with the graph of the network and I store this object as attribute of S
Read more

Summary of work done during GSoC

GSoC17 is at the end and I want to thank my mentors and the Octave community for giving me the opportunity to participate in this unique experience. During this Google Summer of Code, my goal was to implement from scratch the Convolutional Neural Networks package for GNU Octave. It  will be integrated with the already existing nnet package. This was a very interesting project and a stimulating experience for both the implemented code and the theoretical base behind the algorithms treated. A part has been implemented in Octave and an other part in Python using the Tensorflow API. Code repository All the code implemented during these months can be found in my public repository:    https://bitbucket.org/cittiberto/gsoc-octave-nnet/commits/all (my username: citti berto , bookmark enrico ) Since I implemented a completely new part of the package, I pushed the entire project in three commits and I wait for the community approving for preparing a PR for the official packa
Read more