Online augmentation tests

[gngdb]

We're running a number of models using online augmentation as the code to do it is now (just) working. Most importantly, we want to be able to replicate the results of the simple 8 augmentations with normalisation we've already submitted to the leaderboard. The full list is:

• 8aug repeat with three different initial learning rates
• many more augmentations (based on spec from Scott, more info to follow)
• 96 by 96 with rotation and flip augmentations, as with the 8aug
• 128 by 128 as above

[gngdb]

All seem to have diverged/failed. As the results are so far from what we saw without online augmentation suspect some large bug in the ListDataset.

[gngdb]

First of these is now done. Somehow got better score than before; could not think of good explanation why.

[scottclowe]

The online augmentations are as follows:

• flip: Reflects the image in the x-direction
  • 0 or undefined: never flips
  • 1: Flips with probability 50%
• rotate: Rotates the image
  • 0 or undefined: no rotate augmentations
  • [list,of,angles,in,degrees]: pick an angle at random from the list and rotate by that
  • N, where N is integer>1: pick a random augmentation from k/N * 360 for k=0,...,N
  • -1: pick a float from [0,360) with uniform distribution
• shear: Shears (skews) the image
  • 0 or undefined: no shearing
  • [list,of,angles,in,degrees]: pick an angle at random from the list and shear the image by that much
  • S, where S is a float (should be order of 1): pick a shear angle from a Gaussian distribution centred around 0 with standard deviation S
• scale: Scales the image
  • 0 or undefined: no scaling
  • [list,of,proportional,scale,factors]: pick a scale factor at random from the list and scale the image by that scale factor (1 is no change, 0.9 etc is bigger, 1.1 etc is smaller [yes the opposite to expected])
  • S, where S is a float (should be order of 0.1): pick a scale factor from a Gaussian distribution centred around 1.0 with standard deviation S
• scale_asym: Stretches the image by scaling x and y differently. x and y scales use the scale factor from scale, plus a bonus scale each
  • 0 or undefined: no asymmetric scaling
  • S, where S is a float (should be order of 0.01): for both x and y, pick a scale factor from a Gaussian distribution centred around 0 with standard deviation S. Then each is scaled by globalscalefactor+dimensionalasymscalefactor
• shunt: Translates the image in the x-y plane
  • 0 or undefined: no translations
  • [list,of,proportional,translations]: for x and y, pick a translation at random from the list and shift the image by these amounts for x and y respectively
  • S, where S is a float (should be order of 0.1): for both x and y, pick a translation from a Gaussian distribution centred around 0 with standard deviation S. Then each is shifted by this proportion of the image width or height respectively.

Also

• transform_order: Defines the polynomial order of the image warp.
  • 0.5 or undefined: default. Slower, but gives better results than 0 or 1.
  • 0: nearest neighbour
  • 1: bilinear interpolation
  • 2,3,...: spline, cubic, etc
  • X for 0<X<1: weighted average of nearest neighbour and bilinear.

[scottclowe]

I am currently using these augmentations:

                flip: 1,
                rotate: -1,
                shear: 6,
                scale: 0.08,
                scale_asym: 0.008,
                shunt: 0.04