>   Multiple times, the video referred to "an infinite variety of latent
>   images of cats."  The infinite-ness surprised me.  I can understand it
>   would be an astronomically large variety, and thereby *practically*
>   infinite.  But, it would seem to me that the training data and the
>   computer precision would put real finite constraints on this, no?

Great question.  It's true that any digital computer has only a finite
number of states.  But the parameters and outputs of a neural network
are considered to be real numbers, and the number of reals is infinite.

    Brief aside: reals are the infinite ideal; floats are a discrete
    approximation to this ideal.  If you're using 32 bit floats then
    there are fewer than 4.3 billion (2**32) distinct floating point
    values, ranging from -3.4 x 10^38 to 3.4 x 10^38.  The reason there
    aren't exactly 2**32 distinct floats has to do with technical issues
    of the exponent/mantissa representation they use.  In any case, you
    can get more float values by switching to 64 or 128 bit precision,
    but the number of values is still finite.

When a neural net is implemented on a digital computer, what you get is
a discrete approximation to the continuous-valued model.  If 32 bit
floats don't offer enough precision, you can upgrade to 64 or even 128
bits.  And in theory you could keep going as far as you wanted.  There
is nothing about the neural net that limits the number of discrete
states you can use to simulate it.  The neural net itself, and the
manifold it is sampling, are real-valued.

However, just because the neural net model can generate an infinite
variety of images, that doesn't mean it can generate every possible
image.  There will be many images that, due to the training data used,
don't lie anywhere on the manifold.

-- Dave