It doesn't apply to Latvia specifically, but I recommend James Michener's "Poland", for an idea of how middle-european countries were occupied by one side or the other. It is a very engaging novel, of course fictionalized, but it's much more readable than "dry" history :)

The result being "who owns what" can be very muddy indeed.

http://en.wikipedia.org/wiki/Poland_(novel)

http://www.amazon.com/Poland-James-A-Michener/dp/0394531892