After DiGi posted about it, the issue of the missing Katyusha spread around. Even Brickmuppet got in on it (with a cute animated gif).

One thing amuses me about it is the selection of the song. Russians have a whole of two songs, which I find more appropriate for tank anime: the official item, called "March of the Soviet Tankmen", and the alternative popular song, called "Three Tankists". It is the latter which deals with "three tankers, three merry friends" crushing samurai with "the power of fire and steel". It would be a mega troll to use that one. Both songs used to be instantly recognized in Russia, but I suppose "Katyusha" had more pull in Japan overall, so, alas, the two excellent candidates were passed over. Opportunity, missed!

UPDATE: Omo commented:

I actually got CR to comment on this and it comes down to that Japan doesn't want to keep the footage in the episode. It probably won't be CR's place to pay for it, given the type of right required and the sort of licensing agreement they have (remember this is also a Sentai title). [...]

In the same time, the "best part of Taishou Yakyuu Musume" (sung by Koume) was clearead, as were all the remixes in Nichijou. It appears that it was about the foreign song after all. [Omo also posted about it on his own blog.]